{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Comparing Scenarios\n", "\n", "### The Boltzmann Wealth Model " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "If you want to get straight to the tutorial checkout these environment providers:
\n", "(with Google Account) [![Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/projectmesa/mesa/blob/main/docs/tutorials/8_comparing_scenarios.ipynb)
\n", "(No Google Account) [![Binder](https://mybinder.org/badge_logo.svg)](https://mybinder.org/v2/gh/projectmesa/mesa/main?labpath=docs%2Ftutorials%2F8_comparing_scenarios.ipynb) (This can take 30 seconds to 5 minutes to load)\n", "\n", "*If you are running locally, please ensure you have the latest Mesa version installed.*\n", "\n", "## Tutorial Description\n", "\n", "This tutorial extends the Boltzmann wealth model from the [Batch Run tutorial](https://mesa.readthedocs.io/latest/tutorials/2_batch_run.html), by showing some ways in which users can analyze `batch_run` results. \n", "\n", "*If you are starting here please see the [Running Your First Model tutorial](https://mesa.readthedocs.io/latest/tutorials/0_first_model.html) for dependency and start-up instructions*" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### IN COLAB? - Run the next cell " ] }, { "cell_type": "raw", "metadata": { "vscode": { "languageId": "raw" } }, "source": [ "%pip install --quiet mesa[rec]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Import Dependencies\n", "This includes importing of dependencies needed for the tutorial." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false, "jupyter": { "outputs_hidden": false } }, "outputs": [], "source": [ "# Has multi-dimensional arrays and matrices.\n", "# Has a large collection of mathematical functions to operate on these arrays.\n", "import numpy as np\n", "\n", "# Data manipulation and analysis.\n", "import pandas as pd\n", "\n", "# Data visualization tools.\n", "import seaborn as sns\n", "\n", "import mesa\n", "\n", "# Import Cell Agent and OrthogonalMooreGrid\n", "from mesa.discrete_space import CellAgent, OrthogonalMooreGrid" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Base Model\n", "\n", "The below provides the base model from which we conduct a parameter sweep by altering the population parameter and running each variation for 5 scenarios. \n", "\n", "This is from the [Batch Run tutorial](https://mesa.readthedocs.io/latest/tutorials/7_batch_run.html) tutorial. If you have any questions about it functionality please review that tutorial." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def compute_gini(model):\n", " agent_wealths = [agent.wealth for agent in model.agents]\n", " x = sorted(agent_wealths)\n", " n = model.num_agents\n", " B = sum(xi * (n - i) for i, xi in enumerate(x)) / (n * sum(x))\n", " return 1 + (1 / n) - 2 * B\n", "\n", "\n", "class MoneyAgent(CellAgent):\n", " \"\"\"An agent with fixed initial wealth.\"\"\"\n", "\n", " def __init__(self, model, cell):\n", " super().__init__(model)\n", " self.cell = cell\n", " self.wealth = 1\n", " self.steps_not_given = 0\n", "\n", " def move(self):\n", " self.cell = self.cell.neighborhood.select_random_cell()\n", "\n", " def give_money(self):\n", " cellmates = [a for a in self.cell.agents if a is not self]\n", "\n", " if len(cellmates) > 0 and self.wealth > 0:\n", " other = self.random.choice(cellmates)\n", " other.wealth += 1\n", " self.wealth -= 1\n", " self.steps_not_given = 0\n", " else:\n", " self.steps_not_given += 1\n", "\n", "\n", "class MoneyModel(mesa.Model):\n", " \"\"\"A model with some number of agents.\"\"\"\n", "\n", " def __init__(self, n, width, height, seed=None):\n", " super().__init__(seed=seed)\n", " self.num_agents = n\n", " self.grid = OrthogonalMooreGrid(\n", " (width, height), torus=True, capacity=10, random=self.random\n", " )\n", " # Instantiate DataCollector\n", " self.datacollector = mesa.DataCollector(\n", " model_reporters={\"Gini\": compute_gini},\n", " agent_reporters={\"Wealth\": \"wealth\", \"Steps_not_given\": \"steps_not_given\"},\n", " )\n", " self.running = True\n", "\n", " # Create agents\n", " agents = MoneyAgent.create_agents(\n", " self,\n", " self.num_agents,\n", " self.random.choices(self.grid.all_cells.cells, k=self.num_agents),\n", " )\n", "\n", " def step(self):\n", " # Collect data each step\n", " self.datacollector.collect(self)\n", " self.agents.shuffle_do(\"move\")\n", " self.agents.do(\"give_money\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "params = {\"width\": 10, \"height\": 10, \"n\": range(5, 105, 5)}\n", "\n", "results = mesa.batch_run(\n", " MoneyModel,\n", " parameters=params,\n", " iterations=5,\n", " max_steps=100,\n", " number_processes=1,\n", " data_collection_period=1,\n", " display_progress=True,\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We will now extract the results into a pandas dataframe." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "results_df = pd.DataFrame(results)\n", "print(f\"The results have {len(results)} rows.\")\n", "print(f\"The columns of the data frame are {list(results_df.keys())}.\")" ] }, { "cell_type": "markdown", "metadata": { "collapsed": false, "jupyter": { "outputs_hidden": false } }, "source": [ "### Analyzing model reporters: Comparing 5 scenarios\n", "Other insights might be gathered when we compare the Gini coefficient of different scenarios. For example, we can compare the Gini coefficient of a population with 25 agents to the Gini coefficient of a population with 400 agents. While doing this, we increase the number of iterations to 25 to get a better estimate of the Gini coefficient for each population size and get usable error estimations.\n", "\n", "As we look varying the parameters to see the impact on model outcomes, it is critical to again point out that users can set the random seed. Due to the often inherent randomness with ABMs the seed becomes crucial for: \n", "- **Reproducibility** - Being able to replicate the ABM results\n", "- **Sensitivity Analysis** - Identifying how sensitive/robust your model results are to random fluctuations\n", "\n", "Treating the seed as an additional parameter and running numerous scenarios allows us to see the impact of randomness on this model. " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false, "jupyter": { "outputs_hidden": false } }, "outputs": [], "source": [ "params = {\"seed\": None, \"width\": 10, \"height\": 10, \"n\": [5, 10, 20, 40, 80]}\n", "\n", "results_5s = mesa.batch_run(\n", " MoneyModel,\n", " parameters=params,\n", " iterations=25,\n", " max_steps=100,\n", " number_processes=1,\n", " data_collection_period=1, # Need to collect every step\n", " display_progress=True,\n", ")\n", "\n", "results_5s_df = pd.DataFrame(results_5s)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We filter the results to only contain the data of one agent" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false, "jupyter": { "outputs_hidden": false } }, "outputs": [], "source": [ "# The Gini coefficient will be the same for the entire population at any time.\n", "results_5s_df_filtered = results_5s_df[(results_5s_df.AgentID == 1)]\n", "results_5s_df_filtered.head(3)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false, "jupyter": { "outputs_hidden": false } }, "outputs": [], "source": [ "# Create a lineplot with error bars\n", "g = sns.lineplot(\n", " data=results_5s_df,\n", " x=\"Step\",\n", " y=\"Gini\",\n", " hue=\"n\",\n", " errorbar=(\"ci\", 95),\n", " palette=\"tab10\",\n", ")\n", "g.figure.set_size_inches(8, 4)\n", "plot_title = (\n", " \"Gini coefficient for different population sizes\\n\"\n", " \"(mean over 100 runs, with 95% confidence interval)\"\n", ")\n", "g.set(title=plot_title, ylabel=\"Gini coefficient\");" ] }, { "cell_type": "markdown", "metadata": { "collapsed": false, "jupyter": { "outputs_hidden": false } }, "source": [ "In this case it looks like the Gini coefficient increases slower for smaller populations. This can be because of different things, either because the Gini coefficient is a measure of inequality and the smaller the population, the more likely it is that the agents are all in the same wealth class, or because there are less interactions between agents in smaller populations, which means that the wealth of an agent is less likely to change." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise:\n", "Treat the seed as a parameter and see the impact on the Gini Coefficient\n", "\n", "You can also plot the seeds against the Gini Coefficient by changing the \"hue\" parameter in sns.lineplot function." ] }, { "cell_type": "markdown", "metadata": { "collapsed": false, "jupyter": { "outputs_hidden": false } }, "source": [ "### Analyzing agent reporters: Comparing 5 scenarios\n", "From the agents we collected the wealth and the number of consecutive rounds without a transaction. We can compare the 5 different population sizes by plotting the average number of consecutive rounds without a transaction for each population size.\n", "\n", "Note that we're aggregating multiple times here: First we take the average of all agents for each single replication. Then we plot the averages for all replications, with the error band showing the 95% confidence interval of that first average (over all agents). So this error band is representing the uncertainty of the mean value of the number of consecutive rounds without a transaction for each population size." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false, "jupyter": { "outputs_hidden": false } }, "outputs": [], "source": [ "# Calculate the mean of the wealth and the number of consecutive rounds\n", "# for all agents in each episode.\n", "agg_results_df = (\n", " results_5s_df.groupby([\"iteration\", \"n\", \"Step\"])\n", " .agg({\"Wealth\": \"mean\", \"Steps_not_given\": \"mean\"})\n", " .reset_index()\n", ")\n", "agg_results_df.head(3)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false, "jupyter": { "outputs_hidden": false } }, "outputs": [], "source": [ "# Create a line plot with error bars\n", "g = sns.lineplot(\n", " data=agg_results_df, x=\"Step\", y=\"Steps_not_given\", hue=\"n\", palette=\"tab10\"\n", ")\n", "g.figure.set_size_inches(8, 4)\n", "g.set(\n", " title=\"Average number of consecutive rounds without a transaction for \"\n", " \"different population sizes\\n(mean with 95% confidence interval)\",\n", " ylabel=\"Consecutive rounds without a transaction\",\n", ");" ] }, { "cell_type": "markdown", "metadata": { "collapsed": false, "jupyter": { "outputs_hidden": false } }, "source": [ "It can be clearly seen that the lower the number of agents, the higher the number of consecutive rounds without a transaction. This is because the agents have fewer interactions with each other and therefore the wealth of an agent is less likely to change." ] }, { "cell_type": "markdown", "metadata": { "collapsed": false, "jupyter": { "outputs_hidden": false } }, "source": [ "### General steps for analyzing results\n", "\n", "Many other analysis are possible based on the policies, scenarios and uncertainties that you might be interested in. In general, you can follow these steps to do your own analysis:\n", "\n", "1. Determine which metrics you want to analyse. Add these as model and agent reporters to the datacollector of your model.\n", "2. Determine the input parameters you want to vary. Add these as parameters to the batch_run function, using ranges or lists to test different values.\n", "3. Determine the hyperparameters of the batch_run function. Define the number of iterations, the number of processes, the number of steps, the data collection period, etc.\n", "4. Run the batch_run function and save the results.\n", "5. Transform, filter and aggregate the results to get the data you want to analyze. Make sure it's in long format, so that each row represents a single value.\n", "6. Choose a plot type, what to plot on the x and y axis, which columns to use for the hue. Seaborn also has an amazing [Example Gallery](https://seaborn.pydata.org/examples/index.html).\n", "7. Plot the data and analyze the results." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Exercise: \n", "Update the model in some new way (e.g. a new agent attribute, a new model reporter), conduct a batch run with a parameter sweep and visualize your results" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## That is it you have successfully completed Mesa's Introductory Tutorial!" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### More Mesa\n", "\n", "If you are looking for other Mesa models or tools here are some additional resources. \n", "\n", "- Interactive Dashboard: There is a separate [visualization tutorial](https://mesa.readthedocs.io/latest/tutorials/visualization_tutorial.html) that will take users through building a dashboard for this model (aka Boltzmann Wealth Model).\n", "- Example ABMs: Find canonical examples and examples of ABMs demonstrating highlighted features in the [Examples Tab](https://mesa.readthedocs.io/stable/examples.html)\n", "- Expanded Examples: Want to integrate Reinforcement Learning or work on the Traveling Salesman Problem? Checkout [Mesa Examples](https://github.com/projectmesa/mesa-examples/)\n", "- Mesa-Geo: If you need an ABM with Geographic Information Systems (GIS) checkout [Mesa-Geo](https://mesa-geo.readthedocs.io/latest/)\n", "- Mesa Frames: Have a large complex model that you need to speed up, check out [Mesa Frames](https://github.com/projectmesa/mesa-frames)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Happy Modeling!\n", "\n", "This document is a work in progress. If you see any errors, exclusions or have any problems please contact [us](https://github.com/projectmesa/mesa/issues)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "[Comer2014] Comer, Kenneth W. “Who Goes First? An Examination of the Impact of Activation on Outcome Behavior in AgentBased Models.” George Mason University, 2014. http://mars.gmu.edu/bitstream/handle/1920/9070/Comer_gmu_0883E_10539.pdf\n", "\n", "[Dragulescu2002] Drăgulescu, Adrian A., and Victor M. Yakovenko. “Statistical Mechanics of Money, Income, and Wealth: A Short Survey.” arXiv Preprint Cond-mat/0211175, 2002. http://arxiv.org/abs/cond-mat/0211175." ] } ], "metadata": { "anaconda-cloud": {}, "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.5" }, "widgets": { "state": {}, "version": "1.1.2" } }, "nbformat": 4, "nbformat_minor": 4 }