6. Application Flow

While not essential to be able to use MUSE, it is useful to know the sequence of events that a run of MUSE will follow in a bit more detail that the brief overview of the MUSE Overview section. Let’s start with the big picture.

Note

Throughout this section, greyed nodes will be further described in a more detailed chart, so keep reading to find out more about those, probably unclear steps.

6.1. High level sequence

Any MUSE simulation follows the steps outlined in the following graph:

High level sequence of steps in a MUSE simulation

It has two main components, the Initialisation phase when the input settings file is read and, based on it, all the components needed for the simulation are created, and the Run phase when the actual simulation takes place and intermediate outputs are produced along the way.

6.2. initialisation

The initialisation phase is where all the parameters of the simulation are pulled from the Input Files and the relevant objects required to run the simulation are created. If there is any configuration that does not make sense, it should be spotted during this phase and the execution of MUSE interrupted (with a meaningful error message) so no time is wasted in running a simulation that is wrong.

Each of the steps above can be further split into smaller steps, described individually in the following sections:

6.2.1. Read settings

The settings TOML file is where the higher level configuration of the simulation is defined. See Simulation settings for details of its content and structure. During the initialisation, the file is read, merged with default settings included in MUSE to get those parameters that are required and not provided by the user and, finally, the resulting settings are validated.

The validation step covers a wide range of checks (and more that can be added by the user via plugins) that not only asses if the relevant information is correct but that, in some cases, also create the relevant Python objects or normalizes it to some defined format.

Read settings detailed flow chart

6.2.2. Create initial market

As described in Initial Market Projection, MUSE needs an initial market with prices and potential imports and exports of commodities to kick-off the simulation. These prices will be updated as the simulation progresses, or used as a static market throughout the whole timeline of the simulation.

This market object (an xarray Dataset, internally) will be instrumental throughout the simulation and regularly updated with supply, consumption and new prices, if relevant.

Steps when creating the initial market

6.2.3. Create sectors

The sectors manage all the actors that will drive the evolution of the simulation: the technologies available, the commodities consumed and produced and the agents that will invest in the different technologies to ensure that the supply of commodities meets the demand. Sections Key MUSE Components and Input Files provide more information on the different factors that influence sectors and their components.

During the initialisation step, all input files relevant to a sector are loaded, their consistency validated and the agents that will be investing in this sector created. A broad description of the steps involved in the creation of each sector defined in the input file are included in the following chart (there might be other validation and data reformatting steps).

Simplified process of the creation of the sectors

6.2.4. Create the MCA

The last step of the initialisation is also the simplest one. The MCA (market clearing algorithm) is initialized with all the objects created in the previous sections and, specifically, the global simulation parameters, the handling of the carbon budget and the global outputs. Once the MCA is initialized, the simulation is ready to run!

Steps of the creation of the MCA

6.3. Simulation run

If the initialisation is successful, the execution of the simulation will start. Depending on the configuration of the carbon budget and what to do with it, the steps will be slightly different, but in all cases the main part will be the steps for reaching the equilibrium between the demand and the supply based on the investment.

6.3.1. Update carbon prices

One of MUSE core features is to (optionally) consider carbon emission as a constrain for the investment in future technologies. A carbon budget can be defined in the Simulation settings across all years of the simulation and this will result in an increase of prices for those technologies that are less green.

The sequence of steps related to the carbon budget control are as follows:

Description of the carbon budget cycle

The method used to calculate the new carbon price can be selected by the user. There are currently only two options for this method, fitting and bisection, however this can be expanded by the user with the @register_carbon_budget_method hook in muse.carbon_budget.

The fitting method is based in the following algorithm:

Fitting method to calculate the new carbon price in the future year

The bisection method is a custom implementation of the well known bisection algorithm on the carbon price to minimize the difference between the carbon budget and the carbon emissions.

Both methods will run the Find market equilibrium algorithm multiple times and, as a result, the simulation will take significantly longer to complete than if no carbon budget is considered.

6.3.2. Find market equilibrium

This is the main part of MUSE, in which the agents in the different sectors will invest in new - or old - technologies to make sure that the supply of commodities matches their demand in years to come across all the regions of the simulation.

An overall picture of this process can be seen in the following chart, but there are many fine-grained steps related to specific objectives and criteria that heavily influence the results of the calculation. These steps are described in other parts of the documentation.

Main loop to find the market equilibrium

6.3.3. Single year iteration

Both in the carbon budget and in the equilibrium calculation, a single year iteration step is involved. It is in this step where MUSE will go through each sector and use the agents to appropriately invest in different technologies, aiming to match these two factors.

As sectors have different priorities, sectors with lower priorities (larger numbers) will run last and see a market updated by the higher priority sectors. In general, demand sectors should run before conversion sectors and these before supply sectors, such that the later can see the real demand. Running each sector will update their commodities, consumption and production. Balancing them is the purpose of the Find market equilibrium loop described above, where the prices of the commodities are updated due to the change in their demand occurring during the single year iteration.

A chart summarising this process is depicted below:

Steps of a single year iteration

With the run of each sector involving the following steps:

Steps of one period step in a sector

This deeper level of the process is where most of the input options of MUSE are put in use to decide how the agents behave, in what sort of technologies they invest, what metrics are used to make these decisions and how the dispatch of commodities takes place in order to fulfil the demand.

6.3.4. Investment

In the investment step is where new capacity is added to the different assets managed by the agents. This investment might be needed to cover an increase in demand (between now and forecast) or to match decommissioned assets, typically to do both.

The following graph summarises the process.

Investment stage of a subsector calculation

First the demand is distributed among the available agents as requested by the demand_share argument of each subsector in the settings.toml file. This distribution can be done based on any attribute or property of the agents, as included in the Agents.csv file. Demand can also be shared across multiple agents, depending on the “quantity” attribute (defined in Agents.csv). The two built-in options in MUSE are:

  • new_and_retro (default): The input demand is split amongst both new and retro agents. New agents get a share of the increase in demand for the forecast year, whereas retrofit agents are assigned a share of the demand that occurs from decommissioned assets.

  • standard_demand: The demand is split only amongst new agents (indeed there will be an error if a retro agent is found for this subsector). New agents get a share of the increase in demand for the forecast years well as the demand that occurs from decommissioned assets.

Then, each agent select the technologies it can invest in based on what is needed and the search rules defined for it in the Agents.csv file. The possible search rules are described in muse.filters. These determine the search rules for each replacement technology.

For those selected replacement technologies, an objective function is computed. This value is a well defined economic concept, like LCOE or NPV, or a combination of them, and will be used to prioritise the investment of some technologies over others. As above, these objectives are defined in the Agents.csv file for each of the agents. Available objectives are described in muse.objectives.

Then, a decision is computed. Decision methods reduce multiple objectives into a single scalar objective per replacement technology. The decision method to use is selected in the Agents.csv file. They allow combining several objectives into a single metric through which replacement technologies can be ranked. See muse.decisions.

The final step of preparing the investment process is to compute the constrains, e.g. factors that will determine how much a technology could be invested in and include things like matching the demand, the search rules calculated above, the maximum production of a technology for a given capacity or the maximum capacity expansion for a given time period. Available constrains are set in the subsector section of the settings.toml file and described in muse.constrains. By default, all of them are applied. Note that these constrains might result in unfeasible situations if they do not allow the production to grow enough to match the demand. This is one of the common reasons for a MUSE simulation not converging.

With all this information, the investment process can proceed. This is done per sector using the method described by the lpsolver in the settings.toml file. Available solvers are described in muse.investments

If the investment succeeds, the new installed capacity will become part of the agents’ assets.

6.3.5. Dispatch

The dispatch stage when running a sector can be described by the following graph:

Dispatch stage of a sector calculation

After the investment stage is completed, then the new capacity of the sector is obtained by aggregating the assets of all agents of the sector. Then, the supply of commodities is calculated as requested by the dispatch_production argument defined for each sector in the settings.toml file.

The typical choice used in most examples in MUSE is share, where the utilization across similar assets is the same in percentage. However, there are other options available, like

  • costed: assets are ranked by their levelised costs and the cheaper ones are allowed to service the demand first up to their maximum production. Minimum service can be imposed if present.

  • maximum: all the assets dispatch their maximum production, regardless of the demand.

  • match: supply matches the demand within the constrains on how much an asset can produce while minimizing the overall associated costs. match allows the choice between different metrics to rank assets, such as levelised costs and gross margin. See muse.demand_matching for the mathematical details.

Once the supply is obtained, the consumed commodities required to achieve that production level are calculated. The cheapest fuel for flexible technologies is used.

Finally, the cost associated with that supply is calculated as the weighted average annual LCOE over assets, where the weights are the supply. This is later used to set the new prices.