muse.outputs package

Submodules

muse.outputs.mca module

Output quantities.

Functions that compute MCA quantities for post-simulation analysis should all follow the same signature:

@register_output_quantity
def quantity(
    sectors: List[AbstractSector],
    market: xr.Dataset, **kwargs
) -> Union[pd.DataFrame, xr.DataArray]:
    pass

The function should never modify it’s arguments. It can return either a pandas dataframe or an xarray xr.DataArray.

class AggregateResources(commodities=(), metric='consumption')

Bases: object

Aggregates a set of commodities.

class FiniteResources(limits_path, commodities=(), metric='consumption')

Bases: AggregateResources

Aggregates a set of commodities.

OUTPUTS_PARAMETERS

Acceptable Datastructures for outputs parameters

alias of str | Mapping

OUTPUT_QUANTITIES = {'AggregateResources': <class 'muse.outputs.mca.AggregateResources'>, 'Aggregateresources': <class 'muse.outputs.mca.AggregateResources'>, 'Capacity': <function capacity>, 'CapitalCosts': <function metric_capital_costs>, 'Consumption': <function consumption>, 'EAC': <function metric_eac>, 'Eac': <function metric_eac>, 'EmissionCosts': <function metric_emission_costs>, 'FiniteResources': <class 'muse.outputs.mca.FiniteResources'>, 'Finiteresources': <class 'muse.outputs.mca.FiniteResources'>, 'FuelCosts': <function metric_fuel_costs>, 'LCOE': <function metric_lcoe>, 'Lcoe': <function metric_lcoe>, 'MetricCapitalCosts': <function metric_capital_costs>, 'MetricConsumption': <function metric_consumption>, 'MetricEac': <function metric_eac>, 'MetricEmissionCosts': <function metric_emission_costs>, 'MetricFuelCosts': <function metric_fuel_costs>, 'MetricLcoe': <function metric_lcoe>, 'MetricSupply': <function metric_supply>, 'MetricyConsumption': <function metricy_consumption>, 'MetricySupply': <function metricy_supply>, 'Prices': <function prices>, 'Supply': <function supply>, 'TimesliceConsumption': <function metric_consumption>, 'TimesliceSupply': <function metric_supply>, 'YearlyConsumption': <function metricy_consumption>, 'YearlySupply': <function metricy_supply>, 'capacity': <function capacity>, 'capital-costs': <function metric_capital_costs>, 'capitalCosts': <function metric_capital_costs>, 'capital_costs': <function metric_capital_costs>, 'capitalcosts': <function metric_capital_costs>, 'consumption': <function consumption>, 'emission-costs': <function metric_emission_costs>, 'emissionCosts': <function metric_emission_costs>, 'emission_costs': <function metric_emission_costs>, 'emissioncosts': <function metric_emission_costs>, 'finite-resources': <class 'muse.outputs.mca.FiniteResources'>, 'finiteResources': <class 'muse.outputs.mca.FiniteResources'>, 'finite_resources': <class 'muse.outputs.mca.FiniteResources'>, 'finiteresources': <class 'muse.outputs.mca.FiniteResources'>, 'fuel-costs': <function metric_fuel_costs>, 'fuelCosts': <function metric_fuel_costs>, 'fuel_costs': <function metric_fuel_costs>, 'fuelcosts': <function metric_fuel_costs>, 'metric-capital-costs': <function metric_capital_costs>, 'metric-consumption': <function metric_consumption>, 'metric-eac': <function metric_eac>, 'metric-emission-costs': <function metric_emission_costs>, 'metric-fuel-costs': <function metric_fuel_costs>, 'metric-lcoe': <function metric_lcoe>, 'metric-supply': <function metric_supply>, 'metricCapitalCosts': <function metric_capital_costs>, 'metricConsumption': <function metric_consumption>, 'metricEac': <function metric_eac>, 'metricEmissionCosts': <function metric_emission_costs>, 'metricFuelCosts': <function metric_fuel_costs>, 'metricLcoe': <function metric_lcoe>, 'metricSupply': <function metric_supply>, 'metric_capital_costs': <function metric_capital_costs>, 'metric_consumption': <function metric_consumption>, 'metric_eac': <function metric_eac>, 'metric_emission_costs': <function metric_emission_costs>, 'metric_fuel_costs': <function metric_fuel_costs>, 'metric_lcoe': <function metric_lcoe>, 'metric_supply': <function metric_supply>, 'metriccapitalcosts': <function metric_capital_costs>, 'metricconsumption': <function metric_consumption>, 'metriceac': <function metric_eac>, 'metricemissioncosts': <function metric_emission_costs>, 'metricfuelcosts': <function metric_fuel_costs>, 'metriclcoe': <function metric_lcoe>, 'metricsupply': <function metric_supply>, 'metricy-consumption': <function metricy_consumption>, 'metricy-supply': <function metricy_supply>, 'metricyConsumption': <function metricy_consumption>, 'metricySupply': <function metricy_supply>, 'metricy_consumption': <function metricy_consumption>, 'metricy_supply': <function metricy_supply>, 'metricyconsumption': <function metricy_consumption>, 'metricysupply': <function metricy_supply>, 'prices': <function prices>, 'supply': <function supply>, 'timeslice-consumption': <function metric_consumption>, 'timeslice-supply': <function metric_supply>, 'timesliceConsumption': <function metric_consumption>, 'timesliceSupply': <function metric_supply>, 'timeslice_consumption': <function metric_consumption>, 'timeslice_supply': <function metric_supply>, 'timesliceconsumption': <function metric_consumption>, 'timeslicesupply': <function metric_supply>, 'yearly-consumption': <function metricy_consumption>, 'yearly-supply': <function metricy_supply>, 'yearlyConsumption': <function metricy_consumption>, 'yearlySupply': <function metricy_supply>, 'yearly_consumption': <function metricy_consumption>, 'yearly_supply': <function metricy_supply>, 'yearlyconsumption': <function metricy_consumption>, 'yearlysupply': <function metricy_supply>}

Quantity for post-simulation analysis.

OUTPUT_QUANTITY_SIGNATURE

Signature of functions computing quantities for later analysis.

alias of Callable[[Dataset, List[AbstractSector], Any], DataArray | DataFrame]

capacity(market, sectors, **kwargs)

Current capacity across all sectors.

consumption(market, sectors, **kwargs)

Current consumption.

factory(*parameters)

Creates outputs functions for post-mortem analysis.

Each parameter is a dictionary containing the following:

• quantity (mandatory): name of the quantity to output. Mandatory.

• sink (optional): name of the storage procedure, e.g. the file format or database format. When it cannot be guessed from filename, it defaults to “csv”.

• filename (optional): path to a directory or a file where to store the quantity. In the latter case, if sink is not given, it will be determined from the file extension. The filename can incorporate markers. By default, it is “{default_output_dir}/{sector}{year}{quantity}{suffix}”.

• any other parameter relevant to the sink, e.g. pandas.to_csv keyword arguments.

For simplicity, it is also possible to given lone strings as input. They default to {‘quantity’: string} (and the sink will default to “csv”).

metric_capital_costs(market, sectors, **kwargs)

Current capital costs across all sectors.

metric_consumption(market, sectors, **kwargs)

Current timeslice consumption across all sectors.

metric_eac(market, sectors, **kwargs)

Current emission costs across all sectors.

metric_emission_costs(market, sectors, **kwargs)

Current emission costs across all sectors.

metric_fuel_costs(market, sectors, **kwargs)

Current lifetime levelised cost across all sectors.

metric_lcoe(market, sectors, **kwargs)

Current emission costs across all sectors.

metric_supply(market, sectors, **kwargs)

Current timeslice supply across all sectors.

metricy_consumption(market, sectors, **kwargs)

Current yearly consumption across all sectors.

metricy_supply(market, sectors, **kwargs)

Current yearlysupply across all sectors.

prices(market, sectors, drop_empty=True, keep_columns='prices', **kwargs)

Current MCA market prices.

register_output_quantity(function=None)

Registers a function to compute an output quantity.

round_values(function)

Rounds the outputs to given number of decimals and drops columns with zeros.

sector_capacity(sector)

Sector capacity with agent annotations.

sector_capital_costs(sector, market, **kwargs)

Sector capital costs with agent annotations.

sector_consumption(sector, market, **kwargs)

Sector fuel consumption with agent annotations.

sector_eac(sector, market, **kwargs)

Net Present Value of technologies over their lifetime.

sector_emission_costs(sector, market, **kwargs)

Sector emission costs with agent annotations.

sector_fuel_costs(sector, market, **kwargs)

Sector fuel costs with agent annotations.

sector_lcoe(sector, market, **kwargs)

Levelized cost of energy () of technologies over their lifetime.

sector_supply(sector, market, **kwargs)

Sector supply with agent annotations.

sectory_consumption(sector, market, **kwargs)

Sector fuel consumption with agent annotations.

sectory_supply(sector, market, **kwargs)

Sector supply with agent annotations.

supply(market, sectors, **kwargs)

Current supply.

muse.outputs.cache module

Output cached quantities.

Functions that output the state of diverse quantities at intermediate steps of the calculation.

The core of the method is the OutputCache class that initiated by the MCA with input parameters defined in the TOML file, much like the existing output options but in a outputs_cache list, enables listening for data to be cached and, after each period, saved into disk via the consolidate_cache method.

Anywhere in the code, you can write:

cache_quantity(quantity_name=some_data)

If the quantity has been set as something to cache, the data will be stored and, eventually, save to disk after - possibly - aggregating the data and removing those entries corresponding to non-convergent investment attempts. This process of cleaning and aggregation is quantity specific.

See documentation for the muse.outputs.cache.cache_quantity() function as well as how to setup the toml input file to cache quantities. Users can customize and create further output quantities by registering with MUSE via muse.outputs.cache.register_cached_quantity().

CACHE_TOPIC_CHANNEL = 'cache_quantity'

Topic channel to use with the pubsub messaging system.

OUTPUT_QUANTITIES = {'Capacity': <function capacity>, 'Lcoe': <function lcoe>, 'LifetimeLevelizedCostOfEnergy': <function lcoe>, 'Production': <function production>, 'capacity': <function capacity>, 'lcoe': <function lcoe>, 'lifetime-levelized-cost-of-energy': <function lcoe>, 'lifetimeLevelizedCostOfEnergy': <function lcoe>, 'lifetime_levelized_cost_of_energy': <function lcoe>, 'lifetimelevelizedcostofenergy': <function lcoe>, 'production': <function production>}

Quantity for post-simulation analysis.

OUTPUT_QUANTITY_SIGNATURE

Signature of functions computing quantities for later analysis.

alias of Callable[[List[DataArray]], DataArray | DataFrame]

class OutputCache(*parameters, output_quantities=None, sectors=None, topic='cache_quantity')

Bases: object

Creates outputs functions for post-mortem analysis of cached quantities.

Each parameter is a dictionary containing the following:

• quantity (mandatory): name of the quantity to output. Mandatory.

• sink (optional): name of the storage procedure, e.g. the file format or database format. When it cannot be guessed from filename, it defaults to “csv”.

• filename (optional): path to a directory or a file where to store the quantity. In the latter case, if sink is not given, it will be determined from the file extension. The filename can incorporate markers. By default, it is “{default_output_dir}/{sector}{year}{quantity}{suffix}”.

• any other parameter relevant to the sink, e.g. pandas.to_csv keyword arguments.

For simplicity, it is also possible to given lone strings as input. They default to {‘quantity’: string} (and the sink will default to “csv”).

Raises:

ValueError – If unknown quantities are requested to be cached.

cache(data)

Caches the data into memory.

If the quantity has not been selected to be cached when configuring the MUSE simulation, it will be silently ignored if present as an input to this function.

Parameters:

• data (Mapping[str, xr.DataArray]) – Dictionary with the quantities and

• save. (DataArray values to)

consolidate_cache(year)

Save the cached data into disk and flushes cache.

This method is meant to be called after each time period in the main loop of the MCA, just after market and sector quantities are saved.

Parameters:

year (int) – Year of interest.

cache_quantity(function=None, quantity=None, **kwargs)

Cache one or more quantities to be post-processed later on.

This function can be used as a decorator, in which case the quantity input argument must be set, or directly called with any number of keyword arguments. In the former case, the matching between quantities and values to cached is done by the function ‘match_quantities’. When used in combination with other decorators, care must be taken to decide the order in which they are applied to make sure the appropriate output is cached.

Note that if the quantity has NOT been selected to be cached when configuring the MUSE simulation, it will be silently ignored if present as an input to this function.

Example

As a decorator, the quantity argument must be set:

>>> @cache_quantity(quantity="capacity")
... def some_calculation():
...     return xr.DataArray()

If returning a sequence of DataArrays, the number of quantities to record must be the same as the number of arrays. They are paired in the same order they are given and the ‘name’ attribute of the arrays, if present, is ignored.

>>> @cache_quantity(quantity=["capacity", "production"])
... def other_calculation():
...     return xr.DataArray(), xr.DataArray()

For a finer control of what is cached when there is a complex output, combine the DataArrays in a Dataset. In this case, the ‘quantity’ input argument can be either a string or a sequence of strings to record multiple variables in the Dataset.

>>> @cache_quantity(quantity=["capacity", "production"])
... def and_another_one():
...     return xr.Dataset(
...         {
...             "not cached": xr.DataArray(),
...             "capacity": xr.DataArray(),
...             "production": xr.DataArray(),
...         }
...     )

When this function is called directly and not used as a decorator, simply provide the name of the quantities and the DataArray to record as keyword arguments:

>>> cache_quantity(capacity=xr.DataArray(), production=xr.DataArray())

Parameters:

• function (Optional[Callable]) – The decorated function, if any. Its output must be a DataArray, a sequence of DataArray or a Dataset. See ‘match_quantities’

• quantity (Union[str, List[str], None]) – The name of the quantities to record.

• **kwargs (xr.DataArray) – Keyword arguments of the form ‘quantity_name=quantity_value’.

Raises:

• ValueError – If a function input argument is provided at the same time than

• keyword arguments. –

Returns:

(Optional[Callable]) The decorated function (or a dummy function if called directly).

capacity(cached, agents)

Consolidates the cached capacities into a single DataFrame to save.

Parameters:

• cached (List[xr.DataArray]) – The list of cached arrays

• agents (MutableMapping[Text, MutableMapping[Text, Text]]) – Agents’ metadata.

Returns:

DataFrame with the consolidated data.

Return type:

pd.DataFrame

consolidate_quantity(quantity, cached, agents)

Consolidates the cached quantity into a single DataFrame to save.

Parameters:

• quantity (Text) – The quantity to cache.

• cached (List[xr.DataArray]) – The list of cached arrays

• agents (MutableMapping[Text, MutableMapping[Text, Text]]) – Agents’ metadata.

Returns:

DataFrame with the consolidated data.

Return type:

pd.DataFrame

extract_agents(sectors)

_summary_.

Parameters:

sectors (List[AbstractSector]) – _description_

Returns:

_description_

Return type:

Mapping[Text, Text]

extract_agents_internal(sector)

Extract simple agent metadata from a sector.

Parameters:

sector (AbstractSector) – Sector to extract the metadata from.

Returns:

A dictionary with the uuid of each agent as keys and a dictionary with the name, agent type and agent sector as values.

Return type:

Mapping[Text, Text]

lcoe(cached, agents)

Consolidates the cached LCOE into a single DataFrame to save.

Parameters:

• cached (List[xr.DataArray]) – The list of cached arrays

• agents (MutableMapping[Text, MutableMapping[Text, Text]]) – Agents’ metadata.

Returns:

DataFrame with the consolidated data.

Return type:

pd.DataFrame

match_quantities(quantity, data)

Matches the quantities with the corresponding data.

The possible name attribute in the DataArrays is ignored.

Parameters:

• quantity (Union[str, Sequence[str]]) – The name(s) of the quantity(ies) to cache.

• data (Union[xr.DataArray, xr.Dataset, Sequence[xr.DataArray]]) – The structure containing the data to cache.

Raises:

• TypeError – If there is an invalid combination of input argument types.

• ValueError – If the number of quantities does not match the length of the data.

• KeyError – If the required quantities do not exist as variables in the dataset.

Returns:

(Mapping[str, xr.DataArray]) A dictionary matching the quantity names with the corresponding data.

production(cached, agents)

Consolidates the cached production into a single DataFrame to save.

Parameters:

• cached (List[xr.DataArray]) – The list of cached arrays

• agents (MutableMapping[Text, MutableMapping[Text, Text]]) – Agents’ metadata.

Returns:

DataFrame with the consolidated data.

Return type:

pd.DataFrame

register_cached_quantity(function)

Registers a function to compute an output quantity.

muse.outputs.sector module

Output quantities.

Functions that compute sectorial quantities for post-simulation analysis should all follow the same signature:

@register_output_quantity
def quantity(
    capacity: xr.DataArray,
    market: xr.Dataset,
    technologies: xr.Dataset
) -> Union[xr.DataArray, DataFrame]:
    pass

They take as input the current capacity profile, aggregated across a sectoar, a dataset containing market-related quantities, and a dataset characterizing the technologies in the market. It returns a single xr.DataArray object.

The function should never modify it’s arguments.

OUTPUTS_PARAMETERS

Acceptable Datastructures for outputs parameters

alias of str | Mapping

OUTPUT_QUANTITIES = {'Capacity': <function capacity>, 'Consumption': <function consumption>, 'Costs': <function costs>, 'Supply': <function supply>, 'capacity': <function capacity>, 'consumption': <function consumption>, 'costs': <function costs>, 'supply': <function supply>}

Quantity for post-simulation analysis.

OUTPUT_QUANTITY_SIGNATURE

Signature of functions computing quantities for later analysis.

alias of Callable[[Dataset, DataArray, Dataset, Any], DataFrame | DataArray]

capacity(market, capacity, technologies, rounding=4)

Current capacity.

consumption(market, capacity, technologies, sum_over=None, drop=None, rounding=4)

Current consumption.

costs(market, capacity, technologies, sum_over=None, drop=None, rounding=4)

Current costs.

factory(*parameters, sector_name='default')

Creates outputs functions for post-mortem analysis.

Each parameter is a dictionary containing the following:

• quantity (mandatory): name of the quantity to output. Mandatory.

• sink (optional): name of the storage procedure, e.g. the file format or database format. When it cannot be guessed from filename, it defaults to “csv”.

• filename (optional): path to a directory or a file where to store the quantity. In the latter case, if sink is not given, it will be determined from the file extension. The filename can incorporate markers. By default, it is “{default_output_dir}/{sector}{year}{quantity}{suffix}”.

• any other parameter relevant to the sink, e.g. pandas.to_csv keyword arguments.

For simplicity, it is also possible to given lone strings as input. They default to {‘quantity’: string} (and the sink will default to “csv”).

market_quantity(quantity, sum_over=None, drop=None)

register_output_quantity(function=None)

Registers a function to compute an output quantity.

supply(market, capacity, technologies, sum_over=None, drop=None, rounding=4)

Current supply.

muse.outputs.sinks module

Sinks where output quantities can be stored.

Sinks take as argument a DataArray and store it somewhere. Additionally they take a dictionary as argument. This dictionary will always contains the items (‘quantity’, ‘sector’, ‘year’) referring to the name of the quantity, the name of the calling sector, the current year. They may contain additional parameters which depend on the actual sink, such as ‘filename’.

Optionally, a description of the storage (filename, etc) can be returned.

The signature of a sink is:

@register_output_sink(name="netcfd")
def to_netcfd(quantity: DataArray, config: Mapping) -> Optional[Text]:
    pass

exception FiniteResourceException

Bases: Exception

Raised when a finite resource is exceeded.

OUTPUT_SINKS = {'Aggregate': <class 'muse.outputs.sinks.YearlyAggregate'>, 'Csv': <function to_csv>, 'Excel': <function to_excel>, 'FiniteResourceLogger': <function finite_resource_logger>, 'Nc': <function to_netcdf>, 'Netcdf': <function to_netcdf>, 'ToCsv': <function to_csv>, 'ToExcel': <function to_excel>, 'ToNetcdf': <function to_netcdf>, 'Xlsx': <function to_excel>, 'YearlyAggregate': <class 'muse.outputs.sinks.YearlyAggregate'>, 'Yearlyaggregate': <class 'muse.outputs.sinks.YearlyAggregate'>, 'aggregate': <class 'muse.outputs.sinks.YearlyAggregate'>, 'csv': <function to_csv>, 'excel': <function to_excel>, 'finite-resource-logger': <function finite_resource_logger>, 'finiteResourceLogger': <function finite_resource_logger>, 'finite_resource_logger': <function finite_resource_logger>, 'finiteresourcelogger': <function finite_resource_logger>, 'nc': <function to_netcdf>, 'netcdf': <function to_netcdf>, 'to-csv': <function to_csv>, 'to-excel': <function to_excel>, 'to-netcdf': <function to_netcdf>, 'toCsv': <function to_csv>, 'toExcel': <function to_excel>, 'toNetcdf': <function to_netcdf>, 'to_csv': <function to_csv>, 'to_excel': <function to_excel>, 'to_netcdf': <function to_netcdf>, 'tocsv': <function to_csv>, 'toexcel': <function to_excel>, 'tonetcdf': <function to_netcdf>, 'xlsx': <function to_excel>}

Stores a quantity somewhere.

OUTPUT_SINK_SIGNATURE

Signature of functions used to save quantities.

alias of Callable[[DataArray | DataFrame, int, Any], str | None]

class YearlyAggregate(final_sink=None, sector='', axis='year', **kwargs)

Bases: object

Incrementally aggregates data from year to year.

factory(parameters, sector_name='default')

finite_resource_logger(data, year, early_exit=False, **kwargs)

register_output_sink(function=None)

Registers a function to save quantities.

sink_to_file(suffix)

Simplifies sinks to files.

The decorator takes care of figuring out the path to the file, as well as trims the configuration dictionary to include only parameters for the sink itself. The decorated function returns the path to the output file.

standardize_quantity(function)

Helps standardize how the quantities are specified.

This decorator adds three keyword arguments to an input function:

• set_index: A dictionary or any argument accepted by pandas.DataFrame.set_index(). Ignored if not specified. If specified, a call to pandas.DataFrame.reset_index() is made first.

• sort_index: A dictionary or any argument accepted by pandas.DataFrame.sort_index(). Ignored if not specified.

• keep_columns: a string or a list of strings with the names of the columns to keep. Ignored if not specified.

The three functions are applied in the order given, assuming an input is specified.

to_csv(quantity, filename, **params)

Saves data array to csv format, using pandas.to_csv.

Parameters:

• quantity – The data to be saved

• filename – File to which the data should be saved

• params – A configuration dictionary accepting any argument to pandas.to_csv

to_excel(quantity, filename, **params)

Saves data array to csv format, using pandas.to_excel.

Parameters:

• quantity – The data to be saved

• filename – File to which the data should be saved

• params – A configuration dictionary accepting any argument to pandas.to_excel

to_netcdf(quantity, filename, **params)

Saves data array to csv format, using xarray.to_netcdf.

Parameters:

• quantity – The data to be saved

• filename – File to which the data should be saved

• params – A configuration dictionary accepting any argument to xarray.to_netcdf

Module contents

register_output_sink(function=None)

Registers a function to save quantities.