M5

M5 dataset

M5

M5(source_url='https://github.com/Nixtla/m5-forecasts/raw/main/datasets/m5.zip')

M5.download

download(directory)

Downloads M5 Competition Dataset.

Parameters:

Name	Type	Description	Default
directory	str	Directory path to download dataset.	required

M5.load

load(directory, cache=True)

Downloads and loads M5 data.

Parameters:

Name	Type	Description	Default
directory	str	Directory where data will be downloaded.	required
cache	bool	If True, saves and loads a parquet cache under {directory}/.cache.	True

Returns:

Type	Description
Tuple[DataFrame, DataFrame, DataFrame]	Tuple[pd.DataFrame, pd.DataFrame, pd.DataFrame]: Target time series with columns [‘unique_id’, ‘ds’, ‘y’], Exogenous time series with columns [‘unique_id’, ‘ds’, ‘y’], Static exogenous variables with columns [‘unique_id’, ‘ds’] and static variables.

M5.source_url

source_url: str = 'https://github.com/Nixtla/m5-forecasts/raw/main/datasets/m5.zip'

Evaluation class

M5Evaluation

M5Evaluation.aggregate_levels

aggregate_levels(y_hat, categories=None)

Aggregates the 30_490 series to get 42_840.

Parameters:

Name	Type	Description	Default
y_hat	DataFrame	Forecasts as wide pandas dataframe with columns [‘unique_id’].	required
categories	DataFrame	Categories of M5 dataset (not used). Defaults to None.	None

Returns:

Type	Description
DataFrame	pd.DataFrame: Aggregated forecasts as wide pandas dataframe with columns [‘unique_id’].

M5Evaluation.evaluate

evaluate(directory, y_hat, validation=False)

Evaluates y_hat according to M5 methodology.

Parameters:

Name	Type	Description	Default
directory	str	Directory where data will be downloaded.	required
validation	bool	Whether perform validation evaluation. Default False, return test evaluation.	False
y_hat	Union[DataFrame, str]	Forecasts as wide pandas dataframe with columns [‘unique_id’] and forecasts or benchmark url from https://github.com/Nixtla/m5-forecasts/tree/main/forecasts.	required

Returns:

Type	Description
DataFrame	pd.DataFrame: DataFrame with column wrmsse and Level_id (plus a ‘Total’ row) as index.

Examples:

m5_winner_url = 'https://github.com/Nixtla/m5-forecasts/raw/main/forecasts/0001 YJ_STU.zip'
winner_evaluation = M5Evaluation.evaluate('data', m5_winner_url)

m5_second_place_url = 'https://github.com/Nixtla/m5-forecasts/raw/main/forecasts/0002 Matthias.zip'
m5_second_place_forecasts = M5Evaluation.load_benchmark('data', m5_second_place_url)
second_place_evaluation = M5Evaluation.evaluate('data', m5_second_place_forecasts)

M5Evaluation.levels

levels: dict = dict(Level1=['total'], Level2=['state_id'], Level3=['store_id'], Level4=['cat_id'], Level5=['dept_id'], Level6=['state_id', 'cat_id'], Level7=['state_id', 'dept_id'], Level8=['store_id', 'cat_id'], Level9=['store_id', 'dept_id'], Level10=['item_id'], Level11=['state_id', 'item_id'], Level12=['item_id', 'store_id'])

M5Evaluation.load_benchmark

load_benchmark(directory, source_url=None, validation=False)

Downloads and loads benchmark forecasts.

Parameters:

Name	Type	Description	Default
directory	str	Directory where data will be downloaded.	required
source_url	str	Optional benchmark url obtained from https://github.com/Nixtla/m5-forecasts/tree/master/forecasts. If None returns the M5 winner.	None
validation	bool	Whether return validation forecasts. Default False, return test forecasts.	False

Returns:

Type	Description
ndarray	np.ndarray: Numpy array of shape (n_series, horizon).

Example:

winner_benchmark = M5Evaluation.load_benchmark('data')
winner_evaluation = M5Evaluation.evaluate('data', winner_benchmark)

URL-based evaluation

The method evaluate from the class M5Evaluation can receive a url of a submission to the M5 competition.

The results compared to the on-the-fly evaluation were obtained from the official evaluation.

m5_winner_url = 'https://github.com/Nixtla/m5-forecasts/raw/main/forecasts/0001 YJ_STU.zip'
winner_evaluation = M5Evaluation.evaluate('data', m5_winner_url)
# Test of the same evaluation as the original one
test_close(winner_evaluation.loc['Total'].item(), 0.520, eps=1e-3)
winner_evaluation

Pandas-based evaluation

Also the method evaluate can receive a pandas DataFrame of forecasts.

m5_second_place_url = 'https://github.com/Nixtla/m5-forecasts/raw/main/forecasts/0002 Matthias.zip'
m5_second_place_forecasts = M5Evaluation.load_benchmark('data', m5_second_place_url)
second_place_evaluation = M5Evaluation.evaluate('data', m5_second_place_forecasts)
# Test of the same evaluation as the original one
test_close(second_place_evaluation.loc['Total'].item(), 0.528, eps=1e-3)
second_place_evaluation

By default you can load the winner benchmark using the following.

winner_benchmark = M5Evaluation.load_benchmark('data')
winner_evaluation = M5Evaluation.evaluate('data', winner_benchmark)
# Test of the same evaluation as the original one
test_close(winner_evaluation.loc['Total'].item(), 0.520, eps=1e-3)
winner_evaluation

Validation evaluation

You can also evaluate the official validation set.

winner_benchmark_val = M5Evaluation.load_benchmark('data', validation=True)
winner_evaluation_val = M5Evaluation.evaluate('data', winner_benchmark_val, validation=True)
winner_evaluation_val