`epibench create`¶

Running epibench create --config-path "../.." creates model-input ground truth from a file you specify in a forecasting hub. It filters and vintages that file for each requested reference date, then writes one standardized ground truth CSV per date.

Config file¶

The configuration file for an epibench create run takes in the following keys:

hub_path: a path to a local hub repository clone, or a hub GitHub repository URL
challenge_name: whatever name you would like to give the "challenge" you are defining
target: the single target for this create run. If the source file contains target, it must exactly match this value.
ground_truth_file: a relative, hub-root path to the CSV or Parquet ground truth file. Absolute paths and paths outside the hub are rejected.
observed_column_name: the source column containing observed values, as a string or an ordered list of alternative names.
location_column_name: the source column containing locations, as a string or an ordered list of alternative names.
date_column_name: the source column containing target end dates, as a string or an ordered list of alternative names.
dates: which dates of reference you want to fetch ground truth data for (YYYY-MM-DD)
- this can be passes as a list of individually-specified dates,
- or as a dictionary with three keys: start_date, end_date, freq
  - freq format must be "<n> weeks" or "<n> week"
- important: your dates must match the submission cadence of the hub you have provided for a given season. If your epibench create run spans more than a season, the process will exit and ask that you limit to one season at a time. The boundary between seasons is set at June 30 (end of season) and July 1 (start of new season).
vintaging: TRUE or FALSE. Set TRUE to prepare a distinct ground truth vintage for every reference date.
vintaging_method: when vintaging, choose "as_of" or "checkout".
- "as_of" reads the configured current file. It requires an as_of column and uses the latest revision at or before each cutoff date.
- "checkout" checks out the hub at each cutoff date, then reads the configured file from that historical repository state. The file must exist at every requested checkout. If it has an as_of column, the same latest-eligible-revision rule is applied.
vintaging_offset: if vintaging, elect an integer vintaging offset
- many hubs take forecasting submission a few days before the reported reference date (e.g., forecasts are made on Wednesday, but the reference date is the following Saturday, creating an offset of -3 days). For maximum realism, you can inform epibench create of this and it will ensure you do not get any ground truth data that would have been filled in (e.g.) between the Wednesday and the Saturday.
- if you would like no offset, pass 0
- when possible, epibench create will match you vintaging_offset value with the submisison cadence of the hub and give warning if your value does not match the hub

See our configuration templates for a copy-pasteable template of the epibench create config.

Column name fallbacks¶

If source column names change over time, supply an ordered list for any of observed_column_name, location_column_name, or date_column_name. You can mix strings and lists in the same config:

observed_column_name: "observation"
location_column_name: "location"
date_column_name: ["date", "target_end_date"]

For each file read, EpiBench selects the first listed name that exists for each field independently. In this example, it uses date when present and otherwise tries target_end_date. With vintaging_method: "checkout", selection happens separately for each historical checkout, so different vintages can use different column names. This also works with "as_of" and with vintaging: FALSE, for both CSV and Parquet files.

Names must match exactly, including capitalization. Each setting must be a non-empty string or a non-empty list of non-empty strings. If multiple candidates exist, the first wins; values from different columns are not combined. A column with missing or invalid data does not trigger a fallback. The three fields must resolve to distinct source columns. If no candidate exists, the run fails with an error listing the attempted names and available columns.

Existing string settings continue to work. List order sets priority and affects the output folder's challenge hash.

Source-file Requirements¶

Every source file read during the run must contain at least one configured candidate for each of the observed, location, and date fields. The as_of method also requires an as_of column.

The source target column is optional. If it is present, EpiBench filters the file to the configured target and preserves that column. If it is absent from every file, EpiBench adds a target column with the configured target value and logs a warning that it could not verify the target. A run fails if some vintages have a target column and others do not.

When multiple revisions share a date and location, EpiBench retains the latest as_of version available at the cutoff. If target is present, it is also part of this revision key. When no as_of column exists, duplicate revision keys are rejected because there is no reliable way to select a vintage.

Output¶

Upon a successful epibench create run, you can expect a single folder to appear at your --output-path. The folder will be named by whatever string you set as challenge_name in the configuration file, followed by a ten-character hash that is both unique to your specifications and reproducible. Within the folder, you would find the following nested structure:

output_path/
└── my-covid-challenge_b7bb9fbf7c/
    ├── task_list.csv
    └── gt/ 
        ├── 2026-01-01/
        │   └── 20260101_gt.csv
        ├── 2026-01-08/
        │   └── 20260108_gt.csv
        ├── 2026-01-15/
        │   └── 20260115_gt.csv
        ├── 2026-01-22/
        │   └── 20260122_gt.csv
        └── 2026-01-29/
            └── 20260129_gt.csv

Where each requested date of reference has its own folder and file within the gt/ directory, and the task_list.csv file give relative paths to ground truth data files for each date of reference.

Each generated ground truth file uses exactly these standardized columns: target_end_date, location, target, and observed. The configured source date, location, and observed columns are renamed during output preparation. For each task, only observations with target_end_date from July 1 of its season through that task's reference date are returned.

example usage¶

An example config could look like this:

epibench create --config-path "path/to/create-config.yml"

where --config-path is the path to your YAML configuration file.

epibench create¶