Skip to content

B1: finalized history, Wednesday reports, and known-final inputs

Current work: three-candidate cluster handoff. The fromB0 attribution experiments are retired. The architecture descriptions below describe implemented models; the flag-aware direct and parallel-head candidates in the handoff still require implementation.

B1 retains B0's MLP, convolution, multiscale, sharing and spatial formulations. It predicts reference finals for two completed weeks, then four future weeks from each sampled correction. The default fits six independent targets; pathogen-grouped and shared six-target fits are also available. Every component receives all six observed histories and shares parameters across locations.

Precomputed data

Like B0, training reads a compressed NumPy dataset, without querying providers or the explorer. B0 stores one finalized [week, channel, value/mask, location] panel. B1 stores finalized older history and, for the two recent weeks, Wednesday reports where available and explicitly flagged reference finals otherwise:

Array Axes
X_values, X_available, X_final episode, 12 context weeks, 6 channels, 52 locations
Y_recent, Y_recent_valid episode, 2 recent weeks, 6 channels, 52 locations
Y_future, Y_future_valid episode, 4 future weeks, 6 channels, 52 locations
X_provenance, X_reason same as X
Y_provenance, Y_reason episode, 6 output weeks, 6 channels, 52 locations

Issuance dates, exact context/target dates, ordered locations and JSON metadata are saved alongside the arrays. Missing values use zero storage with a false mask; a true observed zero has a true mask. Training and inference zero hidden values before transforms, anchors, or derived features. The pinned label arrays are separate from input arrays. X_final identifies supplied reference finals. Episode X has axes [week, channel, (value, available, known_final), location]; Y retains its separate value/mask fields. The raw label mask records truth availability; the effective nowcast loss/score mask also excludes visible known finals.

uv run python -m tapestry.model_data build-wednesday \
  --data-root data --start 2023-08-09 --end 2026-09-16 \
  --truth-cutoff 2026-09-16 \
  --calendar-dataset data/processed/build_b_finalized.npz \
  --output data/processed/build_b1_wednesday_calendar.npz

uv run python -m tapestry.models.b1_report \
  --dataset data/processed/build_b1_wednesday_calendar.npz \
  --output data/processed/b1-audit

The model calendar has one shared default, provenance.CALENDAR_START = 2023-09-02, next to the three SEASONS. B0's builder and CLI use it. For a paired B0/B1 experiment, pass --calendar-dataset as above: B1 records B0's exact week list and NPZ hash, including a custom start or shorter calendar. The manager verifies that source and pins its hash when planning. A changed B0 NPZ requires a new B1 build and experiment. --calendar-start is an alternative for an explicit Saturday lower bound; it cannot be combined with the B0 file.

The NPZ retains all issuance rows with the current mixed-input policy. WednesdayDataset.load() returns the model-calendar view by default: out-of-calendar context and labels are masked and zeroed, and issuances with no overlap are removed. Folds, native prediction and audits therefore agree. load(path, archive=True) exposes raw archive arrays for issuance diagnostics; .episodes() still applies the model calendar. The model view covers the three configured seasons, not prospective 2026–2027 work. Extending the research calendar requires an explicit change to SEASONS.

The current dataset and checkpoint schemas are version 3. Rebuild the dataset and refit models before using this representation; older artifacts are rejected. The local build_b1_wednesday_calendar.npz is replaced in place. Existing manager plans pin the previous hash and require a new experiment. Remote datasets and running jobs are not changed by rebuilding this local checkout.

The September 17 local rebuild has 154/154 calendar episodes with reference labels and inputs, up from 151. Among the 90,896 labeled recent cells, 49,961 have eligible Wednesday reports and 40,935 receive supplied finals after the Git-history integration. Of the recent inputs, 560 use Git vintages. Compared with the earlier native-Hub/Delphi-only build, Git adds 61 reported cells and its authoritative coverage masks 249 previously used Delphi cells; those 249 receive flagged finals. These changes are all influenza ED and follow the existing no-fallback-through-Hub-omissions rule. Natural nowcast scoring excludes those supplied answers. COVID and RSV admissions have zero natural nowcast scoring cells in 2023–2024 under this rule; their future forecasts remain scored. All six channels have both fitting and validation supervision in every fold.

Held-out season Fitting episodes Validation episodes Evaluation episodes
2023–2024 105 54 49
2024–2025 102 54 57
2025–2026 101 48 58

Support counts are in data/processed/b1-final-support.json; input coverage and revision plots are in data/processed/b1-audit/. No model was refit for these counts. Version availability is not revision frequency. Delphi advertises changes by report_time; the last eligible value for an observation remains available until its next change. A Hub commit is ground truth for the complete repository state, including unchanged files and rows. The last eligible complete Hub state remains available until replaced. No Wednesday change is required; explicit nulls and removals are different from absence of a new change.

The source-coverage*.png plots reconstruct these information states from the same pinned canonical archives, before provider selection and final-value filling. Gray means no valid eligible version in any source, not “no new revision today.” Every combination of Hub as_of, Hub Git, and Delphi has its own color. In the aggregate plot, overlaps are measured for the same week/location cell. Weekly existence aggregates over locations, so its overlaps can also span locations. source-coverage-weeks.png has one square per context week and issuance; source-coverage-locations.png shows two rows per location, offset −2 above −1, over continuous Wednesday time. White means outside the model calendar. The aggregate plot's denominator is up to 12 eligible context weeks × 52 locations (states, DC, US) per issuance and target, with equal cell weights and no padding. The location plot keeps both recent weeks separate, without averaging over time.

input-source-coverage.png and its CSV separately show what the model uses after source precedence and the supplied-final policy. Older weeks use finals by policy, even when an historical version exists. A source version may also be excluded by Hub precedence. Neither case is evidence that the archive has no version. The source CSVs retain overlapping provider availability; the location CSV also records model_supplied_final. location-coverage-check.json reports geographic support, and version-state-dataset-check.json compares every saved recent-report mask and value with the reconstructed latest eligible source state. The standalone b1_report --data-root data audit requires archives matching the dataset manifests; training reports without archives produce model-input diagnostics only. The September 17 check reconstructed all saved inputs with zero differences in values, availability, or final flags. Of 49,961 recent selected report cells, 14,712 already use a release from before their issuance date. Version persistence therefore needs no dataset-value change or Longleaf dataset replacement. The source-state audit additionally finds 41,430 recent cells with multiple available providers before B1 applies its source precedence.

This reads SelectedData.selected_tables, canonical Hub time-series products, and native unsmoothed Delphi archives. It does not read the plotting cache. Sources must already be acquired locally. The materializer retains snapshot IDs, source paths, release timestamps, manifest hashes, missingness reasons and the selector's quarantine audit. Provenance and missingness reasons are audit metadata; X_final is a model input. A cell's integer provenance index refers to the saved metadata table; a Delphi entry stores the fallback reason absent_hub_historical_coverage.

The builder rejects a dataset with zero usable supervised episodes: at least one Wednesday must have both an available context value and a valid recent or future reference label. It raises ValueError in Python; build-wednesday prints the error to stderr and exits with status 2 before writing the NPZ or JSON. The error includes the requested dates, data root, and counts of episodes with inputs and labels. Existing output files remain untouched. Missing channels and leading gaps remain allowed when there is usable overlap; an observed zero is still an available value. Copying a precomputed dataset does not populate the source archives needed to rebuild it.

Calendar and source assumptions

  • The requested July 30, 2023 and November 5, 2023 are interpreted as week starts, giving first supported Saturday ends August 5 and November 11. The latter restriction applies to RSV ED inputs and labels, not other channels.
  • Native Hub event dates and Delphi weekly reference_time are treated as week ends. The reader requires Saturdays and rejects other weekdays, rather than shifting dates heuristically. Current local records passed this check. Hub Parquet schemas may expose target_end_date despite a catalog date alias; both are supported explicitly.
  • For Wednesday issuance d, s=d−4 days. Context ends at s; outputs are s−7, s, s+7, s+14, s+21, s+28 days. Offsets from the following Saturday are [-2,-1,0,1,2,3]. November 22, 2023 therefore predicts finals for November 11 and 18, followed by November 25 and December 2, 9 and 16.
  • The date-based cutoff includes all of Wednesday. Timestamps normalize to UTC; naive timestamps are assumed UTC. This follows the explorer's end-of-day convention and does not establish availability before an intraday deadline. Retrieval dates never substitute for missing historical release dates.
  • Admissions remain counts. Hub ED observations are proportions. Delphi ED percentages are divided by 100. Only native states/DC and US are retained; no geographic aggregation, interpolation or filled Delphi variants are used. Delphi fill_method=source is the native observed series.
  • Resolve the latest eligible complete Hub as_of snapshot per source, and independently resolve each Delphi observation's latest eligible report_time. Thursday information is excluded from Wednesday inputs, even when on disk. Canonical Git snapshots of historical files without as_of are used outside native Hub coverage, before Delphi. A commit on the first-parent branch is ground truth for repository state; its committer timestamp determines the dated state in this reconstruction. It is not the provider's release clock, and authorship and retrieval dates do not establish availability. Full-file omissions and deletions remain missing, including in Git fallback.
  • Conservative coverage assumption: for each channel/location, the earliest observation in any eligible Hub snapshot establishes coverage from that week onward. Absent earlier periods may use Delphi. Interior holes, trailing gaps, explicit nulls, conflicts and disappearances in later snapshots remain missing in the vintage resolver. The input builder subsequently applies the explicit reference-final policy below; it never claims those replacements were available on Wednesday. This deliberately avoids resurrecting a retraction through an older release or Delphi. It can retain missingness where archive coverage cannot be distinguished from a provider omission. No later release establishes earlier Wednesday coverage.
  • Reference finals are the latest eligible revisions under the same Hub-first policy at the explicitly supplied truth cutoff. They are not guaranteed immutable. Genuine source support gaps remain false masks; no synthetic labels.
  • All issuance rows are retained in the raw archive for auditing. The default model view selects calendar-overlapping rows. Episode iteration skips rows with no conditioning data or no permitted labels; inference does not require labels. Incomplete initial contexts are allowed and padded with false masks.

Missing vintages and the nowcasting task

Current assumption: reference finals are supplied as known conditioning data.

  • All context weeks older than the two nowcast weeks use reference finals pinned to truth_cutoff, with X_final=True where a valid final exists. Missing finals remain missing; no interpolation is used.
  • In the last two weeks, keep any genuine report available by the Wednesday UTC cutoff. An exact Wednesday release is not required: the latest eligible vintage still applies. These cells have X_final=False, even if equal to the later final.
  • When such a recent report is absent for any reason, supply its pinned reference final and set X_final=True. If both are absent, leave availability and finality false. Source support restrictions still apply.
  • A visible known recent final bypasses the recent correction head and is returned exactly in native units. Forecasts condition on that value using the existing working-space transforms (including the positive anchor floor and bounded ED transform); no new numerical finality rule is inferred from source values.
  • Its reference label remains in the archive, but it contributes no nowcast loss or nowcast score. It can still condition and train the future forecast stage.
  • Artificial dropout and blocked CV weeks hide the value and its known-final flag. If a recent final is hidden, it becomes an ordinary nowcast target again, provided its label is permitted by the fold. Hidden older finals add no loss.
  • This is a retrospective conditional-forecast experiment: older and fallback recent inputs may contain revisions released after issuance. Scores do not establish Wednesday-operational performance. No improvement is assumed before rerunning experiments.

A missing archive does not establish that the provider had stopped reporting. Release timestamps still control which recent values count as real Wednesday reports; retrieval timestamps never establish historical availability.

First vintage by channel

Verified from X_available and the corresponding X_provenance release records in the September 16 truth-cutoff archive. Vintage date is the source release date contributing the first usable B1 history; first Wednesday is the first issuance allowed to use it. Release dates are UTC, not acquisition dates.

Channel Vintage date First Wednesday Sources at first Wednesday Locations with any context
Flu admissions 2023-09-23 2023-09-27 FluSight Hub 52/52
COVID admissions 2024-11-20 2024-11-20 COVID Hub + Delphi NHSN 52/52
RSV admissions 2024-11-27 2024-11-27 RSV Hub 52/52
Flu ED 2024-04-19 2024-04-24 Delphi NSSP 49/52
COVID ED 2024-04-19 2024-04-24 Delphi NSSP 49/52
RSV ED 2024-04-19 2024-04-24 Delphi NSSP 49/52

These dates describe the pinned local archive under B1's source-selection policy, not the earliest release a provider may ever have published. A location counts when any week in its 12-week focal history is available; this does not establish availability of the two recent weeks or complete coverage of every context week. The 52 locations are the 50 states, DC and US.

These dates describe genuine vintage coverage. Earlier inputs can now be supplied from pinned reference finals, explicitly marked by X_final.

The masks have distinct meanings:

Field or option Meaning
X_available / A A usable report or supplied final exists
X_final / F The input was supplied from reference truth under this policy
X_reason, X_provenance Why absent / which release supplied the value
artificial mask D Hide inputs; visibility is V = A & ~D; effective finality is F & V
Y_recent_valid A reference label exists; visible known finals are additionally excluded from supervision
--pipeline two_stage (pipe_two) Correct unknown recent values, pass through known finals, then forecast
--pipeline direct (pipe_direct) Forecast from the same mixed inputs using B0; no recent head or bypass

X_final records our conditioning choice, not a provider guarantee of finality. Missing focal history can still use other channels and calendar/location features. True absence of reference labels cannot be repaired by this representation.

Historical strict-vintage support audit

The counts below describe the superseded strict Wednesday representation, not the current finalized-history/fallback inputs. Current support is generated in data/processed/b1-audit/audit.json by the audit command.

These historical counts use the September 17 rebuild, September 16 reference truth, the actual Longleaf B0 NPZ calendar, and natural availability before training dropout. An episode contains all six channels and 52 locations. A missing focal channel does not remove it; zero inputs everywhere or no valid labels does.

Comparison on the shared three-season calendar Finalized input Wednesday B1 Lost
Same B1 truth/support, two recent + four future outputs 154 151 3 (1.9%)
Actual B0 NPZ, same four future horizons and issuance dates 152 149 3 (2.0%)

Both lose September 6, 13 and 20, 2023. The first row isolates vintage availability using the same B1 reference support. The second uses the actual B0 file downloaded from Longleaf, so source/label differences remain. Two trailing B1 episodes have recent labels only; the difference between 151 and 149 is not an additional vintage loss. These are eligible development episodes, not counts used to fit any single fold. Removing the three episodes loses 3,542 of 270,608 labeled output cells (1.3%), or 628 of 46,417 issuance/target/location examples. Overlapping origins repeat observation weeks; these are not independent epidemics.

Strict-vintage leave-one-season-out fitting support, compared to counterfactual finalized histories on the same B1 label support and with identical held/hidden weeks removed:

Held-out season Finalized eligible Wednesday fitting Lost
2023–2024 105 104 1 (1.0%)
2024–2025 102 99 3 (2.9%)
2025–2026 101 98 3 (3.0%)

The first fold loses August 7, 2024: its surviving training-season context has no Wednesday input after removing held/hidden weeks. The other folds lose the three September dates. Validation loses 1/3/3 episodes respectively; held-out evaluation loses 3/0/0. Fold counts overlap and must not be added as distinct samples.

Missing input cells

Against B1's own finalized-label support, 148,150 of 531,752 input cells (27.9%) have no Wednesday value. This is not missingness relative to B0: cells already absent from B1 reference truth are excluded from the denominator. B0 also has missing channels, and has a different source policy.

Channel Reference-supported input cells Unavailable Wednesday Missing
Flu admissions 91,452 16,687 18.2%
COVID admissions 92,180 37,164 40.3%
RSV admissions 77,252 23,775 30.8%
Flu ED 91,884 25,800 28.1%
COVID ED 92,612 25,467 27.5%
RSV ED 86,372 19,257 22.3%

A cell is one issuance/context-week/channel/location, counted again in every window. Counts include the three unusable candidate episodes, exclude calendar padding and missing reference labels, and precede artificial dropout. They do not attribute missingness entirely to lack of archive coverage: delayed reports, nulls and conservative treatment of retractions can also contribute.

Reproduce the complete counts, including per-fold losses and per-target/season history coverage, in b1-calendar-support.json:

uv run python scripts/audit_b1_support.py \
  --dataset data/processed/build_b1_wednesday_calendar.npz \
  --b0-dataset data/processed/build_b_finalized.npz \
  --output docs/design/b1-calendar-support.json

The JSON's short_run_fitting_partition is a legacy date-bounded diagnostic, not today's season-CV training split. On the current calendar it has 88/91 eligible episodes and 135,271/297,648 missing input cells (45.4%). The previous 163/156 and 28.2% full-archive counts are preserved in b1-training-support.json; they used August and trailing weeks outside the model calendar and must not be quoted as current fold support.

Model formulations

The successful B0 formulations are available through named presets and explicit architecture switches. These are B0-derived candidates for B1, not evidence that their B0 ranking will hold with Wednesday inputs and sampled nowcasts.

Preset Encoder Independently fitted groups Spatial exchange Output-head sharing
target_mlp (default) MLP Six targets None Shared within component
pathogen_mlp MLP Three admission/ED pathogen pairs None Shared within component
target_conv Two temporal convolutions Six targets None Shared within component
target_multiscale Multiscale temporal convolutions Six targets None Shared within component
spatial_conv Convolution All six together Shared context attention Shared
joint_mlp MLP All six together Joint location/target attention Per pathogen

Independent-target/pathogen MLPs and independent-target multiscale correspond to the leading B0 results. The joint-attention MLP with pathogen heads was also among the leading configurations. Convolution and shared spatial convolution are retained as useful contrasting B0 formulations, not claimed B0 winners. Standalone presets default to cap 100, except joint_mlp cap 300 matching that B0 reference. A multi-preset comparison uses a common training budget from its base configuration (default cap 100); --epochs 300 changes it for all.

The implementation reuses B0's convolution/multiscale encoders, spatial block, transforms, loss scales and scientific weights. It supports:

  • --encoder mlp|conv|multiscale_conv and --fit-partition target|pathogen|all.
  • --spatial none|attention|pathogen_spatial|target_spatial|joint_location_target.
  • --decoder legacy|residual2, --head-sharing shared|pathogen|target, and --heads shared|state_us.
  • --noise global|local (local adds a per-location latent), plus --us-error none|shared_factor. Recent and future stages have independent draws; validation freezes global, local and national noise separately.
  • Admission transforms sqrt|fourth_root|log1p; ED transforms linear|logit|fourth_root. Scores always remain in native units.
  • Calendar, geography and dynamics switches; location embeddings; lookback, width, latent size, learning rate, weight decay and training/stopping budgets.

The failed stochastic-trend decoder stays removed. Raw/rate-only admissions remain excluded following the B0.1 design. The default uses width 64, latent 16, fourth-root admission rates per 100,000 and logit ED inputs. Adjacent slopes and acceleration are recomputed after masking with validity flags. The two-stage B1 excludes observation-age and release-age/revision-history features. Direct mode calls B0 itself and therefore retains B0's observation-age dynamics feature.

Population denominators remain fixed across seasons. Checkpoints save the mapping and population-file hash. The default is B0's frozen data/metadata/locations.csv; a different CSV can be supplied explicitly. A shorter --lookback crops the precomputed context before scale fitting and masking. A longer one requires materializing at least that many weeks; the runner rejects insufficient context rather than synthesizing it. Episodes with no conditioning observations in the selected context are excluded before fitting.

In two-stage mode, each output has its own focal-history encoding and explicit recent-anchor and any-history flags. Visible preliminary values anchor their corresponding recent corrections. Missing values use the last visible focal value, or a learned context baseline when that history is entirely missing. Positive transformed outputs use B0's softplus residual with a .001 normalized anchor floor; ED uses a sigmoid for linear/logit inputs or a bounded inverse fourth-root transform.

Every forecast receives the original masked Wednesday encoding and only its own target's paired two sampled recent values, including in grouped fits. Member m always feeds member m; no averaging, shuffling or teacher forcing. Forecast gradients pass through the sampled corrections. Shared components can represent within-group dependence through their shared noise. Separate fitted components use independent draws; common member numbering across them does not represent learned dependence. Marginal scores do not establish joint calibration.

Scenario strings and masking controls

B1 uses complete, round-trippable b1:v2: strings using B0's field vocabulary, extended with pipeline and masking settings. Example, obtained with python -m tapestry.models b1 scenario --preset target_conv --mask-rate 0.25:

b1:v2:h12:tr_4rt:ed_logit:geo1:dyn1:lw_obj:enc_conv:sp_none:hd_sh:dec_leg:nz_glob:us_none:z16:w64:ep100:pat30:bs8:m128:lr0.001:hs_sh:cal1:id0:fit_targ:vm256:wd0.0:pipe_two:mask0.25:mr0.5:mg0.3:mo0.2

enc_conv selects convolution, fit_targ independently fits six targets, pipe_two selects the two-stage model, and mask0.25 masks 25% of episodes in expectation. All architecture switches, training settings and masking settings are encoded; no omitted fields silently become defaults when parsing a string. Floating-point values round-trip without truncation. Seeds, data snapshots and date partitions are recorded separately in the run manifest, as in B0.

Use --preset target_conv to start from a readable preset, or --scenario 'b1:v2:…' to load an exact configuration. Explicit flags override either base and the resolved string is saved in the checkpoint and manifest. Changed settings produce a different identifier. Output folders use a readable short run_id plus a 12-character digest of the complete string, avoiding filesystem name-length limits. The full string is also included in per-cell score files and the comparison manifest. B1Scenario.from_string and .flags() support programmatic round trips and invocation.

--mask-rate p is the probability an episode receives an artificial missingness pattern, not the fraction of individual cells hidden. 0 preserves natural missingness only; .5 is the original B1 default; 1 assigns every episode a pattern, which may overlap already-missing inputs. Conditional pattern fractions are controlled by --mask-recent, --mask-gap, and --mask-outage; they must sum to one and default to .5/.3/.2.

Mask rate Natural Recent reports Local gap Channel outage
0 100% 0% 0% 0%
.25 75% 12.5% 7.5% 5%
.5 50% 25% 15% 10%
1 0% 50% 30% 20%

These probabilities are assumptions for experiments, not optimized choices. They apply to fresh masks for each component and epoch. Labels remain stored, but hiding a known recent final restores its supervision, so loss-cell weights are recomputed across the full partition after masking. Both direct and two-stage models accept any masking rate in single-model training. The comparison grid uses an unmasked direct control.

Availability, dropout and known-final masks are distinct:

A = available mixed inputs (reports or supplied finals; precomputed)
D = artificially hidden available inputs (drawn every training epoch)
V = A & ~D (visible inputs)
F_visible = X_final & V (known finals visible to the network)

Default episode probabilities are 50% natural, 25% missing recent reports, 15% local gaps, 10% source outage. These are configurable starting assumptions:

  • Recent reports hide one or two ending weeks, across all locations. Half choose one channel; half choose the entire admissions or ED family.
  • A local gap hides one random 1–3 week block in one location/channel history.
  • An outage hides one channel across all context weeks and locations.
  • D intersects A; no artificial flag is supplied to the network. All-hidden histories remain possible and exercise the learned baseline.

Packed training D arrays are saved for each epoch and fitted component in masks-*.npz, with shape and issuance order. Validation D arrays are saved explicitly. A is recoverable from the pinned dataset; V is their Boolean difference. Validation uses fixed masks and latent draws, with separate generators from training. Natural/two-stage controls use natural validation; masked B1 uses a fixed draw from its training-mask mixture. Inference defaults to natural availability; the Python API accepts explicit dropout, and the CLI exposes reproducible stress scenarios. Masked older inputs do not create a historical reconstruction loss: supervision is still two recent finals plus four future weeks.

Fitting and prediction

For each fitted component, minimize one-half the weighted recent native-unit fair CRPS plus one-half the weighted future native-unit fair CRPS. Each task is independently normalized over its valid cells. Fitting-only unique-label-week Q95 scales and B0 season/geography weights are retained (equal target-date seasons, 80% states/DC, 20% native US, renormalizing absent geography groups). Visible known finals are excluded from recent supervision. Cell weights are recomputed over the complete partition after dropout, preserving the same season/target/geography rule. A wholly absent task retains zero weight; its half is not transferred to the other task. A component with only forecast supervision can still train. B0 target coefficients [1,1,1,.5,.5,.5] apply within shared or pathogen-grouped fits and cancel within single-target fits. A completely unsupported target raises an error rather than yielding an untrained component.

This is the training objective. It is deliberately not the selection score: ranking is B0's ensemble-relative WIS, described below.

Input scales use only fitting mixed histories: the latest permitted fitting-episode value for each context date, counted once. Native loss scales use unique permitted reference-label dates. Validation does not refit scales.

One command fits one leave-one-season-out fold and writes that fold's held-out season forecasts under the four evaluation input scenarios:

uv run python -m tapestry.models b1 train \
  --held-out-season 2025-2026 --retrospective \
  --preset target_conv --mask-rate 0.5 \
  --output data/experiments/b1-probe/s42/eval_2025-2026

uv run python -m tapestry.models b1 predict \
  --checkpoint data/experiments/b1-probe/s42/eval_2025-2026/model.pt \
  --issuance 2025-08-06 --members 256 \
  --output data/experiments/b1-probe/s42/predictions.npz

The defaults are a 100-epoch cap, patience 30, 128 training members and 256 fixed validation members, Adam at .001 and no weight decay. Components use seed + 10000i. Each component retains its best validation checkpoint; --patience 0 instead retains the final epoch of a fixed-budget fit. The runner performs no final refit* on validation data. Prediction NPZs contain the paired six-week samples, quantiles, exact dates, A/D and input/training provenance. They are native research artifacts, not Hub submissions; a Hub exporter must select only that Hub's supported horizons.

Leave-one-season-out CV pins reference truth after each fold's own dates, exactly as B0's finalized CV does, so --retrospective is required and these are retrospective development runs, never operational ones. There is no chronological split: the one B1.0 used is described in the log below and was removed.

One manager, one scorer

B1 and B0 are managed and scored by the same code, so a B1 number and a B0 number mean the same thing and can be compared directly:

Concern Shared implementation
Plan, run, resume, status, rank, compare tapestry.models.manager
GPU dispatch and launcher tapestry.models.dispatch, scripts/jlessler.sbatch
Forecast scoring and ranking tapestry.evaluation.totals
Hub task export tapestry.evaluation.hubs.export
EpiBench comparison, plots, fans tapestry.evaluation.sweep
Fold calendar and hidden validation weeks season_cv.SEASONS, VALIDATION_WEEKS/SPACING/OFFSET
Encoders, spatial block, transforms, fair CRPS, loss scales and weights b0.py, architecture.py, objective.py
Population denominators data/metadata/locations.csv

What genuinely differs is confined to src/tapestry/models/backends.py: how a suite expands into scenarios, how a fold's fit is launched, and which artifacts prove a seed finished. The manager itself contains no model conditionals.

Direct B1 wraps the actual B0 module and selects its fitted target channels; it reuses B0's dynamics, heads, residual anchors and fixed missing-history prior. Two-stage B1 uses SampleHead with an explicit member axis for paired sampling, recent-anchor features and a learned missing-history baseline.

Forecast ranking: B0's objective, unchanged

manager rank scores B1 exactly as it scores B0. B1's future quantiles are matched to the frozen Hub tasks by target, reference Saturday, target-end date, location and horizon. Reference Saturday is the Saturday after Wednesday issuance, with horizons 0–3; recent offsets are never relabelled as Hub forecasts. Each fold contributes only its own held-out season, since a fold's six-week output window reaches into seasons it trained on.

The score is then B0's, verbatim: per target/season/location, total native model WIS over total ensemble WIS; states/DC ratios average equally with 80% weight and native US takes 20%; within a season, available targets average with admissions 1 and ED .5; season composites average equally; configurations report mean and SD across seeds.

--frozen is required, not optional. A run that cannot be scored against the ensemble is a failed run, and fails at planning time rather than after fitting.

The scored observations come from the frozen units.parquet files, exactly the same later-vintage truth used for B0. They do not come from Wednesday's inputs or substitute B1's own label values. Scoring requires every frozen task; missing focal history is not a reason to drop a forecast. The composite covers the nine ensemble-supported target/season cases, not every cell in the full archive. Native diagnostics separately score the broader B1 reference-label support.

Assumption: B1 conditions on the Wednesday information state, not the Hub's original submission cutoff, so this is a retrospective benchmark and not an operational submission comparison. A B1-versus-B0 difference therefore mixes the model change with the information change — which is exactly what the two attribution experiments below are designed to separate.

Nowcast ranking: relative to preliminary-value persistence

Offsets -2 and -1 are reference finals for weeks that have already happened, so no Hub ensemble forecasts them and B0's denominator does not exist. The denominator is instead the latest visible context value for the same channel/location, repeated for both recent offsets. It need not be the report for the target week itself. That is season_cv.persistence, the same rule B0 uses for its own persistence diagnostic.

Everything above the denominator is unchanged: totals.quantile_scores computes the metrics and B0's season_scores/run_scores/configuration_ranking do the aggregation, so a nowcast rank uses the same aggregation as a forecast rank with a different reference. rank writes it under ranking-<hash>/nowcast/, and nowcast-support.json records scored and excluded cell counts, also broken down by target and season. history-stratified-scores-*.csv separately reports native WIS/CRPS/coverage with and without focal history, by task, target, season, geography and stress condition. Every valid label remains in native per-cell scores, even when it has no persistence baseline. These diagnostics are not an alternative composite ranking. history-support.csv from the data audit also distinguishes an observed recent report from only older focal history.

The 2023-2024 nowcast support is thin, and so is the forecast support. Measured on the calendar-pinned dataset, among model-eligible nowcast label cells (episodes with some input anywhere), the share that has focal Wednesday-visible history — and therefore a persistence baseline — is:

Target 2023-2024 2024-2025 2025-2026
Flu admissions 94.6% 72.7% 99.9%
COVID admissions 0% 72.7% 100%
RSV admissions 0% 74.8% 100%
Flu ED 32.1% 95.2% 97.6%
COVID ED 32.1% 95.5% 98.4%
RSV ED 38.4% 95.3% 98.4%

Early Wednesday vintages for COVID and RSV admissions do not exist in the local archives under the conservative coverage policy, so those two targets receive no weight at all in the 2023-2024 nowcast composite, and the ED targets are scored on about a third of their cells. Since seasons average equally, that season's composite rests on four targets and a non-random subset of locations — the ones that reported early.

This is not specific to nowcasting. The frozen Hub support that the forecast score uses is thinner still: 2023-2024 has ensemble-scored tasks for flu admissions only, and 2024-2025 for flu and COVID admissions only. B0's published scores already carry that property, which is why season_composites renormalizes over available targets rather than assuming six. Read both scores as "equal-weighted over seasons, renormalized within each season over whatever that season actually supports", and read 2023-2024 in particular as a narrow slice.

  • Cells with no visible history for their own channel and location have no persistence baseline and are excluded from both relative-score sums. Other channels can still condition the model and native scores remain available. This is a large exclusion early in a season, and it is reported, not hidden.
  • A direct configuration produces no nowcasts, writes no nowcast-totals.csv, and is simply absent from that ranking.
  • A ratio below 1 means the model beats latest-visible-value persistence. It does not mean the model beats a competitive nowcasting method; persistence is a naive reference.
  • Forecast and nowcast scores share weights and aggregation but not support. They are reported separately and must never be averaged into one number.

Named experiments and cluster launch

.venv/bin/python -m tapestry.models.manager plan -e B1-onlynowcast-calendar \
  --suite B1-onlynowcast --seeds 42 43 44 --retrospective --device cuda \
  --dataset data/processed/build_b1_wednesday_calendar.npz

mkdir -p output/slurm
sbatch --job-name=B1-onlynowcast-calendar --array=0-3 scripts/jlessler.sbatch B1-onlynowcast-calendar

.venv/bin/python -m tapestry.models.manager status -e B1-onlynowcast-calendar
.venv/bin/python -m tapestry.models.manager rank -e B1-onlynowcast-calendar
sbatch scripts/b0_compare.sbatch B1-onlynowcast-calendar

plan writes experiment.json, jobs.csv, preflight.json and a code/ source snapshot, and pins input hashes. The preflight also verifies that every frozen location has positive ensemble WIS, so an undefined denominator is caught in seconds instead of after an experiment. Do not refresh that snapshot while workers run. A changed data/protocol setting requires a new experiment name. Attempts live at <run_id>/s42/attempt-001/b1/, with run.json, run.log, one eval_<season>/ per fold, totals.csv and nowcast-totals.csv.

Each GPU allocation runs several lanes from one shared queue. Seeds of one configuration stay sequential; different configurations share a GPU. LANES=4 is the default and LANES=6 is the tested upper end. This range and the two-day walltime are resource choices, not measured B1 capacity guarantees.

The attribution ladder

B1.0 was a 24-configuration grid that moved architecture, task and inputs all at once, so a gap against B0 had no single cause. It was discarded. Instead, B1 offers B0's four best configurations under each task/input setting. All four are h12:tr_4rt:ed_logit:geo1:dyn1:lw_obj:enc_mlp:sp_none:hd_sh:dec_leg:nz_glob:z16:w64:pat30:bs8:m128:lr0.001, differing only in target partition and epoch cap:

B0.1 rank B0 combined score fit_partition epochs
1 .8834 target 100
2 .8948 pathogen 300
3 .8949 target 300
4 .8968 pathogen 100

The active model-selection experiment uses only B0's rank-1 configuration, with ten seeds per candidate: masked direct, masked direct with finality flags, and the latter with a parallel auxiliary nowcast head. See the cluster handoff for fixed assumptions, scoring requirements and the decision rule. The existing B1-onlymask-refit provides three potentially reusable baseline seeds.

The old unmasked fromB0 attribution experiments and their launch suite are retired. The shared B0 predictor, rank-1 recipe and select-then-refit implementation remain in use. Historical B0 scores are not a controlled estimate of the vintage input effect because training labels and usable support also differ.

EpiBench comparison

manager compare runs the same B0 EpiBench config scorer, R scoringutils, Hubverse exporter, diagnostic plots and projection fans, for either model, via scripts/b0_compare.sbatch <experiment>. EpiBench's separate all-target ranking uses B0's equal-target geometric WIS-ratio convention, which is not the manager's arithmetic location-relative rank. Pinned mirrors of the frozen commits, EpiBenchmark and the R dependencies are required for this optional comparison.

Stress scenarios and native diagnostics

Every fold also writes forecasts under four evaluation input scenarios — natural, recent, gap, outage — with fixed stress masks identical across configurations for each seed and issuance. Training mask rates do not change evaluation masks. Ranking uses natural only; the other three are diagnostics of robustness to missing inputs, saved as forecasts-<run>-<stress>.npz beside their evaluation-masks-*.npz.

b1_report additionally writes native per-target summaries, source-coverage and missingness audits, preliminary-versus-reference-final plots, paired sampled paths and score/coverage graphs. These are native-unit diagnostics in different units per target; raw admissions and ED scores must not be pooled into an unnormalized overall score. Hub-relative comparison is the ranking above.

Scoring when a Hub benchmark is unavailable

Hub forecast availability determines what can be ranked, not what can be measured. The frozen support has nine scored target/season cases; the native diagnostics cover all six targets on every date with a reference label, independently of whether a matching Hub forecast exists.

Available data for a target/date/location Evaluation
Reference observation and matching Hub forecast Enters the ensemble-relative ranking
Reference observation, no matching Hub forecast Native CRPS/WIS/coverage diagnostics only
No reference observation No score; reported as missing label support

An ED score measures ED prediction; it does not substitute for an unobserved admissions outcome.

Protocol: B0's leave-one-season-out cross-validation

B1.0 used a chronological split whose validation window was nine summer issuances (June–July 2025). Measured on that run, validation medians sat 10–13× below both the fitting and evaluation data (RSV ED .0001 against .0013 fitting and .0011 evaluation). Selection therefore optimized prediction of a seasonal floor: 41% of component fits peaked at epoch ≤5, the median best epoch was 7, and every one of the 24 configurations was badly under-dispersed (95% coverage .62–.77, with 93% of RSV ED's WIS attributable to underprediction). A control fit at a fixed 300 epochs did not repair this, so the cause was the selection signal, not the stopping rule.

B1 therefore reuses B0's protocol exactly, and it is now B1's only protocol, sharing season_cv.SEASONS and the VALIDATION_WEEKS/SPACING/OFFSET constants:

  • Three leave-one-season-out folds over 2023-2024, 2024-2025 and 2025-2026. Each seed fits all three; a partial 2026-2027 season is excluded, as in B0.
  • The fold calendar starts at B0's first panel week, 2023-09-02. B1's archive reaches back to 2023-08-05, but those four weeks are not in B0's panel, so including them would give B1's 2023-2024 fold both extra history and, because the 3-in-16 pattern counts from the start of a season, a different set of hidden weeks. Weeks outside the three seasons are blocked from context as well as from labels.
  • Within each fold's two training seasons, early stopping hides three consecutive target weeks out of every sixteen — season start, winter and spring. Hidden weeks now include the December peak, giving selection real epidemic signal.
  • Held-out and hidden weeks are removed from labels and zeroed inside every fitting episode's context (X_available → False), mirroring B0 zeroing its panel. Episodes are retained rather than dropped, since B1 already models missing inputs.
  • Validation scores only hidden weeks; evaluation scores only the held-out season and keeps the unmodified Wednesday information state.

Assumption: B1 conditions on real Wednesday vintages, so zeroing a context week removes information genuinely available at issuance. This is the protocol's cost, accepted so that B1 and B0 hold out the same calendar weeks under the same leakage rule. Fold membership and hidden weeks are decided by target date.

Measured effect on one fold (target_mlp, mask .5, seed 42, held-out 2025-2026): selected epochs moved from 3/17/243/51/152/3 under the chronological split to 212/208/163/183/269/107, and validation losses rose about tenfold because validation is no longer a summer trough.

Historical strict-vintage fold sizes after calendar alignment, in episodes:

Held-out season Fitting Validation Evaluation Hidden weeks (B0 and B1)
2023-2024 104 53 46 19
2024-2025 99 51 57 19
2025-2026 98 45 58 18

The hidden-week sets are identical to B0's in all three folds, checked directly against season_cv.validation_split. Training/validation dates are not accepted: folds come from the season calendar. Each seed writes one eval_<season>/ per fold, as B0 does.

Log and local execution, 2026-09-16

  • September 17: versions persist without revisions. Source coverage now resolves complete information states before model-input selection. Unchanged Delphi observations and Hub states remain available between changes; source overlaps are explicit. The previous plots counted selected non-final model inputs, incorrectly suggesting that finalized older context lacked versions. Keep that distinct diagnostic as input-source-coverage, and verify stored recent dataset values and masks against the pinned archives.

  • September 17: separate temporal and geographic coverage. Audit actual per-week location counts and add date-existence and location-by-time report plots to b1_report, alongside the aggregate context-cell plot. No-report cells are gray even when supplied finals exist; weekly source overlaps have distinct colors, and geographic time plots retain both recent offsets separately.

  • September 17: canonical Git target histories. Materialize first-parent history of configured Hub targets without as_of, preserve commit/path/blob provenance, and expose it as a distinct explorer variant. B1 reads the same canonical stream, using Git outside native as_of coverage and before Delphi. The finalized older-history and flagged recent-final policy remains in force.

  • September 17: supplied reference finals with explicit flags. Replace older inputs with pinned finals and fill absent recent reports with flagged finals. Preserve genuine Wednesday recent reports. Visible recent finals bypass the nowcast and are excluded from recent loss/scoring; fold masks and dropout hide flags and restore permitted supervision. Add schema 3 and require refitting. Natural and stress exports carry X_final and exclude supplied answers from nowcast scores; input-coverage figures distinguish reports from supplied finals. This deliberately assumes final values known after issuance and is a retrospective conditional-forecast experiment. Local rebuild only; no training or remote update is implied.

  • Converged B1 onto B0's tools and objective. B1 had drifted into a parallel universe: its own scoring backend, its own native normalized-CRPS ranking, its own launcher and its own copy of the locations file, so a B1 number could not be compared with a B0 number and every tool had two implementations. The manager, the scorer, the ranker and the launcher are now single model-agnostic implementations. Deleted b1_experiment.py and b1_hubs.py along with the native normalized-CRPS objective, SCORE_VERSION b1-task-normalized-crps-v2, b1 compare, the chronological split, scripts/b1_jlessler.sbatch, scripts/b01_jlessler.sbatch and data/metadata/b1_locations.csv (byte-identical to B0's, now data/metadata/locations.csv). Added backends.py for the little that really differs, provenance.py to break a circular import, and evaluation/nowcast.py. Verified by rescoring a completed B0.1 run: totals.csv is byte-identical to the stored one, so the refactor did not move B0's numbers.

  • September 17: exact B0 predictor for the direct control. Found that the previous direct B1 retained 24 dynamics features, recent-anchor flags and a learned baseline, unlike B0's 30 dynamics features and fixed missing-history prior. Direct mode now calls B0 itself; fixed-noise predictions match exactly, including absent histories and hidden NaNs. The calendar-pinned Wednesday dataset remains the B1 default. The subsequent fromB0 attribution runs were retired when the project moved to direct model selection.

  • September 17: archive start dates and Longleaf cleanup. Documented first usable release dates separately from first Wednesday issuances for all six channels, including sources and location coverage. Removed the superseded Longleaf NPZ/JSON at the user's request after checking the replacement hash and the empty job queue; the calendar-pinned dataset remains.

  • September 17: one calendar and explicit missing-history support. Moved the default start to provenance.py; added exact B0-calendar pinning and a shared model view for folds, prediction and audits. Rebuilt to a new NPZ with the same September 16 truth cutoff and byte-identical raw arrays, preserving running experiments' inputs. Added native score strata and relative-score exclusion counts by target/season. Chose strict Wednesday availability: absent vintages stay missing, recent labels remain supervised, and no final-value proxy or inferred finality flag is introduced. Current support is 151/154 six-output episodes; historical full-archive counts are kept only in the old audit JSON.

  • B1's primary score is now B0's ensemble-relative WIS, on the same nine frozen Hub cases, making B1 directly comparable with B0's .8834 top score. --frozen became required rather than optional. Nowcasting at offsets -2/-1 has no ensemble, so it is ranked separately against preliminary-value persistence using B0's identical weights and aggregation. Assumption: the earlier native normalized-CRPS scores are not migrated. Nothing had been run at full budget, so there was nothing to migrate.

  • Fixed a training-season mismatch that would have confounded the comparison. B1's archive reaches four weeks earlier than B0's panel (2023-08-05 onward), so B1's 2023-2024 season had 52 weeks against B0's 48. Because the 3-in-16 hidden validation pattern counts weeks from the start of a season, this also shifted which weeks were hidden: two of the three folds hid a completely different set of 19 and 18 weeks. The shared fold calendar now starts at B0's first panel week (CALENDAR_START = 2023-09-02) and weeks outside the three modelled seasons are blocked from context as well as labels. Verified: B0 and B1 now hold out and hide identical calendar weeks in all three folds.

  • Attribution experiments retired, September 17. Earlier experiments attempted to explain the historical B0/B1 score gap, but changes in input policy, training support and refit procedure limited causal interpretation. The user chose to remove the fromB0 runs and their dedicated tooling, and prioritize a three-candidate model-selection experiment. The masked direct baseline remains. A separate matched-final control is no longer scheduled.

  • ntfy notifications are on by default. scripts/jlessler.sbatch queues its own afterany summary job, so a timeout or cancellation still reports; NTFY=0 disables it and NTFY_URL retargets the topic. Replaced the B0.1-specific scripts/b01_notify.sbatch with the model-agnostic scripts/notify.sbatch.

  • Replaced B1's chronological split with B0's exact leave-one-season-out protocol after B1.0 showed summer-only validation drove selection. Added b1_seasons.py and --held-out-season; one seed now fits three folds. The earlier chronological B1.0 results were discarded rather than kept, since their selection step is not scientifically usable. The chronological split is now removed entirely rather than kept as an option.

  • build-wednesday now rejects zero usable episodes before saving, including missing-source builds. The CLI exits nonzero with input/label support counts instead of writing an all-missing NPZ/JSON and reporting success.

  • Renamed the score columns to B0's convention before launching B1.0. They were backwards: variant held the configuration while scenario held the stress mask, so groupby(['scenario','task']).rank() read as ranking stress scenarios. The arithmetic was correct and is unchanged; only the names were misleading. Now config_id (as in B0) is the configuration, scenario_string its full b1:v2: string, and stress the evaluation input scenario. This renames columns in scores-*.parquet, configuration_ranking.csv, run_scores.csv, target_scores.csv and summary.csv. SCORE_VERSION is bumped to b1-task-normalized-crps-v2 so pre-rename score files cannot be pooled with new ones. Assumption: no full-budget B1 scores existed yet, so nothing is migrated; any pre-rename smoke artifacts must be rescored, not reused.

  • Integrated B1 with the shared manager/dispatcher and replaced the separate fixed-slice launcher. Six one-epoch manager runs (27 component fits) completed; resume reused all six, and rank/compare produced distinct nowcast/forecast results. The full R/EpiBench adapter, exports, rankings and plots completed on an eight-task synthetic frozen-Hub check. An additional one-epoch pathogen run exercised plan/run/rank/compare with that optional benchmark enabled. The 29 focused B1/objective/totals checks passed. These are execution checks, not skill evidence or a full-budget result. Source/data snapshots and attempts use the B0 management workflow; the B1 chronological scientific protocol is retained.

  • Current formulation smoke comparison completed 18 configurations / 69 component fits (six presets × direct control and two-stage mask rates 0/.5), one seed and one epoch. Outputs are under data/experiments/b1-formulations-smoke/. All six horizons were scored; natural and stress curves are graphed separately. This is execution validation only; the prepared 72-seed-run default grid remains unrun.

  • Restored B0 formulations: MLP/conv/multiscale, target/pathogen/all fitting, spatial scopes, residual heads, local/national noise, separate US heads, transforms, calendar/geography/dynamics switches and location embeddings.

  • Added canonical schema-v2 scenario strings, named presets, episode-level mask rate and conditional mask mix, and a multi-formulation comparison grid with a plan-only mode. All scenario settings are stored alongside checkpoints/scores.
  • Kept each forecast conditioned on its own paired corrections. Grouped objectives retain B0 target weights; prediction bundles restore canonical channel order. Validation fixes every enabled stochastic term. Scientific checks cover masking leakage, group weights/alignment, native-unit transform inverses and pairing.

  • Confirmed that direct scoring already includes all six targets irrespective of Hub benchmark availability. Documented reference-label eligibility, unavailable outcomes, ED evaluation and the separate support needed for Hub-relative skill.

  • Audited training-support losses with explicit episode/input/output denominators; added scripts/audit_b1_support.py and the checked-in numerical report. Clarified that B1 has implemented models and one-epoch checkpoints, but no full-budget result.

  • Implemented Wednesday materialization, B0-derived independent two-stage MLPs, training masks, joint-through-samples loss, training/prediction/comparison CLI, provenance, numerical audits and graphs.

  • Current local metadata reports Delphi NHSN releases beginning November 19, 2024 and NSSP releases beginning April 18, 2024. Older observation dates do not establish historical release availability. Hub archives add earlier coverage.
  • Built data/processed/build_b1_wednesday.npz: 163 Wednesday rows from August 9, 2023 through September 16, 2026; shape [163,12,6,52]; 156 usable episodes. First usable issuance: September 27, 2023. Truth pinned September 16, 2026.
  • Focused checks cover source units/date aliases, Thursday leakage, whole-snapshot retractions, same-release conflicts, Sunday rejection, RSV support, hidden NaNs through anchors/dynamics, task/geography weights, future-to-nowcast gradients, exact member pairing and independent future noise.
  • Ran all three variants, six targets each, for one epoch on real local data: 88 fitting episodes through May 28, 2025; chronological validation June 4–July 30; evaluation August 6–September 10. Seed 42, 8 training/validation members and 32 evaluation members. This verifies execution, not model quality. Artifacts and graphs are under data/experiments/b1-smoke/; data-only graphs under data/processed/b1-audit/.
  • The B0 population file was absent locally. The short run used a separately saved data/metadata/locations.csv from the local FluSight mirror at 97e758419ae817f6a6b5e3d0a9a60dd1378c86bf, auxiliary-data/locations.csv. Its source/hash are saved next to it. It is not relabeled as B0's frozen table.
  • Removed the stochastic-trend decoder and joint_trend recipes. The active B0.1 JSON grid now has 135 configurations; historical 172-configuration result tables retain their original numbers. The joint_trend reference scored 4.104 versus .883 for the leading independent-target MLP cap-100 recipe. Joint spatial exchange is retained because it also appears in a high-ranking configuration. Removed-decoder checkpoints/scenario strings are intentionally unsupported.

Training-label audit (2026-09-17)

The B1 dataset uses Wednesday inputs and later-vintage reference labels pinned at 2026-09-16, not Wednesday preliminary labels. Those labels come from B1's Hub/Delphi archive policy, not directly from build_b_finalized.npz. Frozen scoring truth is a separate input and does not replace training labels.

An audit aligned unique future target dates (157 dates in the B1 model view), channels, and locations against B0's finalized panel. On mutually valid cells, COVID admissions, RSV admissions, COVID ED, and RSV ED agree to absolute tolerance 1e-7. Flu admissions differ in 431/7,660 shared cells (mean absolute difference 0.756 admissions); flu ED differs in 358/7,695 (mean absolute difference 0.00005754 in proportion units). B0 has respectively 468, 260, 260, 459, 255, and 723 additional valid cells across the six channels. These are unique panel cells, not fold-specific training counts; the audit assumes the recorded local NPZ files are the experiment inputs. These findings do not establish the cause of the score degradation. In particular, shared COVID labels agree despite the large COVID score degradation. An exact input-only control requires B0 labels and an explicit decision about training episode/support alignment.

Current interpretation and cleanup (2026-09-17)

The archived fromB0 attribution runs and their dedicated reports were removed at the user's request. Their earlier causal explanations are not retained as established findings. The shared trainer now selects epochs on inner validation, then refits fresh models on full permitted training partitions. The direct model still ignores finality flags; the proposed comparison tests adding these flags and a parallel recent-value reconstruction head.

The retired ladder report mixed native-unit WIS across targets for stress conditions, rather than reproducing the frozen scientific objective. Recompute stress comparisons through the shared scorer before drawing robustness conclusions. The three-candidate handoff specifies this requirement. Keep forecast ensemble-relative scores separate from nowcast persistence-relative scores, and report scored and excluded nowcast cells as separate counts.