Skip to content

Complete B6 evaluation inventory

All44 original recipes and38 extension recipes have five completed seeds44–48:410 two-fold fits in total. This final inventory scores299 standalone/addition/replacement ensembles. summary.csv contains equal-season admissions/log scores and recent-season ED; matched-anchor-comparisons.csv directly compares each extension to its corresponding original recipe alone and added to A+B. Negative admission deltas mean improvement. Joint broader-ensemble checks remain separate.

All three-season models learn finalized future labels. Evaluation on2025–26 trains on2022–23,2023–24,2024–25. Evaluation on2024–25 trains on2022–23,2023–24,2025–26 and is retrospective. Nested two-season arms remove only2022–23. Evaluation inputs are archived reports with finalized fallback and recipe-specific correction. Admissions relative WIS gives80% to mean state/DC and20%US, then equal seasons; ED relative comparison is supported only in2025–26. Lower is better. These repeatedly used development folds are not an untouched test set.

All six nested season-count comparisons favor the original three-season recipe on aggregate admissions. This supports keeping the oldest season for the evaluated recipes, but does not prove the effect of a fourth season. Production will use all four completed seasons as authorized.

The16 wide-source arms retrain A/B at widths192/256 with half the original learning rates, adding wastewater flu, clinical-lab positivity, FluSurv or outpatient flu one at a time. Sources enter training and evaluation histories, not prediction labels. A retains Kinsa. No wide-source arm beats its source-free anchor on standalone aggregate admissions. The small exception inside A+B is width256 B+FluSurv: admissions0.793515 versus0.793993 without FluSurv, recent ED0.830826 versus0.847090, and log admissions slightly worse0.796901 versus0.795734. It needs a joint broad-ensemble contribution check.

The ED pairs hold reporting-error augmentation and real-tree admission correction constant. A additionally applies ED correction to cross-fitted training histories and evaluation inputs; B applies ED correction only at evaluation, retaining joint reconstruction training on uncorrected histories. Trees use training-season report-to-mature pairs only; ED retains proportion units. A96 and A192 correction worsen the paired aggregate metrics; A256 correction helps its augmented counterpart. Both B96 and B256 correction help their paired counterpart. These effects are architecture-dependent, and the A experiment cannot isolate training correction from evaluation correction. Actual-report B correction helps modestly but its recent ED still trails original A+B.

System1 production fits now retrain original A+B with ten seeds per recipe on2022–23 through2025–26 and finalized future labels. This preserves a conservative primary while System2's joint comparison finishes. Ten seeds are a variance-reduction choice, not a measured pure seed-count guarantee. Existing A/B checkpoints were scenario-matched, but new pinned-code fits avoid assuming full numerical compatibility.

System2 comparison adds four extension recipes to the previously evaluated eight: A96 with ED augmentation but no ED correction; B96 with ED augmentation and evaluation correction; A256 with ED augmentation and training/evaluation correction; B256+FluSurv. It checks the assembled12-recipe ensemble, each leave-one-out and both quantile averaging/distribution mixing. None of these additions is promoted just from its isolated A+B score.