airmosaic-ai

Common evaluation and the mean baseline

English 中文

The training-mean baseline is a common evaluation reference for all supported numeric regression candidates, not a separate research-model family. It predicts the training target mean for every holdout row. It never estimates that mean from validation/test outcomes.

Existing R/Python behavior

Both retain the current candidate selection rule: validation loss can select the baseline if it performs best. The shared comparison is not restricted to forests. Candidate identities, parameter keys and selection calculations are unchanged.

Configure, run and inspect

Use any implemented model example to obtain the baseline and candidate scores together. Prepare the R environment or Python task/environment, then follow plan review.

An optional baseline-only Python engineering check remains available as an evaluation fixture: task, runner parameters. It is not the main model-selection entry.

python -m runtime.cli run-tool ml_python_regression_workflow --params-file config/examples/run-tool/ml-evaluation-python-baseline.json --dry-run
python -m runtime.cli run-tool ml_python_regression_workflow --params-file config/examples/run-tool/ml-evaluation-python-baseline.json --timeout 180

Check can_execute and blockers, then actual process/artifact status. Resolve candidates, keyed predictions and metrics through artifacts.json; independently recalculate metrics on the same retained cohort and verify model reload. Preview does not train or read data rows. Keep fresh outputs within the selected runner’s existing scope. Scientific comparisons need the evaluation method, not engineering fixture settings.