sysone-typed-decisions-0.4B

A typed decision model: Ettin-encoder-400m, fully fine-tuned with sysone on the train split of typed-decisions. It is a specialist (fitted on that benchmark's train split), not a zero-shot model. Probabilities are temperature-calibrated on a held-out 10% of train.

Results on test (400 cases, 2,000 decisions)

Scored with the benchmark's own formulas (bench/score_with_jobs.py). Raw predictions are in predictions.jsonl. One training run (seed 0), 59 minutes on a Colab L4; 40 ms per case (5 questions) as a single request on the L4. Answers do not depend on option order.

Use

pip install "sysone>=0.4"

from sysone import SystemOneModel

model = SystemOneModel.load("forward-metric/sysone-typed-decisions-0.4B")
model.predict(state, questions)   # questions in the typed-decisions / System One format

Training

Full fine-tuning; backbone_lr=3e-5, batch_size=32, up to 10 epochs with early stopping, seed 0. Only typed-decisions train was used for training and calibration; test was used only for the scores above.