Add Typed Decisions benchmark eval results
#2
by codelion - opened
Adds Typed Decisions benchmark results for this model.
Evaluated on LocalLLaMA/typed-decisions test split, 400 cases, 2,000 decisions. Results: accuracy 0.791, KL from gold 0.281, Brier 0.114, ECE 0.093.
Run on 2026-10-22 through the Decision Models hosted API using model quyet-1-large, which serves this model in FP8. One question per request, since that endpoint takes one question per call. Most rows on the leaderboard send all five questions in one request.
Scored with the benchmark's reference scorer; ECE uses 15 bins.
Feel free to close this PR if you prefer to report your own numbers.
