Crivo-17M

The smallest Crivo: a 17M-parameter typed-decision encoder for CPUs, edge devices and very high request volumes.

Paper: Joint or Per-Option? Compact Encoders for Closed-Set Decisions from 17 Million to 1 Billion Parameters (arXiv:2610.30970)

Crivo-17M reads a state and a set of typed questions (choice, noul, score) and returns a probability for every option in a single encoder pass. It does not generate text. It is the smallest member of the Crivo family, below Crivo-32M: 7 layers and a hidden size of 256, light enough to sit inside every step of an agent loop on a CPU. It was trained on the data of Crivo-400M, distilled from Crivo-150M.

Crivo follows the typed-decision interface popularized by TypeSafe's Jev (a state plus typed questions in, a probability for every option out) and runs unmodified in the open Decision Index kit. It is an independent open model, not affiliated with or endorsed by TypeSafe, and not trained on Jev outputs. Crivo is Portuguese for "sieve"; passar pelo crivo, to pass through the sieve, means to be judged and sorted.

Resumo em português

Crivo-17M é o menor modelo da família Crivo: um encoder de decisões tipadas com 17M de parâmetros (Ettin/ModernBERT, 7 camadas), treinado com os dados do Crivo-400M e destilado do Crivo-150M. Ele lê um state e perguntas choice, noul ou score e devolve uma probabilidade para cada opção numa única passada, sem gerar texto. No índice público do Decision Index 0.3 marca 22,15 (run completo publicado, sem truncamento), com mediana de 7,7 ms por requisição numa A100. No Decision Index 0.4, que conta só os testes privados, o Full score é 11,11 (mesmas skills 15,88; domínios novos 3,22). Foi treinado só com dados abertos, com verificação de contaminação contra a suíte. Funciona em inglês. Licença CC BY-NC 4.0 (uso não comercial).

Results

Decision Index 0.4 — Full score (board)

Scored by the maintainers on the board in the Decision Index 0.4 batch of 2026-10-20. In edition 0.4 the Full score is 0.60 × private tests of the same skills + 0.40 × private tasks from new domains, both mapped onto the public scale using untuned stock models. The public index is still run and shown, but it no longer counts toward the Full score, and the new-domain tasks were revised. Edition 0.3.2 (0.20 public + 0.50 same skills + 0.30 new domains) stays available as an archive. Models within 0.9 points of the first model of a group share a rank group.

Rank group 47 is shared with LiquidAI d1-omni-600M (Full 11.46), vera-spark (Full 11.15) and Bosun v3.1 0.6B (Full 10.61).

Compared with models of similar size (board, Decision Index 0.4):

Crivo-17M is the smallest model on the board's Pareto frontier of size × Full score: no model with a listed parameter count is smaller, and every larger model above it is at least 1.9× its size. Prumo, a rule-based engine with no learned parameters, scores 12.21 but has no parameter count on the board, so it is not on the size chart.

Decision Index 0.3 — public index (self-run, complete)

Run with the official kit (apolinario/decision-index, commit 9eb2dbe, edition 0.3) on the complete public suite, with the engine shipped in this repository: every one of the 140,178 scoreable requests answered, none unsupported, nothing truncated. Scores are chance-corrected skill (0 = random guessing, 100 = perfect), weighted as the board does. Full results: tardellirs/decision-index-results → runs/crivo-17m.

In edition 0.4 the board shows this public index next to the model, but it no longer counts toward the Full score.

Per-benchmark skill (0–100, chance-corrected) — click to expand

★ = gold benchmark (weight 1.2 within its area). The index uses the native metric listed, chance-corrected.

Benchmark	Native metric	Crivo-17M	Crivo-32M
Knowledge & Reasoning
GPQA Diamond ★	accuracy	0.0	0.0
GSM8K	accuracy	0.9	0.3
ChessBench	accuracy	0.0	0.0
MuSR	accuracy	1.3	0.0
SATA-Bench	case exact accuracy	0.0	0.0
CRUXEval	accuracy	5.1	8.4
CLadder	accuracy	0.0	8.4
HLE ★	accuracy	0.0	0.0
MMLU-Pro ★	accuracy	0.0	1.1
BBH fixed-option tasks ★	accuracy	2.0	1.4
WinoGrande ★	accuracy	4.3	3.4
Language Understanding
ContractNLI	macro-F1	59.0	59.0
ANLI ★	macro-F1	0.0	8.4
HellaSwag ★	accuracy	7.4	11.0
ACOS	per-review F1	8.0	25.4
FinEntity	macro-F1	52.1	52.2
iSarcasmEval	sarcasm F1, track A English	0.0	0.0
VAST	macro-F1	39.7	51.0
NLI4CT	macro-F1	1.9	0.0
RAGTruth	F1 on hallucinated class	27.9	30.0
Retrieval & Classification
BANKING77 ★	macro-F1	79.3	86.7
CLINC150+OOS ★	macro-F1	79.6	86.2
BRIGHT ★	nDCG@10	13.3	17.5
Amazon ESCI	macro-F1	6.9	21.3
PhishNChips	accuracy (verdict)	16.1	0.0
HoVer	accuracy	34.9	58.5
Tools & Automation
BFCL ★	case exact accuracy	47.6	54.2
ToolRet	nDCG@10	39.5	49.4
API-Bank ★	accuracy	49.0	7.9
Home appliance simulator	case exact accuracy	3.4	27.3
When2Call	accuracy	38.1	47.2
Arts & Human Taste
BPoMP	accuracy	61.6	67.3
Humicroedit	accuracy	2.4	0.0
POP909-CL	accuracy	1.0	0.0
cfcolor	accuracy	2.6	0.0
Habermas Machine	accuracy	7.7	11.6
New Yorker caption matching	accuracy	26.9	32.3

• Strong: intent classification (BANKING77 79.3, CLINC150 79.6), long-document NLI (ContractNLI 59.0, level with Crivo-32M), entity sentiment (FinEntity 52.1), tool and API selection (API-Bank 49.0, BFCL 47.6, ToolRet 39.5) and original-vs-perturbed text (BPoMP 61.6). Against Crivo-32M it is ahead on API-Bank (49.0 vs 7.9) and PhishNChips.
• Weak: anything that needs world knowledge or reasoning (Knowledge & Reasoning 1.2), adversarial NLI and commonsense completion (ANLI 0.0, HellaSwag 7.4), smart-home control (3.4), product relevance (ESCI 6.9), claim verification (HoVer 34.9), aspect sentiment (ACOS 8.0), sarcasm and subjective "taste" benchmarks.

Quick start

Standalone (no kit)

from huggingface_hub import snapshot_download
import sys; path = snapshot_download("tardellirs/crivo-17m"); sys.path.insert(0, path)
from crivo import Crivo

model = Crivo(path)                                 # CUDA if available, else CPU
out = model.answer(
    state={"ticket": "My card was charged twice for the same order."},
    questions={
        "intent": {"type": "choice", "instructions": "What does the customer want?",
                   "criteria": {"refund": "Money back for the duplicate charge",
                                "cancel": "Cancel the order",
                                "info": "Just asking for information"}},
        "urgent": {"type": "noul", "instructions": "The customer needs a fast response."},
    })
print(out["intent"]["choice"], out["intent"]["probabilities"])   # refund {'refund': 0.85, 'cancel': 0.06, 'info': 0.09}
print(out["urgent"]["noul"])                                     # 0.10

state and instructions can be strings or JSON objects (rendered as compact JSON with sorted keys, exactly as in training).

Decision Index kit

pip install "decision-index[transformers] @ git+https://github.com/apolinario/decision-index@9eb2dbe"
hf download tardellirs/crivo-17m --local-dir crivo-17m && cd crivo-17m
python -m decision_index pipeline --engine decision_index_engine:DecisionEngine --edition 0.3 --out runs/crivo-17m

The engine defaults are the settings of the reported run: one option order (tta=1), fp32 scoring head, SDPA attention, batch_size=32, token_budget=65536, max_length=32768, bf16 autocast on CUDA. The state is tokenized once per request and all questions are batched. It raises Unsupported only for a question above max_length tokens, and there is none in the 0.3 suite.

Model details

Same design as the other Crivo models. Each question is one sequence

[CLS] instructions [SEP] state [SEP] [OPT] option_1 [OPT] option_2 … [OPT] option_n

(option_i = key: description). The hidden state at each [OPT] marker plus a question-type embedding goes through a small MLP scorer (run in fp32) to one logit per option; a softmax over the question's options gives the distribution. Options of a question see each other, and option order was randomized during training. Training sequences were up to 8,192 tokens; longer inputs are read whole with rotary positions up to 32,768.

Files: model.safetensors (fp32 weights), config.json (ModernBERT config + decision block), tokenizer files, model.py, crivo.py (standalone API), decision_index_engine.py (kit engine).

Training

Full fine-tune of cross-encoder/ettin-reranker-17m-v1 in three stages, on exactly the data of Crivo-400M, with a larger Crivo as the teacher in stages 1 and 2. Every example is one question with a target distribution over its options.

Data

No data labeled by Jev or any other proprietary decision API was used.

Distillation and the choice of teacher

The loss is

α · (CE + 0.5 · Brier) + (1 − α) · τ² · KL(teacher ‖ student), with α = 0.5,

τ = 2 for questions with more than two options and τ = 1 otherwise. Teacher logits were computed once over the stage-1 and stage-2 training rows. Sources where the teacher agrees with the gold label on fewer than 60% of their training rows, plus smart-home control, CLadder and RAGTruth, train on the gold label only. Stage 3 (long context) trains on the gold label only.

Crivo-400M is 23× the size of this model, so three students were trained in parallel, one per teacher, with the same data, stages and hyperparameters:

The panel columns are our own held-out panels (not board numbers): the core of a development set of the board's skills and a proxy built from held-out new-domain decision families. Arm C was selected: it led on the same-skill panel and on a public-suite sample, while arms A and B were slightly ahead on the new-domain panel, which is noisy at this size. With Crivo-150M as the teacher, 302 sources in stage 1 and 60 in stage 2 trained on gold only, among them the code-generated families added for Crivo-400M.

Recipe (released arm, C)

AdamW (weight decay 0.01), linear warmup (5% stage 1, 3% after) and linear decay, bf16 autocast, no gradient accumulation, option order shuffled for every choice example. The released weights are the end of stage 3. Every checkpoint choice (stage-1 final vs EMA as the stage-2 init; stage 2, stage 3 or their average; interpolations of 10% and 20% toward the base model, WiSE-FT; and the arm) was made on our own panels: held-out new-domain decision families, a development set of the board's skills, an external evaluation-only benchmark and a public-suite sample. None of them used the board.

Compute. One NVIDIA A100 40 GB (Lambda Cloud). Crivo-17M and Crivo-68M were trained and evaluated in the same run of about 7 hours: teacher logits, the 68M, the three 17M arms (in parallel on the same GPU), the panels and both complete public suites.

Contamination firewall

The training files are the ones of Crivo-400M, checked at three levels:

1. Suite test rows. Every training example was checked against the 0.3 suite: same normalized state and question, same distinctive state, or a shared passage of ≥120 characters were dropped.
2. Our own panels. Strict leaf-level filter (any string of ≥25 normalized characters) against the held-out panels used for checkpoint selection, so the selection stayed honest.
3. Final audit of the exact training files: every question against all 563,686 questions of the 0.3 suite (normalized state + instructions): 0 matches; the rule above re-applied to every row: 0 hits.

The API-Bank train split was not used: its dialogues and API catalogs repeat those of the test pool. The API-Bank skill comes only from the synthetic dialogues above, built from Hermes function-calling data and filtered against the suite. Benchmarks whose sources have no clean train split were used for evaluation only. The teachers only labeled these same training rows; they never saw suite items.

Intended use and limitations

Intended use: fast, local decisions over text or JSON state — routing, intent and topic classification, policy gates, tool selection, relevance checks — where even a 32M model is too heavy (CPU, edge, very high request volume).

• Weak on new domains. On the board's private tasks from new domains it scores 3.22, the lowest of the Crivo models. Validate on your own data before relying on it for an unfamiliar task.
• English only.
• Almost no world knowledge and no step-by-step reasoning: at or near chance on expert QA, math, commonsense and causal-inference benchmarks. Use Crivo-150M or Crivo-400M for those.
• Smart-home control is weak (3.4) even though synthetic smart-home cases were in training; do not use it for device-control decisions without testing.
• Distilled model: it inherits Crivo-150M's biases and errors; where the teacher was wrong, the student often is too.
• Long inputs beyond 8,192 tokens are read without truncation but were never seen in training.
• No temperature fitted: treat probabilities as scores to threshold on your own data.
• Not for high-stakes decisions about people without human review.

License

CC BY-NC 4.0, the same as its teacher, Crivo-150M: several training sources are licensed for non-commercial use only (for example ANLI, CC BY-NC 4.0, and some Bekko dataset subsets). Upstream dataset licenses and terms continue to apply to the data. The base model (Ettin reranker 17M) is Apache-2.0.

See also

• The Crivo family, by size: Crivo-17M · Crivo-32M · Crivo-68M · Crivo-150M (v1, rev. 7a6c745, its teacher) · Crivo-400M (Full 0.4 22.96) · Crivo-1B (Full 0.4 28.96).
• Crivo collection.

Acknowledgements

• Ettin (JHU CLSP) and the Ettin reranker 17M used as the starting point.
• Bekko System One (hotchpotch) for the open training dataset.
• Dinah-0 (Lukitaduarte) for the [OPT] readout design this model follows.
• tasksource for procedural and multi-task typed-decision data.
• Decider (Mapika) for its open teacher data.
• NousResearch, MadeAgents, Team-ACE, Glaive, Galileo, The Atticus Project, bitext and BigCode for the open datasets behind the sibling sources.
• The Decision Index maintainers and its open kit, which made a reproducible comparison possible.

Citation

@misc{stekel2026jointperoption,
  title         = {Joint or Per-Option? Compact Encoders for Closed-Set Decisions from 17 Million to 1 Billion Parameters},
  author        = {Tardelli Ronan Coelho Stekel},
  year          = {2026},
  eprint        = {2610.30970},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  url           = {https://arxiv.org/abs/2610.30970}
}

@misc{crivo17m2026,
  title        = {Crivo-17M: a small typed-decision encoder distilled from Crivo-150M},
  author       = {Tardelli Stekel},
  year         = {2026},
  howpublished = {Hugging Face model card},
  url          = {https://huggingface.co/tardellirs/crivo-17m}
}