Sieve
Collection
Decision models that return calibrated probabilities for typed questions about text or JSON, without generating text. • 24 items • Updated • 10
Dataset Viewer
The dataset viewer is not available because its heuristics could not detect any supported data files. You can try uploading some data files, or configuring the data files location manually.
Sieve-31B-v2: Decision Index results
The complete run of Sieve-31B-v2 (a full fine-tune of Gemma-4-31B-it with bidirectional attention in its 10 global layers) on the public suite of the Decision Index 0.3, made with the official kit
apolinario/decision-index at 62d2f51 (0.3); the kit's own scorer produced the scores.
| This run | Decision Index 0.3: public index 69.97 (raw 76.95, breadth 69.44) |
| Requests | 140,178 of 140,178 scored requests ok: 0 unsupported, 0 errors, nothing truncated |
| Leaderboard | not on the Decision Index leaderboard yet; the score is self-reported |
| area | skill (×100) |
|---|---|
| Knowledge & Reasoning | 59.1 |
| Language Understanding | 71.0 |
| Retrieval & Classification | 72.8 |
| Tools & Automation | 85.9 |
| Arts & Human Taste | 60.7 |
Files (runs/Sieve-31B-v2/)
| file | contents |
|---|---|
scores.json |
the kit's scores: "complete": true, edition 0.3, index, areas, per-benchmark values |
index.json, benchmark-summary.json |
per-benchmark raw, skill and coverage; native metrics and counts |
results.jsonl.gz |
one result per request: run id, status, the model's answers with a probability for every option |
RUN_NOTES.md |
how the run was made, the published engine's parity with it, latency and checksums |
parity/ |
the RTX PRO 6000 parity and latency run: environment, status and comparison report |
code/ |
the batched evaluator, its engine classes, the attention masks, the weight average, the temperature re-read and the scoring script, as used for this run |
The results contain only request ids, answers and probabilities: no suite payloads and no raw model output.
How the run was made
- Run. On 2026-10-22/23 on 2× NVIDIA A100 80GB PCIe, with a batched evaluator that uses the engine's prompts,
readout, bidirectional global attention and refusal rules and packs many question rows into one forward pass. Probabilities were stored at
T = 1 and re-read at T = 8.5495 before scoring. Details are in
runs/Sieve-31B-v2/RUN_NOTES.md. - Engine.
sieve.decision_index_engine:SieveEnginefrom github.com/sthanika-ai/Sieve with--option model=sthanika-ai/Sieve-31B-v2runs the model one request at a time; its parity with this run is inRUN_NOTES.md. - Latency (published engine, 1× RTX PRO 6000 Blackwell Server Edition, single process, eager, 400 random public rows): median 110 ms, mean 437 ms, 80th percentile 462 ms.
- Downloads last month
- 78
