Sieve-Vision-4B

Sieve-Vision-4B is a vision decision model from the Sieve-Vision collection. You give it one or more images and a multiple-choice question, and it returns a calibrated probability for every option. It reads the images and the question once, scores the options directly, and never generates text.

It is a full fine-tune of Qwen/Qwen3.5-4B-Base (revision 1001bb4d) with a decision head. The inference code is in this repository (sieve_vision/).

Summary

How it works

• Backbone. Qwen3.5-4B-Base's vision encoder and text model. The vision encoder is unchanged; the model cannot produce text.
• Readout. A decision head reads the hidden states at the decision point and of each option, and scores each option. A calibrated softmax gives the probabilities.
• One pass. One forward pass per question, with the options in the order given.

Use

pip install "torch>=2.8" "transformers>=5.16" safetensors pillow huggingface_hub
pip install flash-linear-attention   # recommended: the fused kernels our runs used (0.5.2)

import sys
from huggingface_hub import snapshot_download

path = snapshot_download("sthanika-ai/Sieve-Vision-4B")
sys.path.insert(0, path)
from sieve_vision import SieveVisionEngine

engine = SieveVisionEngine(model=path)
response, _ = engine(
    {},
    {"answer": {"type": "choice", "instructions": "Is the parcel in the photo damaged?",
                "criteria": {"A": "Yes, the box is torn or crushed", "B": "No, it looks intact"}}},
    images=["parcel.jpg"])
print(response["answers"]["answer"])  # {'type': 'choice', 'choice': 'A', 'probabilities': {'A': ..., 'B': ...}}

• Loading. SieveVisionEngine(model=...) takes a Hub id (optionally repo@revision) or a local folder.

• GPU memory. About 9.3 GB of weights in bf16, plus activations.

• Images. Paths, data: URLs, raw base64 or bytes. Images above 1.6 MP are downscaled (LANCZOS, aspect kept); smaller ones are used at their own size. Any number of images and options; nothing is truncated.

• Batching. engine.batch([{"state": ..., "questions": ..., "images": [...]}, ...]); batch_tokens (default 32,000) sets the padded-token budget of one batch.

• Decision Index.

  hf download sthanika-ai/Sieve-Vision-4B --local-dir Sieve-Vision-4B
  PYTHONPATH=Sieve-Vision-4B decision-index vision pipeline --dir work/vision \
      --engine sieve_vision_engine:SieveVisionEngine --option model=sthanika-ai/Sieve-Vision-4B \
      --out runs/Sieve-Vision-4B
  

• flash-linear-attention. Without it, transformers falls back to slower PyTorch kernels, whose probabilities can differ slightly. Install it to reproduce our numbers.

• Temperature. decision_config.json stores T = 1.817, fitted on our own held-out data. T never changes an answer.

Results

Decision Index

Our own run of the public vision suite of the Decision Index, with the official kit at f5a6e4f (edition 0.3) and its scorer, gives a public vision index of 64.95.

• Coverage. All 12,210 requests over 11 benchmarks were answered, with no unsupported request and no error.
• How the run was made. One complete run on one A100 80GB PCIe GPU, on a suite rebuilt from the pinned sources that passes decision-index vision verify.
• Results. sthanika-ai/Sieve-Vision-4B-decision-index-results.
• Leaderboard. On the Decision Index vision leaderboard (edition 0.4, board data of 2026-10-26), Sieve-Vision-4B has a Full score of 58.60 and a public score of 65.17. The vision board's Full score is half the public vision suite and half the maintainers' private vision tests of the same skills.

The index is chance-corrected: 0 is random guessing and 100 is perfect.

Latency

• Worst case. Four images of 1.6 MP each (6,489 input tokens), one request at a time, in-process wall time (A100 80GB PCIe, bf16): median 778 ms, mean 779 ms, 80th percentile 781 ms. This is not the maintainers' RTX PRO 6000 measurement.

Training

• Recipe.
  • Full fine-tune of the 32 text decoder layers, with the decision head trained from scratch. The vision encoder, the vision-text merger and the token embeddings are frozen.
  • The head is trained first on its own, then together with the text model, then once more on its own.
  • AdamW, lr 5e-4 for the head on its own, then 5e-6 for the text model and 1e-4 for the head; cosine schedule with 3% warm-up, 128 questions per step, bf16 compute with fp32 master weights.
  • Two A100 80GB PCIe GPUs (FSDP2).
  • Options shuffled and re-lettered every time a question is drawn, with a cross-entropy loss.
• Selection. The checkpoint with the best score on our own held-out data. The benchmarks were never used for training or selection.
• Data.
  • Sources. Public datasets, converted to multiple-choice questions over images. Every training image was checked against the images of the public vision suite, and question texts against its question texts; no exact duplicate image was found.
  • Not published. The training files are not part of this release; their SHA-256 is in decision_config.json.

Files

License

Apache-2.0, see LICENSE, as the backbone. Data notices are in NOTICE.