Sieve-Vision-27B

Sieve-Vision-27B is a vision decision model from the Sieve-Vision collection. You give it one or more images and a multiple-choice question, and it returns a calibrated probability for every option. It reads the images and the question once, scores the options directly, and never generates text.

It is a fine-tune of Qwen/Qwen3.8-27B (revision 1d4bf0f2) with a decision head; the LoRA adapter is merged into the weights. The inference code is in this repository (sieve_vision/).

Summary

How it works

• Backbone. Qwen3.8-27B's vision encoder and text model. The vision encoder is unchanged; the model cannot produce text.
• Readout. A decision head reads the hidden states at the decision point and of each option, lets the options see each other, and scores each option. A calibrated softmax gives the probabilities.
• One pass. One forward pass per question, with the options in the order given.

Use

pip install "torch>=2.8" "transformers>=5.16" safetensors pillow huggingface_hub
pip install flash-linear-attention   # recommended: the fused kernels our runs used (0.5.2)

import sys
from huggingface_hub import snapshot_download

path = snapshot_download("sthanika-ai/Sieve-Vision-27B")
sys.path.insert(0, path)
from sieve_vision import SieveVisionEngine

engine = SieveVisionEngine(model=path)
response, _ = engine(
    {},
    {"answer": {"type": "choice", "instructions": "Is the parcel in the photo damaged?",
                "criteria": {"A": "Yes, the box is torn or crushed", "B": "No, it looks intact"}}},
    images=["parcel.jpg"])
print(response["answers"]["answer"])  # {'type': 'choice', 'choice': 'A', 'probabilities': {'A': ..., 'B': ...}}

• Loading. SieveVisionEngine(model=...) takes a Hub id (optionally repo@revision) or a local folder.

• GPU memory. About 54 GB in bf16 (53.9 GB peak in our latency run), so one 80 GB GPU is enough.

• Images. Paths, data: URLs, raw base64 or bytes. Images above 1.6 MP are downscaled (LANCZOS, aspect kept); smaller ones are used at their own size. Any number of images and options; nothing is truncated.

• Batching. engine.batch([{"state": ..., "questions": ..., "images": [...]}, ...]); batch_tokens (default 32,000) sets the padded-token budget of one batch.

• Decision Index.

  hf download sthanika-ai/Sieve-Vision-27B --local-dir Sieve-Vision-27B
  PYTHONPATH=Sieve-Vision-27B decision-index vision pipeline --dir work/vision \
      --engine sieve_vision_engine:SieveVisionEngine --option model=sthanika-ai/Sieve-Vision-27B \
      --out runs/Sieve-Vision-27B
  

• flash-linear-attention. Without it, transformers falls back to slower PyTorch kernels, whose probabilities can differ slightly. Install it to reproduce our numbers.

• Temperature. decision_config.json stores T = 1.396, fitted on our own held-out data. T never changes an answer.

Results

Decision Index

Our own run of the public vision suite of the Decision Index, with the official kit at f5a6e4f (edition 0.3) and its scorer, gives a public vision index of 73.12.

• Coverage. All 12,210 requests over 11 benchmarks were answered, with no unsupported request and no error.
• How the run was made. One complete run on one A100 80GB PCIe GPU, on a suite rebuilt from the pinned sources that passes decision-index vision verify.
• Results. sthanika-ai/Sieve-Vision-27B-decision-index-results.
• Full score. The Vision board's Full score adds the maintainers' private tests of the same skills.

The index is chance-corrected: 0 is random guessing and 100 is perfect.

Latency

• Per Decision Index request. In-process wall time, one request at a time, over 200 requests sampled across the suite (A100 80GB PCIe, bf16): median 462 ms, mean 514 ms, 80th percentile 732 ms, maximum 1,319 ms. This is not the maintainers' RTX PRO 6000 measurement.
• Worst case. Four images of 1.6 MP each (6,489 input tokens): 2,528 ms.

Training

• Recipe.
  • LoRA r=64, α=128, dropout 0.05 on the attention, MLP and linear-attention projections (12 modules) of the text model, with the decision head trained from scratch. The vision encoder, the vision-text merger and the token embeddings are frozen.
  • The head is trained first on its own, then together with the adapter.
  • AdamW, lr 5e-4 for the head on its own, then 1e-4 for the adapter and the head; cosine schedule with 3% warm-up, 128 questions per step, bf16.
  • Two A100 80GB PCIe GPUs.
  • Options shuffled and re-lettered every time a question is drawn, with a cross-entropy loss.
• Selection. The checkpoint with the best score on our own held-out data, which chose step 1,500. The benchmarks were never used for training or selection.
• Data.
  • Sources. Public datasets, converted to multiple-choice questions over images. Each source was checked against the images and question texts of the public vision suite when it was built; a final check of the whole training set against the suite was not run.
  • Not published. The training files are not part of this release; their SHA-256 is in decision_config.json.

Files

License

CC BY-NC 4.0 (non-commercial), see LICENSE. Built with Qwen: the backbone's Apache-2.0 licence is in LICENSE-Qwen.