Qwen3-VL-2B, Jevified (Tier 2, decoder LoRA)

A Jevify Tier 2 model: Qwen/Qwen3-VL-2B-Instruct used as a vision-language model: state may carry images (PIL, path, URL, data URI or bytes) and the same three typed questions are asked about them, with a rank-16 LoRA confined to the decoder half of the model, trained on the readout itself — the restricted log-softmax over the allowed answers at the answer position — with the primitive's proper scoring rule, on the aokvqa train split only. The adapter (17,432,576 parameters) is merged into the backbone at load, so it serves at the plain checkpoint's speed. The calibration recipe was fitted on validation splits of the adapted model. pope, ai2d never appeared in training.

At a glance — macro accuracy 0.804, macro ECE 0.035 over POPE, A-OKVQA and AI2D (the untrained readout: 0.779 / 0.047)

jev-bench · leaderboard · findings · code

Use it

from jevify import load_jevified

model = load_jevified("Praveenrajus/jevify-qwen3-vl-2b-t2")
model.ask({"image": "https://example.com/photo.jpg", "question": "Is there a cat?"},
          {"q": {"type": "noul", "instructions": "Is the answer to `question` yes, based on `image`?"}})

Serve it as a drop-in for the TypeSafe SDK: jevify-serve --model Praveenrajus/jevify-qwen3-vl-2b-t2 then TYPESAFE_BASE_URL=http://localhost:8000.

Results on jev-bench test splits

Macro accuracy 0.804, macro ECE 0.035 over 3 sources.

Recipe: mode index, permutations {'choice': 1, 'score': 1, 'noul': 1}, temperature {'choice': 1.001, 'score': 1.0, 'noul': 1.3129}, Noul bias {'noul': 1.4}. Predictions, metrics and figures: results/qwen3vl-2b-t2-decoder/ on Praveenrajus/jev-bench. Findings and method: docs/FINDINGS.md.

How it was built

A rank-16 LoRA confined to the decoder (the language half; the vision tower is frozen) is trained on the readout itself — the restricted log-softmax over the allowed answers — with the primitive's proper scoring rule, on 2,000 A-OKVQA training records. POPE and AI2D never appear in training, and AI2D improves anyway (0.652 → 0.707). The recipe is re-fitted on validation splits of the adapted model; its Choice temperature comes out at 1.001, so the proper scoring rule has already calibrated the trained primitive.

Related models

• The untrained readout of the same backbone (Tier 0)
• Why the decoder and not the vision tower: FINDINGS 9.10

Limitations

• One seed per LoRA scope, trained on one source (A-OKVQA).
• Measured on three vision sources (POPE, A-OKVQA, AI2D).
• English only.