Sieve-Vision
Collection
Vision decision models: images and a multiple-choice question in, a calibrated probability per option out, in one forward pass. • 4 items • Updated • 1
Dataset Viewer
The dataset viewer is not available for this split.
Cannot load the dataset split (in streaming mode) to extract the first rows.
Error code: StreamingRowsError
Exception: CastError
Message: Couldn't cast
median_ms: double
mean_ms: double
p80_ms: double
image_tokens: list<item: int64>
child 0, item: int64
input_tokens: int64
device: string
threshold_ms: int64
pass: bool
note: string
public_vision_index_without_touched_rows: double
near_duplicates_blank_or_single_colour: int64
near_duplicates: int64
compared: string
public_vision_index_all_rows: double
thresholds: string
suite_rows_touched: list<item: string>
child 0, item: string
exact_matches: int64
to
{'compared': Value('string'), 'thresholds': Value('string'), 'exact_matches': Value('int64'), 'near_duplicates': Value('int64'), 'near_duplicates_blank_or_single_colour': Value('int64'), 'suite_rows_touched': List(Value('string')), 'public_vision_index_all_rows': Value('float64'), 'public_vision_index_without_touched_rows': Value('float64')}
because column names don't match
Traceback: Traceback (most recent call last):
File "/src/services/worker/src/worker/utils.py", line 149, in get_rows_or_raise
return get_rows(
dataset=dataset,
...<4 lines>...
column_names=column_names,
)
File "/src/libs/libcommon/src/libcommon/utils.py", line 272, in decorator
return func(*args, **kwargs)
File "/src/services/worker/src/worker/utils.py", line 129, in get_rows
rows_plus_one = list(itertools.islice(safe_iter(ds, dataset=dataset), rows_max_number + 1))
File "/src/services/worker/src/worker/utils.py", line 544, in safe_iter
yield from ds.decode(False) if ds.features else ds
File "/usr/local/lib/python3.14/site-packages/datasets/iterable_dataset.py", line 3052, in __iter__
for key, example in ex_iterable:
^^^^^^^^^^^
File "/usr/local/lib/python3.14/site-packages/datasets/iterable_dataset.py", line 2559, in __iter__
for key, pa_table in self._iter_arrow():
~~~~~~~~~~~~~~~~^^
File "/usr/local/lib/python3.14/site-packages/datasets/iterable_dataset.py", line 2584, in _iter_arrow
for key, pa_table in self.ex_iterable._iter_arrow():
~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
File "/usr/local/lib/python3.14/site-packages/datasets/iterable_dataset.py", line 558, in _iter_arrow
for key, pa_table in iterator:
^^^^^^^^
File "/usr/local/lib/python3.14/site-packages/datasets/iterable_dataset.py", line 440, in _iter_arrow
for key, pa_table in self.generate_tables_fn(**gen_kwags):
~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
File "/usr/local/lib/python3.14/site-packages/datasets/packaged_modules/json/json.py", line 361, in _generate_tables
self._cast_table(pa_table, json_field_paths=json_field_paths),
~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.14/site-packages/datasets/packaged_modules/json/json.py", line 132, in _cast_table
pa_table = table_cast(pa_table, self.info.features.arrow_schema)
File "/usr/local/lib/python3.14/site-packages/datasets/table.py", line 2381, in table_cast
return cast_table_to_schema(table, schema)
File "/usr/local/lib/python3.14/site-packages/datasets/table.py", line 2309, in cast_table_to_schema
raise CastError(
...<3 lines>...
)
datasets.table.CastError: Couldn't cast
median_ms: double
mean_ms: double
p80_ms: double
image_tokens: list<item: int64>
child 0, item: int64
input_tokens: int64
device: string
threshold_ms: int64
pass: bool
note: string
public_vision_index_without_touched_rows: double
near_duplicates_blank_or_single_colour: int64
near_duplicates: int64
compared: string
public_vision_index_all_rows: double
thresholds: string
suite_rows_touched: list<item: string>
child 0, item: string
exact_matches: int64
to
{'compared': Value('string'), 'thresholds': Value('string'), 'exact_matches': Value('int64'), 'near_duplicates': Value('int64'), 'near_duplicates_blank_or_single_colour': Value('int64'), 'suite_rows_touched': List(Value('string')), 'public_vision_index_all_rows': Value('float64'), 'public_vision_index_without_touched_rows': Value('float64')}
because column names don't matchNeed help to make the dataset viewer work? Make sure to review how to configure the dataset viewer, and open a discussion for direct support.
Sieve-Vision-4B: Decision Index (vision) results
The complete run of Sieve-Vision-4B on the public vision suite of the Decision Index 0.3, made
with the official kit apolinario/decision-index at f5a6e4f; the kit's own scorer produced the scores.
| Leaderboard | vision Full score 58.60, public score 65.17 (edition 0.4, board data of 2026-10-26) |
| This run | public vision index 64.95 |
| Requests | all 12,210 answered: 0 unsupported, 0 errors |
| Suite | rebuilt from the pinned sources; vision verify passes |
The vision board's Full score is half the public vision suite and half the maintainers' private vision tests of the same skills.
Files
| file | contents |
|---|---|
runs/Sieve-Vision-4B/vision-scores.json |
the kit's scores ("complete": true) |
runs/Sieve-Vision-4B/results.jsonl |
one result per request: run id, status, the chosen key with a probability for every option, timings (--compact form) |
runs/Sieve-Vision-4B/environment.json, status.json |
the runner's environment (engine, options, image budget) and final status |
runs/Sieve-Vision-4B/RUN_NOTES.md |
how the suite was built and the run made, with checksums |
reports/contamination_check.json |
our image check against the suite: no exact match; the index without the 3 rows near-duplicates touch is 64.94 |
reports/latency.json |
our latency measurement: 4 images × 1.6 MP, batch 1, one A100 |
Scoring with decision-index vision score --dir work/vision --results runs/Sieve-Vision-4B/results.jsonl reproduces
vision-scores.json exactly.
- Downloads last month
- 72
