Saihai v1 310M training dataset

日本語での紹介記事はこちら: 日本語 Decision モデル "saihai" (采配) と、日本語 Decision モデルベンチマークの公開

This dataset is used to train Saihai's Japanese small encoder Typed Decision Model, saihai-v1-310m, based on Ruri v3 reranker 310M. It combines a fixed selection of Bekko System One v0 training data with Japanese dataset-derived and synthetic decisions. It provides Choice, Noul and numeric Score supervision over shared state.

The larger saihai-v1-8b model is trained on a separate dataset, saihai-v1-dataset.

Loading and storage

from datasets import load_dataset

dataset = load_dataset("hotchpotch/saihai-v1-310m-dataset", "all-mixed")
train = dataset["train"]
validation = dataset["validation"]

Select the all-mixed configuration explicitly. Parquet uses Zstandard compression and dictionary encoding; decoded values and row order match the original mixed training pool. training-manifest.json defines active membership. The release contains train and validation only; no test split is included. Source attribution, revisions and license scope are retained in SOURCES.md and _sources/. The leading YAML block lists the configuration and its shards.

A case can contain several judgments. These are counts in this release, not upstream dataset sizes. Train includes 840,947 judgments selected from Bekko v0 and 131,171 Japanese judgments: 116,771 dataset-derived/template judgments plus 14,400 Japanese synthetic cases in 720 subcategories, 20 cases per subcategory. The Japanese synthetic dataset has no validation cases in this release. English GPT-6 Sol synthetic dataset is not included; synthetic sources already present in Bekko v0 are retained.

Columns

The compact schema is system_one.v1. Preserve soft targets, decision/candidate order, candidate IDs and explicit numeric Score values. Score values come from the authored criteria, not arbitrary label names. Noul retains authored false/true definitions. Targets and source metadata must not enter inference input. See schema.json for the full nested schema.

Selection and training

Bekko v0's capped train pool contains 8,421,771 judgments across 153 sources. Its per-source 300,000-judgment cap is already applied. This release selected approximately 10% using alpha=0.5 square-root source allocation and seed 42, without replacement and with complete related groups. The target budget was 842,177 judgments; the actual selection is 840,947, leaving 1,230 judgments unused because groups are indivisible. All Japanese release train cases are included. Source validation membership is retained separately.

The one globally mixed pool preserves the original row order. Train uniformly with sampling: uniform, epoch_fraction: 1.0 and seed 42; do not apply another 10% fraction or alpha weighting. Source configurations and original row indices remain in _sources/train-lineage.jsonl.gz and _sources/validation-lineage.jsonl.gz. Japanese synthetic dataset subcategories remain in target metadata, source records and original lineage.

See selection.json, release-receipt.json and the model's training/config.yaml for the selected pool and training conditions.

Sources

Japanese synthetic dataset (720 subcategories) was created by @hotchpotch and is licensed under CC BY-SA 4.0.

License

Each source follows its original dataset's license and usage terms. Check the linked original sources before using or redistributing the data. This collection does not change those licenses. The Hub other license metadata means source-specific terms, not a new umbrella license.

Where noted, code, annotations and source documents have different terms. CC BY / CC BY-SA attribution, ShareAlike conditions and MIT notices remain applicable to their respective material. Some Bekko sources restrict use to non-commercial research or have unknown/conflicting license declarations; these qualifications remain in the table and source records. An unknown license is not permission to reuse the data.

For Open-Jev customer-control, generated conversations have a CC0-1.0 declaration, but upstream question wording has no verified license and is not relicensed as CC0. Synthetic data does not automatically waive source-material or generation-service conditions. Retain the recorded synthetic dataset notices.

The trained model's Apache-2.0 license does not relicense this training dataset. Consult SOURCES.md and the detailed records for each source's scope and terms.

Split and overlap checks

Train and validation have no shared case_id, group_id or input_hash. Japanese evaluation examples and related parent groups were excluded during source preparation; original train from the same source families was allowed. The Japanese synthetic dataset's catalog is train-only by explicit selection, with no independent validation claim. Bekko inherits the pinned v0 release's targeted training removals; this package adds no claim of comprehensive semantic decontamination.

No S1MB evaluation input/target files are bundled. Training shares source families with S1MB japanese-v1, so these checks do not establish unseen-family generalization or absence of pretraining exposure. Historical JDMB references in archived Japanese provenance are the old name of S1MB japanese-v1 and are retained for traceability. Historical source counts are not this release's active membership; use the manifests, table and row lineage above.