StartLux-Decision-v1.1-9B-AutoThink-MLX-4bit

StartLux-Decision-v1.1-9B-AutoThink as a 4-bit MLX build for Apple silicon. StartLux-Decision-v1.1-9B-AutoThink is a 9B decision model that thinks only where its one-pass answer is uncertain. Given a state (text or JSON) and typed questions about it (pick an option, yes or no, or a level on a scale of 2 to 10), it reads a probability for every option from the option letters in one pass; where the top probability is below 0.90 it writes a thought of at most 4,096 tokens and reads the answer again, and the default answer is 0.25 × the first read-out + 0.75 × the second. Requests and responses use the TypeSafe /v1/systemone format.

Formats: BF16 · GGUF · MLX 8-bit · MLX 4-bit · Code: StartLuxLabs/StartLux-Decision · Paper: arXiv 2610.17309

Decision Index 0.3, public subset, as of 11 October 2026: the StartLux-Decision Instant models against other decision models of each size. Figure 1 of the technical report.

This build

Converted from the BF16 weights with mlx-vlm 0.7.6 (python -m mlx_vlm convert -q --q-bits 4 --q-group-size 64). The text model has affine 4-bit weights in groups of 64; the vision tower stays in BF16 and is not quantized, so the folder reads images as well as text. 5.96 GB in total, of which about 0.9 GB is the vision tower. The multi-token-prediction layer is left out. The text weights are byte-identical to our earlier text-only build. decision_config.json (option letters, read-out temperatures) is the BF16 model's, and the tokenizer gives the same token ids.

Run

git clone https://github.com/StartLuxLabs/StartLux-Decision
cd StartLux-Decision && pip install -e '.[mlx]'          # the package and mlx-vlm, on Apple silicon
hf download startlux-models/StartLux-Decision-v1.1-9B-AutoThink-MLX-4bit --local-dir ~/models/StartLux-Decision-v1.1-9B-AutoThink-MLX-4bit
startlux-decide serve --model ~/models/StartLux-Decision-v1.1-9B-AutoThink-MLX-4bit --backend mlx --port 8090

curl -s localhost:8090/v1/systemone -H 'Content-Type: application/json' -d '{
  "state": {"ticket": "I was charged twice for order #4411 and the app still shows it as unpaid."},
  "questions": {
    "team": {"type": "choice", "instructions": "Which team should handle this ticket?",
             "criteria": {"billing": "Payments and refunds", "shipping": "Delivery", "technical": "App and login"}},
    "urgent": {"type": "noul", "instructions": "Should this ticket be answered today?"}
  }
}'

The server thinks where the first read-out's top probability is below the gate, 0.6 by default (think_mode auto); a request can set think_mode to never or always, and gate, think_budget and mix. Loaded with plain mlx-vlm or mlx-lm the folder only generates text: the option-letter read-out, the gate and the thought loop are in the package. The server reads text and up to 8 images per request, each resized to at most 1,048,576 pixels (images: base64 strings or data URIs, next to state and questions).

Agreement with BF16

Before release we read 100 questions of Decision Index requests (92 with prompts under 1,000 tokens, 8 with 1,000 to 4,000 tokens) in one pass with this build and with the BF16 weights, both in MLX. This build gave the same answer to 97 of the 100. Per question, the largest change of an option's probability was 0.053 on average and 0.83 at most. Not all of the changed answers were close calls: in BF16 the top two options of one of them were 0.94 apart. This comparison was made with the earlier text-only build of this model, whose text weights are byte-identical to this one's.

This check covers the first read-out, which also decides whether the model thinks.

In an earlier study against BF16 served by vLLM on an H200, on 700 to 800 questions per build, 8-bit builds of our 0.8B, 2B and 4B Instant models chose the same option for 99.5% to 99.6% of the questions, and 4-bit builds (0.8B and 4B Instant, 4B AutoThink) for 96.3% to 97.5%.

4-bit costs accuracy. On 650 DecisionPivot items, both in MLX, the 4-bit build of the 4B AutoThink model scored 8.2 points below BF16 (59.9 against 68.1, 95% interval −15.1 to −1.4). Use the 8-bit build where memory allows.

Speed

On a MacBook Pro with an Apple M5 Pro (20-core GPU, 64 GB of memory) and mlx 0.32.3, one read-out of a prompt of 1,024 tokens takes 993 ms with this build (BF16: 965 ms), and one of 256 tokens 169 ms. A thought is generated at 25.6 ms per token at batch size 1 (BF16: 70.8 ms). These timings are from the earlier text-only build of this model, whose text weights are byte-identical to this one's.

License

This is a quantized version of StartLux-Decision-v1.1-9B-AutoThink. The weights are released under CC BY-NC 4.0, as the BF16 weights are: free for research and other non-commercial use, with attribution. Commercial use requires a separate license from StartLux Labs (contact@startlux.com). The inference package startlux-decide is Apache-2.0. See LICENSE and NOTICE.