StartLux-Decision-v1.1-4B-AutoThink-GGUF

StartLux-Decision-v1.1-4B-AutoThink as GGUF files for llama.cpp, in Q8_0 (8-bit) and Q4_K_M (4-bit). StartLux-Decision-v1.1-4B-AutoThink is a 4B decision model that thinks only where its one-pass answer is uncertain. Given a state (text or JSON) and typed questions about it (pick an option, yes or no, or a level on a scale of 2 to 10), it reads a probability for every option from the option letters in one pass; where the top probability is below 0.90 it writes a thought of at most 4,096 tokens and reads the answer again, and the default answer is 0.25 × the first read-out + 0.75 × the second. Requests and responses use the TypeSafe /v1/systemone format.

Formats: BF16 · GGUF · MLX 8-bit · MLX 4-bit · Code: StartLuxLabs/StartLux-Decision · Paper: arXiv 2610.17309

Decision Index 0.3, public subset, as of 11 October 2026: the StartLux-Decision Instant models against other decision models of each size. Figure 1 of the technical report.

Files

The DecisionPivot column is the mean chance-corrected accuracy over the seven layers of the benchmark's scorer on its 7,800 core items, from the report's measurements: the GGUF files on llama.cpp (release b11549), the BF16 weights on vLLM. For an AutoThink model the rows therefore also compare two engines. Q8_0 is the recommended file; Q4_K_M lost up to 3.8 points across the sizes.

The Q8_0 and Q4_K_M files hold the text decoder only, without the multi-token-prediction layer of the BF16 checkpoint. The vision encoder is the separate F16 file mmproj-StartLux-Decision-v1.1-4B-AutoThink-F16.gguf (672 MB), which lets llama-server read images (section Images). Plain chat with a GGUF file does not give these decisions, because the prompt format, the option-letter read-out and its temperatures are in the package, which takes the tokenizer, config.json and decision_config.json from the BF16 repository.

Run

hf download startlux-models/StartLux-Decision-v1.1-4B-AutoThink-GGUF StartLux-Decision-v1.1-4B-AutoThink-Q8_0.gguf --local-dir ~/models/StartLux-Decision-v1.1-4B-AutoThink-GGUF
hf download startlux-models/StartLux-Decision-v1.1-4B-AutoThink --exclude "*.safetensors" --local-dir ~/models/StartLux-Decision-v1.1-4B-AutoThink   # tokenizer and configs
git clone https://github.com/StartLuxLabs/StartLux-Decision && pip install -e ./StartLux-Decision
llama-server -m ~/models/StartLux-Decision-v1.1-4B-AutoThink-GGUF/StartLux-Decision-v1.1-4B-AutoThink-Q8_0.gguf -ngl 99 -c 65536 --port 8081 &
startlux-decide serve --model ~/models/StartLux-Decision-v1.1-4B-AutoThink --backend llamacpp --llamacpp-url http://127.0.0.1:8081 --port 8090

The package runs the whole decision path on llama-server: the first read-out, the gate, the thought in chunks of 1,024 tokens with the loop check between them, and the read-out after it. A request can set think_mode to never or always, and gate, think_budget and mix.

For Q4_K_M, replace Q8_0 in both places. -ngl 99 puts every layer on the GPU (CUDA or Metal); leave it out on a machine without one. Tested with llama.cpp b11549.

curl -s localhost:8090/v1/systemone -H 'Content-Type: application/json' -d '{
  "state": {"ticket": "I was charged twice for order #4411 and the app still shows it as unpaid."},
  "questions": {
    "team": {"type": "choice", "instructions": "Which team should handle this ticket?",
             "criteria": {"billing": "Payments and refunds", "shipping": "Delivery", "technical": "App and login"}},
    "urgent": {"type": "noul", "instructions": "Should this ticket be answered today?"}
  }
}'

Images

mmproj-StartLux-Decision-v1.1-4B-AutoThink-F16.gguf (672 MB) is the vision encoder of this model in one F16 file; the encoder is not quantized, so Q8_0 and Q4_K_M use the same file. Download it next to the language model and give it to llama-server with --mmproj. The decision engine (startlux-decide, --backend llamacpp) sends the images to llama-server, resizes them to multiples of 32 pixels (65,536 to 1,048,576 pixels, 64 to 1,024 tokens per image) and checks the image token count of every call.

hf download startlux-models/StartLux-Decision-v1.1-4B-AutoThink-GGUF mmproj-StartLux-Decision-v1.1-4B-AutoThink-F16.gguf --local-dir ~/models/StartLux-Decision-v1.1-4B-AutoThink-GGUF
llama-server -m ~/models/StartLux-Decision-v1.1-4B-AutoThink-GGUF/StartLux-Decision-v1.1-4B-AutoThink-Q8_0.gguf \
    --mmproj ~/models/StartLux-Decision-v1.1-4B-AutoThink-GGUF/mmproj-StartLux-Decision-v1.1-4B-AutoThink-F16.gguf \
    -ngl 99 -fa on --parallel 4 -c 81920 -b 16384 -ub 2048 --ctx-checkpoints 0 --cache-ram 0 \
    --image-min-tokens 64 --image-max-tokens 1024 --port 8081 &
startlux-decide serve --model ~/models/StartLux-Decision-v1.1-4B-AutoThink --backend llamacpp --llamacpp-url http://127.0.0.1:8081 --port 8090

Every call of a request, the read-outs and the thought, carries the images. -c 81920 with --parallel 4 gives four slots of 20,480 tokens.

License

This is a quantized version of StartLux-Decision-v1.1-4B-AutoThink. The weights are released under CC BY-NC 4.0, as the BF16 weights are: free for research and other non-commercial use, with attribution. Commercial use requires a separate license from StartLux Labs (contact@startlux.com). The inference package startlux-decide is Apache-2.0. See LICENSE and NOTICE.