StartLux-Decision-v1.1-27B-Instant-MLX-4bit

StartLux-Decision-v1.1-27B-Instant as a 4-bit MLX build for Apple silicon. StartLux-Decision-v1.1-27B-Instant is a 27B decision model. Given a state (text or JSON) and typed questions about it (pick an option, yes or no, or a level on a scale of 2 to 10), it returns a probability for every option of every question from one forward pass, without generating text. Requests and responses use the TypeSafe /v1/systemone format.

Formats: BF16 · GGUF · MLX 8-bit · MLX 4-bit · Code: StartLuxLabs/StartLux-Decision · Paper: arXiv 2610.17309

Decision Index 0.3, public subset, as of 11 October 2026: the StartLux-Decision Instant models against other decision models of each size. Figure 1 of the technical report.

This build

Converted from the BF16 weights with mlx-vlm 0.7.6 (python -m mlx_vlm convert -q --q-bits 4 --q-group-size 64). The text model has affine 4-bit weights in groups of 64; the vision tower stays in BF16 and is not quantized, so the folder reads images as well as text. 16.07 GB in total, of which about 0.9 GB is the vision tower. The multi-token-prediction layer is left out. decision_config.json (option letters, read-out temperatures) is the BF16 model's, and the tokenizer gives the same token ids.

Instant models read their prompt bidirectionally in the full-attention layers (one in four; the linear-attention layers stay causal), and config.json keeps the flag "jev_prompt_attention": "bidirectional". Plain mlx-vlm or mlx-lm ignore the flag and read every prompt causally, which makes it a different model: read that way, our 0.8B and 4B Instant models chose the same option as BF16 served by vLLM for 96.5% and 95.5% of the questions, against 99.6% and 99.9% with the bidirectional prompt. Serve this build with an engine that reads the prompt bidirectionally. The MLX backend of startlux-decide applies this mask and checks at load time that it is on; a version without it refuses the folder instead of reading it causally.

Run

git clone https://github.com/StartLuxLabs/StartLux-Decision
cd StartLux-Decision && pip install -e '.[mlx]'          # the package and mlx-vlm, on Apple silicon
hf download startlux-models/StartLux-Decision-v1.1-27B-Instant-MLX-4bit --local-dir ~/models/StartLux-Decision-v1.1-27B-Instant-MLX-4bit
startlux-decide serve --model ~/models/StartLux-Decision-v1.1-27B-Instant-MLX-4bit --backend mlx --port 8090

curl -s localhost:8090/v1/systemone -H 'Content-Type: application/json' -d '{
  "state": {"ticket": "I was charged twice for order #4411 and the app still shows it as unpaid."},
  "questions": {
    "team": {"type": "choice", "instructions": "Which team should handle this ticket?",
             "criteria": {"billing": "Payments and refunds", "shipping": "Delivery", "technical": "App and login"}},
    "urgent": {"type": "noul", "instructions": "Should this ticket be answered today?"}
  }
}'

Every question comes back with a probability for each option. The server reads text and up to 8 images per request, each resized to at most 1,048,576 pixels (images: base64 strings or data URIs, next to state and questions). A request that asks this model to think gets status 422. Loaded with plain mlx-vlm or mlx-lm the folder only generates text: the option-letter read-out and its temperatures are in the package.

Agreement with BF16

We have not compared this build with BF16, which does not fit in the 64 GB of our Mac. In an earlier study against BF16 served by vLLM on an H200, on 700 to 800 questions per build, 8-bit builds of our 0.8B, 2B and 4B Instant models chose the same option for 99.5% to 99.6% of the questions, and 4-bit builds (0.8B and 4B Instant, 4B AutoThink) for 96.3% to 97.5%.

4-bit costs accuracy. On 650 DecisionPivot items, both in MLX, the 4-bit build of the 4B AutoThink model scored 8.2 points below BF16 (59.9 against 68.1, 95% interval −15.1 to −1.4). Use the 8-bit build where memory allows.

License

This is a quantized version of StartLux-Decision-v1.1-27B-Instant. The weights are released under CC BY-NC 4.0, as the BF16 weights are: free for research and other non-commercial use, with attribution. Commercial use requires a separate license from StartLux Labs (contact@startlux.com). The inference package startlux-decide is Apache-2.0. See LICENSE and NOTICE.