[mmangkad]

Summary

Serving an MXFP8 model with FlashInfer A2A and --flashinfer-a2a-dispatch-type mxfp8 dies during init:

TypeError: FlashInfer A2A + TRT-LLM Gen FP8 MoE requires a BF16 dispatch payload, got torch.float8_e4m3fn.

#31470 made the dispatcher send an already-quantized float8_e4m3fn payload with its activation scales, and taught the runner to consume it — but the entry point's BF16-only guard still rejects it. Gate that guard on the quantization payload instead of the dtype alone.

Verified on 4x GB300:

sglang serve --model-path mmangkad/Qwen3-30B-A3B-Instruct-2507-MXFP8 \
  --tp-size 4 --dp-size 4 --enable-dp-attention --ep-size 4 \
  --moe-a2a-backend flashinfer --flashinfer-a2a-dispatch-type mxfp8

sgl-eval run gsm8k --base-url http://127.0.0.1:30000/v1 --num-threads 1319

	Server	GSM8K (1319, single-shot)
Before	dies during init	n/a
After	starts, 0 exceptions	96.06%, error_rate 0.00%, truncated 0.00%

---

CI States

Latest PR Test (Base): ❌ Run #34981974603
Latest PR Test (Extra): ❌ Run #34981974173
Latest PR Test (AMD ROCm 10): ❌ Run #34981974508

[mmangkad]

/rerun-test test/registered/backends/test_flashinfer_trtllm_gen_moe_backend.py

[github-actions]

⚠️ Rebase Required Before Re-run

A major update has landed on main. Your PR is diverged relative to required base commit ddd4600197a4.

Re-run was not dispatched. What to do:

• Rebase your branch onto the latest main and push again
• Follow issue #21065 for context
• CI-fix PRs may request the bypass-maintenance label to skip this check

[github-actions]

Results for /rerun-test test/registered/backends/test_flashinfer_trtllm_gen_moe_backend.py:

🚀 4-gpu-b200 (1 test): ✅ View workflow run

cd test/ && python3 registered/backends/test_flashinfer_trtllm_gen_moe_backend.py