Repository navigation
Conversation
Add MODEL_EXPRESS_URL environment variable and deployment instructions to all 4 Dynamo inference examples (agg, agg-multi-nodes, pd-disagg, pd-disagg-multi-nodes), enabling accelerated P2P model distribution via the Model Express server. Signed-off-by: 玖宇 <jiuyu.zyq@alibaba-inc.com> Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: 玖宇 <guotongyu.gty@alibaba-inc.com>
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
cheyang
left a comment
There was a problem hiding this comment.
Looks good overall. Clean documentation addition — consistent across all four Dynamo example YAMLs, well-commented, and non-breaking by design (graceful fallback when MX Server is unavailable).
What was reviewed:
- Documentation accuracy and completeness of Model Express integration instructions
- Consistency across
agg.yaml,agg-multi-nodes.yaml,pd-disagg.yaml,pd-disagg-multi-nodes.yaml - Env var placement and naming conventions
- CI status (all green)
Minor observation (non-blocking):
MODEL_EXPRESS_URL lives in commonTemplate, which means it propagates to all roles including the processor/router that doesn't download models. This is perfectly harmless (the var is simply ignored by non-downloading components), and for example YAMLs the simplicity trade-off makes sense. Just flagging for awareness.
LGTM — ready to merge.
cheyang
left a comment
There was a problem hiding this comment.
LGTM. Docs-only change, consistent structure across all example YAMLs, all CI green.
Summary
MODEL_EXPRESS_URLenvironment variable to all 4 Dynamo inference examples (agg.yaml,agg-multi-nodes.yaml,pd-disagg.yaml,pd-disagg-multi-nodes.yaml)Context
Model Express is Dynamo's accelerated model distribution service that coordinates P2P model weight transfers (GPU-to-GPU via NIXL RDMA) across nodes. When using the Dynamo Operator,
MODEL_EXPRESS_URLis automatically injected into all pods. However, RBG deployments bypass the Dynamo Operator, so users need to configure this manually in their RBG YAML files.Test plan
🤖 Generated with Claude Code