Skip to content

docs: add Model Express integration to Dynamo SGLang examples - #356

Merged
cheyang merged 1 commit into
sgl-project:mainfrom
Syspretor:feat/add-model-express-to-dynamo-examples
Jun 10, 2026
Merged

cheyang merged 1 commit into
sgl-project:mainfrom
Syspretor:feat/add-model-express-to-dynamo-examples

Conversation

@Syspretor

Copy link
Copy Markdown
Collaborator

Summary

  • Add MODEL_EXPRESS_URL environment variable to all 4 Dynamo inference examples (agg.yaml, agg-multi-nodes.yaml, pd-disagg.yaml, pd-disagg-multi-nodes.yaml)
  • Add Model Express server deployment instructions (Helm chart and manual) to the prerequisites section of each example
  • Add inline comments explaining Model Express functionality, fallback behavior, and project link

Context

Model Express is Dynamo's accelerated model distribution service that coordinates P2P model weight transfers (GPU-to-GPU via NIXL RDMA) across nodes. When using the Dynamo Operator, MODEL_EXPRESS_URL is automatically injected into all pods. However, RBG deployments bypass the Dynamo Operator, so users need to configure this manually in their RBG YAML files.

Test plan

  • Verify YAML syntax is valid for all 4 modified example files
  • Confirm examples deploy correctly with and without Model Express server

🤖 Generated with Claude Code

Add MODEL_EXPRESS_URL environment variable and deployment instructions
to all 4 Dynamo inference examples (agg, agg-multi-nodes, pd-disagg,
pd-disagg-multi-nodes), enabling accelerated P2P model distribution
via the Model Express server.

Signed-off-by: 玖宇 <jiuyu.zyq@alibaba-inc.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: 玖宇 <guotongyu.gty@alibaba-inc.com>
@gemini-code-assist

Copy link
Copy Markdown

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

@cheyang cheyang left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good overall. Clean documentation addition — consistent across all four Dynamo example YAMLs, well-commented, and non-breaking by design (graceful fallback when MX Server is unavailable).

What was reviewed:

  • Documentation accuracy and completeness of Model Express integration instructions
  • Consistency across agg.yaml, agg-multi-nodes.yaml, pd-disagg.yaml, pd-disagg-multi-nodes.yaml
  • Env var placement and naming conventions
  • CI status (all green)

Minor observation (non-blocking):
MODEL_EXPRESS_URL lives in commonTemplate, which means it propagates to all roles including the processor/router that doesn't download models. This is perfectly harmless (the var is simply ignored by non-downloading components), and for example YAMLs the simplicity trade-off makes sense. Just flagging for awareness.

LGTM — ready to merge.

@cheyang cheyang left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Docs-only change, consistent structure across all example YAMLs, all CI green.

@cheyang
cheyang merged commit 04dc9a8 into sgl-project:main Jun 10, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants