Skip to content

model: support EmbeddingGemma - #32375

Merged
mickqian merged 5 commits into
mainfrom
agent/support-embeddinggemma-bcg
Jul 27, 2026
Merged

mickqian merged 5 commits into
mainfrom
agent/support-embeddinggemma-bcg

Conversation

@mickqian

@mickqian mickqian commented Jul 25, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • add native EmbeddingGemma dispatch and normalized mean pooling
  • honor its bidirectional attention configuration and run CUDA prefill through Breakable CUDA Graph
  • disable incompatible radix-prefix reuse and chunked prefill, with unit coverage for dispatch, BCG configuration, and mean pooling

Validation

  • git diff --check
  • ruff check --ignore F722,F841 on the changed Python and test files
  • commit hooks, including AST, import, Ruff, Black, codespell, and registered-test CI registry checks

pytest collection could not run in the current local environment because its Python dependencies are incomplete/incompatible (msgspec is missing and the global Hugging Face Hub package is incompatible with this checkout).


CI States

Latest PR Test (Base): 🚫 Run #30183386286
Latest PR Test (Extra): ✅ Run #30231131067

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@github-actions github-actions Bot added the Multi-modal multi-modal language model label Jul 25, 2026
@mickqian
mickqian force-pushed the agent/support-embeddinggemma-bcg branch 2 times, most recently from af2093d to 27108bb Compare July 25, 2026 03:07
@mickqian
mickqian force-pushed the agent/support-embeddinggemma-bcg branch from 27108bb to e4252b5 Compare July 25, 2026 03:11
@mickqian
mickqian marked this pull request as ready for review July 25, 2026 05:35
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@mickqian

Copy link
Copy Markdown
Collaborator Author

/tag-and-rerun-ci

@github-actions github-actions Bot added the run-ci CI: run the baseline test suite on this PR label Jul 25, 2026
@mickqian mickqian changed the title Support EmbeddingGemma with breakable CUDA graph model: support EmbeddingGemma Jul 25, 2026
mickqian and others added 4 commits July 25, 2026 14:33
cuda_graph_config.prefill.max_bs is Optional[int] and defaults to None
when the phase config is built without an explicit value, so the
`max_bs <= 0` check raised `TypeError: '<=' not supported between
instances of 'NoneType' and 'int'`. Treat None as unset (same as <= 0)
using the existing `max_bs or 0` idiom.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@mickqian
mickqian merged commit abb8f4b into main Jul 27, 2026
366 of 405 checks passed
@mickqian
mickqian deleted the agent/support-embeddinggemma-bcg branch July 27, 2026 02:40
Atituiset pushed a commit to Atituiset/sglang that referenced this pull request Sep 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

high priority Multi-modal multi-modal language model run-ci CI: run the baseline test suite on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants