Repository navigation
Conversation
When get_provider_models returns names that already carry a provider prefix (e.g. ollama/gemma3:1b) and the user-defined model_name prefix differs from that provider (e.g. ollama_server1/*), the code was unconditionally prepending the user prefix, producing a double prefix like ollama_server1/ollama/gemma3:1b. Before adding the user-defined prefix, strip the existing provider prefix when it does not match the user-defined one. Raw (unprefixed) model names are unaffected, and when the wildcard prefix happens to match the provider prefix the existing startswith guard still applies. Fixes BerriAI#30358
Greptile SummaryThis PR fixes a double-prefix bug in
Confidence Score: 5/5The change is narrowly scoped to wildcard model expansion and does not touch the auth critical path; the three regression tests cover the fixed scenario and boundary conditions. The fix correctly targets the double-prefix bug: it only executes when the model already failed the startswith guard, so models that already carry the right prefix are never touched. The partial-filter path was reviewed and the new stripping still produces the correct result there. No regressions were found in the changed logic. The unconditional split("/", 1)[-1] in model_checks.py (noted in a prior review thread) is worth a follow-up if providers start returning HuggingFace-style org/model-name strings, but it does not affect current behavior for the providers that use this path today.
|
| Filename | Overview |
|---|---|
| litellm/proxy/auth/model_checks.py | Adds prefix-stripping before prepending the user-defined wildcard prefix to prevent double-prefixed model names (e.g. ollama_server1/ollama/gemma3:1b). The split("/", 1)[-1] approach is unconditional and may silently drop legitimate namespaces for non-provider slashes (noted in prior thread). |
| tests/test_litellm/proxy/auth/test_model_checks.py | Adds three regression test cases covering the bug scenario, raw model names, and matching-prefix pass-through. No network calls; all dependencies are monkeypatched. |
Reviews (2): Last reviewed commit: "fix: strip existing provider prefix in g..." | Re-trigger Greptile
| # Strip any existing provider prefix (e.g. "ollama/gemma3:1b" -> "gemma3:1b") | ||
| # before prepending the user-defined wildcard prefix. | ||
| # This prevents double prefixes like "ollama_server1/ollama/gemma3:1b" | ||
| _model = model.split("/", 1)[-1] if "/" in model else model |
There was a problem hiding this comment.
Unconditional stripping of first
/-segment may drop legitimate namespace
model.split("/", 1)[-1] unconditionally discards the first slash-delimited segment, treating it as a provider prefix. For models whose names genuinely contain a / (e.g. HuggingFace-style organization/model-name returned by a provider), this will silently drop organization, producing wildcard_prefix/model-name instead of the intended wildcard_prefix/organization/model-name. Consider checking whether the first segment is a recognised LiteLLM provider name before stripping it, so the stripping only fires for true provider-prefixed strings.
| def test_wildcard_expansion_handles_prefixed_provider_models(monkeypatch): | ||
| """ | ||
| Regression test: when get_provider_models returns names with a provider prefix | ||
| (e.g. "ollama/gemma3:1b"), the wildcard expansion must NOT double-prefix them. |
There was a problem hiding this comment.
Docstring issue URL points to author's fork, not the upstream repo
The docstring says https://github.com/ylcnymn/litellm/issues/30358 (the contributor's personal fork) instead of https://github.com/BerriAI/litellm/issues/30358. Anyone reading the test later and trying to follow the link to understand the original bug report will land on the wrong repository.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Closes #30358
Root cause
get_known_models_from_wildcardreceives model names fromget_provider_modelsthat may already carry a provider prefix (e.g.ollama/gemma3:1bfromOllamaModelInfo). When the user-definedmodel_nameprefix differs from that provider (e.g.ollama_server1/*), the code unconditionally prepends the user prefix, producing a double prefix likeollama_server1/ollama/gemma3:1b.Fix
Before adding the user-defined prefix, strip the existing provider prefix when it does not match the user-defined one. Raw (unprefixed) model names are unaffected. When the wildcard prefix happens to match the provider prefix, the existing
startswithguard still applies.Test
Added a regression test with three cases:
Type
Bug Fix