[lehmanju]

Changes

• Crate Datasource with custom registryUrl does not respect config.json if using a sparse crate index #41452.

This PR implements proper handling of cargo registries by respecting their config.json file. Each cargo registry serves this file at the root. It contains base urls for downloading and optionally for the web api.

https://doc.rust-lang.org/cargo/reference/registry-index.html

The change implements this behavior by always consulting config.json to retrieve the correct base urls. Additionally, it removes most of the special treatment for "crates.io" since it is just a normal cargo registry.

A potentially breaking change is, that the user-defined registry "https://crates.io" does not work anymore since it has been replace with "sparse+https://index.crates.io". For most users this shouldn't be a problem, since they just use the default registry. A possible solution is to automatically map the old url to the new sparse version.

Also, the registryUrl field for a detected dependency changes as can be seen in the test files.

Context

Please select one of the following:

• This doesn't close an Issue, but I accept the risk that this PR may be closed if maintainers disagree with its opening or implementation

AI assistance disclosure

Did you use AI tools to create any part of this pull request?

Please select one option and, if yes, briefly describe how AI was used (e.g., code, tests, docs) and which tool(s) you used.

• No — I did not use AI for this contribution.
• Yes — minimal assistance (e.g., IDE autocomplete, small code completions, grammar fixes).
• Yes — substantive assistance (AI-generated non‑trivial portions of code, tests, or documentation).
• Yes — other (please describe):

Documentation (please check one with an [x])

• I have updated the documentation, or
• No documentation update is required

How I've tested my work (please select one)

I have verified these changes via:

• Code inspection only, or
• Newly added/modified unit tests, or
• No unit tests, but ran on a real repository, or
• Both unit tests + ran on a real repository

[cla-assistant]

All committers have signed the CLA.

[viceice]

nearly LGTM

@zharinov can you check the caching thing? maybe we can use one of our decorator functions to simplify?

[zharinov]

Memory cache doesn't have decorator, and I think it's not worth it anyways

[altaiezior]

@lehmanju when using a custom registry the getCrateMetadata method fails as it tries to query the metadata using /api/v1 but in registries such as of nexus and artifact-keeper the metadata API is not available, it is only part of crates.io and hence can be one of the reasons why crates.io was earlier being used here.

[lehmanju]

Yes, I know. Thats why I have merged a logging fix recently. The /api/v1 endpoint is still part of the spec but as you say not implemented for most custom implementations. It should fail silently with the latest version.

[altaiezior]

@lehmanju do you think failed to download crate metadata should also be suppressed as it would otherwise create confusion of the call being failure

Secondly can we somehow make a quick check if the repository supports metadata API, cache that for repository and don't waste our calls if it is not supported.

In metadata we usually fetch the source repository / homepage currently, so when using custom registry a user would miss these data points, do you think falling back to crates.io for metadata would be a better choice.

We can have feature controls via configurations

[lehmanju]

I don't understand, the warning suppression is already merged (#41601) and afaik there is no way of checking the implemented registry features.

[altaiezior]

I don't understand, the warning suppression is already merged (#41601) and afaik there is no way of checking the implemented registry features.

I just thought if we can save some network calls, usually when I have to debug why a dependency is not getting updated I use the debug mode and I see these failure logs before I reliased may be they don't even matter.