Repository navigation
Fix intermittent "context canceled" token failures in managed identity retry() - #642
Merged
Merged
Conversation
retry() deferred every per-attempt context cancel until it returned, so the context bound to the returned resp.Body was already canceled when the caller read it. On slow identity-endpoint responses the caller's io.ReadAll raced the cancel and surfaced 'context canceled' on a successful HTTP 200. Buffer the response body into memory while the attempt context is still alive on both the success and final-attempt paths, and manage per-attempt cancels explicitly (canceling the previous attempt after its body is drained) instead of stacking defers in the loop. Fixes #634 Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 3c006218-507e-4cf9-8cdb-d12204c25b5c
Bogdan Gavril (bgavrilMS)
approved these changes
Jul 28, 2026
Gladwin Johnson VR (gladjohn)
approved these changes
Jul 28, 2026
Luís Sousa (LuisLSousa)
approved these changes
Jul 28, 2026
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.



What's going on
Fixes #634.
Managed identity token acquisitions were intermittently failing with
context canceledright after a successful HTTP 200, especially when the identity endpoint (IMDS/App Service) was responding slowly. That is exactly when the retry loop kicks in, so the bug showed up precisely in the situation the retry logic is supposed to handle.The root cause
Inside
retry()inapps/managedidentity/managedidentity.go, each attempt created a per-attempt context withdefer tryCancel()inside the loop. Becausedeferruns when the function returns (not at the end of each iteration), every stacked cancel fired the momentretry()returned, including the one tied to the response body that was being handed back to the caller.The caller then did
io.ReadAll(resp.Body)on a body whose context had already been canceled. Whether that read succeeded was a race against the transport having buffered the body. Small token JSON over a fast connection usually won the race, which is why this was intermittent and got much worse when the endpoint was slow.The fix
defercalls in the loop. The previous attempt's context is canceled only after its body has been drained, so connections can still be reused. This also clears up the classic "defer inside a loop" resource retention that was called out in the issue.Tests
TestRetrySlowBodyDoesNotCancelContext, a regression test using a realhttptestserver that returns a retryable 500 first and then a slow body. It fails on the old code withcontext canceledand passes with the fix.TestRetryPolicyDisabledSlowBodyto confirm the single-request (retry disabled) path handles the same slow body, isolating the bug toretry().TestRetryFunctionandTest_RetryPolicy_For_AcquireTokenstill pass, including the "max retries reached" case that reads the body of a retryable response.context canceledon the pre-fix code and passes after the fix.