Repository navigation
dra: reject device counts whose sum overflows int64 - #12897
Conversation
✅ Deploy Preview for kubernetes-sigs-kueue canceled.
|
|
Hi @thc1006. Thanks for your PR. I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with Tip We noticed you've done this a few times! Consider joining the org to skip this step and gain Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
📝 WalkthroughWalkthroughAdds saturating arithmetic to per-device-class device counting in ChangesDevice Count Overflow Protection
Estimated code review effort: 2 (Simple) | ~10 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
6e03862 to
e9d6f45
Compare
|
/ok-to-test |
| // Device counts are user-controlled and effectively unbounded (the | ||
| // apiserver accepts up to MaxInt64), so guard the running sum against | ||
| // int64 overflow instead of letting it wrap to a negative count. | ||
| if q > math.MaxInt64-out[dc] { |
There was a problem hiding this comment.
Wdyt about using the "Amount" structure we use for scheduling. This strucutre instead of validation error caps at int64. I'm ok with both, but maybe worth to consider using Amount more in DRA code for better resilience instead of checking individual operations, wdyt @sohankunkerkar ?
Validation is also ok, just a matter of preference, because values near 64 don't have much of practical use anyway.
There was a problem hiding this comment.
Good point. SaturatingAdd is what the scheduler and cache code already use everywhere via Amount, and countDevicesPerClass is really the outlier still doing raw int64. Aligning it with that pattern is cleaner than a one-off validation error, and I agree counts near MaxInt64 have no practical use.
I've switched the accumulation to utilmath.SaturatingAdd(out[dc], q) and updated the test to assert it saturates rather than wraps negative. Happy to go back to the validation error if you or @sohankunkerkar would rather keep an explicit signal.
There was a problem hiding this comment.
Yeah, both approaches have some merit, but I think Amount is ok and more consistent with the rest of the codebase
There was a problem hiding this comment.
Thanks for confirming. The latest revision (f0fef9b) already accumulates with SaturatingAdd, which is the same helper that Amount.AddInt64 wraps, so countDevicesPerClass now uses the scheduler saturating arithmetic while keeping the Requests int64 return type unchanged. If you would prefer this call site to use the Amount type directly, I am glad to switch it.
countDevicesPerClass summed per-request device counts into an int64 with no overflow guard, and device counts are unbounded (the apiserver accepts up to MaxInt64). A ResourceClaimTemplate whose counts sum past MaxInt64 wrapped to a negative value, which admitted a Workload that exceeds quota and left a negative used-quota in the ClusterQueue status. Accumulate with utilmath.SaturatingAdd, matching the Amount arithmetic the scheduler and cache already use, so the sum saturates at MaxInt64 instead of wrapping negative.
e9d6f45 to
f0fef9b
Compare
|
/lgtm |
|
@mimowo: once the present PR merges, I will cherry-pick it on top of DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
LGTM label has been added. DetailsGit tree hash: 3409fee07535b4daed59768ddfb2a9e6b6d59574 |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: mimowo, thc1006 The full list of commands accepted by this bot can be found here. The pull request process is described here DetailsNeeds approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
@mimowo: #12897 failed to apply on top of branch "release-0.17": DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
@mimowo: new pull request created: #12901 DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
|
/release-note-edit |
What type of PR is this?
/kind bug
What this PR does / why we need it:
countDevicesPerClasssums per-request device counts into an int64 with no overflow guard, and device counts are unbounded (the apiserver only requirescount > 0, so it accepts up to MaxInt64). A ResourceClaimTemplate whose counts sum past MaxInt64 wraps to a negative value: the quota usage goes negative, which admits a Workload that exceeds quota and leaves a negative used-quota in the ClusterQueue status.This accumulates with
utilmath.SaturatingAdd(matching theAmountarithmetic the scheduler and cache already use), so the sum saturates at MaxInt64 instead of wrapping negative. A huge count then stays a large positive that cannot fit quota, rather than being admitted on a negative. The count is turned into aresource.Quantityright after this, so the saturated value is handled by overflow-safe Quantity arithmetic downstream.A unit test covers both the normal sum and the saturation case.
Which issue(s) this PR fixes:
Fixes #12896
Special notes for your reviewer:
Switched from an explicit validation error to
SaturatingAddper review feedback, to match the resilientAmountarithmetic used across the scheduler and cache. The extended-resource path already uses Quantity arithmetic and is unaffected.Does this PR introduce a user-facing change?