[ReubenBond]

Problem

Firestore can return native ABORTED: Transaction lock timeout when a membership commit contends with a reader even though the caller's table token is unchanged. The provider treated that transient abort as a rejected conditional write. Transactional ReadAllAsync can also abort while consuming its streamed snapshot.

Solution

Retry whole aborted membership operations up to five attempts with cancellation-aware exponential backoff. Inserts and updates replay their original atomic batch with identical create, row-existence, and table-update-time preconditions, so a concurrent canonical write still fails the CAS. Aborted reads restart a fresh transaction; the SDK's inner attempt count is one to keep the total bound at five. Final write-contention logs retain the native exception.

Regression cases inject the native lock-timeout response and check unchanged-token success, concurrent-write rejection without partial effects, bounded retries, cancellation, and fresh snapshots after streamed read aborts.

Refs #11402. Matching main fix for #11613.

Microsoft Reviewers: Open in CodeFlow

[Copilot]

🟢 Approval recommended

No active review issues are reported for the Firestore membership retry changes.

No open issues 🎉

What changed in this PR

Improves Firestore membership reliability by retrying transient native ABORTED failures without weakening conditional-write guarantees. This addresses the contention scenario tracked in #11402 and mirrors the 10.x fix in #11613.

Changes:

• Retries aborted reads and writes up to five attempts with cancellation-aware exponential backoff.
• Restarts reads in fresh transactions, preserves original write preconditions, and includes native exceptions in final contention logs.
• Adds regression tests for retry success, concurrent-write rejection, bounded attempts, cancellation, and fresh read snapshots.

File	Description
test/​Extensions/​Orleans.Clustering.Firestore.Tests/​FirestoreMembershipHeartbeatTests.cs	Adds deterministic abort-injection tests for membership reads and writes.
src/​Google/​Orleans.Clustering.Firestore/​FirestoreMembershipTable.cs	Adds bounded operation retries and improves contention logging.

_{Review effort: Balanced}

[github-actions]

Code coverage

Metric	Pull request	Current main	Variance
Lines	83.82% (120,837 / 144,158)	83.83% (120,850 / 144,155)	-0.0108 pp
Branches	73.65% (36,255 / 49,226)	73.67% (36,263 / 49,222)	-0.0222 pp

Report-only conclusion: regressed.

The current-main baseline is commit 30137dd94f and uses the same reviewed coverage matrix.

Coverage combines every CI test matrix job, including providers, CodeGen, .NET 8/10, Linux, Windows, and macOS, using canonical physical source and branch identities.

The comparison remains report-only while normal line and branch variance is calibrated.

Coverage details

• Pull request commit: 57238a68dc
• View the pull request test run and coverage artifacts
• View the trusted coverage reporting run