Coder Legion - Recent posts tagged idempotency

Why Pressing the Button Twice Shouldn't Double Your Bill: Understanding Idempotency

Sun, 13 Sep 2026 08:32:47 +0000

Have you ever clicked a "Submit Payment" button on an online shop, watched the page freeze, and felt a sudden wave of panic? You wonder: Did it go through? Should I click it again, or will I be charged twice? The technology that protects you from this exact nightmare is called idempotency.

In computer science, idempotency is a property of an action or design where repeating the action multiple times produces the exact same result as doing it just once. In other words, if an operation is idempotent, running it a hundred times has no more effect on the system's final state than running it a single time. This concept acts as a vital safety guarantee across modern software architectures, preventing accidental duplicates and keeping data consistent.

The Elevator Analogy

To understand how this works in the physical world, consider an elevator call button. When you walk up to an elevator bank and press the "Up" button, the button illuminates, sending a signal to the elevator's controller to dispatch a car to your floor. If you get impatient and press that same button ten more times, the system does not dispatch ten separate elevators, nor does it charge the building's electrical system ten times over. The final state is exactly the same: one elevator is scheduled to stop at your floor. The elevator button is an everyday example of an idempotent interface.

Why it Matters in Software Engineering

In the technology sector, engineers rely on idempotency to build resilient systems that can survive unstable network connections. When you buy something online, your web browser communicates with a payment gateway (a secure service that processes credit cards). If your internet connection briefly drops after you press "Buy," your browser might fail to receive the confirmation signal from the payment gateway. Because the browser doesn't know if the transaction succeeded, it is programmed to automatically retry the request.

Without idempotency, the payment gateway would treat each retry as a brand-new transaction, charging your credit card multiple times for a single order. To prevent this, software developers design API (Application Programming Interface, which is how systems talk to one another) endpoints to be idempotent. They do this by requiring an "idempotency key"—a unique, one-time identifier generated by the client application for that specific transaction. When the server receives a request, it checks its database to see if it has already processed a request with that specific key. If it has, it simply returns the cached response from the first successful attempt instead of running the billing logic again.

Idempotency in Code

Here is a simple example in JavaScript demonstrating how this mechanism operates in code:

// A simulated database of processed transaction IDs
const processedTransactions = new Set();

function processPayment(idempotencyKey, amount) {
  // Check if we have already handled this exact request
  if (processedTransactions.has(idempotencyKey)) {
    return {
      status: "success",
      message: "Duplicate request ignored. Returning previous transaction details.",
      amount: amount
    };
  }

  // If it is a new request, process the payment and store the key
  processedTransactions.add(idempotencyKey);
  
  // Imagine complex credit card processing logic happens here
  return {
    status: "success",
    message: `Successfully charged $${amount} for new transaction.`,
    amount: amount
  };
}

The Final Takeaway

Ultimately, idempotency is the silent architecture of reliability in a chaotic digital landscape. It shifts the burden of managing unpredictable internet dropouts away from anxious users and places it squarely on smart system design, ensuring that patience-testing screen freezes never translate into administrative or financial disasters.

Resources

GitHub Repository: react-hook-lab
react-hook-lab: npm package
Connect with me on LinkedIn: Saurav Pandey

Originally published on my blog. You can read the alternative breakdown here.

Idempotency in APIs - Why Your Retry Logic Can Break Everything (And How to Fix It)

Sat, 12 Sep 2026 13:30:46 +0000

Modern applications rely heavily on APIs. But networks are unreliable - requests fail, time out, or get retried.

Now imagine this scenario:

User clicks “Pay Now”
Request sent to server
Payment processed successfully
But response is lost due to network issue
Client retries the request

💥 Payment gets processed again. User is charged twice.

This is one of the most dangerous and common problems in distributed systems.

The solution? Idempotency.

What is Idempotency?

An operation is idempotent if performing it multiple times produces the same result as performing it once.

In simple terms:

Same request → same effect → no duplicate side effects

Example:

Create Order (without idempotency):
Request sent twice → 2 orders created ❌

Create Order (with idempotency):
Request sent twice → only 1 order created ✅

Idempotency ensures system correctness even when retries happen.

Why Do Retries Happen?

Retries are extremely common due to:

Network timeouts
Server crashes
Client retries
Load balancers retrying requests
Mobile networks instability
API gateway retries
Reverse proxy retries

Retries are normal. Duplicate side effects are not.

Which HTTP Methods Are Idempotent?

According to HTTP specification:

Method	Idempotent	Explanation
GET	✅ Yes	Fetch data
PUT	✅ Yes	Replace resource
DELETE	✅ Yes	Delete resource
POST	❌ No	Create resource
PATCH	❌ No	Partial update

POST is NOT idempotent by default.

Example:

POST /orders

Calling twice → creates 2 orders.

This must be handled explicitly.

Real-World Example: Payment API Problem

Client sends:

POST /payments
{
  "user_id": "123",
  "amount": 100
}

If client retries due to timeout:

POST /payments

Without idempotency → duplicate payment ❌

With idempotency → single payment ✅

Solution: Idempotency Key

Client sends a unique key with request:

POST /payments
Idempotency-Key: abc123

Server behavior:

If key is new → process request
If key already exists → return previous result

This guarantees single execution.

How Idempotency Works Internally

Flow:

Client generates unique key
Sends request with key
Server checks database/cache for key
If key exists → return stored response
If not exists → process request
Store key and response
Return response

Go Implementation Example

Let’s implement idempotent payment API using Go.

Database Table

idempotency_keys table:

key (primary key)
response
created_at

Go Example (Basic Version)

type IdempotencyRecord struct {
    Key      string
    Response string
}

Handler Implementation

func PaymentHandler(w http.ResponseWriter, r *http.Request) {

    key := r.Header.Get("Idempotency-Key")

    if key == "" {
        http.Error(w, "Idempotency-Key required", http.StatusBadRequest)
        return
    }

    // Check if key exists
    record, exists := getIdempotencyRecord(key)

    if exists {
        w.Header().Set("Content-Type", "application/json")
        w.Write([]byte(record.Response))
        return
    }

    // Process payment
    paymentID := processPayment()

    response := fmt.Sprintf(`{"payment_id": "%s"}`, paymentID)

    // Store key and response
    saveIdempotencyRecord(key, response)

    w.Header().Set("Content-Type", "application/json")
    w.Write([]byte(response))
}

Example Flow

First request:

POST /payments
Idempotency-Key: abc123

Response:
{
  "payment_id": "pay_001"
}

Retry request:

POST /payments
Idempotency-Key: abc123

Response returned from storage:

{
  "payment_id": "pay_001"
}

No duplicate payment created.

Production Implementation Using Redis (Recommended)

Redis is ideal because:

Extremely fast
Supports expiration (TTL)
Perfect for temporary idempotency storage

Redis Example in Go

func PaymentHandler(w http.ResponseWriter, r *http.Request) {

    key := r.Header.Get("Idempotency-Key")

    val, err := redisClient.Get(ctx, key).Result()

    if err == nil {
        w.Write([]byte(val))
        return
    }

    paymentID := processPayment()

    response := fmt.Sprintf(`{"payment_id": "%s"}`, paymentID)

    redisClient.Set(ctx, key, response, time.Hour*24)

    w.Write([]byte(response))
}

Important: Handle Race Conditions

Two requests may arrive simultaneously with same key.

Solution: Use atomic operations.

Redis example:

SET key value NX

NX = set only if not exists

Important Best Practice: Store Full Response

Do not just store key.

Store:

status code
response body
timestamp

Because retry must return exact same response.

Example Database Schema (PostgreSQL)

CREATE TABLE idempotency_keys (
    key TEXT PRIMARY KEY,
    response JSONB,
    status_code INT,
    created_at TIMESTAMP DEFAULT NOW()
);

Primary key ensures uniqueness.

Real-World Systems That Use Idempotency

Critical systems use idempotency extensively:

Payment APIs
Order creation APIs
Financial transactions
Booking systems
Wallet transfers
Stripe payments
Banking APIs

Without idempotency, financial systems would break.

Common Mistake Developers Make

❌ Using retry without idempotency

retry 3 times on failure

This multiplies side effects.

Example:

Retry logic: 3 times
Result: 3 payments

Dangerous bug.

Idempotency vs Duplicate Prevention Using Unique Constraints

Example:

UNIQUE(user_id, order_id)

This prevents duplicates at DB level.

But idempotency is better because:

Works at API level
Handles retries cleanly
Returns same response

Best approach: use BOTH.

When Should You Use Idempotency?

Use idempotency for operations that create side effects:

Payment creation
Order creation
Wallet transfer
Booking
Resource creation

Not required for:

GET requests
Read-only operations

How Client Should Generate Idempotency Key

Use UUID:

Example:

550e8400-e29b-41d4-a716-446655440000

Go example:

uuid.New().String()

Unique per operation.

Idempotency Key Expiration Strategy

Keys should expire after some time.

Common TTL:

24 hours
48 hours
7 days (depending on business need)

Redis makes expiration easy.

Example Architecture

Client → API Gateway → Service → Redis → Database

Redis handles idempotency keys
Database handles actual data

Fast and reliable.

Real Production Scenario: Mobile Network Retry

Mobile user taps "Place Order"

Network slow → timeout
Client retries automatically

Without idempotency → duplicate orders

With idempotency → safe retry

Important Difference: Safe Retry vs Unsafe Retry

Unsafe retry:

POST /orders

Safe retry:

POST /orders
Idempotency-Key: unique-key

Idempotency vs Exactly-Once Execution

Exactly-once execution is impossible in distributed systems.

Idempotency provides practical solution.

It ensures same final state.

Best Practices Checklist

Always:

Use idempotency keys for POST APIs
Store keys in Redis or database
Store full response
Use expiration
Handle race conditions
Use UUID keys
Combine with database constraints

Key Takeaway

Retries are normal.

Duplicate side effects are dangerous.

Idempotency ensures your APIs remain safe, reliable, and production-ready.

It is not optional for critical APIs - it is essential.

Final Thought

If your API handles payments, orders, bookings, or financial transactions - idempotency is mandatory.

Without it, retries can silently corrupt your system.

With it, your APIs become reliable, predictable, and safe.

The "Press-It-Twice" Problem: Why Idempotency is Your API's Best Friend

Fri, 04 Sep 2026 06:30:04 +0000

What is Idempotency?

Idempotency is a core design property of software systems where performing an operation multiple times produces the exact same result as running it once. In plain terms, it guarantees that if a command is accidentally repeated, the system behaves as if it only happened the first time. The server safely ignores any duplicate instructions while still confirming that the job was successfully done.

The Elevator Button Analogy

Imagine you are waiting in a lobby and press the button to call the elevator. The button lights up. If you get impatient and press that same button five more times, what happens? The elevator does not arrive any faster, and five different elevators do not suddenly descend to pick you up. The first press changed the state of the elevator system (it registered your call), and every subsequent press was safely ignored because the desired state had already been reached. The button is idempotent.

In contrast, think of a non-idempotent action like buying a snack from a vending machine. If you press the button for a bag of chips once, you get one bag. If you press it five times, you will be charged five times and receive five bags. In software, we want critical actions—like checking out of an online store—to behave like the elevator button, not the vending machine.

Why Idempotency Matters in Tech

In modern web development, networks are inherently unreliable. When you click "Buy Now" on a website, a request travels across the internet to a server. If the server processes your payment but your internet drops before you receive the confirmation screen, your browser doesn't know if the transaction succeeded. If you (or your browser) retry the request, a poorly designed system might charge your credit card a second time.

By implementing idempotency, engineers prevent these costly duplicate operations. When a request is sent, it includes a unique identifier called an "idempotency key" (often a random string generated by the client). The server records this key. If it sees the same key again, it simply returns the saved response from the first attempt rather than processing the transaction a second time. This is vital for payment gateways, database migrations, and background email dispatchers.

Idempotency in Action (JavaScript)

Here is a simple example of how you can implement idempotency in a payment handler using a key-value store to track processed requests:

const processedPayments = new Map();

function processPayment(idempotencyKey, amount, accountId) {
  // 1. Check if we have already processed this exact request
  if (processedPayments.has(idempotencyKey)) {
    console.log("Duplicate request detected. Returning cached result.");
    return processedPayments.get(idempotencyKey);
  }

  // 2. Perform the actual operation (mock transaction)
  console.log(`Processing fresh payment of $${amount} for account: ${accountId}`);
  const transactionResult = {
    status: "success",
    transactionId: Math.floor(Math.random() * 100000),
    amount: amount,
    processedAt: new Date().toISOString()
  };

  // 3. Store the result associated with the unique key
  processedPayments.set(idempotencyKey, transactionResult);

  return transactionResult;
}

// First attempt: Processes successfully
const key = "unique-order-xyz-123";
processPayment(key, 150.00, "user_abc");

// Second attempt (e.g. user double-clicks or network retries): Returns cached data safely
processPayment(key, 150.00, "user_abc");

The Takeaway

Building idempotent APIs is the ultimate insurance policy against the chaos of the internet. It transforms fragile, duplicate-prone web transactions into bulletproof operations by ensuring that no matter how many times a client retries, your application state remains consistent, reliable, and trustworthy.

Resources

GitHub Repository: react-hook-lab
react-hook-lab: npm package
Connect with me on LinkedIn: Saurav Pandey

Originally published on my blog. You can read the alternative breakdown here.

Idempotency: The Secret to Preventing Double Payments and Network Glitches

Thu, 03 Sep 2026 14:19:27 +0000

Idempotency is a property of an operation where executing it multiple times has the exact same effect as executing it once. It means that if a network glitch or an impatient user triggers the same action repeatedly, the system state doesn't change after the first successful attempt. In simple terms, it is a design guarantee that safety-checks repetitive requests so they don't cause duplicate side effects.

The Elevator Button Analogy

Think about an elevator button. When you walk up to the elevator lobby and press the button for floor 5, the button lights up, registering your request. If you get impatient because the elevator is taking too long and press the button five more times, nothing different happens. The elevator doesn't queue up five separate trips to the fifth floor, nor does it spin out of control. The action of pressing the button is idempotent: one press or ten presses yield the exact same result.

Conversely, a non-idempotent real-life action is buying a soda from a vending machine. If you press the button for a drink three times, you will be charged three times, and three sodas will drop out.

Why Idempotency Matters in Software Engineering

In the real world of software engineering, network connections are inherently unreliable. When you buy a pair of shoes online, your browser sends a request to the server to process your payment. If your Wi-Fi drops at that exact millisecond, your browser might not receive the confirmation message, even if the payment went through. Without idempotency, if the website automatically retries the payment—or if you manually click "Submit" again out of frustration—you would be charged twice.

Software engineers use idempotency to build robust APIs (Application Programming Interfaces) that can safely receive the exact same payment request multiple times without charging the customer's credit card more than once. It is the ultimate shield against duplicate transactions, duplicate user sign-ups, and scrambled database records.

Implementing Idempotency in Code

Here is a simple example of how engineers implement this safety net in JavaScript using an in-memory cache of unique identifiers, often called "idempotency keys":

// A simplified system to process payments safely
const processedPayments = new Set();

function chargeUser(paymentId, amount, userId) {
  // Check if we have already successfully processed this specific payment ID
  if (processedPayments.has(paymentId)) {
    return {
      status: "success",
      message: "Duplicate request ignored. Transaction was already completed."
    };
  }

  // Simulate the actual credit card charging logic
  console.log(`Charging user ${userId} the amount of $${amount}...`);
  
  // Save the unique payment ID so we never process it again
  processedPayments.add(paymentId);

  return {
    status: "success",
    message: `Successfully charged $${amount}.`
  };
}

In this code, the paymentId acts as a unique token generated by the client app. If the network drops and the client sends the exact same paymentId again, the server recognizes it instantly, bypasses the actual charging logic, and returns the original successful state without executing a double charge.

The Takeaway

Ultimately, idempotency is the cornerstone of defensive programming in distributed networks. Rather than hoping that connections never drop or that users never double-click, professional developers design their systems to expect failure and handle retries gracefully. Making your systems idempotent is the single best way to ensure data integrity and build trust with your users, transforming fragile, error-prone operations into robust, bulletproof services.

Resources

GitHub Repository: react-hook-lab
react-hook-lab: npm package
Connect with me on LinkedIn: Saurav Pandey

Originally published on my blog. You can read the alternative breakdown here.

The Golden Rule of API Design: Understanding Idempotency

Tue, 01 Sep 2026 01:11:43 +0000

Have you ever clicked a "Submit" button on a website, experienced a sudden freeze, and wondered if clicking it again would charge your credit card twice? In the world of software development, engineers prevent this exact nightmare using a core design concept called idempotency.

Idempotency is a property of an action or system design where performing an operation multiple times produces the exact same result as performing it once. In simple terms, it means that if an instruction is repeated, the system is smart enough to ensure that the outcome does not change after the first successful attempt. If you send the same command ten times, the system state behaves as if you only sent it once.

To understand this, imagine an elevator call button. When you walk up to an elevator and press the "Up" button, the button lights up, and the elevator is summoned to your floor. If you become impatient and press that same button five more times, the elevator does not speed up, nor does it summon five different elevators. The elevator system registered your initial request, and all subsequent presses are ignored because they are idempotent. The end result remains identical: one elevator arrives at your floor.

In the tech industry, idempotency is a daily necessity because computer networks are inherently unreliable. When a mobile application communicates with a server to process a $50 payment, three things must happen: the app sends the request, the server processes the payment, and the server sends back a confirmation. If the network drops after the payment is processed but before the confirmation reaches the app, the user's phone will report an error.

Without idempotency, when the app automatically retries the payment request—or when the user gets frustrated and taps the payment button again—the server would charge the user another $50. By designing the payment API to be idempotent, engineers protect users from duplicate charges. The server recognizes a unique "idempotency key" sent with the transaction, notices that this specific transaction has already been completed, and safely returns the original success message without running the payment again.

Here is a simple JavaScript code snippet demonstrating how an idempotent payment processor might look in backend software:

// A database simulation of already processed payment identifiers
const completedPaymentKeys = new Set();

function processTransaction(idempotencyKey, amount) {
  // 1. Check if we have already successfully processed this exact request
  if (completedPaymentKeys.has(idempotencyKey)) {
    return {
      status: "ignored",
      message: "This transaction was already processed. No extra charge applied."
    };
  }

  // 2. Perform the actual work if it is a new request
  console.log(`Charging the customer $${amount}...`);
  
  // 3. Save the key so we recognize it if it gets sent again
  completedPaymentKeys.add(idempotencyKey);

  return {
    status: "success",
    message: `Successfully charged $${amount}.`
  };
}

// First attempt succeeds
console.log(processTransaction("tx_98765", 50)); 

// Second attempt (retry due to network lag) is safely ignored
console.log(processTransaction("tx_98765", 50));

Ultimately, idempotency is the ultimate defensive programming practice for distributed networks where things can and will go wrong. By ensuring that your actions are safely repeatable, you eliminate the risk of duplicate data creation, incorrect billing, and system inconsistencies, resulting in a predictable and trustworthy user experience.

Resources

GitHub Repository: react-hook-lab
react-hook-lab: npm package
Connect with me on LinkedIn: Saurav Pandey

Originally published on my blog. You can read the alternative breakdown here.

Retry Is Not Re-Decide: Idempotency-by-ID Is the First Invariant for LLM Pipelines

Fri, 28 Aug 2026 04:45:50 +0000

Against an LLM endpoint there is no “again.” A retry is a re-sample, and if the call is a decision, your fault tolerance is quietly moonlighting as a sampling policy nobody chose.

Researchers at Thinking Machines Lab recently ran an experiment that every engineer with a max_retries=3 in their LLM pipeline should sit with. They sent the same prompt to Qwen3-235B one thousand times, at temperature zero, greedy decoding, the setting everyone calls “deterministic.” They got back 80 distinct completions. The outputs were identical for the first 102 tokens. At token 103, the paths split: some completions said “Queens, New York,” others said “New York City,” and from there the texts diverged for good.

Eighty different answers to one question, at the temperature that is supposed to mean “always the same answer.”

The cause is the genuinely surprising part. It is not floating-point folklore, and it is not information leaking between requests. The forward pass of a production inference server lacks what the authors call batch invariance: the numerical result of the computation depends on the batch it ran inside. In their words, “when a non-batch-invariant kernel is used as part of a larger inference system, the system can become nondeterministic.” Your request gets batched with whatever other traffic hit the server that millisecond, the kernel's reduction order shifts with batch shape, tiny numerical differences change one token, and autoregression amplifies one token into a different document. Which means the answer you got depends, in a small but real way, on who else was talking to the model at the time. Nobody reasons about that when they add a retry loop. Almost nobody knows it is true.

Now put that fact next to the humble retry, the most reflexive piece of plumbing in distributed systems. The whole mental model of a retry is: the operation failed, so run the same operation again. Against an LLM endpoint, that operation does not exist. There is no “again.” Attempt two runs in a different batch, against different concurrent load, and measurably often returns a different answer. The word “retry” is false advertising. What you built is a re-sample. And if the call you are re-sampling is a decision (a judge model scoring an eval item, a classifier routing a ticket, an extractor committing a value), then your retry loop is not fault-tolerance machinery. It is a second-opinion machine that silently overwrites first opinions, with the tiebreak decided by whichever attempt happened to complete.

Classical pipelines have their own version of duplicate trouble, and it is well-trodden territory: a recovery process requeues finished work, duplicates fan out, you pay twice and clean up the mess. We have written about that failure shape before, and the distributed-systems literature has owned it for decades, back to the impossibility results about exactly-once delivery. But notice what makes the classical case survivable: a duplicate execution of a deterministic task produces a copy. Wasteful, embarrassing, semantically harmless. Run the same video transcode twice and you get the same video. The LLM case breaks that comfort. A duplicate execution produces a disagreement. The duplicate is not a copy of your data; it is a competing claim about the truth, and unless you built something to prevent it, the loser of that competition vanishes without a trace.

The retry loop is a biased sampler

Here is the part I have not seen anyone say about eval pipelines, so take it as this essay's argument rather than a citation.

Nobody retries uniformly. You retry on failure: the timeout, the rate-limit response, the malformed JSON that would not parse. And those failure modes are not independent of content. The long, gnarly, ambiguous items are precisely the ones that blow past token limits, time out on slow decodes, or produce output mangled enough to fail parsing. Easy items complete on attempt one. Hard items complete, disproportionately, on attempt two or three, in a different batch, as a different sample from the model's answer distribution.

Follow that to its conclusion. The verdicts in your results table are not “the model's judgment on each item.” They are “the model's judgment conditioned on the attempt that happened to complete,” and the conditioning is strongest on exactly the difficult items you built the eval to probe. Your dataset of verdicts has quietly become a weighted draw from the model's opinion distribution, where the weights were assigned by infrastructure noise: server load, batch composition, whoever else was querying that second. Two runs of the “same” eval suite can disagree, not because the model changed, but because the lottery of which attempts landed came out differently. When someone then asks why this week's pass rate moved two points, you will look for a model regression, a data change, a prompt bug. The real culprit can be that your retry policy is a sampling policy, and nobody chose it.

It helps to be concrete about what the pollution looks like downstream, because “phantom duplicates” sounds abstract until you have debugged one. Two committed verdicts for one item means your aggregation query either double-counts the item, keeps whichever row it met last, or crashes on a uniqueness assumption someone made three services away. Pass rates drift depending on query order. A dashboard shows 1,043 judgments for a 1,000-item suite and nobody can say which 43 items are twins, or which of each twin's two opinions is the one a human already acted on. The cleanup is a forensic exercise in deciding, after the fact, which of two model opinions was “real.” There is no principled answer to that question. The only good time to answer it is before the second opinion exists.

That is why the fix matters more than its size suggests. It is tempting to call it a one-line check: “this ID already has a committed verdict, skip.” As a first approximation, yes. Once a decision is committed for an item, later attempts, stragglers, requeues, and duplicate messages hit the check and become no-ops. You pay for the compute once, and, more importantly, exactly one opinion per item enters the record. The retry loop goes back to being fault-tolerance instead of a hidden second-opinion generator.

But the one-line framing undersells both the pattern's pedigree and its subtlety, and the subtlety is where implementations quietly go wrong. Two corrections, both standard, both worth the paragraph.

The key is a design, and the check is a constraint

First: what the ID keys on. This pattern is not folk wisdom; it has a written standard. The IETF's httpapi working group produced a draft specifically for it, the Idempotency-Key HTTP header, which exists to “make non-idempotent HTTP methods such as POST or PATCH fault-tolerant,” was explicitly inspired by the production idempotency machinery at Stripe and PayPal (with a nod to Mark Nottingham's old “POST Once Exactly” draft), and recommends a UUID or similar random identifier as the key. The draft's most recent revision dates to late 2025; drafts lapse, but the pattern it codifies is running in every serious payment API on earth. And the sentence in it that earns its keep for our purposes is this one: “The idempotency key MUST be unique and MUST NOT be reused with another request with a different request payload.”

Read that clause against the naive implementation, which keys on item_id alone. The moment your prompt template changes, your judge model rolls to a new version, or your rubric gets edited, the request payload for that item is different, and the correct behavior is a new verdict. A bare item-ID check cannot tell the difference; it will pin the stale verdict forever and silently serve you last month's judgment against this month's rubric. The spec already told you the naive version is wrong. The key that honors it is composite: something like (item_id, prompt_version, model_id, rubric_version). Same payload, same verdict, forever; changed payload, fresh decision. The idempotency key is not a deduplication hack. It is a declaration of which changes are supposed to change the answer, which makes designing it a small act of epistemology, not plumbing.

Second: how the check executes. “Check if a verdict exists, then write one” reads correctly and is a race. Two workers, both holding a retry for the same item, both run the existence check, both see nothing, both write. Under exactly the concurrency that motivated your retries, check-then-write manufactures the duplicate decisions it was built to prevent. The real one-liner is not an if statement; it is a uniqueness guarantee in the storage layer: a UNIQUE constraint on the composite key with an INSERT ... ON CONFLICT DO NOTHING, a conditional put, a transactional upsert. Let the database refuse the second opinion atomically. The difference between the if and the constraint is invisible in the demo and decisive in production, which is the profile of most distributed-systems bugs worth knowing about.

One placement note, since the IETF draft describes a client-server header and your pipeline probably is not one. The header pattern exists because Stripe cannot trust its clients; the key travels with the request so the server can deduplicate. Inside your own eval pipeline you own both sides, which simplifies the answer: enforce at the layer where the truth lives, the verdict store, and let every producer be as dumb as it likes. Workers do not coordinate, do not check, do not care; they attempt writes, and the constraint arbitrates. Pushing the invariant to the data layer is what makes it survive the next refactor, the second worker pool, and the colleague who adds a new retry path without reading this essay.

There is also a systemic reason to get this right that goes beyond your own bill. The distributed-systems community has a name, metastable failure, for outages that sustain themselves after the trigger is gone, and a HotOS '21 study by Bronson and colleagues found retry storms to be a classic engine of them: retries amplify load, which causes failures, which cause retries. Every failure they studied came from real production cloud systems. LLM pipelines add a nasty twist, and flag this as inference rather than citation: the resource being exhausted is a metered token quota, and rate-limit responses are themselves the retryable error, so an undisciplined retry loop burns exactly the budget the pipeline needs to recover while generating more of the signal that triggers it. Idempotency keys, retry budgets, and backoff with jitter are the standard triad; the key is the piece that caps the decision damage while the other two cap the load.

What idempotency does not buy

Honesty about the limit, because it points at the deeper practice. Idempotency-by-ID commits a verdict per key. Not the right verdict. If attempt one drew an unlucky sample from those eighty possible completions, the constraint now faithfully preserves the unlucky answer forever. Idempotency buys reproducibility and cost control. It does not buy accuracy.

If accuracy on contested items is what you need, the answer is not retrying harder; it is the mirror image of a retry. Sample the model deliberately: run five judgments on purpose, with an aggregation rule you wrote down, a budget you chose, and the variance recorded in your results instead of hidden behind whichever attempt survived. A retry is unplanned resampling with the disagreement discarded. An ensemble is planned resampling with the disagreement measured. The same API calls, opposite epistemics. You can also, at the infrastructure level, buy determinism outright: the Thinking Machines team shipped batch-invariant kernels that made all one thousand completions bitwise identical, at a real performance cost, 26 seconds for standard vLLM versus 42 with their improved deterministic attention kernel. Determinism is purchasable, and for some compliance-shaped workloads it is worth the price. For everyone else, the cheap resolution of the whole essay is the one-row version: the call whose result you never re-request costs nothing extra, drifts nowhere, and disagrees with nobody.

So, the practical insight, in the order you should apply it. Give every decision in your pipeline a committed identity: a composite key naming the item and everything that legitimately changes the answer. Enforce it with an atomic constraint, not a check. Let retries hit the constraint and die as no-ops, so your fault tolerance stops moonlighting as an unlogged sampling policy. And when a decision is important enough that one draw from eighty possible answers is not good enough, do not retry your way to confidence. Decide, deliberately, how many opinions to buy, and write down how much they disagreed. The retry loop was never qualified to make that call. At token 103, it does not even know it is making one.

Sources

Thinking Machines Lab, “Defeating Nondeterminism in LLM Inference.” (1,000 temperature-0 completions from Qwen3-235B yielding 80 distinct outputs, identical for the first 102 tokens; lack of batch invariance: “when a non-batch-invariant kernel is used as part of a larger inference system, the system can become nondeterministic”; batch-invariant kernels producing 1,000 bitwise-identical completions at a performance cost: 26s standard vLLM vs 55s unoptimized / 42s improved deterministic attention.)
IETF httpapi Working Group, “The Idempotency-Key HTTP Header Field” (draft-ietf-httpapi-idempotency-key-header-07, most recent revision October 2025; draft has since lapsed). Purpose: “make non-idempotent HTTP methods such as POST or PATCH fault-tolerant.” Key clause: “The idempotency key MUST be unique and MUST NOT be reused with another request with a different request payload”; UUID recommended; acknowledged inspiration from PayPal and Stripe practice and Mark Nottingham's “POST Once Exactly.”
Bronson, N., Aghayev, A., Charapko, A., & Zhu, T. (2021). “Metastable Failures in Distributed Systems.” HotOS '21. (Self-sustaining failure states; retry storms as amplification; all failures studied drawn from production cloud systems; mitigations including backoff with jitter and retry budgets.) The token-quota feedback specific to LLM APIs is this essay's inference, not the paper's claim.
Fischer, M., Lynch, N., & Paterson, M. (1985). JACM. (The classical impossibility backdrop for exactly-once delivery: referenced, not re-argued; exactly-once processing via idempotent consumers remains achievable, which is this essay's subject.)
The biased-sampler analysis of retry-on-failure conditioning eval verdicts on completion is presented as this essay's argument.

Give every decision in your pipeline a committed identity, and let exactly one opinion per item enter the record.

That is the invariant Chain of Consciousness is built to hold. Every decision an agent commits gets a tamper-evident record keyed to what it was deciding about and the version of everything that legitimately changes the answer, so a retry, a straggler, or a duplicate message hits a committed identity instead of overwriting a first opinion with a second. One decision, one durable entry, reproducible on replay, with the disagreement measured on purpose when you choose to sample rather than discarded by whichever attempt happened to land.

See Hosted Chain of Consciousness · Read the Theory of Agent Trust

pip install chain-of-consciousness · npm install chain-of-consciousness

Or the whole trust stack at once: pip install agent-trust-stack / npm install agent-trust-stack

Originally published at vibeagentmaking.com.

Understanding Idempotency in Software Engineering

Mon, 24 Aug 2026 12:14:11 +0000

Idempotency

In software engineering, an operation is considered idempotent if applying it multiple times has the same result as applying it once.

For example, an API endpoint PUT /user/1 that updates a user's name is idempotent because making the same request twice leaves the system in the same state. A POST /user endpoint that creates a new user each time is not idempotent.

This is a test post from the react-hook-lab publishing workflow!