Coder Legion - Recent posts tagged serverless

Organization-Wide EC2 DNS Automation with EventBridge & AWS Config Aggregator: One Rule Per Account

Tue, 27 Oct 2026 19:19:28 +0000

🌐 Organization-Wide EC2 DNS Automation with EventBridge and AWS Config Aggregator: One Rule Per Account

TL;DR

Detecting EC2 launches and terminations across a whole AWS Organization for DNS is not the five-minute single-account job it looks like.

The EC2 state-change event carries only an instance ID and a state, no IP, so it is useless for DNS on its own.

A hub-and-spoke design keeps the per-account footprint to a single EventBridge rule; all logic lives in one hub account.

The hub enriches each event by querying the AWS Config Aggregator you already run, which holds every instance's IP, tags, and state org-wide. No new data pipeline.

An aws:PrincipalOrgID resource policy on the hub's custom bus gives cross-account fan-in without per-account config.

SQS between the rule and the Lambda absorbs bulk launches and gives you retries plus a dead-letter queue.

How to detect every EC2 launch and termination across an entire AWS Organization, and turn it into automated DNS registration, without deploying repeated infrastructure in every account.

I spent longer than I would like to admit staring at this problem before the answer got simple. So if you are wiring up DNS across an AWS Organization right now, maybe this saves you a weekend.

Here is the short version of what I landed on. I needed near real-time DNS registration for every EC2 instance across a multi-account, multi-region org, feeding an on-prem enterprise DDI system (think Infoblox). The two obvious ways to do it both annoyed me: one puts infrastructure in every account, the other leans on a fragile cross-account API-scraping job.

What I used instead is a hub-and-spoke event setup:

Each spoke account gets exactly one EventBridge rule. That is the whole footprint.
One hub account does everything else: a custom event bus, an SQS queue, one Lambda.
The Lambda fills in the missing details by querying the AWS Config Aggregator, which, and this is the part I almost missed, already has every instance's IP, tags, and state for the entire org. No new data pipeline required.

Onboarding a new account came down to a one-line deploy. Below is the whole thing: the architecture, why I made the calls I made, and the code.

The problem (why the single-account version does not scale)

In one account this is a five-minute job. EC2 fires a state-change event, you point an EventBridge rule at a Lambda, done. I have written that Lambda a dozen times.

Across an organization it falls apart for a few reasons:

Instances come up in dozens of accounts and several regions.
The people who run DNS are not the people who own the workload accounts, so "just add it to the account" is not really on the table.
The EC2 state-change event does not include the IP address. You get an instance ID and a state. That is it. Useless for DNS on its own.
Nobody wants to deploy, version, and babysit a Lambda plus queue plus IAM role in every account just to catch these events.

The usual reason you end up here is a move to an enterprise DDI platform. Every instance that boots needs a DNS record, every instance that dies needs its record cleaned up, org-wide, in near real-time, with as little junk in each account as you can manage.

So two things really drove the design:

Adding an account should be almost free.
The IP is not in the event, so I have to look it up, and that lookup cannot be flaky.

The approaches I threw out

Option A: put the whole stack in every account. Rule, queue, Lambda, IAM role, shipped everywhere with StackSets. It works. But now you own N copies of the same code, and every bug fix or IAM tweak is an N-account rollout. I have maintained fleets like that. No thanks.

Option B: a central poller that assumes a role into the source account and calls DescribeInstances for the IP. Logic is centralized, which is nice, but you need an assume-role trust in every account and you are paying a cross-account API hop on every single event. More IAM surface, more things to break.

Option C: hub-and-spoke, enriched from the Config Aggregator. This is the one. All the logic sits in the hub. Spokes just forward. And instead of building anything to fetch the IP, the hub asks the Config Aggregator, which is already sitting there aggregating EC2 config (IP, tags, state) from every account and region.

Option C won for a boring but decisive reason: the hard part, the enrichment data, already existed. I did not have to build it. I just had to notice it was there.

How it fits together

Spokes forward EC2 state changes to the hub's custom bus; the hub enriches via the org Config Aggregator, then calls the DDI system. Spokes are dumb, the hub is smart, and the data already exists. That is the whole idea in one line.

That second hub rule is easy to forget, by the way. If you only listen on the custom bus, the hub account's own instances never get DNS records, because they never leave home to reach the custom bus. Ask me how I know.

Filling in the blanks

Most single-account tutorials skip this bit, which is exactly the bit that matters at scale. Here is what the event actually gives you:

{
  "source": "aws.ec2",
  "detail-type": "EC2 Instance State-change Notification",
  "detail": {
    "instance-id": "i-1234567890",
    "state": "running"
  }
}

An ID and a state. No IP, no tags, no hostname. So the Lambda goes and gets the rest, from the aggregator, not from the source account.

The nice thing: the aggregator is org-level, so one query resolves an instance no matter which account or region it lives in. No assume-role, no per-account permissions to manage.

The code

Everything here is vendor-neutral and anonymized. Swap the Infoblox call for whatever you actually talk to: Route 53, a CMDB, an IPAM API.

The spoke's one and only rule (Terraform)

This is the entire per-account footprint. Ship it with StackSets or a Terraform module.

# --- Deployed in EVERY spoke account/region ---

resource "aws_cloudwatch_event_rule" "forward_ec2_state" {
  name           = "forward-ec2-state-to-hub"
  description    = "Forward EC2 running/terminated events to the hub account"
  event_bus_name = "default"

  event_pattern = jsonencode({
    source        = ["aws.ec2"]
    "detail-type" = ["EC2 Instance State-change Notification"]
    detail = {
      state = ["running", "terminated"]
    }
  })
}

resource "aws_cloudwatch_event_target" "to_hub_bus" {
  rule           = aws_cloudwatch_event_rule.forward_ec2_state.name
  event_bus_name = "default"
  arn            = "arn:aws:events:${var.region}:${var.hub_account_id}:event-bus/ec2-dns-hub-bus"
  role_arn       = aws_iam_role.eventbridge_cross_account.arn
}

# Minimal role that lets EventBridge PutEvents to the hub bus
resource "aws_iam_role" "eventbridge_cross_account" {
  name = "eventbridge-forward-to-hub"
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect    = "Allow"
      Principal = { Service = "events.amazonaws.com" }
      Action    = "sts:AssumeRole"
    }]
  })
}

resource "aws_iam_role_policy" "put_events" {
  role = aws_iam_role.eventbridge_cross_account.id
  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Effect   = "Allow"
      Action   = "events:PutEvents"
      Resource = "arn:aws:events:${var.region}:${var.hub_account_id}:event-bus/ec2-dns-hub-bus"
    }]
  })
}

The hub's custom bus, with an org-scoped resource policy

This policy is the thing that makes "one rule per spoke" actually work: any account in the org can publish, and the hub needs zero per-account config.

# --- Deployed ONCE in the hub account ---

resource "aws_cloudwatch_event_bus" "hub" {
  name = "ec2-dns-hub-bus"
}

resource "aws_cloudwatch_event_bus_policy" "allow_org" {
  event_bus_name = aws_cloudwatch_event_bus.hub.name
  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Sid       = "AllowOrgAccountsToPutEvents"
      Effect    = "Allow"
      Principal = "*"
      Action    = "events:PutEvents"
      Resource  = aws_cloudwatch_event_bus.hub.arn
      Condition = {
        StringEquals = {
          "aws:PrincipalOrgID" = var.aws_org_id
        }
      }
    }]
  })
}

Do not skip the aws:PrincipalOrgID condition. It scopes that Principal = "*" down to your organization instead of, you know, the entire internet. It is the one line standing between "elegant pattern" and "open event bus."

Rules, queue, and Lambda wiring (hub)

resource "aws_sqs_queue" "events" {
  name                       = "ec2-dns-events"
  visibility_timeout_seconds = 120
  redrive_policy = jsonencode({
    deadLetterTargetArn = aws_sqs_queue.dlq.arn
    maxReceiveCount     = 5
  })
}

resource "aws_sqs_queue" "dlq" {
  name = "ec2-dns-events-dlq"
}

# Rule on the custom bus (events from spokes)
resource "aws_cloudwatch_event_rule" "hub_custom" {
  name           = "ec2-state-custom-bus"
  event_bus_name = aws_cloudwatch_event_bus.hub.name
  event_pattern = jsonencode({
    source        = ["aws.ec2"]
    "detail-type" = ["EC2 Instance State-change Notification"]
    detail        = { state = ["running", "terminated"] }
  })
}

resource "aws_cloudwatch_event_target" "custom_to_sqs" {
  rule           = aws_cloudwatch_event_rule.hub_custom.name
  event_bus_name = aws_cloudwatch_event_bus.hub.name
  arn            = aws_sqs_queue.events.arn
}

# Second rule on the default bus catches the hub account's OWN instances
resource "aws_cloudwatch_event_rule" "hub_default" {
  name           = "ec2-state-default-bus"
  event_bus_name = "default"
  event_pattern = jsonencode({
    source        = ["aws.ec2"]
    "detail-type" = ["EC2 Instance State-change Notification"]
    detail        = { state = ["running", "terminated"] }
  })
}

resource "aws_cloudwatch_event_target" "default_to_sqs" {
  rule = aws_cloudwatch_event_rule.hub_default.name
  arn  = aws_sqs_queue.events.arn
}

resource "aws_lambda_event_source_mapping" "sqs_to_lambda" {
  event_source_arn = aws_sqs_queue.events.arn
  function_name    = aws_lambda_function.dns_registrar.arn
  batch_size       = 10
}

The enrichment Lambda (Python)

This is the heart of it: resolve the instance against the Config Aggregator, then act on the state.

import json
import boto3

config = boto3.client("config")

AGGREGATOR_NAME = "OrganizationConfigAggregator"

def get_instance_details(instance_id: str) -> dict | None:
    """Resolve IP, tags, account, and region from the org Config Aggregator."""
    expression = (
        "SELECT resourceId, accountId, awsRegion, "
        "configuration.privateIpAddress, "
        "configuration.publicIpAddress, tags "
        f"WHERE resourceId = '{instance_id}' "
        "AND resourceType = 'AWS::EC2::Instance'"
    )
    resp = config.select_aggregate_resource_config(
        Expression=expression,
        ConfigurationAggregatorName=AGGREGATOR_NAME,
        Limit=1,
    )
    results = resp.get("Results", [])
    return json.loads(results[0]) if results else None


def handler(event, _context):
    for record in event["Records"]:
        body = json.loads(record["body"])
        detail = body["detail"]
        instance_id = detail["instance-id"]
        state = detail["state"]

        details = get_instance_details(instance_id)
        if not details:
            # Terminated instances may already be gone from the aggregator;
            # fall back to event data for deregistration.
            if state == "terminated":
                deregister_dns(instance_id)
            continue

        ip = (
            details.get("configuration", {}).get("privateIpAddress")
            or details.get("configuration", {}).get("publicIpAddress")
        )

        if state == "running" and ip:
            register_dns(instance_id, ip, details.get("tags", []))
        elif state == "terminated":
            deregister_dns(instance_id)


def register_dns(instance_id, ip, tags):
    # Replace with your Infoblox / Route 53 / IPAM call.
    # Infoblox example: POST to the WAPI record:a endpoint.
    print(f"REGISTER {instance_id} -> {ip}")


def deregister_dns(instance_id):
    print(f"DEREGISTER {instance_id}")

One thing worth calling out: that SELECT against the aggregator resolves a resource across every account and region in a single call. That is the capability that collapses the whole cross-account enrichment headache into a one-liner. I keep coming back to it for other problems now.

And notice the fallback in the handler. A terminated instance has often already dropped out of the aggregator by the time you look, so you cannot rely on the lookup for deregistration. You fall back to the instance ID from the event. Small thing, but it will bite you in testing if you miss it.

Why there is an SQS queue in the middle

You can target the Lambda straight from the rule, and for a demo I would. In production I put SQS in between on purpose:

A bulk launch, a scale-out, a StackSet rollout, can throw hundreds of events at you at once. SQS soaks that up and lets the Lambda drain at its own pace instead of getting hammered.
When Infoblox or the aggregator has a bad minute, messages retry and eventually land in a DLQ you can actually look at. Nothing quietly disappears.
Batching. Ten events per invocation, fewer cold starts, lower cost.

Decoupling ingest from processing is most of what separates "worked in the demo" from "still working at 2am during a mass launch."

What actually came out of it

Detection and DNS updates happened within seconds, across the whole org.
Adding an account or region was a single forwarding rule. No Lambda, no queue, no Config setup per account.
Registration and cleanup were fully automated across the instance lifecycle, nobody touching DNS by hand.
Cost tracked instance activity, not the clock, because there is no idle compute.
The shape is reusable. Same hub-spoke plus aggregator-enrichment works for CMDB sync, security tagging, IPAM updates, anything org-wide and event-driven.

The part worth stealing

Take Infoblox out of the picture and you are left with a pattern I now reach for often:

Fan org-wide events into one hub, then enrich them from a data source the organization already has, instead of building a new one.

Two moves do the work:

A custom event bus with an aws:PrincipalOrgID resource policy gives you cross-account fan-in and a one-rule spoke footprint.
The Config Aggregator, used as an enrichment lookup, resolves any resource in any account or region without an assume-role fan-out.

So the next time you are about to copy the same Lambda into 40 accounts, stop and ask whether a fan-in plus an aggregator you already run could do it with one rule per account instead. Usually it can.

Wrapping up

Honestly, none of the individual pieces here are clever. Cross-account EventBridge is documented. Config aggregators are documented. The win was in composing them so the per-account cost dropped to near zero and the enrichment data was something we already had lying around. That is where most of the good architecture calls seem to live, not in the shiny new component, but in the one you talked yourself out of building.

If you end up adapting the enrichment step to Route 53, a CMDB, or your own IPAM, I am happy to talk through it in the comments.

Deferring Publication After a Budget-Safe Draft Revision

Fri, 16 Oct 2026 08:38:28 +0000

A Cloudflare Workflow can spend most of its bounded external request budget before publication begins, risking partial failures when generation and publishing share a single run.

Quick solution:

if (!hasPublicationBudget) {
  await commitDraftOnly(revisionData);
  return { promotionDeferred: true, readyPath: revisionData.path };
}

Reserving candidate commit, promotion preflight, live checks, site writes, and vault finalization before model calls prevents generation from creating a draft that the same run cannot safely publish. When an explicit revision-only run cannot reserve the full tail, retain only the five-request draft commit reserve.

Understanding the Request Budget Conflict

Cloudflare documents strict subrequest ceilings on its free plan, while GitHub documents base tree writes for atomic tree changes via REST API endpoints for Git trees as outlined in the GitHub REST API Git trees documentation. When a workflow handles candidate commits, promotion preflights, and final vault writes, it can exhaust its budget before publishing.

For more details on platform limits, consult the Cloudflare Workflows limits documentation. The five-request commit budget and the later publish-only step are application safeguards rather than limits specified by either provider. For related architectural guidance, you can also review Managing Concurrent Git Commits During Automated Publishing.

Implementing a Deferred Publication Fallback

When a revision-only repair occurs, you have a different recovery option available. Instead of failing the entire run, commit the reviewed revision and promote that exact ready draft in a later run.

Keep the full publication reserve unchanged for normal candidates and same-run promotion.
When an explicit revision-only run cannot reserve the full tail, retain only the five-request draft commit reserve.
Generate and review the revision, commit its Markdown, quality record, and raster metadata, then return the resulting ready-draft path with an explicit deferred publication result.
Skip publication selection in that run. After it completes, use a separate publish-only run scoped to the exact ready-draft path.
Keep manual live runs serial so a second workflow cannot race the vault SHA guard.

Verification and Execution Checklist

Verify the five-request reserve constraint during low-budget execution.
Confirm generation and commit succeed after the full publication reserve fails.
Validate that the exact-path deferred result is correctly returned.
Ensure manual live runs remain serial to prevent race conditions on the vault SHA guard.

Leveraging AWS CodePipeline pipeline and Serverless Framework to deploy a Flask API.

Wed, 14 Oct 2026 18:02:28 +0000

For the past couple of days. I have been working with the Serverless framework. I came to know that you can actually host your flask API that integrates several AWS services such as Lambda, API Gateway, and DynamoDB using this framework. I had to spend a lot of time to understand the process at first since there is not a lot of content regarding this implementation. So, here’s me contributing a little.

To make this swift, I wanted to integrate the entire process with a pipeline that contains the code and a build environment. Here’s how everything is carried out. Please understand that this is not a beginner level tutorial and the whole process is written in assumption that the readers have knowledge and experience with AWS services and Python Flask.

First, you need to have a flask api. An API that works. I couldn’t share the actual code that I first used since it is confidential . So, I have decided to go with a template of serverless framework. You can also convert your flask api to be compatible with serverless using the serverless-wsgi plugin. For this, you may need to make some changes to your serverless.yaml configurations. You can check it here to achieve that

Now that you have your api , lets upload it again to an aws codecommit repository. You can clone an empty repo, add the files, commit and push. Check it here.

Next, we need to create a build project in AWS CodeBuild.

In this codebuild project, use the following buildspec.yaml file. You can choose an Ubuntu environment for the build project to run.

version: 0.2
phases:
  install:
    runtime-versions:
      nodejs: 14
    commands:
        - npm install -g serverless
        - npm install serverless-compose
        - serverless --version
        - npm install serverless-deployment-bucket
        - sls plugin install -n serverless-wsgi
        - sls plugin install -n serverless-python-requirements

  build: 
    commands: 
      - serverless deploy

Remember: In this case, Codebuild is only used to run the deploy command. All the resources are built by serverless by automatically running a cloudformation.

Now, we’re also gonna need another yaml file. This one is the configuration file for the serverless framework which will be deployed once we deploy it using the Aws pipeline.

Here’s the configuration code.

service: new12

frameworkVersion: '3'

custom:
  wsgi:
    app: app.hello

provider:
  name: aws
  runtime: python3.9

functions:
  api:
    handler: wsgi_handler.handler
    events:
      - httpApi: '*'

plugins:
  - serverless-wsgi
  - serverless-python-requirements

Also keep your flask api file in the same directory. Make sure to configure codebuild to look for the specific branch in the specific repo.

Once everything is created, we need to make sure that our Codebuild service role has enough permission to use cloudformation and all the other services associated with the resources built through the cloudformation stack.

In my case, I faced a lot of failed builds due to permission issues.Always make sure to check your build logs when your build fails, so that you can get a clear understanding of the underlying reasons and issues.

So, to make it quick I added some permission to the codebuild service role. I suggest to create your own permission policies based on the least privilege access policy rather than assigning managed policies with Full access.

Once the permissions are set, your pipeline will run swiftly. Here's how my pipeline looks like with the two phases.

Here are the outputs from the cloudformation stack.

You invoke URL will be generated in Amazon API Gateway. Here in the build logs, you can see an invoke URL for the api.

The serverless Framework will also help you monitor your resources and their performance through the console. All you need to do is login to the console and connect to your aws account. The console will run another cloudformation stack to integrate with the resources in our aws account. Here is how the dashboard looks like.

Thank you for making it to the end, hope You found what you were looking for!

How to Build a Complete Headless Micro-SaaS Backend in n8n (Auth, DataTables & AI Pipelines)

Mon, 12 Oct 2026 17:09:35 +0000

Every developer building a Micro-SaaS runs into the same wall: before you can monetize your core AI value proposition, you need to spend 40+ hours wiring up the exact same commodity infrastructure.

You write authentication middleware, implement password hashing, configure database migrations, stand up REST endpoints for basic CRUD, wire rate limiters, and connect AI queues. By the time you are ready to test product-market fit, you've burned two weeks writing code that does not differentiate your product in any way.

In this guide, we will break down how to bypass traditional backend infrastructure entirely by turning n8n into a full-scale, headless Micro-SaaS engine equipped with JWT authentication, internal data management, and an asynchronous AI generation pipeline.

1. The Bottleneck: The High Cost of Commodity Infrastructure

Traditional approaches to standing up a SaaS backend fall into two frustrating categories:

Full-stack frameworks (NestJS, FastAPI, Django): High setup overhead. You have to configure ORMs, setup migrations, implement JWT verification, write schema validations, and build custom workers to handle LLM rate limits and timeouts.
Managed BaaS (Firebase, Supabase): Excellent for basic relational data, but as soon as you need complex orchestration—such as chained LLM calls, background image synthesis, dynamic credit deduction, and webhook dispatching—you end up writing disparate edge functions that are difficult to debug and trace.

Using n8n as your core backend orchestrator provides a unified visual control plane. You get native state management, out-of-the-box secret handling, built-in DataTables for storage, and automated retries for flaky 3rd-party AI APIs—all accessible via clean REST endpoints for your frontend (React, Next.js, Svelte, or Vue).

2. The Architecture: Headless n8n SaaS Topology

The entire system operates on a headless decoupled model:

[ Frontend: Next.js / Tailwind ]
          │   (Bearer JWT)
          ▼
[ Gateway Webhook: POST /api/v1/* ]
          │
          ├───> [ Sub-Workflow: Verify-JWT ]
          │           ├── Valid  ──> Extracts `userId`, `tier`
          │           └── Invalid ─> Returns 401 Unauthorized
          │
          ├───> [ Router: Target Action ]
          │           ├── Action: /user/credits   ──> [ DataTables Read ]
          │           ├── Action: /items/crud     ──> [ CRUD DataTables ]
          │           └── Action: /ai/generate    ──> [ Async Queue ]
          │                                                 │
          │                                                 ▼
          │                                      [ AI Engine Pipeline ]
          │                                      - Prompt Sanitizer
          │                                      - Model Generation
          │                                      - S3 / Storage Dispatch
          ▼
[ Webhook Respond: JSON Payload ]

Core Structural Components:

API Gateway Webhook: A single entry point or modular route handlers receiving incoming HTTP requests from the client.
Authentication Sub-Workflow: An isolated execution pipeline executing cryptographic signature checks on incoming JWTs.
Native DataTables Storage: Acting as the operational transactional database for user state, credit balances, and entity storage.
AI Generation Pipeline: An isolated pipeline executing prompt validation, token budgeting, external model API execution (OpenAI, Stable Diffusion, Flux, or Anthropic), and database logging.

3. The Code & Logic

Let's implement the core building blocks required to make this production-ready.

Step 1: JWT Verification via n8n Code Node

When a request hits your Webhook node, the raw headers contain the Authorization: Bearer <token>. Instead of invoking external auth providers for every internal action, you can verify and decode the token directly within a native JavaScript Code node using native WebCrypto APIs.

// Code Node: Verify & Decode JWT
const authHeader = $input.item.json.headers['authorization'] || '';

if (!authHeader.startsWith('Bearer ')) {
  return [{
    json: {
      authorized: false,
      error: 'Missing or malformed Authorization header'
    }
  }];
}

const token = authHeader.split(' ')[1];
const secret = $env.JWT_SECRET; // Sourced from n8n Environment Variables

function base64UrlDecode(str) {
  let base64 = str.replace(/-/g, '+').replace(/_/g, '/');
  while (base64.length % 4) base64 += '=';
  return Buffer.from(base64, 'base64').toString('utf8');
}

try {
  const parts = token.split('.');
  if (parts.length !== 3) throw new Error('Invalid token structure');

  const [rawHeader, rawPayload, signature] = parts;
  const payload = JSON.parse(base64UrlDecode(rawPayload));

  // Check expiration (exp in seconds)
  const now = Math.floor(Date.now() / 1000);
  if (payload.exp && payload.exp < now) {
    return [{ json: { authorized: false, error: 'Token has expired' } }];
  }

  // Cryptographic signature check (HMAC-SHA256)
  const crypto = require('crypto');
  const expectedSig = crypto
    .createHmac('sha256', secret)
    .update(`${rawHeader}.${rawPayload}`)
    .digest('base64url');

  if (signature !== expectedSig) {
    return [{ json: { authorized: false, error: 'Signature mismatch' } }];
  }

  return [{
    json: {
      authorized: true,
      user: payload
    }
  }];
} catch (err) {
  return [{
    json: {
      authorized: false,
      error: err.message
    }
  }];
}

If authorized is false, your workflow branches to a Respond to Webhook node returning status 401 immediately, preventing unauthorized processing.

Step 2: Dynamic Input Sanitization & Credit Check

Before forwarding user prompts to AI models, we must prevent prompt injection, validate token bounds, and confirm the user has available credits in the DataTables layer.

// Code Node: Prompt Sanitize & Deduct Prep
const rawPrompt = $input.item.json.body.prompt || '';
const userCredits = $input.item.json.userData.credits || 0;
const COST_PER_GENERATION = 2;

// 1. Guard against empty/oversized prompts
const cleanPrompt = rawPrompt.trim().replace(/[<>\0]/g, '');

if (cleanPrompt.length < 5 || cleanPrompt.length > 1000) {
  throw new Error('Prompt length must be between 5 and 1000 characters.');
}

// 2. Validate balance
if (userCredits < COST_PER_GENERATION) {
  return [{
    json: {
      canExecute: false,
      error: 'Insufficient credit balance. Top-up required.',
      statusCode: 402
    }
  }];
}

return [{
  json: {
    canExecute: true,
    sanitizedPrompt: cleanPrompt,
    newCreditBalance: userCredits - COST_PER_GENERATION,
    timestamp: new Date().toISOString()
  }
}];

Step 3: Automated CRUD Mapping via DataTables

Instead of provisioning an external PostgreSQL or MongoDB instance during the prototyping phase, leverage n8n DataTables. By setting up dynamic table operations, a single webhook route /api/v1/records can handle all standard operations based on the HTTP method ($input.item.json.method):

GET: Returns records scoped to the extracted userId.
POST: Inserts a new row with automatic UUID generation.
PATCH: Updates metadata or status fields.
DELETE: Marks rows as deleted or removes them entirely.

This keeps your entire persistence layer contained directly within your workflow backups.

4. Deployment, Concurrency, and Error Handling

Running a Micro-SaaS backend in production requires strict reliability controls.

Handling AI Latency & Timeouts

AI image generators (Midjourney APIs, Flux, SDXL) frequently take 5 to 30 seconds to return. Running synchronous HTTP requests causes browser timeouts on client frontends.

The Solution: Webhook Handshake Pattern

Webhook receives the payload and issues a 202 Accepted response back to the client immediately with an execution_id.
The main workflow passes the payload to a sub-workflow via the Execute Sub-Workflow node configured asynchronously (Wait for Sub-Workflow Completion: false).
The worker sub-workflow runs the generation, uploads the resulting image asset to an S3-compatible bucket, and updates the DataTable record status from processing to completed.
The frontend polls a lightweight /api/v1/status?jobId=... endpoint or listens via SSE / WebSockets.

Preventing Rate Limits with Redis Queue Mode

If your n8n instance experiences sudden spikes in user traffic, running workflows in default execution mode can saturate server memory.

Run n8n in Queue Mode using Redis.
Deploy webhook workers independently from your execution workers.
Enforce rate-limiting headers using an upstream reverse proxy (like NGINX, Traefik, or Caddy) to throttle client IPs before they reach the n8n webhook listeners.

5. Conclusion & Ready-to-Use Boilerplate

You now have the architectural blueprint to eliminate traditional backend code and turn n8n into a modular, production-ready Micro-SaaS infrastructure. By centralizing authentication, state management, and AI execution inside n8n, you can deploy a functional product in an afternoon rather than weeks.

You can manually implement the workflows, tables, and scripts described above directly in your n8n editor.

Alternatively, if you want to skip the manual setup and get a fully configured, production-tested boilerplate with all webhook handlers, sub-workflows, token verification nodes, CRUD DataTables, and AI image pipelines pre-wired:

Instant Access on Whop: Get the Full-Stack Micro-SaaS Boilerplate
Direct Download on Gumroad: Download the Workflow Package — Use code EARLYBIRD at checkout for 20% off.

Both versions include the complete importable JSON architecture, API specifications, and test fixtures so you can launch your next SaaS backend immediately.

🛠️ D'autres outils MR Studio qui pourraient vous intéresser :

Fixing Multimodal AI Fallback in Workflows

Sun, 11 Oct 2026 18:03:24 +0000

Multimodal pipelines in serverless publishing workflows often route text drafting to one model and visual review to another. If a text provider fails, falling back to an alternate model keeps the writing phase moving, but reusing that same fallback policy for image verification creates silent failures.

Quick solution:

Validate the required modality explicitly before routing a fallback, ensuring that image-bearing reviews fail closed instead of accepting a text-only substitute.

if (stepType === 'visual_review' && !provider.supportsVision) {
  throw new Error('Visual review requires a multimodal capability; failing closed.');
}

The Failure Mode of Shared Fallbacks

When building distributed content pipelines, a common architectural pitfall is treating model availability as a uniform property. If a text-generation step fails due to a recognized regional availability constraint, routing to a fallback model is standard practice. But if that fallback policy handles visual review stages without checking capabilities, the system encounters a conceptual mismatch.

A text-only model cannot evaluate pixel data. If the fallback mechanism silently routes an image review to a text-only model, the review step receives an unsupported input format or a generic response. The system then misinterprets this as a passing review, bypassing the visual verification gate entirely.

Implementing Capability-Aware Routing

To maintain strict editorial and technical standards, pipelines must separate generation capabilities from review capabilities. Text drafting can leverage secondary providers or bounded fallbacks because the core requirement is generating coherent natural language. Visual review gates require explicit multimodal support.

When the primary vision-capable model is unreachable, the review step must fail closed. Preserving the source artifact for a subsequent run is safer than allowing an unverified asset to pass into production.

Precise Error Classification for Fallbacks

Another frequent pitfall in fallback design is broad error string matching. Searching an entire API response for keywords can cause false positives if a nested diagnostic message mentions an error state without originating from the top-level API status.

To prevent unrelated client errors from triggering unintended fallbacks, systems should parse structured provider errors and match the authoritative field precisely. The following implementation pattern demonstrates how to verify an exact top-level error message.

function isExactLocationError(err: unknown): boolean {
  if (typeof err !== 'object' || err === null) return false;
  const topLevelMessage = (err as { message?: string }).message;
  return topLevelMessage === 'REQUIRED_LOCATION_PRECONDITION_NOT_MET';
}

By restricting the match to the explicit top-level error.message, similar diagnostic strings buried inside nested details will not trigger a fallback. This protects the integrity of the routing logic and ensures unexpected exceptions are surfaced properly rather than masked.

Execution and Verification Checklist

Confirm that text generation steps use bounded fallbacks only after recognizing specific provider error codes.
Ensure visual review steps reject text-only model substitutions and fail closed when multimodal endpoints are unavailable.
Test that nested diagnostic text inside error payloads does not trigger unintended fallback pathways.
Verify that visual review functions make no extraneous text-model invocation calls during execution.
Check that failed visual reviews preserve the source artifacts for later execution attempts.

Half-Open Circuit Breaker Probes in Serverless Workers

Sun, 11 Oct 2026 02:44:43 +0000

Managing dependency recovery in serverless architectures requires coordinating multiple concurrent invocations to prevent a thundering-herd problem when a service heals. Quick solution: Coordinate the half-open lease through a single SQLite-backed Durable Object per dependency, keep the network probe outside the transaction, and defer competing callers.

async function acquireProbeLease(storage: DurableObjectStorage, now: number): Promise<boolean> {
  return await storage.transaction(async (txn) => {
    const state = await txn.get("breaker_state");
    if (state && state.status === "open" && state.cooldownUntil > now) {
      return false;
    }
    const lease = await txn.get("probe_lease");
    if (lease && lease.expiresAt > now) {
      return false;
    }
    await txn.put("probe_lease", { expiresAt: now + 5000 });
    return true;
  });
}

A circuit breaker protects downstream dependencies by failing fast while an upstream service is unhealthy. The recovery window introduces a new risk when independent request handlers attempt recovery simultaneously. This race condition happens whenever multiple concurrent invocations share a dependency and coordinate through shared state. A scheduled invocation runs at its configured time rather than automatically fanning out into multiple triggers, but a sudden influx of traffic can create the same competitive pressure.

Understanding the Circuit Breaker States

A reliable serverless circuit breaker relies on three distinct operational states:

Closed: normal traffic passes directly to a healthy dependency.
Open: calls fail fast or defer while the dependency remains unhealthy.
Half-open: after a cooldown period, the system admits a controlled probe instead of immediately restoring all traffic.

A successful probe closes the circuit breaker. A failed or timed-out probe reopens it and schedules another recovery attempt. A cooldown period represents a probe opportunity rather than a guarantee of full recovery.

Choosing a Coordination Boundary

Choosing the right coordination backend is essential for safe probe admission. Eventually consistent storage layers cannot provide the atomic guarantees required for single-probe coordination.

Option	When to Use It	Trade-off or Failure Mode	Recommendation
Workers KV	Global read-heavy configurations	Eventually consistent, lacks atomic read-modify-write	Do not use for transactional lease locks.
SQLite-backed Durable Object	Single-region state consistency and transactions	Bound to a single location, requires routing	Recommended for single coordination points.
External Relational Database	Existing enterprise database clusters	Adds network latency to serverless invocations	Avoid if serverless cold-start and latency matter.

A SQLite-backed Durable Object provides strong consistency and transactional guarantees for a single coordination point. Use a stable Durable Object identity for each protected dependency to manage lease admission in a short transaction.

Managing Recovery and Failure Handling

Leases must include an expiration timestamp so a crashed or timed-out claimant does not permanently block recovery. Callers that do not receive the lease should return a retryable or deferred outcome rather than spinning in a tight retry loop.

On a successful probe, close the breaker and clear the active lease. On a failure or timeout, reopen the breaker with an appropriate delay and jitter. For further reading on operational safety mechanisms, see Self-Expiring Kill Switches and Human-Readable Alerts.

Verification Checklist

Issue five simultaneous lease requests against the Durable Object and verify that exactly one caller receives permission to probe.
Confirm that expired leases allow a new claimant to acquire the probe slot after the timeout window elapses.
Verify that late probe results from an expired lease do not overwrite newer breaker states in storage.
Check that callers without a lease receive a deferred response rather than executing unauthorized network traffic.

Testing LLM Fallback Tiers Before Outages

Sat, 10 Oct 2026 09:03:31 +0000

When a primary language model provider rejects requests due to location or transient overloads, many teams rely on an in-platform fallback tier. Logging response bodies and separating deterministic blocks from transient failures solves the diagnosis layer, but it leaves a critical question unanswered. A fallback that never runs in production is merely a hope. If you do not test the backup path continuously, your first time seeing it fire will be during an outage, which is the worst possible time to discover a configuration bug.

Running a reliable secondary tier requires more than basic routing rules. You need a reliable warming pattern that does not inflate latency or cost, a strict response-shape contract so consumers never know which provider answered, and a scheduled drill cadence with a clear readiness metric. This guide covers how to implement these mechanisms in a serverless environment without breaking your budget.

Fractional Shadow Traffic With Diff Logging

To ensure a fallback model is ready, you need to send it realistic payloads. However, routing user traffic blindly to an unproven secondary provider risks increased latency and unexpected error rates. The solution is fractional shadow traffic.

In a serverless worker, you can mirror a small, sampled percentage of live requests to the fallback tier asynchronously. By using asynchronous execution primitives like waitUntil, the shadow request fires in the background without affecting the user-visible critical path. Both the primary response and the shadow response are logged, and their normalized shapes are compared.

export async function handleRequest(request: Request, env: Env, ctx: ExecutionContext): Promise<Response> {
  const primaryResponse = await callPrimaryProvider(request, env);
  
  // Sample 2 percent of traffic for shadow warming
  if (Math.random() < 0.02) {
    ctx.waitUntil(runShadowFallback(request, primaryResponse, env));
  }
  
  return primaryResponse;
}

async function runShadowFallback(request: Request, primary: Response, env: Env): Promise<void> {
  try {
    const fallbackRaw = await callFallbackProvider(request, env);
    const diff = compareResponseShapes(primary, fallbackRaw);
    if (diff.hasMismatch) {
      await logShapeMismatch(env, diff);
    }
  } catch (error) {
    await logShadowError(env, error);
  }
}

This approach gives you continuous coverage. If the fallback provider alters its error format or response structure, your diff logs catch the mismatch on a quiet Tuesday rather than during a traffic spike.

Response-Shape Adapter Contracts

Different language model providers return unique JSON envelopes, usage metadata, and finish reasons. If your application code directly handles provider-specific payloads, switching to a fallback tier requires branching your business logic everywhere.

To eliminate this complexity, introduce a response-shape adapter. Neither provider's raw envelope should ever reach your core application. Instead, every provider output passes through a normalization layer that maps data to a single internal structure containing only the generated text, token counts, finish reason, and standardized error classes.

interface NormalizedResponse {
  text: string;
  inputTokens: number;
  outputTokens: number;
  finishReason: 'stop' | 'length' | 'error';
  errorClass?: string;
}

function adaptProviderResponse(raw: any, provider: string): NormalizedResponse {
  if (provider === 'primary') {
    return {
      text: raw.candidates[0].output,
      inputTokens: raw.usage.prompt_tokens,
      outputTokens: raw.usage.completion_tokens,
      finishReason: raw.candidates[0].finish_reason === 'MAX_TOKENS' ? 'length' : 'stop'
    };
  } else {
    return {
      text: raw.choices[0].message.content,
      inputTokens: raw.usage.input_tokens,
      outputTokens: raw.usage.output_tokens,
      finishReason: raw.choices[0].finish_reason
    };
  }
}

By forcing both primary and fallback responses through this adapter, failover becomes a simple routing decision. The rest of your application remains entirely agnostic of which provider fulfilled the prompt.

Scheduled Failover Drills and Readiness Metrics

Asynchronous shadow traffic proves that responses match, but it does not verify that your routing logic actually triggers when the primary provider fails. For that, you need scheduled failover drills.

A cron-triggered background worker can force a small batch of synthetic probe traffic through the fallback tier for a bounded window. The pass bar for these drills is binary. Every probe must succeed, and every response must conform to the normalized shape contract. If a single probe fails, the drill fails. For a related implementation, see Bounded Llm Fallback Chains.

To give the on-call engineer immediate confidence during an incident, combine your telemetry into a single readiness score. This score factors in three underlying data series:

Shadow success rate over the last twenty-four hours.
Shape-conformance rate from differential logging.
Days elapsed since the last successful green drill.

If the drills are skipped or if the shadow diff rate degrades, the readiness score drops automatically. Your incident runbook should require the on-call engineer to verify that this readiness score meets a minimum threshold before manually forcing a failover switch, removing guesswork from high-stress moments. For a related implementation, see Audit Macos System Data Before Deleting.

Bounding Cost and Complexity

Warming a fallback tier and running regular synthetic drills consumes billable API requests. Without cost controls, your reliability engineering efforts can inadvertently trigger a billing incident.

Keep shadow traffic tightly sampled, typically between one and five percent of total production volume. Cap the number of synthetic probes executed during weekly drills to the minimum necessary for statistical confidence. By bounding these operations and scheduling them during off-peak windows, you maintain a continuously verified backup system while keeping expenses entirely predictable.

Nightly Analyst Digest from Slot Activity Logs

Sat, 10 Oct 2026 03:03:48 +0000

Running a high-frequency automation system that triggers dozens of times per day creates a difficult reporting dilemma. If every run sends an immediate notification, your inbox fills with noise. If you turn off notifications entirely, regressions go unnoticed until a user complains. The natural solution is a nightly digest, but building one reliably on serverless infrastructure introduces subtle failure modes. A naive implementation often results in missing reports when the underlying language model fails, misaligned days due to UTC offset handling, or raw log dumps that require too much manual effort to parse.

This article examines how to build a reliable scheduled digest email from activity log data. You will learn how to design a logging contract specifically for downstream aggregation, handle timezone boundaries without drifting, implement a graceful fallback when the analyst model is unavailable, and keep recommendations strictly advisory.

Sixty Emails versus One Digest

When a system executes tasks tens of times a day, individual alerts quickly lose their value. Developers stop reading them, and important failures hide inside a flood of routine success notices. A single daily email solves this fatigue by aggregating metrics, engagement statistics, and notable events into one readable summary. For a related implementation, see Email Account Safety Developers.

However, building this digest introduces three architectural challenges:

Data Availability: Slots that only execute tasks do not inherently preserve the context needed for a narrative summary.
Fragile Dependencies: If the nightly summary relies entirely on a third-party language model to generate text, an API outage will prevent the email from sending entirely.
Timezone Drift: Standardizing cron jobs on UTC while the human reader operates in a local time zone splits calendar days incorrectly, producing reports that mix evening events from the wrong day.

Solving these issues requires treating the nightly digest not as an afterthought, but as a core pipeline with its own data contract and error boundaries.

Logging for the Analyst

To produce a meaningful summary, individual execution slots must leave behind structured data designed for aggregation rather than raw debugging. Instead of unstructured console logs, every slot should append a compact JSON line to a date-keyed store with a multi-day time-to-live (TTL). For a related implementation, see Debugging Silent Skips In Poll Based.

Below is an example of a structured log entry recorded by a worker slot:

{
  "timestamp": "2023-10-25T08:30:00Z",
  "slot_id": "hourly-sync-04",
  "status": "success",
  "model": "gpt-4o-mini",
  "provider": "openai",
  "material_count": 12,
  "reply_count": 3
}

Writing these entries rides on existing key-value storage writes, avoiding the overhead of a dedicated database. To prevent timezone ambiguity, the storage key incorporates the audience local date directly rather than relying on raw UTC timestamps. For example, a key format of digest:2023-10-25 ensures all runs belonging to that calendar day in the target region are grouped together, regardless of UTC shift.

The Degraded Digest Pattern

Relying on a language model to synthesize raw activity logs into human-readable insights introduces a single point of failure. If the provider experiences downtime or rate limits at the cron trigger time, the entire digest fails to send, leaving the team blind.

To make the system resilient, the analyst call must be treated as best-effort. If the model call fails or times out, the service must catch the error, fall back to displaying the computed raw statistics block directly, and label the email accordingly.

Here is a conceptual TypeScript implementation of the execution handler with fallback logic:

async function generateDigestReport(rawLogs: LogEntry[]): Promise<DigestResult> {
  const stats = computeAggregates(rawLogs);
  
  try {
    const analysis = await callAnalystModel(stats);
    return {
      mode: "ai-enhanced",
      summary: analysis.text,
      recommendations: analysis.recommendations,
      stats
    };
  } catch (error) {
    return {
      mode: "degraded-raw-stats",
      summary: "Analyst model unavailable. Displaying raw computed metrics.",
      recommendations: [],
      stats
    };
  }
}

This pattern ensures that an upstream outage degrades the quality of the formatting rather than breaking delivery. The recipient still receives their morning metrics on schedule.

Questions, Not Actions

Another risk in automated reporting is allowing the analyst model to modify system behavior based on its own findings. If an LLM decides that a prompt or execution cadence should change and automatically applies that change, the reporting pipeline transforms into an unsupervised and potentially destructive deployment system.

To maintain safety, recommendations generated by the digest must be phrased strictly as questions for human review. For instance, rather than updating configuration files, the output should suggest adjustments like "Consider increasing the timeout limit for morning sync runs?" or "Cadence for slot B generated high error rates; review prompt constraints." An explicit human decision remains required to apply any configuration change.

Summary of Architectural Patterns

Building a dependable nightly analyst digest requires more than wiring a cron trigger to an AI model. By designing a dedicated logging contract, anchoring date keys to the audience timezone, implementing a non-LLM fallback for the summary step, and keeping recommendations advisory, you can achieve the best of both worlds. You get the readability of an AI-assisted narrative without sacrificing the reliability of traditional serverless infrastructure.

I built email alerts for my app with no login system

Thu, 08 Oct 2026 23:42:57 +0000

My movie discovery site (galaxymovies.app) needed a feature that sounds simple: "email me when something on my watchlist lands on Netflix." Users save titles to a local watchlist, pick their streaming services, and want a nudge the day a title actually arrives.

That feature wants an account system. Passwords, sessions, email verification, password resets, the works. I really did not want to build that. Every line of auth code is a liability I have to maintain forever, and the payoff for users is typing a password into a movie site.

So I asked a different question: what's the minimum credential that makes this work? The answer turned out to be one email address and two tokens. Here's the whole design.

The insight: the confirmation link is already proof

When someone submits the waitlist form, they get a confirmation email with a token link. Clicking it proves they control that inbox. That's it. That's the entire identity system.

But the confirm token can't be the API credential. It's a link secret, meant to be clicked once and done. Syncing a watchlist needs a durable bearer credential the client sends on every edit. So confirm mints a second, separate token: a UUID sync key stored in a Redis hash keyed by email.

export const PENDING_KEY = 'gm:waitlist:pending'       // email => confirm token
export const CONFIRMED_KEY = 'gm:waitlist:confirmed'   // email => unsub token
export const SYNCKEY_KEY = 'gm:alerts:synckey'         // email => sync UUID
export const ALERTS_PREFS_KEY = 'gm:alerts:prefs'      // email => {watchlist, services, region}
export const ALERTS_NOTIFIED_KEY = 'gm:alerts:notified' // email => sent alert keys

Two credentials, two jobs. Leaking or rotating the sync key never touches the unsubscribe token, and vice versa.

Handing out the key through a URL fragment

The interesting part is delivery. The sync key has to get from the confirmation click to the user's browser without ever appearing in my server logs. Query strings get logged; fragments don't.

So the confirm endpoint 302-redirects to a status page with the credentials in the hash:

// Fragment never reaches the server on redirect follow, so the sync
// credential stays out of access logs from this point on.
const syncKey = await getOrCreateSyncKey(kv, email)
return redirect('confirmed', `email=${encodeURIComponent(email)}&key=${syncKey}`)

Which produces /email-status?status=confirmed#email=you%40x.com&key=abc-123. The status query param is logged; the fragment is not, because browsers don't send it. The page reads window.location.hash, writes {email, key} to localStorage, and the credential now exists in exactly two places: the user's browser and one Redis hash field.

Syncing without sessions

The client posts a snapshot of the watchlist and chosen providers whenever they change. Two details keep this cheap:

export async function syncAlertsNow(data, creds = getAlertsCredentials()) {
  if (!creds) return
  const payload = JSON.stringify(buildSnapshot(data))
  if (localStorage.getItem(LAST_SYNCED_KEY) === payload) return
  const ok = await postPrefs(creds, snapshot)
  if (ok) localStorage.setItem(LAST_SYNCED_KEY, payload)
}

A 4 second debounce collapses edit bursts, and the LAST_SYNCED_KEY check skips the request entirely if nothing changed since the last successful sync. Any watchlist edit re-renders the app, so without the diff check every idle re-render would fire a POST.

On the server side, POST /api/alerts-prefs validates the email and key formats, rate limits per IP and per email (30/hour each, keyed by a sha256 of the address rather than the raw email), then safeEquals the key against the hash:

const stored = await kv.hget<string>(SYNCKEY_KEY, email)
if (!stored || !safeEqual(stored, key)) {
  return res.status(403).json({ error: 'Invalid sync credentials' })
}
await kv.hset(ALERTS_PREFS_KEY, { [email]: { watchlist, services, region, updatedAt: Date.now() } })

Notice what's missing: no session, no cookie, no JWT. The sync key is just a bearer token with a 36 character format check.

The cron does the matching

A daily job diffs each tracked provider's catalog (added and coming soon lists are already computed by an earlier cron and cached). Then it loads all three hashes in three HGETALLs and walks subscribers:

for (const item of change[kind]) {
  if (!item.tmdbId || !watchlistKeys.has(`${item.tmdbType}:${item.tmdbId}`)) continue
  const notifiedKey = `${kind}:${item.tmdbType}:${item.tmdbId}@${providerKey}`
  if (notified.has(notifiedKey)) continue
  out.push({ kind, notifiedKey, tmdbId: item.tmdbId, ... })
}

Each subscriber's watchlist becomes a mediaType:id set matched against their chosen providers' diffs. The notifiedKey dedupe (added:movie:550@netflix) means nobody gets the same alert twice, and each user gets at most one email per day with up to 20 matches. The email carries a List-Unsubscribe header and a one-click link.

One subtlety: the cron reads provider diffs with fresh: true to skip the per-instance memo I added to keep pageviews cheap. A memoized diff is exactly wrong here, the whole point is "what changed today."

A second device is just a form resubmit

Syncing another device needs no new UI. If an already confirmed email resubmits the waitlist form, the server mails them an "alerts-link" URL that reissues the fragment flow on whatever device opened the email:

if (await kv.hget(CONFIRMED_KEY, email)) {
  const key = await getOrCreateSyncKey(kv, email)
  await sendLinkDeviceEmail(email, key)
  return res.status(200).json({ ok: true, pending: true })
}

Same pending: true response as a fresh signup, deliberately. Returning "you're already subscribed" would make the form an oracle for checking who's on the list.

Unsubscribing deletes everything

The unsubscribe link uses the original confirm token and wipes all five hash fields: confirmed status, pending, sync key, stored prefs, and the notified set:

await kv.hdel(CONFIRMED_KEY, email)
await kv.hdel(PENDING_KEY, email)
await kv.hdel(SYNCKEY_KEY, email)
await kv.hdel(ALERTS_PREFS_KEY, email)
await kv.hdel(ALERTS_NOTIFIED_KEY, email)

Any client still holding the sync key starts getting 403s immediately, and there's no residual data to leak.

What I gave up, honestly

This is not authentication, and I want to be straight about the limits:

Anyone with the sync key can overwrite that email's watchlist prefs. It's a write only vandalism risk bounded to preference data, not an impersonation risk. There's nothing sensitive to read back, the prefs endpoint never returns stored data.
The confirm and link endpoints skip rate limiting on purpose, because mail scanners prefetch links. The token is the auth. That trade means those endpoints are cheap GETs that do one hash lookup.
If someone loses their browser profile, the fix is resubmitting the form and getting a fresh link. Annoying, but not a support ticket.

The actual lesson

"Needs alerts" does not mean "needs accounts." If the identity proof you're already doing (email confirmation) is strong enough, a second bearer credential for the chatty sync traffic gets you the whole feature. Fragment delivery keeps it out of logs, format checks plus rate limits keep it boring, and the unsubscribe path deleting every trace keeps it honest.

Total auth surface: zero passwords, zero sessions, zero cookies. One email field and a UUID.

The site is galaxymovies.app if you want to see the flow. Sign up, confirm, and watch your browser's localStorage for gm.alerts.v1, that's the whole credential.

My side project's free tiers were dying, so I counted what each service actually bills

Thu, 08 Oct 2026 22:42:45 +0000

Galaxy Movies is a movie discovery side project ~ Next.js Pages Router on Vercel, TMDB for data, Upstash Redis for caching & rate limiting, and OpenAI for a chat assistant & generated synopses. Everything runs on free tiers, which is the whole point of a side project.

A few weeks ago all three of those free tiers started groaning at once. Vercel function usage was climbing, Upstash was burning through its daily command quota well before midnight, and OpenAI spend had a suspicious correlation with traffic spikes that produced no actual readers.

The fixes all came down to the same realization: every "free" tier bills a different unit, and I was counting the wrong things. Here's what was actually billing me, and what I changed.

Act 1: The bots were running up my function bill

Vercel bills function duration, and my movie and TV detail pages were expensive: five parallel TMDB fetches (title, watch providers, credits, ratings, recommendations), plus Redis reads for the AI content slots, all in getServerSideProps.

The expensive traffic wasn't humans. SEO crawlers and AI scrapers sweep the entire TMDB ID space in bulk, and every one of those detail pages ran the full fetch chain. They don't index anything and send zero readers. Pure cost.

The fix was a thin render path just for them:

// Bots that don't index get a minimal render. They sweep the TMDB id
// space in bulk and don't need providers, credits, videos, or AI slots.
if (isBot(ua) && !isSearchEngine(ua)) {
  const titleRes = await fetch(`https://api.themoviedb.org/3/${media}/${id}?api_key=${apiKey}`)
  // on failure return notFound or errorProps; otherwise:
  context.res.setHeader('Cache-Control', 'private, no-store')
  return { props: { ...thinProps, media, title, topCast: [] } }
}

Two details matter here:

The isSearchEngine split. isBot matches a broad allowlist, but Googlebot, Bingbot, and DuckDuckGo still get the full render since they index. Social preview bots get the thin version. Whether a bot indexes is the only question that matters.
private, no-store is critical. If the stripped page entered the shared CDN cache, a cache collision could serve the bot version to a human, or worse, to Googlebot. Cheap responses must never contaminate the real cache.

I also found robots.txt was being rendered by a Next.js page, which meant every crawler's very first request was a function invocation. It moved to a static public/robots.txt, which doubles as an enforcement layer:

User-agent: AhrefsBot
User-agent: SemrushBot
User-agent: MJ12bot
User-agent: Bytespider
User-agent: GPTBot
User-agent: ClaudeBot
User-agent: CCBot
# ...several more
Disallow: /

Yes, Disallow is an honor system, but the big commercial crawlers honor it, and they were most of the volume.

Finally, I lengthened the SSR CDN cache windows: s-maxage from 1 hour to 1 day, stale-while-revalidate from 1 day to 1 week. Repeat crawler hits on popular titles now terminate at the edge instead of running SSR again.

Act 2: Upstash bills every single Redis command

This one caught me off guard. Upstash's free tier bills per command, and a pipeline doesn't save you, because each member of a pipeline is its own billed command. My request paths fanned out into Redis calls that felt free because "Redis is fast."

The audit turned up some embarrassing finds. Cache reads on detail pages were ~97% misses, so I was paying a billed command for a null almost every time. Rate limiting cost up to three commands per check. A dataset updated once a day was being fetched again on every pageview.

The fixes, roughly in order of satisfaction:

Rate limiting became one Lua script. INCRBY plus a conditional EXPIRE plus a TTL recovery check run inside a single EVAL, billed once no matter how many redis.call()s it makes:

local count = redis.call("INCRBY", KEYS[1], ARGV[1])
if count <= tonumber(ARGV[1]) then
  redis.call("EXPIRE", KEYS[1], ARGV[2])
elseif count > tonumber(ARGV[3]) and redis.call("TTL", KEYS[1]) < 0 then
  redis.call("EXPIRE", KEYS[1], ARGV[2])
end
return count

The TTL check handles a real edge case: if a key somehow loses its expiry, a blocked client fixes it on the next attempt instead of being rate limited forever.

Check caches before paying the rate limit write. /api/prices was rate limiting requests that would have been cache hits, paying a write before doing a read. Reorder, and every cache hit is one command instead of two.

Memoize what changes daily. The streaming-changes diffs are rewritten once a day by cron but were being fetched per key on every pageview. A per instance LRU holding results for ten minutes turns repeat page loads into zero Redis commands:

const changesMemo = new LRUCache<string, Record<string, ProviderChanges>>({ max: 64, ttl: 10 * 60 * 1000 })

export async function getProviderChanges(providerKeys = [], { fresh = false } = {}) {
  const memoKey = [...providerKeys].sort().join(',')
  if (!fresh) {
    const memoized = changesMemo.get(memoKey)
    if (memoized) return memoized
  }
  // ...Redis reads, then changesMemo.set(memoKey, changes)
}

The fresh flag exists because the cron itself must never match subscribers against a diff it just replaced. Memoization that's correct most of the time is a bug with extra steps.

Batch the counters. Every HINCRBY is billed, so AI usage and analytics deltas accumulate in a Map and flush every 30 seconds (or 50 fields). Stats lag by up to half a minute and a dying instance loses its unflushed tail, a fine trade for telemetry.

Restructure data to match access patterns. Detail pages fetch both AI content slots in one MGET instead of two GETs. Generated lists moved from one key per slug into a single hash, where HGETALL is one command replacing an unbounded key fanout.

Net result: roughly half the daily command volume, with quota headroom for the first time in weeks.

Act 3: OpenAI spend, or "why is a scraper generating my content?"

The AI features on detail pages (generated synopses, similar title blurbs) trigger an OpenAI call on a cache miss. Guess who was causing cache misses at scale: the same scrapers from Act 1. Bots were literally generating my content.

So bots got read only access: they can consume AI content that's already cached but can never trigger generation (generate: false on the cache lookup). Human traffic is unaffected; scraper sweeps now cost zero tokens.

The other fixes were less dramatic but added up:

Failed cron seeds stopped retrying forever. A broken seed now records failed with an attempt counter and gives up after 3 tries, instead of burning a discover call on every daily run until the heat death of the universe.
Longer cache TTLs. Reviews and similar titles are static per title, so 365 day TTLs mean one generation per title, ever.
Prompt dieting. Similar title prompts dropped to 6 candidates with 200 character overviews; editorial prompts got explicit word budgets; structured JSON output removed the "please don't ramble" tax.
Shared chat cache. Identical chat requests (quick actions, common first questions; the bulk of traffic) now hit a Redis cache shared across instances, plus tighter limits: 10/hr per IP, 1000/day global.
Per task model control. All OpenAI calls route through one module with per task reasoning_effort: none for latency sensitive tasks, low only for the discover generator. Real token usage gets logged so I can see where spend actually goes.

The kill switch

The scariest scenario isn't gradual spend. It's a quota or billing problem silently converting every request into an error, or worse, into retry storms. Billing and auth failures (401/402, insufficient_quota) now trip a shared kill switch with a one hour cooldown:

export function isQuotaError(err: unknown): boolean {
  if (!(err instanceof OpenAI.APIError)) return false
  if (err.status === 401 || err.status === 402) return true
  const code = err.code || err.error?.code
  return code === 'insufficient_quota'
      || code === 'billing_hard_limit_reached'
      || code === 'billing_not_active'
}

While it's tripped: /api/chat returns 503 { code: 'ai_unavailable' } before spending budget, the chat widget checks /api/aiStatus on mount and hides itself entirely instead of showing an error bubble, the generators skip their calls, and the content cron skips the run without burning seed attempts, since an outage isn't a seed's fault. Crucially, ordinary 429 rate_limit_exceeded errors don't trip it; those are transient.

One deliberate choice throughout: degrade in the direction that saves money. When Redis errors on a check that protects spend, the code falls back to a per instance limiter and never fails open. An in memory limiter might be less accurate, but a Redis outage can't be used to run up my OpenAI bill.

The part where I broke things

Honest bookkeeping: the cost cuts shipped two bugs worth their own paragraph.

After moving the chat cache to Redis, identical requests started hitting on every quick action prompt, and the cache hit path returned plain JSON that the SSE client never parsed. Cache hits went from "the fast path" to "the broken path" overnight. Fix: return cache hits through the SSE stream like everything else.

And when bots stopped triggering AI generation, Lighthouse CI got caught in the crossfire. It audits the real detail pages, and its UA matched the bot list, so it was suddenly measuring the thin render. Chrome-Lighthouse went on the search engine side of the check.

Both bugs were the same shape: a filter added at the edge changed behavior for a consumer I forgot about. If your bot detection doesn't have a list of legitimate automated consumers, it will find one for you. In production. On a Sunday.

What I'd tell past me

Count the billed unit, not the abstraction. "Redis call," "function invocation," and "API request" are all wrong. It's commands, gigabyte seconds, and tokens, and the gap between what I thought I was spending and what I was billed lived entirely inside that distinction.
Segment bots by whether they index, not whether they're bots. That one boolean decided which requests deserved the expensive render, the Redis reads, and the cacheable output.
Cache at the cheapest layer that works. CDN edge > shared Redis > per instance LRU > regenerate. I had requests traversing the whole hierarchy when a static file would do (looking at you, robots.txt).
Free tier quotas are architecture constraints, not billing details. "A few Redis commands per request is fine" is a statement that only survives until you do the multiplication.
Instrument spend itself. The usage counters I added to survive the Upstash quota are now how I watch OpenAI costs. The fix for a quota crisis becomes your observability for the next one.

The site is galaxymovies.app if you want to see the thin renders in the wild. View source as a human, please; the bots are expensive enough. Happy to compare notes in the comments if you've fought the same free tier math!

Self-Expiring Kill Switches and Human-Readable Alerts

Thu, 08 Oct 2026 22:16:58 +0000

When an unattended cron system encounters a total provider failure, engineers often reach for a manual kill switch. While effective at stopping runaway requests, the manual-only reset pattern introduces hidden operational costs. If an outage lasts twenty minutes, the system stays dark until an operator manually intervenes. When multiple transient interruptions occur within a single day, teams find themselves repeatedly waking up or context-switching to clear flags for zero new information. Furthermore, when these systems fail, they frequently dump raw JSON status codes into email inboxes. Over time, operators learn to ignore these notifications entirely, missing the genuine incidents hidden beneath the noise. For a related implementation, see Ai Agent Handoff Fallback Context.

Designing resilient background workflows requires solving two distinct problems. First, the system needs an automated mechanism to halt execution during widespread failures without trapping the infrastructure in a permanent locked state. Second, notifications must translate low-level errors into actionable prose so that operators can assess the situation at a glance. For a related implementation, see Fixing Malformed Json Errors In Ci.

The Problem with Manual-Only Resets

Traditional architectural patterns treat a kill switch as a binary toggle. An operator sets a flag, the background jobs stop, and someone must later run a command or update a database record to resume execution. In practice, this creates a mismatch between system recovery and human intervention. Providers frequently recover on their own within minutes. If the kill switch lacks an expiration mechanism, the recovery is bottlenecked by human availability. The operational burden shifts from fixing the infrastructure to administrative housekeeping.

Designing the Expiring Kill Flag

To balance safety with automation, the kill flag can be upgraded from a simple boolean to a timestamped window. When a total provider failure is detected, the system writes a kill key containing the exact engagement time. Subsequent execution cycles check this key before running. If the current time is within a bounded window, such as three hours, execution is suppressed.

Once the timestamp exceeds the expiration threshold, the system automatically clears the flag and resumes normal operations. This ensures that the infrastructure never remains silenced indefinitely due to an forgotten flag.

Security and failure modes require strict handling. If the kill key value is unreadable, malformed, or corrupted, the system must fail closed rather than assuming everything is fine. Expiry logic applies exclusively to well-formed timestamps.

{
  "kill_switch": {
    "status": "active",
    "engaged_at": "2023-10-25T14:00:00Z",
    "expires_at": "2023-10-25T17:00:00Z"
  }
}

When a cron worker evaluates this structure, it compares the current clock against the expiration boundary:

def is_kill_switch_active(flag_data):
    if not flag_data or "expires_at" not in flag_data:
        # Fail closed on corrupted or missing flags
        return True
    
    current_time = get_current_utc_time()
    if current_time < flag_data["expires_at"]:
        return True
        
    return False

Writing Alerts Humans Actually Read

Infrastructure notifications often fail because they optimize for machine parsing rather than human comprehension. Sending raw JSON dumps or raw status codes trains engineers to filter out alerts. A robust fallback reporting system replaces machine payloads with deterministic prose templates.

Instead of dispatching stack traces, the notification service maps known error shapes to human causes. For instance, a connection timeout combined with a specific gateway response gets translated into a clear sentence explaining that the upstream provider is experiencing degraded performance.

Furthermore, auto-clearing events do not require a dedicated notification. When the three-hour window expires and execution resumes, the transition is recorded as ordinary data in the next scheduled summary digest. This keeps communication channels quiet during self-healing incidents while preserving historical visibility.

Conclusion

Reliable background automation relies on bounding manual intervention and respecting human attention limits. By combining self-expiring timestamps with plain-language fallback reports, engineering teams can stop bleeding during provider outages without trapping their systems in permanent manual locks or drowning in unreadable alerts.

Publishing Threads Carousels from Worker-Hosted Images

Thu, 08 Oct 2026 18:41:04 +0000

When you want to publish multi-image carousels to Threads through its API, every item requires a publicly downloadable image URL. Traditional architectures solve this by uploading generated or processed assets to an object storage service like Amazon S3 or Cloudflare R2 before making API requests. If you are running your application entirely within a serverless worker, adding a separate storage bucket introduces extra infrastructure overhead, credentials management, and recurring billing for assets that an external platform downloads only once. This guide explores how to host carousel images directly inside a Cloudflare Worker using KV storage, bypass traditional object storage, and manage the multi-step publishing pipeline successfully. For a related implementation, see Running A Zero Cost Social Auto.

The Missing Image Problem in Serverless

Serverless workers excel at handling stateless API requests, executing webhook logic, and routing web traffic on the edge. However, they lack a persistent filesystem or a static file server. You cannot simply drop generated images into a public folder inside your worker project because the runtime environment evaluates code on demand rather than serving static assets from disk.

To bridge this gap, you need a mechanism to store binary data temporarily or semi-permanently within your existing serverless ecosystem and expose it via a public URL that Meta can reach. While external buckets are the default recommendation, small-scale or ephemeral workloads often do not justify the added configuration overhead.

KV as a One-Download CDN

Cloudflare KV allows you to store key-value pairs at the edge. By treating KV as a rudimentary object store, you can write binary image bytes directly into a key and serve them through a custom worker route such as /img/<asset-name>.

When storing images in KV, keep in mind the platform limits regarding value sizes. For typical compressed JPEGs or WebP thumbnails, individual images easily fit well within the free-tier value cap. When a request hits your public route, the worker fetches the binary payload from KV and responds with the appropriate MIME type and caching headers.

export default {
  async fetch(request: Request, env: Env): Promise<Response> {
    const url = new URL(request.url);
    if (url.pathname.startsWith('/img/')) {
      const imageName = url.pathname.split('/')[2];
      const imageBuffer = await env.IMAGE_KV.get(imageName, 'arrayBuffer');
      
      if (!imageBuffer) {
        return new Response('Not found', { status: 404 });
      }
      
      return new Response(imageBuffer, {
        headers: {
          'Content-Type': 'image/jpeg',
          'Cache-Control': 'public, max-age=86400'
        }
      });
    }
    return new Response('Endpoint not found', { status: 404 });
  }
};

Before initiating any interactions with the Threads API, verify your route manually. Run a curl command to ensure the endpoint returns an HTTP 200 status, the correct content type header, and a non-zero byte count. If your server cannot download the image reliably, Meta servers will fail as well.

The Carousel Pipeline and Processing Wait

Publishing a carousel on Threads is not a single API call. It requires a strict sequence of container creation steps, server-side processing, and final publication. Skipping the required waiting period between container creation and publishing is the most frequent point of failure.

The publishing workflow consists of four distinct phases:

Item Containers: Create an individual media container for each image in your carousel, passing the public worker URL in the request parameters.
Processing Gap: Wait for Meta to download, validate, and process each image server-side. Publishing immediately after container creation results in processing errors because the remote servers have not finished ingesting the assets.
Carousel Parent: Create a parent container that references the child item container IDs.
Publishing: Trigger the final publish action on the parent container to push the carousel live to your profile.

Because status field names and readiness indicators can vary across API versions, implementing a reliable time-based buffer of roughly sixty seconds after container creation provides a pragmatic safety margin before executing the final publish request.

Managing Generative Visual Quality

When your workflow involves generating visuals programmatically before publishing them in a social media carousel, text rendering remains an ongoing challenge. Modern image generation models frequently attempt to render unintended words, letters, or gibberish characters into visuals unless explicitly instructed otherwise.

Even when prompt instructions explicitly state to avoid text entirely, stochastic models occasionally leak artifacts. Incorporating a human review gate into your publishing loop ensures that every generated asset is visually inspected before it enters a carousel container. Automated pipelines should handle byte storage, container creation, and API orchestration, while human verification safeguards final quality.

Summary of Worker-Hosted Carousel Publishing

By leveraging Cloudflare KV as a lightweight storage layer and enforcing a strict processing delay between container creation steps, you can publish Threads carousels entirely from within a serverless worker. This approach removes the need for external object storage buckets while maintaining full control over asset delivery and visual quality.

Measuring Scheduled Job Reliability

Thu, 08 Oct 2026 09:04:08 +0000

A scheduled workflow can start successfully, finish with no eligible work, or commit an article that is not yet live. Counting all three as successes turns a reliability percentage into a trigger counter. Long-running and manually triggered executions also distort the denominator unless they are labeled separately. The useful question is not simply whether the scheduler fired. It is whether the promised user-visible result became available, and how often that happened across scheduled opportunities.

Defining the Denominator and Success Event

To calculate an honest reliability rate, you must first define what constitutes an opportunity. If a cron trigger fires every hour, every one of those scheduled slots is a denominator entry. A slot where the system finds no work to process is still an opportunity. Omitting empty slots from the denominator quietly inflates your success percentage. For a related implementation, see Viewmodel Partial Success Refresh Failure.

Next, tie success to the final user-visible outcome rather than an intermediate API response. If a serverless function initiates a workflow that commits data to a database, success requires the record to be queryable at its canonical public endpoint. If the database write succeeds but a subsequent webhook notification fails, the core publishing action still succeeded. Separating core delivery from secondary notifications prevents false failures from dragging down your reliability metric. For a related implementation, see Designing A Serverless Ai Publishing Workflow.

Handling Manual Triggers and Dry Runs

Operational workflows often require manual intervention, testing, or dry runs. These paths must be isolated from automated metrics:

Automated scheduled runs populate the primary reliability denominator.
Manual live runs are tracked separately to measure operator-initiated actions without polluting automated cron statistics.
Dry runs are excluded entirely because they make no production attempt and produce no live outcome.

Failing to separate these paths leads to misleading reports where a heavy day of manual testing masks an underlying cron failure or vice versa.

Idempotent Ledger Schema and State Tracking

Using a versioned SQLite database like Cloudflare D1 allows you to record execution states reliably. Each scheduled slot receives a deterministic, idempotent run identifier. When the scheduler triggers, the application writes an initial RUNNING marker before starting any asynchronous work. If starting the workflow fails outright, that same row is finalized as a miss. A marker that remains open past a bounded timeout is also classified as a failed run.

Below is an example schema migration demonstrating how to store these execution states while keeping records compact and sanitized.

CREATE TABLE job_reliability_ledger (
    run_id TEXT PRIMARY KEY,
    scheduled_time TEXT NOT NULL,
    execution_type TEXT NOT NULL,
    status TEXT NOT NULL,
    blocker_category TEXT,
    created_at TEXT DEFAULT CURRENT_TIMESTAMP
);

CREATE INDEX idx_scheduled_time ON job_reliability_ledger(scheduled_time);

When writing updates, use an upsert strategy or check the existing status to ensure replaying a terminal update does not create duplicate rows or alter a verified success timestamp.

Operational Reporting Considerations

When displaying reliability metrics to an engineering team, context matters as much as the raw percentage. Always state the data-collection start date next to every rate so partial history is not mistaken for lifetime performance. Calculate summaries using the audience local timezone rather than forcing UTC conversion on operations teams. Store only sanitized blocker categories, such as timeout or validation error, instead of raw provider replies or secrets.

Conclusion on Measuring Reliability

Accurate reliability measurement requires discipline around what you count as an attempt and what you verify as a success. By separating scheduled opportunities from manual tests, anchoring your denominator to every cron slot, and verifying real user-visible outcomes, you build an operational view that reflects actual system health rather than optimistic trigger counts.

Fixing KV Propagation Delay in Media Publishing

Wed, 07 Oct 2026 06:02:44 +0000

When you build a serverless media pipeline that stores generated images in a key-value store and immediately sends those public URLs to a third-party publishing API, you will likely encounter a frustrating failure mode. Your local tests pass, your terminal curl requests return the correct image bytes with a 200 OK status, yet the publishing API responds with a cryptic media not found error. This happens because your own worker reads from the exact edge location that just wrote the key, while the platform fetcher hits a different edge node that has not received the replicated data yet. For a related implementation, see Viewmodel Partial Success Refresh Failure.

Green Locally, Dead Remotely

The most deceptive part of this issue is its misleading diagnostic feedback. When publishing fails, the natural instinct is to examine the container ordering, parameter payloads, or authentication tokens. Developers often curl the generated public URL from their own workstation or test runner, receive the correct MIME type and image bytes, and conclude that the serving layer works fine. This self-verification is a trap. Your edge succeeding proves only that one specific node has the data. It provides zero visibility into whether the global key-value fabric has synchronized across all regional datacenters.

Whose Edge Is It Anyway

Global key-value stores prioritize low-latency writes and eventual consistency. When your worker executes a put operation, the runtime confirms success once the local write succeeds. Replication to other regions happens asynchronously in the background. If a third-party platform attempts to download the asset immediately after receiving the URL, its request routes to an edge server near its own infrastructure. If that specific server has not received the update, the fetch fails. Because distributed platforms do not expose their internal replication state or routing tables, you cannot poll their edge directly to check availability. You must instead build a defensive timing model into your application code.

Two Waits, Two Handoffs

Solving this problem requires separating your publishing pipeline into distinct phases, each guarded by its own deliberate delay. Conflating these phases leads to brittle workarounds.

The Propagation Wait: After storing the asset in key-value storage, introduce a measured pause to allow global replication. This gives remote edge locations time to catch up before any external system attempts a download.
The Processing Wait: After the third-party platform initiates its import or container creation process, allow an additional window for the platform to process the file before you invoke the final publish command.

Hardcoding these delays into your core logic makes your test suite painfully slow. Instead, make your wait durations configurable by injecting milliseconds during testing while using production-appropriate values for live deployments. Keeping these constants clearly named in your source code prevents future maintainers from accidentally optimizing them away.

Implementing the Fix

To manage these handoffs safely, structure your asynchronous worker code to wait explicitly between storage and publishing steps. Here is an example configuration pattern in TypeScript that uses injected wait durations:

interface PublishConfig {
  propagationDelayMs: number;
  processingDelayMs: number;
  maxRetries: number;
}

async function storeAndPublishMedia(
  kv: KVNamespace,
  key: string,
  data: ArrayBuffer,
  config: PublishConfig
): Promise<string> {
  await kv.put(key, data);
  
  // Guard against global edge replication lag
  await new Promise((resolve) => setTimeout(resolve, config.propagationDelayMs));
  
  const mediaId = await initiatePlatformUpload(key);
  
  // Guard against remote processing lag
  await new Promise((resolve) => setTimeout(resolve, config.processingDelayMs));
  
  return await finalizePublish(mediaId);
}

Verifying the Publish Object

Never rely on your own curl command as proof of a successful publish operation. True success is defined solely by the response of the downstream publishing API. When the platform successfully fetches the asset from its local edge, processes it, and returns a valid media object with attached children or confirmation IDs, your pipeline has succeeded. If the call fails, adjust your propagation budget upward or inspect the platform error logs for download timeouts.

Final Thoughts

Eventual consistency models introduce hidden timing assumptions into modern serverless architectures. When your application hands off assets to external consumers, assuming instant global visibility will cause intermittent failures. By acknowledging edge propagation realities, separating your storage waits from your processing waits, and verifying platform objects rather than local URLs, you can make your media publishing pipelines resilient to distributed latency.

Bounded LLM Fallback Chains

Wed, 07 Oct 2026 03:03:52 +0000

When a primary AI provider experiences an outage or a rate limit in the middle of the night, applications often suffer from unexpected downtime. The common reaction is to implement a multi-provider failover chain. However, standard fallback loops create a different kind of emergency: runaway billing events. Unbounded retries across multiple models can exhaust available budgets in minutes during a regional outage.

To prevent failover mechanisms from becoming cost incidents, systems need strict boundaries. A production-ready fallback tier must exhaust the primary provider completely before switching, enforce a strict per-run call cap, treat billing errors as permanent stop signs, and exclude fallbacks from routine dry runs.

The Cost of Unbounded Retries

Chaining multiple language model providers without constraints introduces several hidden failure modes. Without a defined architecture, an outage triggers cascading problems.

Unbounded failover burns capital rapidly. Each fallback call consumes tokens or credits. If a system cascades through four expensive models without a cap, a single failing background job can drain a monthly budget. Furthermore, applications often retry quota-exceeded and billing-restricted errors. These responses are permanent verdicts from the provider, not transient network glitches. Retrying them only multiplies the failure rate and increases error logs.

Another subtle issue involves testing and dry runs. If health checks or dry runs execute the entire fallback path, they consume the scarce fallback quota that production systems rely on during actual emergencies. Finally, silent provider drift occurs when a primary model fails permanently, but the system quietly runs on the expensive fallback tier for weeks without alerting the engineering team.

Designing the Provider Tier Contract

To solve these challenges, an LLM routing layer must enforce specific operational constraints.

First, the system must exhaust tier one completely before initiating tier two. Fallback models should never run in parallel or preemptively. Second, total fallback calls per execution run require a hard cap, such as a maximum of four attempts. This bounds the worst-case financial exposure regardless of how many models exist in the registry. For a related implementation, see Audit Macos System Data Before Deleting.

Third, error classification dictates the next step. Infrastructure issues like 503 service unavailable or gateway timeouts advance the chain. Conversely, quota exhaustion, invalid API keys, and paid-only model restrictions act as immediate stop signs. The runner halts execution and records the exact reason rather than proceeding to the next provider.

Fourth, dry runs must execute against the primary path only. Fallback bindings remain dormant during routine checks unless an explicit integration test flag is provided. Fifth, every successful response must record the provider and model name in the output metadata. This attribution makes silent fallback immediately visible in monitoring dashboards.

Implementing a Bounded Fallback Runner

Below is a TypeScript example demonstrating a bounded fallback executor with error classification and call limits.

type ProviderResult = {
  content: string;
  provider: string;
  model: string;
};

enum ErrorType {
  Transient,
  PermanentBilling,
}

interface Provider {
  name: string;
  model: string;
  execute(prompt: string): Promise<string>;
}

function classifyError(err: any): ErrorType {
  if (err.status === 429 || err.code === 'QUOTA_EXCEEDED') {
    return ErrorType.PermanentBilling;
  }
  return ErrorType.Transient;
}

async.executeWithFallback = async function(
  primary: Provider,
  fallbacks: Provider[],
  prompt: string,
  maxFallbackCalls: number = 2,

): Promise<ProviderResult> {
  try {
    const content = await primary.execute(prompt);
    return { content, provider: primary.name, model: primary.model };
  } catch (primaryError) {
    if (classifyError(primaryError) === ErrorType.PermanentBilling) {
      throw new Error('Primary failed with billing error; halting chain.');
    }

    let callsMade = 0;
    for (const fb of fallbacks) {
      if (callsMade >= maxFallbackCalls) {
        break;
      }
      callsMade++;

      try {
        const content = await fb.execute(prompt);
        return { content, provider: fb.name, model: fb.model };
      } catch (fbError) {
        if (classifyError(fbError) === ErrorType.PermanentBilling) {
          throw new Error('Fallback failed with billing error; halting chain.');
        }
      }
    }

    throw new Error('All providers exhausted without successful execution.');
  }
};

Verifying Fallback Behavior Safely

Proving that a fallback chain works without triggering a real production outage requires targeted unit and integration tests. Tests should simulate a complete primary failure and assert that the fallback tier engages exactly once. Additional test cases must verify that dry runs never invoke the fallback binding unless explicitly enabled, that hitting the per-run call cap throws an exception instead of executing unauthorized calls, and that billing-class errors halt the chain immediately with a documented stop reason.

Architectural Conclusion

Reliable AI infrastructure requires treating failover as a finite budget rather than an open-ended loop. By classifying provider errors, enforcing hard call limits, and maintaining strict attribution, engineering teams can build resilient multi-provider runtimes that withstand outages without risking financial exposure.

Reads That Survive the Kill Switch

Mon, 05 Oct 2026 09:02:25 +0000

When an unattended cron worker misbehaves, operators typically engage a kill switch to halt all execution. This stops harmful mutations, but it frequently blinds the operations team as well. While the switch is active, nobody can see whether requests are piling up, and the team discovers the backlog only after the system recovers. This occurs because writes and reads often share a single gate. The disabled flag blocks publishing and the demand check together, incorrectly conflating the instruction to stop acting with the need to observe system state.

A kill switch that stops a misbehaving cron worker also blinds it. While engaged, nobody can see whether customers are waiting, so the team discovers a backlog only after recovery, or never. Writes and reads share one gate, meaning the disabled flag blocks publishing and the demand check together. Silent slots hide demand because a disabled result with no context reads identically whether zero or fifty replies wait. Recovery starts blind when the flag clears, as the first healthy slot discovers the backlog by accident instead of by report. For a related implementation, see Schema First Gates Ai Publishing Pipelines.

The Peek Pattern

To keep visibility intact during an outage, systems need a separation of concerns between looking at the queue and modifying it. The peek pattern addresses this by placing read operations ahead of any mutation gates. The workflow executes three distinct phases:

Token validation: Load credentials first. If authentication fails, the run stops immediately.
The peek: Run the read-only demand check to count pending items without altering state.
Gate evaluation: Check the kill switch status. If active, skip writing while preserving the read data.

The read operation must never throw, write data, or trigger external notifications. Even if the underlying data store experiences intermittent pressure, the worst-case return value should be an empty count rather than a failed execution slot. For a related implementation, see Duplicate Push Notifications Fcm.

Demand on Every Result

A common monitoring flaw is reporting demand only when tasks succeed. When a kill switch is active, disabled slots often return a silent empty response with no context. Operators cannot tell whether zero tasks are waiting or fifty items are sitting in a blocked queue.

The solution is to attach the pending count to every possible execution outcome. Whether a slot is disabled, skipped, failed, or successfully published, the log entry or audit digest should carry the exact pending count. This guarantees that observability logs show continuous pressure trends even when the system is entirely fenced off from writing.

Answering Stays Gated

Visibility into demand must never weaken the security boundary of the kill switch. Observing the queue does not grant permission to process it. Publishing answers and executing outbound calls must remain safely behind every existing guard, including duplicate detection entries, budget caps, and audit requirements.

Consider this simplified TypeScript configuration pattern showing how a worker separates the read peek from the write action:

type RunResult = { status: string; pendingCount: number };

async function handleCronJob(isKillSwitchActive: boolean): Promise<RunResult> {
  const pendingCount = await fetchPendingDemandSafely();
  
  if (isKillSwitchActive) {
    return { status: "fenced", pendingCount };
  }

  await executeWrites();
  return { status: "published", pendingCount };
}

async function fetchPendingDemandSafely(): Promise<number> {
  try {
    return await getQueueDepth();
  } catch {
    return 0;
  }
}

In this model, the fetchPendingDemandSafely function wraps calls in a try-catch block to guarantee it never throws an error that could abort the inspection. The pending count is captured regardless of whether the kill switch is active.

Maintaining Visibility During Outages

Separating read operations from write mutations prevents the blind spots that typically accompany emergency interventions. By ensuring that demand checks execute independently of write gates, and by attaching pending counts to every outcome, engineering teams can size their recovery efforts accurately the moment restrictions are lifted.

When Gemini Rejects Cloudflare Workers by Location

Sun, 04 Oct 2026 15:45:20 +0000

When a scheduled worker calls the Gemini API around the clock, an unexpected failure can disrupt pipelines for hours. During an outage, error monitoring dashboards often show a mix of failures that appear to stem from a single overarching problem, yet they require opposite responses. Distinguishing transient overloads from static geo-blocking when using serverless egress IPs is essential to keeping automated workflows running without unnecessary provider migrations or wasted retry cycles. For a related implementation, see Normalize Cloudflare Workflows Trigger Payloads.

Everything Fails at Once

In a recent debugging session with the Morgans provider integration, scheduled workers executing on serverless edges began encountering widespread errors. The dashboard showed a wall of failed requests with status codes that looked like a standard capacity outage. However, closer inspection of the raw error payloads revealed that two entirely distinct failure modes were occurring simultaneously within the same incident window.

Understanding this behavior requires looking at how edge platforms route traffic. Serverless execution environments route requests through rotating egress pools. When an upstream provider updates its regional enforcement rules or experiences capacity shifts, certain egress IPs may trigger geographic restrictions while others continue to pass through normally. Without inspecting the specific error message returned by the provider, operations teams risk misdiagnosing the root cause entirely.

Two Errors, Opposite Cures

During the incident, two specific error signatures appeared in the logs. The first was a transient capacity issue, while the second was a deterministic regional rejection.

The first signature returned a standard HTTP 503 status code with a payload message indicating high demand. This error is temporary. It clears with time, and standard exponential backoff retries are the correct operational response. Forcing a migration away from the primary provider for a temporary demand spike introduces unnecessary complexity.

The second signature returned an HTTP 400 status code accompanied by a message stating that the user location is not supported for the API use. This error is deterministic per egress IP. It never clears with retries, and treating it as an overload wastes the entire incident window waiting for a recovery that will not happen on that IP address.

Treating a geo-block as an overload wastes valuable time, while treating an overload as a geo-block triggers an unnecessary provider migration. Correlating failures solely by clock time or status code makes two different problems look like one.

Routing Around Geography

When a geo-block is confirmed, waiting for it to resolve is not a viable architecture. The structural answer involves routing around the geographic restriction by promoting an in-platform fallback tier. For example, routing execution to a serverless model inference engine running on the same edge network avoids external region-gating rules entirely.

To implement this safely, the fallback tier must be exercised against live traffic regularly. A fallback that never runs is merely a hope rather than a reliable tier. Verifying response-shape handling across both the primary provider and the fallback ensures that switching over during an incident does not introduce secondary parsing failures.

Log Bodies, Not Codes

Status-only logging is the primary reason why teams misdiagnose edge routing failures. Relying solely on HTTP status codes obscures the difference between a demand spike and a regional policy rejection. Capturing a short snippet of the response body, such as the first 160 characters of an error payload, provides the exact context needed to automate routing decisions.

Consider a minimal JavaScript implementation running in an edge worker that inspects the provider response before deciding whether to retry or fail over:

async function callGeminiWithFallback(requestData) {
  try {
    const response = await fetch("https://generativelanguage.googleapis.com/v1beta/models/gemini-pro:generateContent", {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify(requestData)
    });

    if (!response.ok) {
      const errorBody = await response.text();
      const snippet = errorBody.substring(0, 160);

      if (response.status === 400 && snippet.includes("user location is not supported")) {
        return await executeInPlatformFallback(requestData);
      }

      if (response.status === 503) {
        throw new TransientDemandError("Model experiencing high demand: " + snippet);
      }

      throw new Error("Unhandled provider error: " + snippet);
    }

    return await response.json();
  } catch (error) {
    if (error instanceof TransientDemandError) {
      return executeWithBackoff(requestData);
    }
    throw error;
  }
}

This pattern ensures that geographic rejections trigger an immediate switch to the secondary tier, while transient capacity limits correctly trigger bounded retries.

Conclusion

Managing external model providers from serverless edge environments requires looking past simple status codes. By capturing error response bodies, distinguishing static geo-blocking from temporary capacity overloads, and maintaining a warm fallback tier, engineering teams can build resilient integrations that survive unexpected regional policy shifts without manual intervention. For a related implementation, see Managing Gemini Overload Intelligent Fallback Patterns.

Demystifying 'Cold Starts' in Serverless: Why Your App Sometimes Shivers

Wed, 02 Sep 2026 11:40:57 +0000

Have you ever clicked a button on a website, only to wait several agonizing seconds for the page to respond, even though it usually loads instantly? This annoying delay is often caused by a backend phenomenon known as a cold start. In serverless computing, a cold start is the delay that occurs when a cloud function is invoked for the first time or after a period of inactivity. Because there is no constantly running server, the cloud provider must spin up a new virtual container, load your code, and initialize the environment before running the function.

The Winter Car Analogy

Imagine driving your car in the dead of winter. If the car has been sitting in your driveway for days, you cannot just hop in and immediately speed down the highway. First, you must start the cold engine, wait for the fluids to warm up, and let the defroster clear the windshield—this initialization process is your "cold start."

However, if you drive to the grocery store, spend ten minutes shopping, and return to the car, the engine is still warm. You turn the key, put it in drive, and immediately head home. In the cloud, a "warm" function acts just like that warm car: it is already running in memory and ready to process your request instantly.

Why It Matters in the Tech Industry

Serverless computing is incredibly popular because engineers only pay for the exact milliseconds their code is executing, rather than renting an expensive virtual server to sit idle 24/7. However, if an e-commerce website experiences a sudden surge of traffic, or if a user accesses a rarely used feature, they will trigger a cold start.

This results in sudden, unpredictable lag spikes. If a customer clicks "Proceed to Checkout" and experiences a five-second cold-start delay, they might assume the website is broken and abandon their shopping cart. Software engineers must deeply understand cold starts to balance infrastructure cost-savings with a fast, reliable user experience.

Optimizing for Cold Starts in Node.js

To minimize cold start times, developers write code that initializes heavy resources (like database connections) outside the main execution handler. This ensures the heavy work is only done once during the "cold start," while subsequent "warm" requests bypass it entirely.

// 1. GLOBAL SCOPE: Runs ONLY during a cold start.
// This heavy database connection is established once and kept in memory.
const dbConnection = connectToDatabase();

exports.handler = async (event) => {
  // 2. HANDLER SCOPE: Runs on EVERY single request.
  // Warm executions skip the global setup above and run this instantly.
  const userId = event.pathParameters.id;
  const user = await dbConnection.find(userId);
  
  return {
    statusCode: 200,
    body: JSON.stringify({ user })
  };
};

The Takeaway

Cold starts are the natural "tax" of serverless architecture, representing the unavoidable trade-off between absolute cost efficiency and instant responsiveness. By optimizing how your code initializes, keeping your deployment packages small, and strategically choosing your runtime environment, you can harness the scale of the cloud without leaving your users waiting in the cold.

Resources

GitHub Repository: react-hook-lab
react-hook-lab: npm package
Connect with me on LinkedIn: Saurav Pandey

Originally published on my blog. You can read the alternative breakdown here.

# Building a Serverless-Safe Realtime Metaverse for MyZubster

Tue, 01 Sep 2026 09:31:49 +0000

Building a Serverless-Safe Realtime Metaverse for MyZubster

We have completed a major infrastructure upgrade for MyZubster World.

The original realtime system used Server-Sent Events and in-memory state. It worked locally, but serverless deployments introduced a problem: different requests could reach different Vercel instances, each with its own memory.

The result? Players could join successfully while the interface remained stuck on RECONNECTING.

What we changed

We moved the shared Metaverse state to MongoDB.

The new architecture supports:

shared player presence across serverless instances;
synchronized movements and emotes;
recent shared chat messages;
automatic expiration of inactive sessions with TTL indexes;
resilient browser synchronization;
automatic recovery after temporary connection failures;
verified MyZubster characters linked to authenticated accounts;
clear separation between verified users and guests.

The React frontend now periodically calls:

POST /api/metaverse/sync
Each synchronization refreshes the session and returns the current players, movements, emotes, and new chat messages.
This replaces the fragile dependency on a long-lived EventSource connection.
Verified identity
Authenticated users enter with the character linked by the server to their MyZubster account.
Browser-supplied names and identifiers cannot override a verified identity.
For example, the character H4x0r is displayed as:
H4x0r
@DanielIoni-creator
MYZ VERIFIED
Guests can still explore the world, but they are clearly marked as guests.
Production results
The implementation was successfully deployed and verified:
- Vercel deployment: ready;
- world API: HTTP 200;
- active transport: shared-polling;
- MongoDB shared presence: operational;
- legacy Metaverse EventSource removed from the frontend;
- 11 automated tests passed.
This architecture gives MyZubster a reliable serverless foundation today while keeping the door open for WebSockets, Redis, or a dedicated realtime network in the future.
Explore MyZubster World:
🌍 https://www.myzubster.com/metaverse
View the implementation:
💻 https://github.com/MyZubster-Ecosystem/myzubster/commit/412babfedfe29a917e5c6802c34defdcbfad75e7
What approach do you prefer for realtime serverless applications: polling, WebSockets, Redis streams, or an external realtime provider?
#MyZubster #OpenSource #Metaverse #ReactJS #NodeJS #MongoDB #Serverless #WebDevelopment #Realtime #Vercel

What happens when your migration agent dies at file 7 of 12

Fri, 31 Jul 2026 16:57:25 +0000

A migration agent spends real money and real minutes per run. It clones a repo, installs its deps, plans a file order, then rewrites files one at a time, running the target's test suite after each one. On a mid-sized repo that is a lot of Bedrock calls and a lot of wall-clock time.

So the question I had to answer early: what happens when the process dies at file 7 of 12?

If the answer is "start over," the agent is a toy. You have already paid for six migrated files and a passing test suite on each, and a spot reclamation or a redeploy throws all of it away. Worse, the API side runs on Lambda, which has a hard 15-minute ceiling. A real migration blows past that. So the long-horizon work cannot live where the request comes in, and the thing doing the work has to survive being killed.

Here is how RepoModernizer handles it, and the bug that resume introduced that I did not see coming.

The split: API on Lambda, worker on Fargate

The API is a FastAPI app behind API Gateway, running on Lambda through Mangum. It takes the request, writes nothing important, and drops a message on SQS. That is it. It never runs the migration.

An SQS consumer Lambda reads the message and calls ecs.run_task to launch a one-shot Fargate task. The Fargate task is the worker. It runs the LangGraph pipeline to completion (or until it dies) and then exits. Nothing stays warm. Idle cost is zero because there is nothing idle.

# consumer_handler.py — Lambda that turns an SQS message into a Fargate run
_ecs.run_task(
    cluster=os.environ["ECS_CLUSTER"],
    taskDefinition=os.environ["ECS_TASK_DEFINITION"],
    launchType="FARGATE",
    overrides={
        "containerOverrides": [{
            "name": "worker",
            "environment": [
                {"name": str(k).upper(), "value": str(v)}
                for k, v in body.items() if v is not None
            ],
        }]
    },
)

The whole reason the worker is on Fargate and not Lambda is that 15-minute cap. Lambda is fine for the request. It is not fine for a job that might run half an hour.

Two kinds of durable, in two different places

The worker has two things it cannot afford to lose when it dies: the repo it is editing, and the graph state that says how far it got. These live in two different stores, on purpose.

The repo lives on EFS, mounted into the Fargate task at /mnt/workspace. The task clones the target repo there, makes a branch, and every diff it applies lands on that filesystem. When a new task picks up the same job, the files are already sitting there in the state the last run left them.

The graph state lives in DynamoDB. Every node the LangGraph pipeline finishes gets written as a checkpoint. That is what lets a fresh process know it already ingested, already installed deps, already migrated files 0 through 6, and should resume at file 7.

Splitting them matters. DynamoDB holds the plan, the cursor, and the per-file status. EFS holds the actual bytes. Neither one alone is enough to resume. You need the state to know where you were and the filesystem to have the work you already did.

The checkpointer is a real LangGraph saver, not a side table

LangGraph has a checkpointer interface, BaseCheckpointSaver. The out-of-the-box options are an in-memory saver (useless across processes) and integrations you may or may not want to run. I wrote one backed by DynamoDB directly, because DynamoDB was already in the stack and I wanted single-table access patterns I controlled.

The core is put and get_tuple. put serializes the checkpoint with LangGraph's own JsonPlusSerializer and writes it under a per-task partition key:

def put(self, config, checkpoint, metadata, new_versions):
    thread_id = config["configurable"]["thread_id"]
    checkpoint_id = checkpoint["id"]
    parent_id = config["configurable"].get("checkpoint_id")
    self._put_blob(
        f"TASK#{thread_id}", f"CKPT#{checkpoint_id}", (checkpoint, metadata),
        extra={"parent_checkpoint_id": parent_id or ""},
    )
    return {"configurable": {
        "thread_id": thread_id,
        "checkpoint_ns": config["configurable"].get("checkpoint_ns", ""),
        "checkpoint_id": checkpoint_id,
    }}

get_tuple does the reverse. Given a task id and no specific checkpoint, it queries the latest one:

resp = self._table.query(
    KeyConditionExpression=Key("PK").eq(f"TASK#{thread_id}")
        & Key("SK").begins_with("CKPT#"),
    ScanIndexForward=False, Limit=1,
)

ScanIndexForward=False, Limit=1 gives the most recent checkpoint for that task. On cold start the worker builds the graph with this checkpointer attached, and LangGraph replays from the last saved node. Every checkpoint carries a 14-day TTL so old task state ages out of the table on its own.

The graph build is where the checkpointer gets wired in:

graph = build_graph(deps, checkpointer=checkpointer)
config = {"configurable": {"thread_id": task_id}}

thread_id is the task id. That single value ties a resuming process back to everything the previous process wrote.

What "resume" actually does, and what it does not

I want to be precise here, because it is easy to oversell this.

Resume works at the node level, not the byte level inside a file. If the worker dies while migrating file 7, it does not continue from the middle of file 7. On the next run, the loop restarts the migrate_file node for the current cursor from the top: it re-reads the file off EFS, asks the model for fresh content, applies the diff, and reruns tests. Files 0 through 6 are already marked done in the checkpointed state, so it does not touch them. It picks up at the current cursor and moves forward.

So "resumes mid-migration" is true in the sense that matters (you do not redo six files of completed, test-passing work), and false in the sense that a single interrupted file gets redone from scratch. That is the honest version.

The bug: resume, done twice, opens the same PR six times

The pipeline has a human-in-the-loop gate. When the planner scores a file's risk above a threshold, the migrate_file node calls LangGraph's interrupt() with the diff, and the run pauses. A human approves or rejects. Approval comes back in as a separate action that resumes the graph with Command(resume=...).

Here is the thing I missed. That migrate_file node re-executes from its top on every resume. Re-reads the model, re-applies, re-runs tests. That is fine when there is genuinely a pending interrupt to satisfy. It is not fine when the resume action fires twice.

And it fires twice more often than you would think. SQS can deliver a message more than once. A user can click approve again while a slow Fargate cold start has not yet updated the checkpoint, so the UI still shows a pending decision. Either way, a second resume lands for a task whose interrupt was already cleared by the first one.

Calling invoke() again does not no-op. It runs the node fresh, produces a new diff, applies it, and drives the run all the way to finalize again, which opens a pull request. I found this live: one approve click produced six to nine identical commits on the PR.

The first fix was to check whether there is actually a pending interrupt before resuming:

snapshot = graph.get_state(config)
has_pending_interrupt = any(t.interrupts for t in snapshot.tasks)

That helped. It did not fix it. Two approve calls sent about a second apart still produced one extra commit, not zero. The reason is that get_state then invoke is check-then-act. Two callers landing in the same instant both read has_pending_interrupt = True before either one resumes. Classic race.

The real fix is atomicity, and DynamoDB is the only thing in this system that offers it. I added a try_claim that does a conditional put: the write only succeeds if the item does not already exist.

def try_claim(self, task_id: str, key: str) -> bool:
    try:
        self._table.put_item(
            Item={"PK": f"TASK#{task_id}", "SK": f"CLAIM#{key}",
                  "ttl": int(time.time()) + _TTL_SECONDS},
            ConditionExpression="attribute_not_exists(PK)",
        )
        return True
    except ClientError as exc:
        if exc.response["Error"]["Code"] == "ConditionalCheckFailedException":
            return False
        raise

The key is the checkpoint id being resumed. Only the first caller to claim that specific checkpoint proceeds to invoke. The second gets False and returns the existing state without re-running anything.

claimed = checkpointer.try_claim(task_id, f"resume:{checkpoint_id}")
if has_pending_interrupt and claimed:
    result = graph.invoke(Command(resume={...}), config=config)
else:
    result = snapshot.values

There is a second guard on the far side too, at finalize: before opening a PR, check whether this task already has a PR URL recorded, and bail if it does. Belt and suspenders, because the cost of a duplicate PR is annoying and public.

The other thing that broke: git, EFS, and root

Two smaller landmines, both worth knowing if you mount EFS into a container that runs git.

The EFS access point pins every file to uid 1000 no matter who writes it. The container runs as root. Git 2.35.2 and up refuses to operate on a repo it sees as owned by a different user (that is the CVE-2022-24765 dubious-ownership check), so the first git command after the clone blows up. Fix is one line at worker startup:

subprocess.run(["git", "config", "--global", "--add", "safe.directory", "*"], check=True)

The second one showed up only on JS repos. The diff step normalizes the target file to end in a newline before diffing. git apply matches context against real bytes, and difflib never emits the "No newline at end of file" marker, so a file without a trailing newline fails to apply cleanly. Python source almost always ends in a newline, so my Python fixtures never caught it. Hand-written JS and JSON often do not, and it broke against a real JS repo.

What I would do differently

The duplicate-commit bug is the interesting one, and the lesson is not "add a claim." The lesson is that I built idempotency in the wrong layer first. I reached for a state check (has_pending_interrupt) when the problem was concurrency, and a state check cannot solve concurrency. Only a conditional write can. If I were doing it again I would put the atomic claim in from the start, at every resume boundary, and treat the state check as a cheap early-out rather than the guarantee.

The other thing: migrate_file re-running from the top on resume is convenient but wasteful. It re-pays for a Bedrock call on a file it may have already generated. A cleaner design caches the generated content against the checkpoint so a resume that is really just a duplicate delivery costs nothing. I have not built that yet.

Durability is not one feature. It is durable state, durable storage, and idempotent resume, and the last one is the one that bit me.

The cost of a serverless agent isn't the Lambda. It's the loop.

Fri, 17 Jul 2026 05:46:41 +0000

When I moved Wingman onto Lambda, I assumed the thing I'd have to watch was compute duration. That's the serverless mental model everyone starts with: you pay per millisecond, so keep the function fast and cheap.

That model was wrong for an agent. The Lambda bill turned out to be the boring, predictable part. The two things that actually move cost and latency are the cold start and the retry loop, and neither of them shows up where you'd look first.

Here's what the code taught me.

The setup: what Wingman actually is

Wingman is a LangGraph worker-evaluator agent. A worker node (running gpt-4o-mini) does the task. An evaluator node (also gpt-4o-mini) grades the answer against a success criteria. If the criteria isn't met, control routes back to the worker for another attempt, up to five times.

It runs as a container image on Lambda, fronted by API Gateway and CloudFront, with session history in DynamoDB. The compute is stateless: every request rebuilds the conversation from DynamoDB, so there's no persistent checkpointer to worry about across cold and warm starts. I wrote about that state design in an earlier post. This one is about what it costs to run.

Cold start #1: the image

A Lambda container cold start begins by unpacking the image. Wingman's image carries langchain, langgraph, langchain-community, langchain-experimental, boto3, plus reportlab and sendgrid for the agent's tools. That's not a small pile of Python.

The one decision I'm glad I made is in the Dockerfile. The project has a local dependency group with playwright, gradio, and uvicorn, and a dev group on top of that. None of those belong in Lambda. So the build exports only the core deps and drops the rest:

RUN uv export \
      --no-dev \
      --no-group dev \
      --no-group local \
      --format requirements-txt \
      -o /tmp/requirements.txt \
    && pip install -r /tmp/requirements.txt --no-cache-dir \
      --target "${LAMBDA_TASK_ROOT}"

Playwright alone would have pulled a browser runtime into the image. Keeping it in the local group means it exists for running the agent on my laptop and never ships to Lambda. Smaller image, faster unpack, faster cold start. This is the kind of thing you only notice you did right when you compare it to the version where you didn't.

Cold start #2: the init you pay for on import

The bigger cold-start cost is in the handler, and it's easy to miss because it happens at import time:

# backend/main.py — runs once per new execution environment
wingman = Wingman()
wingman.setup()

_dynamodb = boto3.resource("dynamodb", region_name=os.getenv("AWS_REGION", "us-east-1"))
_table = _dynamodb.Table(_dynamodb_table_name)

setup() isn't cheap. It builds the tool list, constructs two ChatOpenAI clients, binds tools to the worker, wraps the evaluator in structured output, and compiles the LangGraph:

def setup(self):
    self.tools = get_tools()
    worker_llm = ChatOpenAI(model="gpt-4o-mini")
    self.worker_llm_with_tools = worker_llm.bind_tools(self.tools)
    evaluator_llm = ChatOpenAI(model="gpt-4o-mini")
    self.evaluator_llm_with_output = evaluator_llm.with_structured_output(EvaluatorOutput)
    self._build_graph()

Putting this at module level is the right call. It runs once when a new execution environment spins up, then every warm invocation reuses the same compiled graph and the same clients. If I'd built the graph inside the request handler instead, I'd be paying graph compilation on every single /api/chat call, warm or not. Module-level init trades a heavier cold start for near-zero per-request setup, which is the trade you want when warm invocations vastly outnumber cold ones.

The lever for cold-start speed is memory:

resource "aws_lambda_function" "backend" {
  timeout     = 300   # 5 minutes
  memory_size = 1024
}

On Lambda, CPU scales with memory. At 1024MB you get roughly a full vCPU, so bumping memory doesn't just buy RAM headroom, it makes that import-time init finish faster. Memory is a latency knob here, not only a capacity one.

The real cost: the retry loop, not the duration

Now the part that surprised me.

Look at the timeout: 300 seconds. Lambda's max is 900. That's a generous ceiling for a function whose real job takes a few seconds. So why so high? Because the retry loop can run the worker up to five times, and each turn is a full round of model calls plus whatever tools the worker decides to invoke.

Here's the accounting that matters. turn_count only increments in the evaluator:

def evaluator(self, state: State) -> Dict[str, Any]:
    ...
    return {
        ...
        "turn_count": state.get("turn_count", 0) + 1,
    }

def route_based_on_evaluation(self, state: State) -> str:
    if state["success_criteria_met"] or state["user_input_needed"]:
        return "END"
    if state.get("turn_count", 0) >= self.MAX_TURNS:
        return "END"
    return "worker"

So a single /api/chat request is not one model call. In the worst case it's five worker calls, five evaluator calls, and every tool call the worker makes in between. The Lambda duration for that is trivial to reason about and cheap. The token cost is the thing that swings, and it swings by a factor of five depending on how many times the evaluator sends the work back.

That reframed the whole cost model for me. Moving to stateless compute on Lambda made the infrastructure cost flat and predictable. What it did was push all the cost variance out of the infrastructure and into the model loop. The bill I have to watch isn't Lambda-seconds. It's tokens times retries.

The 29-second cliff

One more thing the config quietly documents. API Gateway has a hard 29-second integration timeout. A five-turn loop with tool calls can blow past that even though the Lambda itself is happily configured for 300 seconds. So the request dies at the gateway while the function is still working.

The fix is in the Terraform: a Lambda Function URL that bypasses API Gateway entirely for long tasks.

resource "aws_lambda_function_url" "backend" {
  function_name      = aws_lambda_function.backend.function_name
  authorization_type = "NONE"
}

Short chats go through CloudFront and API Gateway. Long agent runs go straight to the Function URL and get the full timeout budget. It works, but authorization_type = "NONE" and allow_origins = ["*"] are both flagged in the code as things to tighten after the first deploy, and they're still open. An unauthenticated public URL that invokes a model loop is a bill waiting to happen if someone finds it. That one's on my list.

What I'd do differently

The retry loop has no token budget. It caps at five turns, but a "turn" can be an expensive worker call with several tool invocations, and nothing stops a run from burning far more tokens than the answer is worth. A turn cap is a crude proxy for a cost cap. If I rebuilt this, the evaluator would carry a token budget alongside turn_count, and the loop would stop on whichever it hit first.

Takeaway

For a serverless agent, Lambda duration is the cheap, predictable line on the bill. The cost that actually moves lives in the cold-start init you pay on import and the retry loop you pay per request, so budget those, not the milliseconds.

The Dirty Secret of Cloud Server Scaling!

Sun, 21 Jun 2026 19:22:50 +0000

The True Path to Cloud Scalability: Beyond the Horizontal Band-Aid

Introduction

In the rapidly evolving landscape of cloud computing, the mantra of “scaling horizontally” by simply adding more server instances has long been preached as the ultimate solution for handling increased load. While seemingly effective, this approach often masks a deeper, more pervasive issue: it’s frequently a costly band-aid, not a robust, efficient design. As we look towards 2026, the truly scalable backends are not those over-provisioned with redundant compute, but rather those meticulously crafted with intelligent database management intertwined with a serverless-first backend design. This tutorial will expose the “dirty secret” of cloud scaling and guide you towards building architectures that are not just bigger, but fundamentally smarter, more cost-effective, and performant.

Architectural Layout and Strategy Walkthrough

Moving beyond simplistic horizontal scaling requires a paradigm shift in how we conceive and construct our cloud backends. The “code” for true scalability lies less in boilerplate server logic and more in a strategic architectural layout that leverages cloud-native capabilities.

1. Embrace True Serverless Databases for Dynamic Scalability

The first critical step is to re-evaluate your database strategy. Traditional relational databases, even when hosted in the cloud, often require pre-provisioned capacity that leads to either over-spending or performance bottlenecks.

Conceptual Layout: Instead of defining fixed-size database instances or complex replication clusters you manage, envision a database layer that inherently scales with demand. For example, AWS Aurora Serverless v3 (or similar offerings like Azure Cosmos DB Serverless or Google Cloud Firestore) exemplifies this.
- You define your database schema and initial configuration.
- The database automatically scales compute and memory based on actual load, often down to zero capacity during idle periods.
- Your application’s database connection string remains static, abstracting the underlying scaling complexity.
Strategy: Configure your database to use auto-scaling policies, focusing on metrics like CPU utilization or active connections, but allowing the serverless engine to manage the details. This eliminates the need for manual scaling events and ensures you pay only for the resources consumed.

2. Implement Smart Sharding Based on Actual Access Patterns

Simply distributing data across multiple nodes (sharding) isn’t enough. The intelligence comes from understanding how your data is accessed and partitioning it accordingly to minimize hot spots and optimize query performance.

Conceptual Layout: Consider a microservices architecture where data ownership is clear. Each microservice might interact with its own shard or a specific sharding strategy.
- User Data Service: Shard by user_id to ensure all user-specific data (profiles, preferences) resides on a single shard, optimizing reads and writes for individual users.
- Time-Series Data Service: Shard by date or month to allow efficient querying of data within specific timeframes, often archived to cheaper storage after a period.
- Routing Logic: Implement a lightweight routing layer (e.g., an API Gateway and Lambda function, or a dedicated data access microservice) that intelligently directs queries to the correct shard based on the request’s key or content. This layer abstracts the sharding logic from the core application.
Strategy: Begin with thorough data access pattern analysis. Identify your most frequent queries, the data they touch, and potential hot keys. Use this analysis to design your sharding key and distribution strategy, ensuring even load distribution and minimizing cross-shard queries where possible.

3. Embrace Event-Driven Architectures for Offloading Synchronous Load

Synchronous processing often creates bottlenecks and increases latency. Adopting an event-driven architecture (EDA) allows you to offload non-critical or time-consuming tasks, improving frontend responsiveness and backend resilience.

Conceptual Layout:
- An API Gateway receives an inbound request.
- A lightweight Serverless Function (e.g., AWS Lambda, Azure Function) performs minimal validation and then publishes an event to a Message Queue (e.g., AWS SQS, Kafka, Azure Service Bus). It immediately returns a “202 Accepted” response to the client.
- Another Serverless Function or a dedicated worker processes the event from the queue asynchronously, performing heavy lifting like image processing, report generation, or complex business logic.
- Results are then stored in a database, S3, or pushed back to the client via WebSockets or another event.
Strategy: Identify parts of your application that don’t require immediate feedback to the user. Decouple these processes using message queues and serverless functions. This not only improves user experience by reducing latency but also enhances system resilience, as messages can be retried and processed even if the consumer temporarily fails.

Conclusion

The “dirty secret” of cloud server scaling isn’t about the inability to scale, but rather the industry’s default to inefficient, reactive over-provisioning. True scalability in 2026 demands a shift towards intelligent design: leveraging dynamically scaling serverless databases, implementing smart sharding strategies informed by real-world access patterns, and embracing event-driven architectures to decouple and optimize load. This proactive approach to anticipating bottlenecks and optimizing P99 latencies is where the real engineering magic happens. By moving beyond the horizontal band-aid, you build not just a bigger backend, but a smarter, more resilient, and significantly more cost-effective one—a true differentiator in today’s competitive digital landscape.

Your Serverless Is Lying To You About Scale!

Sun, 21 Jun 2026 05:37:13 +0000

Your Serverless Is Lying To You About Scale!

Introduction

The promise of serverless computing is irresistible: infinite scalability, pay-per-use, and zero operational overhead. We’ve eagerly embraced platforms like AWS Lambda, Google Cloud Run, and Azure Container Apps, pushing them to scale horizontally with unprecedented agility. Yet, a recent surge in backend outages tells a different story. The culprit isn’t typically the compute layer, but a silent, often overlooked bottleneck: database connection storms. While your serverless functions might explode with instances, your underlying relational database often remains a fixed-capacity component, throttling your “elastic” backend and leading to frustrating, intermittent service disruptions.

The “Dirty Secret”: Database Connection Storms

The fundamental disconnect lies in the architecture. Each instance of a serverless function, by default, often attempts to establish its own fresh connection to the database. When a sudden spike in traffic triggers hundreds or thousands of function instances, this translates directly into an equivalent surge of simultaneous connection requests hitting your PostgreSQL, MySQL, or other relational database instance.

Even highly provisioned databases have hard limits on concurrent connections. Once this limit is reached, new connection attempts are queued, rejected, or timeout. This manifests as increased latency, 5xx errors, and ultimately, backend outages, despite your serverless compute scaling perfectly. This “dirty secret” means that while your Cloud Run containers might be ready to serve millions of requests, your humble Postgres instance can only handle so many concurrent sessions before it buckles, silently undermining your entire scalability strategy.

Architectural Layout/Walkthrough: Designing for True Data Elasticity

Overcoming this limitation requires a strategic shift in how we manage database access in serverless environments. The fix isn’t just provisioning a larger database; it’s about intelligent, distributed connection management and a re-evaluation of data consistency models.

1. Intelligent, Distributed Connection Pooling at the Edge

The most immediate and impactful solution is to introduce a dedicated connection pooling layer. This layer acts as an intermediary, multiplexing many client connections (from your serverless functions) over a fewer, persistent pool of connections to the database.

Conceptual Flow: Instead of: Serverless Function (N instances) -> N direct connections -> Database You’d have: Serverless Function (N instances) -> N connections -> Edge Connection Proxy -> M pooled connections (M << N) -> Database

Implementation Considerations:

Cloud-Native Proxies: Services like AWS RDS Proxy or Google Cloud SQL Proxy are designed specifically for this challenge. Your serverless functions connect to the proxy endpoint, which then manages the connection pool to your RDS or Cloud SQL instance. This requires minimal code changes; you simply update your database connection string to point to the proxy.
```
# Example Serverless Function Environment Variable
DATABASE_URL: "postgresql://user:password@<RDS_PROXY_ENDPOINT>:5432/mydb"
```
Third-Party/Specialized Proxies: Solutions like PgBouncer can be deployed as a separate service (e.g., in a container or VM) or integrated into managed database services (like Neon or Supabase) that offer built-in pooling optimized for serverless workloads.
Connection Lifecycle: Configure your proxy for “session pooling” or “transaction pooling” based on your application’s needs. Transaction pooling is generally more efficient for serverless as connections are returned to the pool immediately after each transaction, maximizing reuse.

2. Dynamic Data Proxy Layers

Beyond simple connection pooling, a more advanced data proxy can offer additional benefits for serverless scalability:

Read/Write Splitting: Route read queries to read replicas and write queries to the primary instance, offloading the primary database.
Caching: Cache frequently accessed data at the proxy layer, reducing direct database hits.
Query Rewriting/Optimization: Optimize queries before they reach the database.

These proxies effectively abstract the database topology from your serverless functions, allowing the data layer to scale and adapt independently.

3. Eventual Consistency Where Possible

The most fundamental architectural shift involves questioning the necessity of synchronous database writes for every operation. Many actions don’t require immediate, transactional consistency across all systems.

Conceptual Flow: Instead of: User Action -> Serverless Function -> Synchronous DB Write -> Response You’d have: User Action -> Serverless Function -> Publish Event to Message Queue (e.g., SQS, Pub/Sub) -> Immediate Response Another Serverless Function (triggered by queue) -> Asynchronous DB Write

Implementation Considerations:

Identify Use Cases: Log events, analytics updates, notification sending, inventory decrements (if stock checks happen upstream), order status updates that can tolerate a slight delay.
Messaging Services: Utilize cloud-native message queues (AWS SQS, Google Cloud Pub/Sub, Azure Service Bus) or event streaming platforms (Kafka).

Event-Driven Architecture: Your initial serverless function publishes an event and returns a response quickly, offloading the database interaction to a separate, asynchronous process. This drastically reduces the synchronous load on your database.

# Pseudo-code for Eventual Consistency
import boto3

sqs = boto3.client('sqs')
QUEUE_URL = "your-sqs-queue-url"

def handle_request(event, context):
    # ... process incoming request data ...
    payload = {"user_id": "123", "action": "product_viewed", "product_id": "XYZ"}

    # Publish event for asynchronous processing
    sqs.send_message(
        QueueUrl=QUEUE_URL,
        MessageBody=json.dumps(payload)
    )

    return {
        "statusCode": 202, # Accepted
        "body": "Request accepted for processing."
    }

# Separately, another function handles the queue:
def process_event(event, context):
    for record in event['Records']:
        message_body = json.loads(record['body'])
        # ... connect to DB, insert data ...
        # Ensure proper error handling and retries for DB writes

Conclusion

True serverless elasticity extends beyond just scaling compute. The core challenge often lies in the fixed-capacity nature of traditional relational databases. By intelligently layering distributed connection pools and dynamic data proxies, you can mitigate connection storms and create a robust buffer between your bursting serverless functions and your database. More profoundly, an architectural shift towards eventual consistency for appropriate workloads can dramatically offload synchronous database writes, allowing your backend to handle peak loads gracefully. Stop provisioning for theoretical maximums; design for truly elastic data access from the ground up.

The Truth About Serverless APIs Nobody Mentions

Wed, 27 May 2026 15:51:56 +0000

I've noticed serverless conversations tend to split into two camps:

People who think it solves everything, and people who think it's a mistake.

The reality is much more useful than either extreme.

Everyone who sells you on serverless leads with the same three points: no servers to manage, automatic scaling, and you only pay for what you use.

All of that is technically true.

None of it is the whole story.

I've shipped serverless APIs in production — Go Lambdas behind API Gateway, the kind that power real user-facing products. I'm not here to talk you out of it. For the right workload, it's genuinely excellent.

But the gaps between the pitch and the reality are specific, and knowing them in advance changes the decisions you make early, when they're still cheap to change.

Cold starts are real — just not how you think

Lambda cold starts happen when AWS needs to provision a new execution environment for your function. The runtime boots, your init code runs, and then the handler executes.

For a Go function on arm64, that's typically 50–150ms total. For Node.js or Python with heavy dependencies, it can be 500ms–2 seconds.

The part nobody leads with: cold starts are probabilistic, not guaranteed.

Under steady traffic, most invocations hit warm instances. You'll rarely see them in your P50 or even P95. Where they show up is P99 and beyond — the tail latency your best users experience right after a traffic spike, or the first request after a quiet weekend.

The practical advice: profile in production with real traffic before you add Provisioned Concurrency. It costs money and adds complexity.

For most APIs, the smarter move is optimizing your init code:

lazy-load dependencies
keep your binary small
initialize only what the first request actually needs

Fix the actual problem before reaching for the expensive solution.

The billing math will surprise you

Lambda pricing is simple on paper: you pay per request and per GB-second of execution time.

At low traffic, it's nearly free.

The surprise comes at scale — or in specific usage patterns.

The scenario that bites teams most often: a Lambda that does meaningful work per invocation. Parsing a large payload, doing in-memory transformation, making multiple downstream calls.

These functions run longer, use more memory, and the cost per invocation adds up faster than the pricing page suggests.

A few numbers from real workloads:

A lightweight REST Lambda (Go, 128MB, 20ms avg): ~$0.40 per million requests
A heavier processing Lambda (Node, 512MB, 800ms avg): ~$8.00 per million requests

That 20× difference in cost per million requests adds up quickly when you have real traffic.

It's still cheap compared to running servers — but the assumption that serverless is always the low-cost option needs to be verified, not assumed.

API Gateway pricing is also frequently missed. At $3.50 per million requests for HTTP APIs, a high-traffic public endpoint can make Gateway cost more than the Lambda it fronts.

Know the full cost model before you commit.

Debugging is a different skill

Local development for Lambda is awkward.

You can simulate the execution environment with tools like AWS SAM or custom test harnesses, but you're never truly running the same thing that runs in production.

The gap shows up in exactly the moments you can least afford it.

Observability requires deliberate setup.

A Lambda that exits silently leaves no trace unless you've instrumented it. CloudWatch Logs are the floor, not the ceiling — you need structured logging, correlation IDs, and ideally X-Ray tracing before you're in production, not after something breaks.

The failure modes are also harder to reproduce.

A Cognito authorizer rejecting a token before your code runs looks different from your code returning a 401. A DynamoDB throttle on a hot partition looks different from a Lambda timeout.

Getting fast at debugging serverless systems means learning the AWS console deeply:

CloudWatch Insights queries
X-Ray service maps
reading IAM policy errors without a stack trace to guide you

Statelessness is a constraint, not just a pattern

Every Lambda invocation starts with a clean slate.

That's the feature — it's what makes horizontal scaling effortless.

It's also the constraint that eliminates entire categories of architectural approaches.

No in-memory caches that persist across requests. No background jobs that run between invocations. No WebSocket connections that stay open. No files written to disk that the next request can read.

If your application assumes any of those things, serverless isn't wrong — but it's not your foundation. It's a piece of a larger architecture that needs something else alongside it.

This comes up most often when teams decide mid-project that they suddenly need a feature requiring persistent state.

Adding ElastiCache or SQS to a serverless architecture is straightforward — but it's a different architecture than what you started with, and the integration work is non-trivial.

The most dangerous part is that many of these problems don't show up until the system already matters.

Vendor lock-in is real, and that's okay

A Lambda function isn't a Docker container.

It doesn't run the same way everywhere else.

Your Go binary targeting provided.al2023 on arm64 can technically run in other environments, but your API Gateway routes, Cognito authorizers, DynamoDB single-table design, and SES integration are AWS-specific throughout.

I'm not arguing against it — I use all of those services in production.

But it's worth being honest:

If you go deep on serverless AWS, migrating later means a substantial rewrite.

Make that decision deliberately, not by default.

When not to use it

Serverless is the wrong tool when:

You have persistent connections — WebSocket servers, gRPC streams, database connection pools
You have long-running jobs — Lambda has a 15-minute max timeout
You have very high, sustained concurrency — account-level concurrency limits can affect multiple services simultaneously
You need low and predictable latency at P99 — cold-start tail latency is difficult to eliminate completely

When it's the right call

Event-driven workloads.

APIs with bursty or unpredictable traffic.

Background processing triggered by S3 uploads, SQS messages, or DynamoDB streams.

Any workload where you don't want to manage capacity and the per-request cost model fits your traffic profile.

The API powering this site is Go Lambdas behind API Gateway.

Contact form submissions, resume requests, visitor analytics, admin dashboard — all of it.

It handles the traffic I need, costs less than $1/month to run, and I've never paged for a server that went down.

For that workload, it's the right call.

The key is understanding what you're actually trading.

Serverless isn't simpler than traditional infrastructure.

It's differently complex.

The operational burden shifts from server management to observability, IAM, and distributed system debugging.

That trade is absolutely worth it in the right context.

Go in with your eyes open, and it's one of the most productive architectures available.

The right architecture should help your team maintain momentum as the system grows — not create friction six months later.

At M²S² Engineering Group, we help teams make these decisions before they become expensive to unwind.

Originally published at m2s2.io

AWS Lambda Is Dead. The $0.20 Was Never the Price.

Tue, 12 May 2026 04:27:39 +0000

Last quarter we migrated 47 Lambda functions off AWS. The monthly bill dropped from $8,362 to $1,790. Lambda invocations were 22% of that bill. The rest was the part AWS never put on the pricing page, never put in the AWS Lambda 101 docs, and never came up when our solutions architect ran our forecast workshop in 2024.

We are committing to one position in this piece and we are not flipping at the end. Lambda is dead for the API, webhook, auth, and edge workload that most teams actually deploy. Every comparison article we read while researching the move closed with "but evaluate your needs carefully." That sentence is what kept us on Lambda for an extra year. We are not writing that sentence today.

If you are skimming, the punchline is at the top.

TL;DR

Thing	What it does	Why Lambda loses
INIT billing (Aug 2025)	Cold start init time now bills like duration	The price floor moved up, quietly
The orchestration bundle	API Gateway, CloudWatch, NAT, egress	Lambda is 20 to 40% of your serverless bill
The crossover point	Where Fargate or Workers wins	Moved from 20M to about 2M invocations a month
V8 isolates	Cold start of 2 to 5ms on Workers	No warm-up tax, no provisioned concurrency to buy

1. The $0.20 Per Million Is a Loss Leader

Lambda lists at $0.20 per million requests and $0.0000166667 per GB-second. People build their forecasts on this number. Then the bill arrives and the Lambda line item is a quarter of the total.

A breakdown from a real account, redacted, one month:

Lambda invocations + duration   $1,847   (22%)
API Gateway                     $1,612   (19%)
CloudWatch Logs                 $1,398   (17%)
NAT Gateway hours               $1,287   (15%)
Data egress                     $1,094   (13%)
X-Ray, KMS, Secrets, parameter store   $1,124   (14%)
                              ─────────
Total                          $8,362

Read that table again. Lambda is the fifth column. The wrapper. CloudWatch Logs is bigger. NAT Gateway is bigger. Almost everything is bigger.

Lambda by itself was fine. Lambda inside the AWS bundle was a loss leader for the bundle. CloudWatch Logs at the default ingestion rate eats functions that log a single audit line per call. NAT Gateway hours rack up if your function needs to reach a private RDS or any non-VPC endpoint, which is, you know, basically every real function. Data egress at $0.09 per gigabyte compounds with every response payload.

We have a name for this pattern on our internal docs. The Token Tab. The headline price advertises the wrapper. The real money is in the orchestration around it. AWS did not invent this pattern. AWS just runs the most disciplined version of it.

2. INIT Billing Changed the Floor (And Nobody Noticed)

In August 2025, AWS quietly changed how Lambda billed cold start initialization. Before the change, INIT time was free for managed runtimes. Now it bills as duration.

For a JVM function with 800ms of init, every cold invocation costs an extra 80% on top of the actual work. Boot up Spring Boot? Pay for the boot. Cold start your fat Python ML container? Pay for the import storm.

The change shipped in a release note. Not a keynote. Not a tweet from a Principal Engineer. A release note.

We did not find out about it until April 2026 when we audited a function that was supposed to cost $40 and was billing $110.

Reports floating around early 2026 (source: InfraDecodedOps cold-start teardown):

23% of customer-facing Lambda invocations hit a cold start
p99 cold latency of 1.8 seconds on those endpoints
AWS counters that fewer than 1% of invocations across Lambda are cold

Both numbers are true. The 1% headline reflects hot, internal, async functions running at scale. The 23% reflects what your users actually feel on your auth endpoint at 6:42am UTC when traffic is sparse.

SnapStart helps for Java, Python, .NET. SnapStart does not help for the part of your bill that is API Gateway and NAT.

3. The Crossover Moved (And It Moved a Lot)

Folk wisdom in 2022 was that Lambda was cheaper until you hit roughly 20 million invocations a month. After that, Fargate. After that, real instances.

With INIT billing layered onto the bundle pricing, the crossover is closer to 2 million for typical API workloads. That is an order of magnitude shift in three years. Nobody updated the blog posts.

Monthly cost
   ▲
   │                    ╱── Lambda + bundle
   │              ╱────╱
   │        ╱────╱
   │  ╱────╱─────────── Fargate (1 task, ALB)
   │═════════════════════ Workers + KV/D1
   │
   └──────────────────────►
    0    1M   2M    5M   10M  invocations/mo

Workers stays flat because Workers bills CPU time, not wall clock. If your function waits 200ms on a database response but only burns 15ms of CPU, you pay for 15ms. Lambda bills for the full 200ms.

That is not a small detail. That is the whole game. The typical API spends 80% of its time waiting on a database, a downstream service, or an LLM. Lambda charges you for the wait. Workers does not.

4. The Cope Chart (Optimizations That Do Not Save You)

Things we tried in 2024 and 2025 to "fix" our Lambda bill:

Tactic	Effect on Lambda line	Effect on total bill
Trim function memory	-8% on Lambda	-1.7% on total
Switch Python to Node	-12% on Lambda	-2.6% on total
ARM64 (Graviton2)	-15% on Lambda	-3.3% on total
Add provisioned concurrency	+30% on Lambda	+6.6% on total
SnapStart on JVM functions	-22% on Lambda	-4.8% on total
Audit and prune CloudWatch retention	0	-9% on total
Add VPC endpoints to kill NAT	0	-14% on total

You see the pattern. The Lambda optimizations move the Lambda line a little. The bundle optimizations move the bill a lot. We spent a year picking up dimes on the Lambda line while the bundle was charging us in 50s.

The audit-and-kill-bundle tactics are what actually save money on AWS. Nobody writes blog posts about them because they are not sexy. There is no AWS re:Invent talk about deleting your unused log groups. There should be.

5. Workers vs Lambda, Side by Side

A real example. A signed URL generator for S3, ported to R2 on Workers.

Lambda version:

import { S3Client, GetObjectCommand } from "@aws-sdk/client-s3";
import { getSignedUrl } from "@aws-sdk/s3-request-presigner";

const s3 = new S3Client({ region: "us-east-1" });

export const handler = async (event: any) => {
  const key = event.pathParameters?.key;
  if (!key) return { statusCode: 400, body: "missing key" };

  const cmd = new GetObjectCommand({ Bucket: "assets", Key: key });
  const url = await getSignedUrl(s3, cmd, { expiresIn: 300 });

  return { statusCode: 200, body: JSON.stringify({ url }) };
};

Cold start with the SDK loads at 400 to 700ms. Warm at 30 to 80ms. Bills wall clock. Behind API Gateway. Logs go to CloudWatch by default. Outbound to S3 may hit NAT depending on VPC config.

Workers version with R2:

export default {
  async fetch(req: Request, env: Env): Promise<Response> {
    const key = new URL(req.url).pathname.slice(1);
    if (!key) return new Response("missing key", { status: 400 });

    const obj = await env.ASSETS.get(key);
    if (!obj) return new Response("not found", { status: 404 });

    return new Response(obj.body, {
      headers: { "cache-control": "public, max-age=300" }
    });
  }
};

Cold start at 2 to 5ms (Cloudflare's own dashboards and a six-month production comparison back this up). R2 egress to the public internet is zero. No log retention to forget about. No NAT in the path. The pricing page does not need a footnote.

Same function. Different platform. The bill says everything.

5b. The Platform Comparison

Here is the table that should be at the top of every serverless conversation in 2026. Pricing is approximate and cold starts come from production benchmarks reported by independent teams. The links go to canonical comparison or pricing pages for each platform.

Platform	Cold start	Billing model	Free egress	Sweet spot
AWS Lambda	200-700ms (cold), bills INIT since Aug 2025	Wall clock + bundle	No ($0.09/GB)	AWS-native event glue, GPU jobs
Cloudflare Workers	2-5ms (V8 isolate)	CPU time only	Yes (R2 zero)	HTTP APIs, edge, webhooks
Google Cloud Run	50-200ms	Wall clock	Yes within region	Container portability, ML inference
Azure Functions Premium	Pre-warmed (0)	Flat + per-exec	No	Enterprise, no-cold-start needed
Fly.io Machines	250ms-2s (cold)	Per-second machine	Limited	Stateful regional apps, full VM control
Railway	Container boot	Usage-based	Yes	Solo and small team backend hosting

Baselime reported an 80% cloud-cost drop after migrating from AWS to Cloudflare. Sitepoint published a case study where a team moved their Lambda proxy entirely into the browser and cut their bill from $2,400 to $140 a month. Different workloads, same direction.

6. The Honest Counter (And Why It Does Not Save Lambda)

Three places Lambda still earns its keep. We are not pretending otherwise.

GPU and long-running compute. Workers has a 30-second CPU cap and no GPU. If you are doing inference, video transcoding, or anything past a 30-second budget, Lambda or SageMaker or a real instance is the answer. We kept zero of these. We do not run any.

Heavy AWS-native event glue. If your workload is S3 to DynamoDB to SQS to Step Functions and you never leave AWS, the orchestration bundle is the platform, and Lambda is the right glue. We kept four functions for this. They run for cents.

Sparse async cron jobs. A nightly batch at 3am that runs once a day for 200ms? Lambda is essentially free at that scale. Workers Cron Triggers are also fine. Either works.

Notice what is absent from that list. The typical API. The webhook receiver. The edge function. The auth gateway. The fan-out worker that most teams actually deploy. That whole pile is the dead zone for Lambda in 2026.

If your team's Lambda fleet is mostly the typical API workload, you are in the dead zone whether you have noticed yet or not. The bill will tell you eventually. Usually after a quarterly investor update where someone asks why infrastructure spend grew faster than revenue.

7. The Migration Playbook

If you want to move the workloads that no longer make sense on Lambda:

Pull a 30-day Cost Explorer report. Tag by Lambda function name. Find the top 10 by total cost.
For each, count outbound calls per invocation, average duration, average response size, cold start frequency, and whether it crosses a NAT.
Candidates that move first are HTTP-fronted, low-CPU, high-invocation, network-bound. Those are the ones bleeding wall-clock dollars.
Port to Workers or Cloud Run. Keep the AWS-native event glue on Lambda. Do not try a big-bang migration. Move one function, watch it for two days, move the next.
Watch the bill for two cycles. Decommission API Gateway routes and CloudWatch log groups as functions go quiet. The teardown is where 30% of the savings actually live.

We did this over six weeks. The hardest part was not the rewrites. The hardest part was untangling which CloudWatch log groups still mattered and which ones quietly cost $40 a month to keep indexed. The second hardest part was convincing one of our seniors that the AWS SDK he had written wrappers around for two years was the part we were throwing out.

8. What We Are Not Coming Back For

Lambda was the right shape in 2015. The shape of cloud has moved. V8 isolates that start in 2ms. Container platforms with sub-second cold boots. Edge runtimes that bill CPU instead of wall clock. Those are the new default for the workload Lambda used to own.

We will not migrate back. Not because Workers is perfect, but because the bundle math does not reverse. If AWS dropped Lambda pricing to $0.10 per million tomorrow, our bill would drop by 11%. The other 89% would still be the orchestration tax.

The $0.20 per million was a beautiful marketing line. That was never the price. The price was always the bundle.

The Bottom Line

If your team is building an API in 2026 and Lambda is the first thing on the architecture diagram, ask why. Ask it out loud. The honest answer is almost always organizational momentum, not technical fit. Someone built this pattern at their last job. The first hire wired it up because the AWS reference architecture said to. Nobody questioned it because nobody had time to audit Cost Explorer.

We had that conversation. Six weeks later our bill was 79% smaller and our p99 was 22 times faster.

Lambda is dead for the workload most teams actually run. The pricing page never told you why. The bill always will.

GDS K S · thegdsks.com · follow on X @thegdsks

Lambda is dead for the workloads most teams actually run. The bundle around it was always the real product.

Sources

Sandesh, AWS Lambda Is Your Worst-Performing Cost-Saver: The 2026 Cold Start Data. Source for 23% cold start figure and 1.8s p99.
LeanOps, AWS Lambda Pricing 2026: Costs, Fees and Hidden Traps. Source for 20-40% base Lambda share of total spend.
Alan West, AWS Lambda's Hidden Costs: When to Migrate to Containers. Source for the crossover point math.
Vantage, Cloudflare Workers vs AWS Lambda Cost. Source for CPU-time vs wall-clock pricing breakdown.
Leaper, Cloudflare Workers vs AWS Lambda 2026. Source for 50-80% cost reduction at scale and Baselime case.
Rebal AI, Cloudflare Workers vs AWS Lambda: Six Months of Production Reality. Source for six-month production comparison.
SitePoint, Case Study: Cloud to Local-First AI Migration. Source for the $2,400 to $140 PWA migration.

How to Build a Serverless Data Lake Foundation with AWS Glue

Sat, 02 May 2026 12:41:31 +0000

1. Introduction

Welcome to this comprehensive tutorial on building a Serverless Data Lake Foundation using AWS Glue. By the end of this guide, you will be able to design and implement a robust, automated pipeline that extracts raw data from Amazon S3, transforms it into an optimized analytics-ready format, and makes it available for querying via Amazon Athena. This architectural pattern is highly useful because it completely eliminates the need to manage underlying servers, clusters, or infrastructure, allowing you to focus entirely on your core data logic. Furthermore, it automatically scales out alongside your data volume, ensuring long-term cost-effectiveness since you only pay for the exact compute resources consumed during the active transformation process. Whether you are dealing with daily batch processing operations or aggregating vast amounts of historical data, mastering this foundational architectural pattern is a critical and necessary milestone in modern data engineering.

2. Prerequisites

Before starting this step-by-step implementation, ensure you have the following tools and foundational knowledge prepared:

AWS Account: An active Amazon Web Services account with administrative or sufficient permissions to create S3 buckets, configure IAM roles, and deploy AWS Glue jobs.
Basic Python or PySpark Knowledge: Familiarity with basic data manipulation concepts, variables, and data structures using Python or Apache Spark.
Understanding of Cloud Storage: General knowledge of how object storage systems function, specifically Amazon S3 concepts including buckets and prefixes.
IAM Fundamentals: A foundational understanding of Identity and Access Management principles to allow distinct AWS services to communicate securely.

3. Step-by-Step

Step 1: Architecting the Serverless Data Lake Foundation

What to do: Review the end-to-end data flow and understand the specific roles of each AWS service before provisioning any cloud resources. The architecture requires Amazon S3 for storage, Amazon EventBridge for orchestration, AWS Glue for processing and metadata management, and Amazon Athena for consumption.

Why do it: Establishing a clear mental model and a visual blueprint prevents configuration errors later in the process. It ensures that all AWS services are logically connected, network paths are understood, and the separation between storage and compute is maintained throughout the pipeline design.

Example:

Step 2: Configuring Amazon S3 Storage Zones

What to do: Access the Amazon S3 console and create a new S3 bucket with a unique name. Inside this bucket, create two distinct folders (prefixes). Name the first folder raw/ to represent the landing zone for your incoming, unprocessed data. Name the second folder curated/ to represent the destination for your clean, transformed data.

Why do it: Segregating data by its processing state is a fundamental best practice in data engineering. It prevents accidental overwrites of source data, simplifies access control policies, and optimizes query performance by keeping raw, unoptimized files strictly separated from the analytical query engine.

Example:

# 1. Define the core S3 Data Lake Bucket
resource "aws_s3_bucket" "datalake_production" {
  bucket = "enterprise-datalake-production"

  tags = {
    Environment = "Production"
    Purpose     = "Serverless Data Lake Foundation"
    ManagedBy   = "Terraform"
  }
}

# 2. Enable Bucket Versioning (Recommended Data Lake Best Practice)
resource "aws_s3_bucket_versioning" "datalake_versioning" {
  bucket = aws_s3_bucket.datalake_production.id
  
  versioning_configuration {
    status = "Enabled"
  }
}

# 3. Create the 'raw' subdirectory (Landing Zone)
resource "aws_s3_object" "raw_zone" {
  bucket = aws_s3_bucket.datalake_production.id
  key    = "raw/"
  
  # Defines the object as a directory rather than a standard file
  content_type = "application/x-directory"
}

# 4. Create the 'curated' subdirectory (Gold/Analytics Zone)
resource "aws_s3_object" "curated_zone" {
  bucket = aws_s3_bucket.datalake_production.id
  key    = "curated/"
  
  content_type = "application/x-directory"
}

Step 3: Setting Up IAM Permissions

What to do: Navigate to the AWS Identity and Access Management console and create a new Service Role specifically assigned to AWS Glue. Attach the AWS managed policy named AWSGlueServiceRole. Additionally, create and attach an inline policy that explicitly grants s3:GetObject and s3:PutObject permissions targeted strictly at the S3 bucket you created in the previous step.

Why do it: AWS services operate under the strict principle of least privilege. AWS Glue cannot access your raw data, write processed files to S3, or publish execution logs to Amazon CloudWatch without explicit, cryptographically secure permissions defined by an IAM Role.

Example:

# 1. Define the Trust Relationship (Assume Role Policy) for AWS Glue
data "aws_iam_policy_document" "glue_assume_role_policy" {
  statement {
    actions = ["sts:AssumeRole"]
    
    principals {
      type        = "Service"
      identifiers = ["glue.amazonaws.com"]
    }
  }
}

# 2. Create the IAM Role
resource "aws_iam_role" "glue_execution_role" {
  name               = "GlueDataLakeExecutionRole"
  assume_role_policy = data.aws_iam_policy_document.glue_assume_role_policy.json

  tags = {
    Environment = "Production"
    Purpose     = "Serverless Data Lake Foundation"
    ManagedBy   = "Terraform"
  }
}

# 3. Attach the core AWS Managed Policy for Glue Service operations
# This grants necessary permissions for CloudWatch logging, ENI creation (if VPC bounded), etc.
resource "aws_iam_role_policy_attachment" "glue_service_managed_policy" {
  role       = aws_iam_role.glue_execution_role.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AWSGlueServiceRole"
}

# 4. Define a restrictive inline policy for the specific Data Lake S3 bucket
data "aws_iam_policy_document" "s3_datalake_access_policy" {
  statement {
    sid    = "AllowDataLakeReadWrite"
    effect = "Allow"
    
    actions = [
      "s3:GetObject",
      "s3:PutObject"
    ]
    
    resources = [
      # Restrict access exclusively to the specific bucket and defined prefixes
      "arn:aws:s3:::enterprise-datalake-production/raw/*",
      "arn:aws:s3:::enterprise-datalake-production/curated/*"
    ]
  }
}

# 5. Create and attach the inline S3 policy to the IAM Role
resource "aws_iam_role_policy" "s3_datalake_inline_policy" {
  name   = "RestrictedS3DataLakeAccess"
  role   = aws_iam_role.glue_execution_role.id
  policy = data.aws_iam_policy_document.s3_datalake_access_policy.json
}

Step 4: Orchestrating the Pipeline Execution Flow

What to do: Define the exact sequence of automated events that will trigger and execute your ETL process. Configure an Amazon EventBridge rule that listens for an ObjectCreated event in your S3 raw/ prefix. Set the target of this rule to invoke your AWS Glue Job automatically whenever a new file finishes uploading.

Why do it: Automation entirely removes the need for manual intervention and scheduling guesswork. By structurally mapping the sequence, you ensure that compute resources are only utilized when new data is actually present, maximizing efficiency and minimizing idle costs.

Example:

Step 5: Authoring the AWS Glue Transformation Job

What to do: Open the AWS Glue Studio visual editor and create a new Spark ETL job. Configure the source node to read from your S3 raw/ path. Add a transformation node to clean the data, such as dropping null values, mapping column names to standard conventions, and casting data types correctly. Finally, configure the target node to write the data back to your S3 curated/ path, explicitly selecting Parquet as the output format.

Why do it: The transformation phase is where raw information becomes valuable data. Writing the output in Apache Parquet format is critical. Parquet is a highly optimized, columnar storage format that compresses data heavily, dramatically reduces long-term S3 storage costs, and exponentially speeds up subsequent analytical queries by allowing engines to skip irrelevant columns.

Example (Terraform):

resource "aws_glue_job" "datalake_curation_job" {
  name     = "raw-to-curated-parquet-job"
  role_arn = aws_iam_role.glue_execution_role.arn # From the previous IAM step

  # Defines the execution engine and script location
  command {
    name            = "glueetl"
    script_location = "s3://enterprise-datalake-production/scripts/etl_job.py"
    python_version  = "3"
  }

  default_arguments = {
    "--job-language"                     = "python"
    "--enable-metrics"                   = "true"
    "--enable-continuous-cloudwatch-log" = "true"
  }

  # Cost and performance optimization
  glue_version      = "4.0"
  worker_type       = "G.1X"
  number_of_workers = 2

  tags = {
    Environment = "Production"
    Purpose     = "Serverless Data Lake Foundation"
    ManagedBy   = "Terraform"
  }
}

Example (Glue Job):

import sys
from awsglue.transforms import *
from awsglue.utils import getResolvedOptions
from pyspark.context import SparkContext
from awsglue.context import GlueContext
from awsglue.job import Job

sc = SparkContext.getOrCreate()
glueContext = GlueContext(sc)
spark = glueContext.spark_session
job = Job(glueContext)

# ==============================================================================
# Node 1: S3 Source (Reads raw CSV data)
# ==============================================================================
S3Source_node1 = glueContext.create_dynamic_frame.from_options(
    format_options={"quoteChar": '"', "withHeader": True, "separator": ","},
    connection_type="s3",
    format="csv",
    connection_options={
        "paths": ["s3://enterprise-datalake-production/raw/"], 
        "recurse": True
    },
    transformation_ctx="S3Source_node1"
)

# ==============================================================================
# Node 2: Transform - ApplyMapping (Cleans and maps data types)
# ==============================================================================
ApplyMapping_node2 = ApplyMapping.apply(
    frame=S3Source_node1,
    mappings=[
        ("id", "string", "user_id", "string"),
        ("name", "string", "full_name", "string"),
        ("timestamp", "string", "event_time", "timestamp")
    ],
    transformation_ctx="ApplyMapping_node2"
)

# ==============================================================================
# Node 3: S3 Target (Writes optimized Parquet data)
# ==============================================================================
S3Target_node3 = glueContext.write_dynamic_frame.from_options(
    frame=ApplyMapping_node2,
    connection_type="s3",
    format="glueparquet",
    connection_options={
        "path": "s3://enterprise-datalake-production/curated/", 
        "partitionKeys": []
    },
    format_options={"compression": "snappy"},
    transformation_ctx="S3Target_node3"
)

job.commit()

Example (Glue Studio):

Step 6: Cataloging the Data with AWS Glue Crawlers

What to do: Navigate to the Crawlers section within the AWS Glue console. Create a new crawler and configure its target repository to point directly at your curated/ S3 folder. Assign the IAM role created in Step 3 to the crawler. Set the crawler to run on a daily schedule or to be triggered immediately after the AWS Glue transformation job completes successfully.

Why do it: Data sitting in S3 is useless to business analysts without structural context. The Crawler automatically inspects your Parquet files, infers the underlying schema including column names and data types, and registers this structural definition as a logical table within the centralized AWS Glue Data Catalog.

Step 7: Querying the Transformed Data with Amazon Athena

What to do: Navigate to the Amazon Athena console. Ensure you have set up an Athena query result location in a separate S3 bucket. Select the database created by your Glue Crawler from the dropdown menu. Write and execute standard SQL queries against your newly cataloged table to analyze the data.

Why do it: This final step validates the structural integrity of your entire pipeline. Athena uses the metadata schema stored in the Data Catalog to seamlessly execute SQL queries directly against the compressed Parquet files residing in S3. This proves that your serverless architecture is fully operational and ready for business intelligence workloads.

4. Common Troubleshooting

Problem 1: Out of Memory Errors in Glue Jobs
This issue frequently occurs due to data skew, where one worker node attempts to process significantly more data than others, or when attempting to process thousands of excessively small files simultaneously.
Solution: Resolve this by navigating to the Glue Job details and increasing the worker type from the standard configuration to G.1X or G.2X, providing more memory per executor. Alternatively, utilize Spark's .repartition() command within your script to evenly distribute the workload across the entire serverless cluster before executing wide transformations.

Problem 2: Glue Crawler Creates Multiple Unwanted Tables
If your curated S3 data utilizes a deeply nested, highly partitioned, or logically inconsistent folder structure, the AWS Glue Crawler might misinterpret the subfolders as entirely separate tables instead of partitions of a single dataset.
Solution: To rectify this, edit the Crawler configuration settings. Locate the section labeled "Grouping behavior for S3 data" and explicitly check the option to "Create a single schema for each S3 path". This forces the crawler to treat all files under the root prefix as part of the same table.

Problem 3: Amazon Athena Queries Return Zero Records
If the table successfully appears in the Data Catalog but standard SELECT queries return absolutely empty results, the S3 data is highly likely partitioned, but the Data Catalog remains unaware of the new partition directories.
Solution: Execute the MSCK REPAIR TABLE your_table_name; command directly within the Athena query editor. This command scans the S3 path, loads the missing partition metadata into the Data Catalog, and immediately makes the data queryable.

5. Conclusion

Congratulations on successfully architecting and understanding the Serverless Data Lake Foundation. You have learned how to decouple storage from compute layers, automate data transformations using event-driven principles, and seamlessly bridge raw information into a highly optimized, queryable state. By mastering these core components, you now possess the foundational engineering skills required to build scalable, enterprise-grade data platforms. As a logical next step, consider exploring AWS Step Functions to orchestrate more complex, multi-job dependencies, or experiment with Apache Iceberg within your AWS Glue jobs to introduce robust transactional capabilities and time-travel querying to your data lake.

Your Serverless Data Lake is Lying to You: Add Observability or Lose Data (AWS)

Mon, 20 Apr 2026 12:55:14 +0000

TL;DR

Serverless Data Lakes are scalable and cost-efficient—but they can fail silently
Without observability, you risk making decisions on incomplete data
Adding a Dead Letter Queue (DLQ) ensures no data is lost
Monitoring with Amazon CloudWatch + alerts via Amazon SNS gives you real visibility
Trade-off: slightly more complexity for significantly higher reliability
The Moment Everything “Worked”… But Was Wrong

It was 2 AM.

The pipeline had completed successfully.
Amazon Athena was returning results.

Dashboards were updating.

And yet… the numbers didn’t match.

Walking through logs in Amazon CloudWatch, I found the issue:
messages sitting in a queue that no one was monitoring.

No alarms.
No failures reported.
Just silently missing data.

That’s the uncomfortable truth about serverless systems:

They don’t crash. They drift.

The Typical “Clean” Architecture

Most Serverless Data Lakes follow a familiar pattern:

Amazon S3 as the storage layer
AWS Glue for schema and transformation
Amazon Athena for analytics

It’s elegant:

No servers
Pay-per-use
Infinite scale

But it assumes something dangerous:

That success = correctness

What Was Missing: Observability

The issue wasn’t the architecture.

It was the lack of visibility into:

What failed
What was retried
What never got processed

Without that, your data lake becomes a black box.

And black boxes are risky in data engineering.

The Fix: Designing for Failure (On Purpose)

For an E-commerce analytics workshop, I reworked the architecture with a different mindset:

Assume everything will fail—and make it visible when it does.

1. Decoupling Ingestion

Instead of triggering jobs directly from S3 events:

Amazon S3 emits events
Amazon SQS captures and buffers them

This introduces control:

Retry mechanisms
Backpressure handling
Event durability

2. Dead Letter Queue (DLQ): No Data Left Behind

Failures are inevitable. Silent failures are optional.

By adding a DLQ:

Failed messages are isolated after retries
Nothing is lost
You can inspect and replay events

This turns “unknown issues” into actionable signals.

3. Lightweight Orchestration
AWS Lambda polls SQS
Triggers AWS Glue jobs

Simple, decoupled, and reactive.

4. Optimized Storage for Analytics
Raw data lands in S3 (CSV/JSON)
Transformed into Parquet
Partitioned by date

This reduces cost and improves query performance in Amazon Athena.

Observability: Where Things Get Real

This is the layer most diagrams ignore—and the one that matters most in production.

Metrics That Tell the Truth

Using Amazon CloudWatch:

Queue depth (are messages piling up?)
DLQ size (are failures accumulating?)
Glue job status
Lambda error rates
Alerts That Don’t Wait

With Amazon SNS:

DLQ > 0 → immediate alert
Glue job failure → alert
Pipeline inactivity → alert

No need to “check dashboards.”
You get notified when something breaks.

A Simple Rule

If your system fails and you don’t know immediately:

You don’t have observability.

Trade-offs (Because Nothing Is Free)

Adding observability and DLQs improves reliability—but it’s not without cost.

What You Gain

Data reliability
Failure traceability
Faster incident response
Confidence in your analytics

What You Pay

More components (SQS, DLQ, Lambda)
Slightly higher operational complexity
Additional monitoring configuration
Small increase in cost (messages, metrics, alerts)

The Real Trade-off

You’re not choosing between “simple vs complex.”

You’re choosing between:

Simple system that hides failures
Slightly more complex system that exposes them

In production, that’s an easy decision.

Final Thought

Serverless is powerful. It removes infrastructure, reduces costs, and accelerates development.

But it also removes friction—and friction is often what makes failures visible.

If you don’t design for observability, your system will fail quietly.
And quiet failures are the most expensive ones.

How are you handling failures in your data pipelines today?

Are you capturing them… or hoping they don’t happen?

Stop Managing Clusters: Building a Scalable Serverless Data Lake on AWS

Mon, 06 Apr 2026 20:52:19 +0000

Building data pipelines shouldn't feel like babysitting servers. If you’ve ever managed a dedicated cluster just to run a few SQL queries, you know the pain: capacity planning, idle costs, and the "fun" of scaling infrastructure at 3 AM.

As a Data Engineer, I always follow a simple mantra: Design, then exist. (Or in this case: Design serverless, then relax.)

Today, we’re breaking down how to centralize your fragmented data into a Serverless Data Lake using the "Big Three" of AWS: S3, Glue, and Athena.

⚠️ The Problem: The "Data Swamps"
When an organization grows, its data ends up in silos—CSV reports in one bucket, application logs in another, and transactional exports scattered everywhere.

The result? Data is hard to find, and your analytical queries require heavy ETL before you can even start.

️ The Architecture: Storage ≠ Compute

The beauty of a serverless approach is the decoupling of storage from compute. You only pay for what you store and what you process.

Amazon S3 (The Backbone): This is your central repository. We don't just "dump" data here; we organize it into Layers:
Raw Layer: The "Source of Truth." Data exactly as it arrived.
Curated Layer: Cleaned, partitioned, and optimized data (think Parquet format).
AWS Glue (The Librarian): You don't want to manually define schemas. Glue Crawlers scan your S3 buckets, infer the schema, and populate the Glue Data Catalog.
Amazon Athena (The Engine): A serverless query engine that lets you run standard SQL directly against your files in S3. No clusters to spin up.

️ Mini-Demo: From S3 to SQL in 3 Steps

Ingest: Drop your dataset (CSV or JSON) into your raw S3 bucket.
Catalog: Run a Glue Crawler. It will "detect" your columns and create a table in the metadata catalog.
Query: Head over to the Athena Console and run a query like this:

SQL
-- Aggregating sales data directly from S3 files
SELECT

region, 
SUM(amount) as total_sales

FROM "data_lake_db"."sales_curated"
GROUP BY region
ORDER BY total_sales DESC;

Pro-Tips for the Community
If you're building this for production, keep these two things in mind:

Friends don't let friends use CSV for Analytics: Convert your data to Apache Parquet. Because it’s a columnar format, Athena only reads the columns you need. This can reduce your query costs by up to 90%.
Partitioning is King: Organize your S3 paths by date (e.g., s3://my-bucket/year=2026/month=04/). This limits the amount of data Athena has to scan, making your queries lightning-fast.

Final Thoughts
Serverless Data Lakes allow us to experiment fast. You can build a proof-of-concept in an afternoon and scale it to petabytes without ever touching a Linux terminal.

I’m curious—what’s your biggest headache when managing data at scale? Are you team "Serverless All The Way" or do you still prefer dedicated clusters for specific workloads?

Let’s discuss in the comments!

Stop Burning Money: Master AWS Lambda Provisioned Concurrency with Terraform

Thu, 19 Mar 2026 02:38:34 +0000

You’ve probably heard that AWS Lambda provisioned concurrency eliminates cold starts. What they don’t tell you? It can turn your $50 Lambda bill into $500+ overnight if you’re not careful.

Here’s the thing: provisioned concurrency is incredibly powerful, but most developers either avoid it entirely (suffering cold starts) or implement it carelessly (suffering bill shock). There’s a sweet spot in between, and Terraform makes it possible to hit it consistently.

The Problem: Cold Starts vs. Cost Explosions

Lambda cold starts happen when AWS needs to initialize a new execution environment. For a typical Node.js function, this adds 200-500ms of latency. For Java or .NET? You’re looking at 2-5 seconds.

The traditional solution is provisioned concurrency - keeping execution environments warm and ready. But here’s the catch:

On-demand pricing: ~$0.20 per 1M requests + compute time
Provisioned concurrency: $0.0000041667 per GB-second (always running) + $0.20 per 1M requests

For a 512MB function with 10 provisioned instances running 24/7:

Monthly cost = 10 instances × 0.5 GB × 730 hours × 3600 seconds × $0.0000041667
             = ~$546 per month

Just to keep functions warm. Before you’ve processed a single request.

The Strategy: Dynamic Provisioned Concurrency

The secret is treating provisioned concurrency like autoscaling for EC2 - scale it up when you need it, down when you don’t. Here’s how to do it right with Terraform.

Step 1: Calculate Your Break-Even Point

Not every function needs provisioned concurrency. Use this formula:

Break-even daily invocations = (Provisioned cost per day) / (Cold start cost savings per invocation)

Cold start cost = Cold start duration (s) × Memory (GB) × $0.0000166667

Example: 512MB function, 300ms cold start, 20% of requests are cold starts

Daily provisioned cost (1 instance) = 1 × 0.5 × 86400 × $0.0000041667 = $0.18
Cold start cost per invocation = 0.3 × 0.5 × $0.0000166667 = $0.0000025
Savings per invocation (20% cold) = $0.0000025 × 0.2 = $0.0000005

Break-even = $0.18 / $0.0000005 = 360,000 invocations/day

If you’re processing fewer than 360k requests per day, provisioned concurrency costs you money.

️ Step 2: Implement Smart Scaling with Terraform

Here’s a production-ready Terraform configuration that scales provisioned concurrency based on actual demand:

Basic Lambda Function Setup

# main.tf
resource "aws_lambda_function" "api_handler" {
  filename         = "lambda_function.zip"
  function_name    = "api-handler"
  role            = aws_iam_role.lambda_role.arn
  handler         = "index.handler"
  runtime         = "nodejs20.x"
  memory_size     = 512
  timeout         = 30

  environment {
    variables = {
      ENVIRONMENT = "production"
    }
  }

  publish = true  # Required for provisioned concurrency
}

# Create an alias for stable endpoint
resource "aws_lambda_alias" "live" {
  name             = "live"
  description      = "Live traffic alias"
  function_name    = aws_lambda_function.api_handler.function_name
  function_version = aws_lambda_function.api_handler.version
}

Provisioned Concurrency with Auto Scaling

# provisioned_concurrency.tf

# Enable provisioned concurrency
resource "aws_lambda_provisioned_concurrency_config" "api_handler" {
  function_name                     = aws_lambda_alias.live.function_name
  qualifier                         = aws_lambda_alias.live.name
  provisioned_concurrent_executions = 2  # Minimum instances
}

# Auto Scaling Target
resource "aws_appautoscaling_target" "lambda_target" {
  max_capacity       = 10  # Maximum instances
  min_capacity       = 2   # Minimum instances (must match provisioned concurrency)
  resource_id        = "function:${aws_lambda_function.api_handler.function_name}:${aws_lambda_alias.live.name}"
  scalable_dimension = "lambda:function:ProvisionedConcurrencyUtilization"
  service_namespace  = "lambda"

  depends_on = [aws_lambda_provisioned_concurrency_config.api_handler]
}

# Scale up when utilization > 70%
resource "aws_appautoscaling_policy" "lambda_scale_up" {
  name               = "lambda-scale-up"
  policy_type        = "TargetTrackingScaling"
  resource_id        = aws_appautoscaling_target.lambda_target.resource_id
  scalable_dimension = aws_appautoscaling_target.lambda_target.scalable_dimension
  service_namespace  = aws_appautoscaling_target.lambda_target.service_namespace

  target_tracking_scaling_policy_configuration {
    target_value = 70.0  # Target 70% utilization

    predefined_metric_specification {
      predefined_metric_type = "LambdaProvisionedConcurrencyUtilization"
    }

    scale_in_cooldown  = 300  # Wait 5 min before scaling down
    scale_out_cooldown = 60   # Wait 1 min before scaling up again
  }
}

Scheduled Scaling for Predictable Workloads

If your traffic follows a pattern (business hours, weekend drops), use scheduled actions:

# scheduled_scaling.tf

# Scale up for business hours (Mon-Fri 8 AM - 6 PM EST)
resource "aws_appautoscaling_scheduled_action" "scale_up_business_hours" {
  name               = "scale-up-business-hours"
  service_namespace  = aws_appautoscaling_target.lambda_target.service_namespace
  resource_id        = aws_appautoscaling_target.lambda_target.resource_id
  scalable_dimension = aws_appautoscaling_target.lambda_target.scalable_dimension

  schedule = "cron(0 8 ? * MON-FRI *)"  # 8 AM EST weekdays

  scalable_target_action {
    min_capacity = 5
    max_capacity = 15
  }
}

# Scale down for nights and weekends
resource "aws_appautoscaling_scheduled_action" "scale_down_off_hours" {
  name               = "scale-down-off-hours"
  service_namespace  = aws_appautoscaling_target.lambda_target.service_namespace
  resource_id        = aws_appautoscaling_target.lambda_target.resource_id
  scalable_dimension = aws_appautoscaling_target.lambda_target.scalable_dimension

  schedule = "cron(0 18 ? * MON-FRI *)"  # 6 PM EST weekdays

  scalable_target_action {
    min_capacity = 1
    max_capacity = 3
  }
}

# Weekend minimal capacity
resource "aws_appautoscaling_scheduled_action" "scale_down_weekend" {
  name               = "scale-down-weekend"
  service_namespace  = aws_appautoscaling_target.lambda_target.service_namespace
  resource_id        = aws_appautoscaling_target.lambda_target.resource_id
  scalable_dimension = aws_appautoscaling_target.lambda_target.scalable_dimension

  schedule = "cron(0 0 ? * SAT *)"  # Midnight Saturday

  scalable_target_action {
    min_capacity = 1
    max_capacity = 2
  }
}

IAM Permissions

# iam.tf

resource "aws_iam_role" "lambda_role" {
  name = "api-handler-lambda-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Action = "sts:AssumeRole"
      Effect = "Allow"
      Principal = {
        Service = "lambda.amazonaws.com"
      }
    }]
  })
}

resource "aws_iam_role_policy_attachment" "lambda_basic" {
  role       = aws_iam_role.lambda_role.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
}

# Required for Application Auto Scaling
resource "aws_iam_role" "autoscaling_role" {
  name = "lambda-autoscaling-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [{
      Action = "sts:AssumeRole"
      Effect = "Allow"
      Principal = {
        Service = "application-autoscaling.amazonaws.com"
      }
    }]
  })
}

resource "aws_iam_role_policy_attachment" "autoscaling_policy" {
  role       = aws_iam_role.autoscaling_role.name
  policy_arn = "arn:aws:iam::aws:policy/aws-service-role/LambdaReplicatorPolicy"
}

Step 3: Monitor and Optimize

Create CloudWatch dashboards to track your investment:

# monitoring.tf

resource "aws_cloudwatch_metric_alarm" "high_cost_alert" {
  alarm_name          = "lambda-provisioned-concurrency-high-cost"
  comparison_operator = "GreaterThanThreshold"
  evaluation_periods  = "2"
  metric_name         = "ProvisionedConcurrencyUtilization"
  namespace           = "AWS/Lambda"
  period              = "300"
  statistic           = "Average"
  threshold           = "30"  # Alert if avg utilization < 30%
  alarm_description   = "Provisioned concurrency may be over-provisioned"
  treat_missing_data  = "notBreaching"

  dimensions = {
    FunctionName = aws_lambda_function.api_handler.function_name
    Resource     = "${aws_lambda_function.api_handler.function_name}:${aws_lambda_alias.live.name}"
  }
}

Real-World Cost Comparison

Let’s say you have an API with these characteristics:

Traffic: 50k requests/day during business hours (10 hours), 5k requests overnight
Function: 512MB, 200ms average execution, 400ms cold start
Cold start rate: 20% without provisioned concurrency

Without Provisioned Concurrency

Compute cost: 55k × 0.2s × 0.5GB × $0.0000166667 = $0.92/day
Request cost: 55k × $0.0000002 = $0.01/day
Cold start penalty: 55k × 0.2 × 0.4s × 0.5GB × $0.0000166667 = $0.37/day

Total: ~$40/month

With Always-On Provisioned Concurrency (5 instances)

Provisioned cost: 5 × 0.5GB × 86400s × $0.0000041667 = $0.90/day
Compute cost: 55k × 0.2s × 0.5GB × $0.0000166667 = $0.92/day
Request cost: 55k × $0.0000002 = $0.01/day

Total: ~$56/month

With Scheduled Provisioned Concurrency (5 instances @ 10hrs, 1 instance @ 14hrs)

Provisioned cost: (5 × 0.5 × 36000) + (1 × 0.5 × 50400) × $0.0000041667 = $0.48/day
Compute cost: $0.92/day
Request cost: $0.01/day

Total: ~$43/month

Savings: $13/month vs always-on, while eliminating cold starts during peak hours. Scale this across 20 functions and you’re saving $3,120/year.

Best Practices Checklist

✅ Always use an alias - Provisioned concurrency requires versioning
✅ Start small - Begin with min_capacity = 1 or 2, monitor utilization
✅ Set cooldown periods - Prevent thrashing (scale_in_cooldown = 300s minimum)
✅ Match workload patterns - Use scheduled actions for predictable traffic
✅ Monitor utilization - Alert when average utilization < 40% (you’re wasting money)
✅ Test scaling behavior - Simulate load spikes in staging first
✅ Version carefully - Updating function code requires reprovisioning (brief downtime)

⚠️ Common Gotchas

1. The “Insufficient Data” Trap
When scaling down to zero, CloudWatch alarms can get stuck in “INSUFFICIENT_DATA” state. Solution: Keep min_capacity = 1 or use scheduled scaling to go to zero during known idle periods.

2. Deployment Delays
Updating a function with provisioned concurrency takes 2-3 minutes to reprovision. For CI/CD, either:

Accept the delay
Use blue/green deployments with two aliases
Temporarily disable provisioned concurrency during deploys

3. Region Limits
Default provisioned concurrency limit is 100 instances per account per region. Request increases through AWS Support if needed.

Quick Start Commands

# Initialize Terraform
terraform init

# Preview changes
terraform plan

# Deploy
terraform apply

# Check provisioned concurrency status
aws lambda get-provisioned-concurrency-config \
  --function-name api-handler \
  --qualifier live

# Monitor scaling activity
aws application-autoscaling describe-scaling-activities \
  --service-namespace lambda \
  --resource-id "function:api-handler:live"

When to Use Provisioned Concurrency

Great for:

Customer-facing APIs with strict latency SLAs (<100ms p99)
Functions with >2s cold starts (Java, .NET, large dependencies)
Predictable, high-volume traffic patterns
Functions processing >100k requests/day

Skip it for:

Internal tools and batch processing
Low-traffic endpoints (<10k requests/day)
Functions with fast cold starts (<200ms)
Unpredictable, sporadic workloads

Complete Example Repository

Here’s a complete working example structure:

lambda-provisioned-concurrency/
├── main.tf
├── provisioned_concurrency.tf
├── scheduled_scaling.tf
├── iam.tf
├── monitoring.tf
├── variables.tf
├── outputs.tf
└── lambda_function/
    ├── index.js
    └── package.json

variables.tf:

variable "function_name" {
  description = "Lambda function name"
  type        = string
  default     = "api-handler"
}

variable "min_capacity" {
  description = "Minimum provisioned concurrency instances"
  type        = number
  default     = 2
}

variable "max_capacity" {
  description = "Maximum provisioned concurrency instances"
  type        = number
  default     = 10
}

variable "target_utilization" {
  description = "Target utilization percentage for auto scaling"
  type        = number
  default     = 70
}

outputs.tf:

output "function_arn" {
  value = aws_lambda_function.api_handler.arn
}

output "function_url" {
  value = aws_lambda_alias.live.invoke_arn
}

output "estimated_monthly_cost" {
  value = "Min: \({var.min_capacity * 0.5 * 730 * 3600 * 0.0000041667}, Max: \){var.max_capacity * 0.5 * 730 * 3600 * 0.0000041667}"
}

Final Thoughts

Provisioned concurrency isn’t an all-or-nothing decision. The key is using it strategically:

Calculate your break-even point
Implement dynamic scaling with Terraform
Monitor utilization and costs
Optimize based on real data

With this approach, you can eliminate cold starts during peak hours while avoiding the $500+ monthly surprise on functions that only need it 20% of the time.

Remember: The goal isn’t zero cold starts - it’s optimal cost for your latency requirements.

Have you implemented provisioned concurrency? What’s your biggest cost optimization win with Lambda? Share in the comments!

Found this helpful? Follow me for more AWS cost optimization tips with Terraform!

Your First Azure Function: HTTP Triggers Step-by-Step

Fri, 13 Feb 2026 07:31:47 +0000

Theory doesn't ship features. You learned the why in Part 1—when serverless makes sense, the economics, how it compares to App Service and Container Apps. Now let's build something.

In this second part of the Azure Functions for .NET Developers series, we'll create a project from scratch, understand every line of generated code, and build three HTTP trigger patterns that cover most real-world API scenarios. By the end, you'll have a working HTTP API running locally on your machine. No Azure subscription required.

All code from this article is available in the azure-functions-samples repository.

Prerequisites

You need four tools installed before we start. Here's what to check and where to get them:

Tool	Verify	Expected
.NET 10 SDK	`dotnet --version`	10.x
Azure Functions Core Tools	`func --version`	4.x
VS Code + Azure Functions extension	—	Latest
Azurite	`npx azurite --version`	Any

.NET 10 SDK is required to build and run our function app. It's an LTS release, supported through November 2028. Grab it from dot.net/download if you don't have it.

Azure Functions Core Tools v4 lets you create, run, and debug functions locally. Install via npm, Homebrew, or Chocolatey:

# npm (any OS)
npm install -g azure-functions-core-tools@4

# Homebrew (macOS)
brew tap azure/functions
brew install azure-functions-core-tools@4

# Chocolatey (Windows)
choco install azure-functions-core-tools-4

VS Code with the Azure Functions extension gives you project templates, debugging, and deployment in one place. Visual Studio and Rider work too—use whatever you're comfortable with. The CLI commands in this article work regardless of editor.

Azurite emulates Azure Storage locally. The Functions runtime needs a storage connection even for HTTP triggers (it uses storage internally for features like function key management and, in production, for coordinating across instances). Install it globally or run it on demand:

# Install globally
npm install -g azurite

# Or run on demand without installing
npx azurite

If the verification commands all return version numbers and you have VS Code with the Azure Functions extension installed, you're ready.

Creating Your First Project

With your tools in place, open a terminal and scaffold a new function app:

func init HttpTriggerDemo --worker-runtime dotnet-isolated --target-framework net10.0
cd HttpTriggerDemo
func new --template "HTTP trigger" --name Hello

The first command creates the project. --worker-runtime dotnet-isolated selects the isolated worker model (the modern default—we'll explain what "isolated" means in Part 5). --target-framework net10.0 targets .NET 10.

Once inside the project directory, func new adds an HTTP-triggered function called Hello from a built-in template.

Your project now looks like this:

HttpTriggerDemo/
├── Hello.cs              # Your function code
├── Program.cs            # Application entry point
├── HttpTriggerDemo.csproj # Project file
├── host.json             # Runtime configuration
└── local.settings.json   # Local environment variables

The Project File

Open HttpTriggerDemo.csproj:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>net10.0</TargetFramework>
    <AzureFunctionsVersion>v4</AzureFunctionsVersion>
    <OutputType>Exe</OutputType>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
  </PropertyGroup>

  <ItemGroup>
    <FrameworkReference Include="Microsoft.AspNetCore.App" />
    <PackageReference Include="Microsoft.Azure.Functions.Worker" Version="2.51.0" />
    <PackageReference Include="Microsoft.Azure.Functions.Worker.Sdk" Version="2.0.7" />
    <PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.Http.AspNetCore" Version="2.1.0" />
  </ItemGroup>
</Project>

A few things to note:

OutputType: Exe — the isolated worker model runs as a standalone executable, not a class library loaded into the Functions host. This is what gives you full control over the process.
FrameworkReference: Microsoft.AspNetCore.App — brings in ASP.NET Core types like HttpRequest and IActionResult. Without this, you'd use the lower-level HttpRequestData API.
Extensions.Http.AspNetCore — the bridge that maps ASP.NET Core's HTTP abstractions to the Functions runtime. This is what makes the developer experience feel like writing a regular web API.
ImplicitUsings: enable — the compiler automatically includes common using statements (System, System.Collections.Generic, System.Linq, System.Threading.Tasks, and others). That's why our code files don't start with a block of using directives.
Nullable: enable — turns on nullable reference types. The compiler warns you when code might dereference null without checking. You'll see this in action when we read query string values—req.Query["name"] returns string?, not string.

The Entry Point

Program.cs is minimal:

using Microsoft.Azure.Functions.Worker.Builder;

var builder = FunctionsApplication.CreateBuilder(args);

builder.ConfigureFunctionsWebApplication();

builder.Build().Run();

ConfigureFunctionsWebApplication() is the key line. It enables ASP.NET Core integration—without it, you'd need to use HttpRequestData and HttpResponseData instead of the familiar HttpRequest and IActionResult. For HTTP-triggered functions, this is almost always what you want.

FunctionsApplication.CreateBuilder(args) is a convenience factory introduced in Worker SDK 2.0. It returns a pre-configured IHostApplicationBuilder with Functions defaults already wired up—logging, JSON serializer options, and middleware. The key using to remember is Microsoft.Azure.Functions.Worker.Builder, which brings the factory method into scope. You may also see an older pattern that starts with new HostBuilder() and calls .ConfigureFunctionsWorkerDefaults()—it still works, but requires more manual setup (for example, explicit ConfigureAppConfiguration() calls to load appsettings.json). func init scaffolds FunctionsApplication.CreateBuilder by default, so stick with it for new projects.

This is also where you'd register services for dependency injection, add middleware, or configure logging. We'll keep it simple for now.

Understanding the Generated Code

With the project structure in place, let's look at the function itself. The func new command created a file called Hello.cs with a complete working function. Before we run it, let's understand what each part does. Open Hello.cs:

public class Hello(ILogger<Hello> logger)
{
    [Function("Hello")]
    public IActionResult Run(
        [HttpTrigger(AuthorizationLevel.Function, "get", "post")] HttpRequest req)
    {
        logger.LogInformation("C# HTTP trigger function processed a request.");
        return new OkObjectResult("Welcome to Azure Functions!");
    }
}

Let's break this apart.

public class Hello(ILogger<Hello> logger) uses a C# 12 primary constructor. The ILogger<Hello> parameter is injected by the dependency injection container automatically—no need to write a separate constructor or field assignment. The logger is available throughout the class.

[Function("Hello")] registers this method with the Functions runtime. The string "Hello" becomes the function's name and appears in the URL path: http://localhost:7071/api/Hello. If you rename the class but keep the attribute string the same, the URL stays the same.

[HttpTrigger(AuthorizationLevel.Function, "get", "post")] is where the behavior is defined:

AuthorizationLevel.Function means callers need a function-specific API key to invoke this endpoint. Other options are Anonymous (no key required) and Admin (requires the master host key). When running locally, authorization is bypassed regardless of the level you set.
"get", "post" lists the allowed HTTP methods. Requests using other methods receive a 405 Method Not Allowed.

HttpRequest req is the full ASP.NET Core request object. Query strings, headers, body, route values—it's all there. If you've written ASP.NET Core controllers or minimal APIs, this is the same type.

IActionResult is the return type. OkObjectResult maps to a 200 response. The runtime serializes the object to JSON (or returns it as plain text for strings) and sends it back to the caller. All the familiar result types work: BadRequestResult, NotFoundResult, CreatedResult, and so on.

Run It

Before we modify anything, let's see this function in action. Start Azurite in a separate terminal (or in the background):

npx azurite --silent --location /tmp/azurite &

Then start the Functions runtime from your project directory:

func start

You should see output listing your function's URL:

Functions:

        Hello: [GET,POST] http://localhost:7071/api/Hello

Test it in another terminal:

curl http://localhost:7071/api/Hello
# → Welcome to Azure Functions!

That's a working HTTP endpoint, running locally, no Azure subscription needed. Keep func start running—we'll build on this function next.

HTTP Trigger Deep Dive

With a running function under your belt, let's build three patterns that cover the majority of real-world HTTP function scenarios. After each change, stop func start with Ctrl+C and restart it to pick up the new code.

Pattern 1: GET with Query String

The simplest pattern—read a value from the URL and return a response:

public class HelloFunction(ILogger<HelloFunction> logger)
{
    [Function("Hello")]
    public IActionResult Run(
        [HttpTrigger(AuthorizationLevel.Function, "get")] HttpRequest req)
    {
        logger.LogInformation("Hello function triggered");

        string? name = req.Query["name"];
        return new OkObjectResult($"Hello, {name ?? "world"}!");
    }
}

req.Query["name"] reads the name parameter from the query string. If it's missing, the value is null, so we fall back to "world" with the null-coalescing operator. Note the string? — the query collection returns a nullable type, and with <Nullable>enable</Nullable> in the project file, the compiler enforces this.

Test it:

curl "http://localhost:7071/api/Hello?name=Azure"
# → Hello, Azure!

curl "http://localhost:7071/api/Hello"
# → Hello, world!

This pattern works well for simple lookups—search queries, filtering lists, or any GET request where the parameters are short and cacheable.

Pattern 2: POST with JSON Body

For creating or updating resources, you'll typically accept a JSON request body. C# records and ASP.NET Core model binding make this clean:

using FromBodyAttribute = Microsoft.Azure.Functions.Worker.Http.FromBodyAttribute;

public record CreateOrderRequest(string ProductId, int Quantity);

public class OrderFunction(ILogger<OrderFunction> logger)
{
    [Function("CreateOrder")]
    public IActionResult CreateOrder(
        [HttpTrigger(AuthorizationLevel.Function, "post", Route = "orders")] HttpRequest req,
        [FromBody] CreateOrderRequest order)
    {
        logger.LogInformation("Order for {ProductId} x{Quantity}", order.ProductId, order.Quantity);
        return new CreatedResult($"/orders/{Guid.NewGuid()}", order);
    }
}

Several things happening here:

record CreateOrderRequest defines an immutable data type. Records are ideal for request and response DTOs—they give you value equality, a clean ToString(), and immutability out of the box. No need to write a class with properties and a constructor.

[FromBody] CreateOrderRequest order tells the ASP.NET Core model binder to deserialize the JSON request body into a CreateOrderRequest instance. If the JSON doesn't match (missing required fields, wrong types), the runtime returns a 400 Bad Request automatically. One gotcha: with ASP.NET Core integration enabled, FromBody exists in two namespaces—Microsoft.Azure.Functions.Worker.Http and Microsoft.AspNetCore.Mvc. You need the Functions Worker version. The using alias at the top of the file resolves the ambiguity so you can use [FromBody] without a fully qualified name.

Route = "orders" overrides the default route. Without it, the URL would be /api/CreateOrder (derived from the function name). With it, the URL becomes /api/orders—cleaner and more RESTful.

CreatedResult returns HTTP 201 with a Location header pointing to the new resource. Standard REST semantics.

Test it:

curl -X POST http://localhost:7071/api/orders \
  -H "Content-Type: application/json" \
  -d '{"productId":"SKU-001","quantity":3}'
# → {"productId":"SKU-001","quantity":3}
# (with 201 status and Location header)

The Content-Type: application/json header is required. Without it, model binding won't kick in and you'll get a deserialization error.

Pattern 3: Route Parameters

For resource-oriented APIs, you'll want parameters embedded in the URL path rather than the query string:

public class ProductFunction(ILogger<ProductFunction> logger)
{
    [Function("GetProduct")]
    public IActionResult GetProduct(
        [HttpTrigger(AuthorizationLevel.Function, "get",
            Route = "products/{category:alpha}/{id:int?}")] HttpRequest req,
        string category, int? id)
    {
        logger.LogInformation("Looking up {Category}, id={Id}", category, id);
        return new OkObjectResult(new { category, id });
    }
}

Route parameters are defined in curly braces inside the Route string and captured as method parameters matched by name. The runtime extracts category and id from the URL and passes them directly to your method.

Route constraints validate parameters before your code runs:

Constraint	Meaning	Example
`:alpha`	Letters only	`electronics` matches, `123` doesn't
`:int`	Integer	`42` matches, `abc` doesn't
`:int?`	Optional integer	Parameter can be omitted
`:guid`	GUID format	`{id:guid}`
`:length(5)`	Exact string length	`{code:length(5)}`
`:min(1)`	Minimum integer value	`{page:min(1)}`

If a constraint fails, the runtime returns a 404—your function never executes.

Test it:

curl http://localhost:7071/api/products/electronics/42
# → {"category":"electronics","id":42}

curl http://localhost:7071/api/products/electronics
# → {"category":"electronics","id":null}

curl http://localhost:7071/api/products/123/42
# → 404 (constraint :alpha rejects "123")

You now have three working patterns that cover most real-world HTTP function scenarios.

Running and Testing Locally

Restart func start now that all three functions are in place. You should see all of them listed:

Azure Functions Core Tools
Core Tools Version: 4.6.0

Functions:

        CreateOrder: [POST] http://localhost:7071/api/orders

        GetProduct: [GET] http://localhost:7071/api/products/{category:alpha}/{id:int?}

        Hello: [GET] http://localhost:7071/api/Hello

For quick reference, here are all three test commands together:

# Pattern 1: GET with query string
curl "http://localhost:7071/api/Hello?name=Azure"

# Pattern 2: POST with JSON body
curl -X POST http://localhost:7071/api/orders \
  -H "Content-Type: application/json" \
  -d '{"productId":"SKU-001","quantity":3}'

# Pattern 3: Route parameters
curl http://localhost:7071/api/products/electronics/42

You can also use Postman, the REST Client extension for VS Code, or HTTPie—whatever fits your workflow.

A note on authorization: when running locally, AuthorizationLevel.Function is bypassed. You don't need to pass a function key. This makes local development frictionless. Once deployed, clients will need to include the key as a query parameter (?code=<key>) or in the x-functions-key header.

Troubleshooting

If things don't work on the first try, check these common issues:

"Can't determine project language" or "No job functions found"

The .csproj file is missing required settings. Verify it targets net10.0, has <AzureFunctionsVersion>v4</AzureFunctionsVersion>, and includes the Worker SDK packages. Run dotnet build separately to check for compilation errors.

Port 7071 already in use

Another Functions host (or another process) is using the default port. Either kill it or start on a different port:

func start --port 7072

"Value cannot be null: provider" or storage connection errors

Azurite isn't running. The Functions runtime needs a storage emulator even for HTTP triggers. Start Azurite, or verify that local.settings.json contains:

{
  "IsEncrypted": false,
  "Values": {
    "AzureWebJobsStorage": "UseDevelopmentStorage=true",
    "FUNCTIONS_WORKER_RUNTIME": "dotnet-isolated"
  }
}

UseDevelopmentStorage=true tells the SDK to connect to Azurite on its default ports.

JSON deserialization errors on POST requests

Two common causes: missing the Content-Type: application/json header, or property name mismatches between your JSON and your C# record. By default, the serializer is case-insensitive, so productId and ProductId both work. But the property names must exist on the target type.

"The listener for function X was unable to start"

Usually a runtime version mismatch. Confirm func --version returns 4.x and your .csproj has <AzureFunctionsVersion>v4</AzureFunctionsVersion>. If you recently upgraded the Core Tools, also run dotnet clean and rebuild.

Conclusion

We went from an empty directory to a running HTTP API with three production patterns:

Query strings for simple reads (GET /api/Hello?name=Azure)
JSON bodies for mutations (POST /api/orders)
Route parameters for resource-oriented endpoints (GET /api/products/electronics/42)

All running locally, no Azure subscription needed.

The isolated worker model with ASP.NET Core integration gives you familiar types—HttpRequest, IActionResult, [FromBody]—so writing Azure Functions feels like writing any other .NET web API. The main difference is that Azure handles the hosting, scaling, and infrastructure.

The code samples in this article are deliberately simple. In a real project, you'd add input validation, error handling, and probably connect to a database or external service. But the patterns are the same—the [HttpTrigger] attribute, the route configuration, the model binding. Once you understand these building blocks, everything else is regular C#.

All code from this article is available in the azure-functions-samples repository—clone it, run func start, and experiment.

What's Next in This Series

Part 3: Beyond HTTP Triggers (coming soon) covers timer triggers for scheduled jobs, queue triggers for message processing, and blob triggers for reacting to file uploads. Same project structure, new event sources.

Part 4: Local Development Setup (coming soon) digs into the day-to-day workflow—debugging in VS Code, hot reload, and productivity tips.

Part 5: Understanding the Isolated Worker Model (coming soon) explains what "isolated" means, why Microsoft created this model, how it differs from the legacy in-process model, and what it means for your code.

And yes—we'll cover deploying to Azure with CI/CD via GitHub Actions later in the series, so everything you're building locally will make it to the cloud.

Why Azure Functions? Serverless for .NET Developers

Wed, 11 Feb 2026 07:29:29 +0000

You've just finished building a webhook handler. Maybe fifty lines of code—parse some JSON, hit a database, return a response. Now comes the fun part: provisioning a VM, configuring IIS, setting up SSL certificates, writing health checks, and figuring out how to scale when traffic spikes at 3 AM. Sound familiar?

This is the problem serverless computing solves. Instead of managing infrastructure, you write the code that matters—your business logic—and let the platform handle everything else. In 2026, with cloud costs under constant scrutiny, the pay-per-execution model makes even more sense: why pay for idle servers when you can pay only for the milliseconds your code actually runs?

For .NET developers, Azure Functions has matured into a first-class serverless platform. The isolated worker model, combined with .NET 10's performance improvements, means you get the full power of modern .NET without compromise.

Every major cloud provider offers a serverless compute platform—AWS has Lambda, Google Cloud has Cloud Functions. For .NET developers, Azure Functions is the most natural choice: first-class support in the .NET SDK, seamless integration with Visual Studio and Rider, and deep ties to the broader Azure ecosystem you're likely already using. If your team runs on .NET, you won't fight the tooling here.

In this series, we'll explore Azure Functions from the ground up—when to use them, how they work, and the patterns that make serverless development practical. Let's start with the fundamentals.

What Azure Functions Actually Is

Azure Functions is Microsoft's event-driven serverless compute service. Instead of managing servers or worrying about infrastructure, you write small, focused pieces of code that run in response to events. Azure handles scaling, patching, and availability automatically.

Triggers: What Starts Your Code

A trigger defines what event causes your function to execute. Common triggers include:

HTTP Trigger - Respond to web requests (APIs, webhooks)
Timer Trigger - Run on a schedule (cron expressions)
Service Bus Trigger - Process messages from queues or topics
Cosmos DB Trigger - React to document changes in your database
Blob Trigger - Execute when files are uploaded to storage

Bindings: Declaring Inputs and Outputs

Bindings connect your function to other Azure services declaratively. Instead of writing SDK code to read from a queue or write to a database, you declare the connection through attributes—and Azure Functions handles the plumbing.

There are two types:

Input bindings - Read data when your function executes (e.g., fetch a document from Cosmos DB, read a blob from storage)
Output bindings - Write data after your function completes (e.g., add a message to a queue, insert a row into Table Storage)

Your function's return value can serve as an output binding directly. Return an object, and Azure Functions writes it to the configured destination—no explicit SDK calls needed.

Your First Function

Here's a minimal HTTP-triggered function using the .NET 10 isolated worker model:

public class HelloFunction(ILogger<HelloFunction> logger)
{
    [Function("Hello")]
    public IActionResult Run([HttpTrigger(AuthorizationLevel.Function, "get")] HttpRequest req)
    {
        logger.LogInformation("Processing request");
        return new OkObjectResult("Hello from Azure Functions!");
    }
}

The [HttpTrigger] attribute defines this as an HTTP GET endpoint with function-level authentication (we'll cover authentication levels in the next article). The OkObjectResult you return becomes the HTTP response sent back to the caller—Azure Functions maps ASP.NET Core's familiar result types to actual HTTP responses for you. That's all you need to deploy a working API endpoint.

When to Use Azure Functions (and When Not To)

Azure Functions excels in specific scenarios, but it's not a universal solution. Here's an honest breakdown to help you choose wisely.

Where Functions Shine

APIs with unpredictable traffic. If your endpoint gets 100 requests one hour and 10,000 the next, you pay only for what you use. No idle VMs burning budget overnight.

Event-driven workloads. Processing messages from Service Bus, reacting to blob uploads, or handling Event Grid notifications—Functions integrates natively with Azure's eventing ecosystem. A few lines of code and you're processing events at scale.

Scheduled jobs. Timer triggers replace fragile cron servers. Need to clean up stale data every night or send weekly reports? Functions handles the scheduling infrastructure for you.

Glue code between services. Moving data from one system to another, transforming payloads, or orchestrating simple workflows—Functions keeps these small integrations from becoming maintenance burdens.

Prototypes and MVPs. When validating an idea quickly, Functions lets you ship without configuring infrastructure. You can always migrate to containers later.

When to Look Elsewhere

Latency-critical paths. Cold starts on the Consumption plan can add seconds of delay. If you need consistent sub-10ms response times, consider App Service or Container Apps with always-on instances.

Long-running processes. The 10-minute timeout (or 230 seconds on HTTP triggers) won't work for heavy batch processing. Look at Durable Functions for orchestration or Container Apps Jobs for extended compute.

Heavy, sustained compute. If you're running CPU-intensive workloads continuously, reserved container instances often cost less than millions of function executions.

Stateful applications. WebSocket connections, in-memory caches, or persistent sessions don't fit the ephemeral Functions model.

A Note on Cost Protection

Pay-per-execution pricing has a flip side: you pay for every request, including malicious ones. A DDoS attack against an unprotected Function endpoint means you're footing the bill for traffic you never wanted.

For public-facing APIs, consider these safeguards:

Azure API Management — rate limiting, throttling, and request quotas
Azure Front Door or Application Gateway — WAF rules and DDoS protection
Function authorization levels — AuthorizationLevel.Function or AuthorizationLevel.Admin require keys, blocking anonymous traffic

The Consumption plan's cost model works best when you control who can call your endpoints. For truly public APIs expecting untrusted traffic, budget for a protection layer—or consider the Premium plan's predictable pricing.

Quick Comparison: Functions vs App Service vs Container Apps

Choosing the right Azure compute option can significantly impact your costs and application architecture. Here's how Azure Functions stacks up against the other popular choices for .NET workloads:

Feature	Azure Functions	App Service	Container Apps
Scale trigger	Events (HTTP, queue, timer)	CPU/memory rules	Events (KEDA), CPU/memory
Pricing	Pay-per-execution*	Fixed/reserved	Per-resource (vCPU/memory)
Cold starts	Yes (Consumption)	No	Configurable
Scale-out limit	200 (Consumption) / 1,000 (Flex)	Up to 30 instances	300–1,000 replicas
Best for	Event-driven, APIs	Web apps, APIs	Microservices

\Consumption plan. Premium and Dedicated plans use fixed or memory/CPU-based pricing, similar to Container Apps.*

Choose Azure Functions when you have event-driven workloads, sporadic traffic patterns, or want to minimize costs for low-volume APIs. Opt for App Service when you need predictable performance without cold starts for traditional web applications. Consider Container Apps when building microservices architectures that require more control over the runtime environment while still benefiting from managed scaling.

A Note on .NET 10

Throughout this series, we'll use .NET 10 with the isolated worker model—the recommended stack for new Azure Functions projects. It's an LTS release (supported through November 2028) and gives you full ASP.NET Core integration with familiar types like HttpRequest and IActionResult. We'll cover the setup in Part 2 and what "isolated worker model" means in detail in Part 5.

What's Next in This Series

This article covered the "why" of Azure Functions—now it's time for the "how." Over the next several parts, we'll build real-world skills you can apply immediately.

Part 2: Your First Azure Function — HTTP Triggers Step-by-Step (coming soon) walks through project creation, running locally, and testing with curl. You'll have a working HTTP endpoint in minutes.

Part 3: Beyond HTTP Triggers (coming soon) explores timer, queue, and blob triggers—the event-driven patterns that make Functions powerful.

Part 4: Local Development Setup (coming soon) covers the tools, debugging techniques, and productivity tips that make daily development smooth.

Part 5: Understanding the Isolated Worker Model (coming soon) explains the architecture behind modern Azure Functions—why Microsoft created it, how it works, and what it means for your code.

Part 6: Production Realities — When Serverless Stops Being Serverless (coming soon) covers what actually happens when enterprise requirements like VNet integration and private endpoints push you off the Consumption plan—and when to consider alternatives.

Conclusion

Azure Functions has become a practical choice for .NET developers who want to ship features without managing infrastructure. The isolated worker model with .NET 10 gives you modern language features, familiar patterns, and genuine pay-per-use economics.

Start small. Create an HTTP-triggered function, deploy it to Azure, and see how quickly you can go from code to production. The barrier to entry has never been lower.

Follow this series to build your serverless skills step by step. Next up: setting up your development environment and deploying your first function. Let's build something.

Handling Cold Starts in Serverless AI: Why Your First Request Fails (And How to Fix It)

Sun, 18 Jan 2026 12:08:41 +0000

First request to your AI model: timeout. Second request: instant success. If you've integrated AI APIs into serverless applications, you've probably hit this wall.

Here's what's happening, why it matters for user experience, and how I solved it without forcing users to manually retry.

The Problem: Cold Starts Kill First Impressions

I was testing LogicVisor (a code review platform using Gemini AI) when I noticed a pattern: after a few hours of inactivity, the first API call would consistently fail with "Model is temporarily unavailable. Please try again later". Trying again just a few seconds later always worked.

For a new user trying the platform for the first time, their experience would be:

Submit code for review
See an error message
Get told to "try again"

As you can expect, this isn't a great first experience. Even if the issue would be resolved on their second try, many would just leave.

Why This Happens: Resource Management in Serverless

On free/low-cost tiers of cloud AI services, providers deallocate resources during inactivity. When your request comes in after idle time, the model needs to "wake up":

Allocate compute resources
Load the model into memory
Initialize the runtime environment

This cold start adds latency—sometimes 2-10 seconds depending on model size. Your request times out before the model is ready.

This doesn't happen on premium tiers because you're paying for dedicated resources. But for no-cost/low-cost MVPs and proof-of-concept apps like mine, you'll deal with cold starts.

The Standard Solution: Exponential Backoff

The industry-standard approach is exponential backoff retry logic:

First retry: wait 2 seconds
Second retry: wait 4 seconds
Third retry: wait 8 seconds
Fourth retry: wait 16 seconds

This works well for distributed systems handling network congestion or database deadlocks where you don't know how long the issue will persist.

Why I Chose Linear Backoff Instead

For my specific use case, I knew:

The error was transient (always resolved on second attempt)
This was a user-facing application (waiting 16 seconds is unacceptable)
Maximum 3 retries was reasonable

Linear backoff fit better: 2s → 4s → 6s progression instead of exponential growth.

Here's the implementation:

// Helper function to call AI with linear backoff retry logic
async function callAIWithRetry(maxRetries = 3) {
  for (let attempt = 0; attempt < maxRetries; attempt++) {
    try {
      // Call Gemini or Groq AI service (simplified for clarity)
      const response = await ai.models.generateContentStream({...});
      return response;
    } catch (error) {
      const errorMessage = error instanceof Error ? error.message : String(error);
      const is503Error = 
        errorMessage.includes("503") || 
        errorMessage.includes("overloaded") ||
        errorMessage.includes("temporarily unavailable");
      
      if (is503Error && attempt < maxRetries - 1) {
        // Linear Backoff: 2s → 4s → 6s (not exponential 2s → 4s → 8s → 16s)
        const waitTime = 2000 * (attempt + 1); 
        
        // Notify user via Server-Sent Events (the UX fix)
        if (!coldStartNotified) {
          controller.enqueue(
            encoder.encode(`data: ${JSON.stringify({
              type: "cold_start",
              message: "Waking up sleepy reviewer... ☕"
            })}\n\n`)
          );
          coldStartNotified = true;
        }
        
        console.log(`Retrying in ${waitTime}ms (Attempt ${attempt + 1}/${maxRetries})`);
        await new Promise((resolve) => setTimeout(resolve, waitTime));
        continue;
      }
      
      throw error; // Not a 503 or max retries exceeded
    }
  }
  throw new Error("Max retries exceeded");
}

The key differences from exponential backoff:

Fixed increment (2 seconds) instead of exponential growth
User-facing messaging during retries via Server-Sent Events
Early exit after 3 attempts to avoid hanging

Making Delays Transparent: Frontend Handling

Backend retry logic solves the technical problem, but users still experience a delay. I added cold start detection on the frontend:

const response = await submitCode(
  code,
  language,
  problemName || "Code Review",
  selectedModel,
  (content: string, eventType?: string) => {
    // Handle cold start event
    if (eventType === "cold_start") {
      setIsColdStart(true);
      setSubmitting(false);
      return;
    }

    // Handle streaming content
    setStreaming(true);
    setStreamedContent((prev) => prev + content);
  }
);

When a cold start is detected, the UI shows:

{isColdStart && (
  <div className="mb-4 p-3 bg-amber-50 dark:bg-amber-900/20 
                  border border-amber-200 dark:border-amber-800 rounded-lg">
    <p className="text-sm text-amber-800 dark:text-amber-200">
      ☕ Waking up sleepy reviewer... This may take a few extra seconds.
    </p>
  </div>
)}

This turns a confusing timeout into an understandable loading state. Users know something is happening, not that the app is broken.

Alternative Strategies (And Why I Didn't Use Them)

1. Keep-Alive Mechanisms
Set up a cron job to ping your endpoint every 5 minutes, preventing cold starts entirely.

Why I skipped it: Adds infrastructure complexity and still incurs API costs even when no real users are active.

2. Upgrade to Premium Tier
Pay for dedicated resources, eliminate cold starts.

Why I skipped it: Not viable for an MVP with zero revenue. This is the eventual solution once the platform proves itself.

Results

With linear backoff + transparent messaging:

First-time users no longer see raw error messages
Retries happen automatically and transparently
Average additional latency: ~2-4 seconds on cold starts only
Warm requests: no change in performance

Takeaway

Cold starts are an infrastructure constraint you can't eliminate on free tiers, but you can handle them gracefully:

Implement retry logic appropriate to your error pattern (linear for transient errors, exponential for unknown duration)
Make delays visible and understandable to users through status messaging
Design for the 80% case (warm starts) while handling the 20% (cold starts)

User experience isn't just about speed—it's about managing expectations during unavoidable delays.

Have you dealt with cold starts in your serverless applications? What strategy worked for you?

Why I Ditched Redis for Cloudflare Durable Objects in My Rate Limiter?

Sun, 31 Aug 2025 04:02:18 +0000

Have you ever watched your serverless application crumble under unexpected traffic? Last month, our AI-powered image generator went viral on social media, and within hours we were drowning in requests. Our traditional rate limiting setup couldn't keep up with the distributed load across Cloudflare's edge network.

This experience taught me that rate limiting in serverless environments requires a fundamentally different approach. Here's how I built a production-ready rate limiter using Cloudflare Durable Objects that handles thousands of concurrent requests while running at the edge.

The Traditional Approach Falls Short

Most developers reach for Redis when implementing rate limiting. It's fast, reliable, and has excellent libraries. But in a serverless environment, especially one distributed across multiple edge locations, Redis introduces several challenges:

Latency Issues: Every rate limit check requires a round trip to your Redis instance, adding 50-200ms of latency depending on geographic distance.

Single Point of Failure: Your entire rate limiting system depends on Redis availability. If Redis goes down, you either block all traffic or allow unlimited requests.

Cold Start Problems: Serverless functions need to establish Redis connections on cold starts, further increasing response times.

Geographic Complexity: Running Redis replicas across multiple regions for low latency gets expensive quickly and introduces data consistency challenges.

Enter Durable Objects

Cloudflare Durable Objects solve these problems elegantly by providing stateful compute primitives that run directly at the edge. Each Durable Object instance maintains its own persistent state and can handle concurrent requests while ensuring strong consistency.

Think of Durable Objects as tiny, persistent computers that live at Cloudflare's edge locations. They wake up when needed, maintain state between requests, and automatically migrate to follow your traffic patterns.

Building the Rate Limiter

Here's the complete implementation of a production-ready rate limiter using Durable Objects:

import { DurableObject } from "cloudflare:workers";

interface ThrottleState {
  limitTimes: number;
  limitEndTimeMs: number;
  executedTimesCurrentCycle: number;
  currentCycle: number;
}

export interface TryApplyOptions {
  limitCycleExecutionTimes: number;
  limitCycleTimeMs: number;
}

export interface ThrottlerResponse {
  granted: boolean;
  state: ThrottleState;
}

export class ThrottlerDO extends DurableObject {
  limitCycleExecutionTimes = 10; // Default: 10 requests per cycle
  limitCycleTimeMs = 10 * 60 * 1000; // Default: 10 minutes

  constructor(ctx: DurableObjectState, env: Env) {
    super(ctx, env);
  }

  async getState(): Promise<ThrottlerResponse> {
    let state = await this.ctx.storage.get('throttle_state') as ThrottleState | null;
    
    if (!state) {
      state = {
        limitTimes: 0,
        limitEndTimeMs: 0,
        executedTimesCurrentCycle: 0,
        currentCycle: 0,
      };
    }
    
    const currentMs = Date.now();
    
    // Reset state if cycle has expired
    if (state.limitEndTimeMs > 0 && currentMs > state.limitEndTimeMs) {
      state = {
        ...state,
        limitEndTimeMs: 0,
        executedTimesCurrentCycle: 0,
      };
    }
    
    const granted = state.executedTimesCurrentCycle < this.limitCycleExecutionTimes;
    return { granted, state };
  }

  async tryApply(options?: TryApplyOptions): Promise<ThrottlerResponse> {
    if (options) {
      this.limitCycleExecutionTimes = options.limitCycleExecutionTimes;
      this.limitCycleTimeMs = options.limitCycleTimeMs;
    }
    
    let granted = false;
    let state = await this.ctx.storage.get('throttle_state') as ThrottleState | null;
    
    if (!state) {
      state = {
        limitTimes: 0,
        limitEndTimeMs: 0,
        executedTimesCurrentCycle: 0,
        currentCycle: 0,
      };
    }

    const currentMs = Date.now();
    
    // Reset cycle if expired
    if (state.limitEndTimeMs > 0 && currentMs > state.limitEndTimeMs) {
      state.limitEndTimeMs = 0;
      state.executedTimesCurrentCycle = 0;
    }
    
    // Check if request can be granted
    if (state.executedTimesCurrentCycle < this.limitCycleExecutionTimes) {
      state.executedTimesCurrentCycle++;
      granted = true;
    } else {
      state.limitTimes++;
      granted = false;
    }
    
    // Initialize new cycle if needed
    if (state.limitEndTimeMs === 0) {
      state.limitEndTimeMs = currentMs + this.limitCycleTimeMs;
      state.currentCycle++;
      
      if (state.currentCycle >= 65535) {
        state.currentCycle = 1;
      }
    }
    
    await this.ctx.storage.put('throttle_state', state);
    return { granted, state };
  }
}

Using the Rate Limiter

Integrating this rate limiter into your Worker is straightforward:

export default {
  async fetch(request: Request, env: Env) {
    // Create DO instance based on user identifier
    const userId = request.headers.get('user-id') || 'anonymous';
    const id = env.THROTTLER.idFromName(userId);
    const throttler = env.THROTTLER.get(id);
    
    // Check rate limit
    const result = await throttler.tryApply({
      limitCycleExecutionTimes: 5,
      limitCycleTimeMs: 60 * 1000, // 1 minute
    });
    
    if (!result.granted) {
      return new Response('Rate limit exceeded', { 
        status: 429,
        headers: {
          'Retry-After': '60'
        }
      });
    }
    
    // Process the actual request
    return processRequest(request);
  }
};

Real-World Performance

I've been running this rate limiter in production for several months now. The results have been impressive:

Latency: Average rate limit checks complete in under 5ms, compared to 80-150ms with our previous Redis setup.

Reliability: Zero downtime related to rate limiting failures since deployment. Durable Objects automatically handle failover and state migration.

Cost Efficiency: Running at roughly 30% of our previous Redis costs while serving 10x more requests.

Geographic Performance: Users in Asia Pacific see the same low latency as users in North America, thanks to Cloudflare's global network.

This architecture has been battle-tested at Fastjrsy, where we process thousands of AI-generated jersey design requests daily. During traffic spikes, the rate limiter seamlessly scales to handle 500+ concurrent requests per second while maintaining fair usage policies across our global user base.

Key Takeaways

Durable Objects excel when you need:

Stateful logic at the edge
Strong consistency guarantees
Automatic scaling and geographic distribution
Low latency for global applications

They may not be the right choice if you need:

Cross-platform compatibility beyond Cloudflare
Complex query capabilities like traditional databases
Shared state across multiple applications

What's Your Experience?

Have you implemented rate limiting in serverless environments? I'd love to hear about your approach and any challenges you've encountered. Are there specific use cases where you think Durable Objects would be particularly valuable?

Drop a comment below and let's discuss the future of edge computing architecture!

DevLog 20250523: Firebase Auth

Fri, 23 May 2025 19:06:57 +0000

The Authentication Problem

Let's face it: develop a custom authentication service is conceptually straightforward but practically troublesome. You either allow user registration or not, then authenticate by taking a look at username and password (likely hashed) against database, then provide user profile results for authorized users. Throughout the process you might set browser cache, deal with (generated) API keys for app usages (so password need not be sent every time). It's a handful of steps, and lots of work to code everything piece by piece. Worse, there is a whole database to worry about. One would naturally expect, such a thing should just exist as a service. Yes, there is.

Quick Comparison

|Platform|What it is|Free tier|Pros|Cons|
|-|-|-|-|-|
|AWS Cognito | Amazon’s identity and user‑management service for web and mobile apps, integrating natively with API Gateway/Lambda and Amplify JS. | 10,000 MAUs for direct/social sign‑in + 50 MAUs for SAML/OIDC federation; Identity Pools are free | No extra vendor to learn; pay only for MAUs above free tier; built‑in social and SAML/OIDC support; AWS‑managed hosted UI. | Learning curve around User Pools vs Identity Pools and IAM policies.|
|Auth0 | Popular identity platform with SDKs for nearly every framework. | 25,000 MAUs; unlimited social & Okta connections; 5 organizations; 5 actions/forms; passwordless & custom domain | Intuitive, developer‑friendly dashboard; extensible via Rules (JavaScript); excellent documentation. | Pricing increases sharply above free tier.|
|Firebase Authentication | Google’s auth service, part of Firebase/GCP. | Unlimited email/password, anonymous, and most social sign‑ins; pay only for phone‑SMS verification | Fastest setup for web/mobile; pre‑built UI widgets; seamless integration with other Firebase services. | SMS‑based authentication incurs per‑message cost.|
|Supabase Auth | Open‑source Firebase alternative built on PostgreSQL. | Unlimited API requests; 50,000 MAUs; 500 MB database; 5 GB bandwidth; 1 GB file storage; community support | True SQL under the hood; you own your data; robust JS client. | Free projects pause after 1 week of inactivity; limited to 2 active projects.|
|Netlify Identity | Netlify’s built‑in auth service powered by GoTrue. | Up to 1,000 registered users per site | Seamless drop‑in for Netlify‑hosted. | Limited to Netlify sites; basic compared to full‑blown IDaaS; deprecated as of Feb 28, 2025 .|
|Okta Developer Edition | Enterprise‑grade identity platform in a developer sandbox. | 100 MAUs | Enterprise features: OpenID Connect federation, SCIM provisioning, lifecycle hooks, universal directory | Console can feel heavy for hobby projects.|

Recommendations:

Full AWS: Use AWS Cognito (with Amplify libraries) to stay within the AWS ecosystem and enjoy a free tier up to 10 K direct/social MAUs and 50 MAUs for SAML/OIDC.
Rapid prototyping: Firebase Authentication for the fastest setup (unlimited email/password/social; pay only for SMS).
SQL‑power & control: Supabase Auth if you need open‑source Postgres‑based auth with a 50 K MAU free tier and 500 MB of built‑in storage.

For reasons I would not disclose, we picked Firebase for Divooka and other Methodox services.

The Setup

For Firebase Auth, the "API Key" just identifies the application and can totally be shown on the client side script.

<!DOCTYPE html>
<html>
<head>
  <meta charset="utf-8"/>
  <title>Custom Email/Password Auth</title>
  <script type="module">
    import { initializeApp } from "https://www.gstatic.com/firebasejs/11.8.1/firebase-app.js";
    import {
      getAuth,
      createUserWithEmailAndPassword,
      updateProfile,
      signInWithEmailAndPassword,
      signOut
    } from "https://www.gstatic.com/firebasejs/11.8.1/firebase-auth.js";

    // Firebase web config
    // For Firebase JS SDK v7.20.0 and later, measurementId is optional
    const firebaseConfig = {
        apiKey: "*********************",
        authDomain: "*********************",
        projectId: "*********************",
        storageBucket: "*********************",
        messagingSenderId: "*********************",
        appId: "*********************",
        measurementId: "*********************"
    };
    // Initialize Firebase
    const app = initializeApp(firebaseConfig);
    const auth = getAuth(app);

    // Register a new user
    async function registerUser() {
      const email = document.getElementById("regEmail").value;
      const password = document.getElementById("regPass").value;
      const username = document.getElementById("regName").value;
      try {
        const cred = await createUserWithEmailAndPassword(auth, email, password);
        // Set a displayName on the Firebase user record
        await updateProfile(cred.user, { displayName: username });
        alert("Registration successful! You can now log in.");
      } catch (e) {
        console.error(e);
        alert("Registration error: " + e.message);
      }
    }

    // Login existing user
    async function loginUser() {
      const email = document.getElementById("loginEmail").value;
      const password = document.getElementById("loginPass").value;
      try {
        const cred = await signInWithEmailAndPassword(auth, email, password);
        const idToken = await cred.user.getIdToken();

        // Call your API
        const resp = await fetch("https://myapi.com/profile", {
          headers: { "Authorization": "Bearer " + idToken }
        });
        if (!resp.ok) throw new Error(await resp.text());
        const profile = await resp.json();
        document.getElementById("output").textContent = JSON.stringify(profile, null, 2);
      } catch (e) {
        console.error(e);
        alert("Login error: " + e.message);
      }
    }

    // Logout
    function logoutUser() {
      signOut(auth);
      document.getElementById("output").textContent = "";
      alert("Signed out");
    }

    window.onload = () => {
      document.getElementById("btnRegister").onclick = registerUser;
      document.getElementById("btnLogin").onclick    = loginUser;
      document.getElementById("btnLogout").onclick   = logoutUser;
    };
  </script>
</head>
<body>
  <h2>Register</h2>
  <input id="regName"  type="text"     placeholder="Username"/><br/>
  <input id="regEmail" type="email"    placeholder="Email"/><br/>
  <input id="regPass"  type="password" placeholder="Password"/><br/>
  <button id="btnRegister">Register</button>

  <hr/>

  <h2>Login</h2>
  <input id="loginEmail" type="email"    placeholder="Email"/><br/>
  <input id="loginPass"  type="password" placeholder="Password"/><br/>
  <button id="btnLogin">Log In</button>
  <button id="btnLogout">Log Out</button>

  <h3>Profile:</h3>
  <pre id="output"></pre>
</body>
</html>

Create API Key

To create API key, navigate to the project then find "Add App" in Project Settings, which is a hidden menu.

References

Official doc on API key, which is not very clear without pictures: here

DevLog 20250522: Serverless & Serverside vs Client Side Rendering

Fri, 23 May 2025 08:33:11 +0000

Full Title: Serverless vs Client-Side Rendering vs Dynamic Content (Server-Side Rendering) vs Compiled/Code Generation for Static Sites

Prelude

There is a lot of confusion about the term "dynamic website" for laypeople. Let's clear that up first: "dynamic" here doesn't mean sleek animation or any sort of table filtering—it refers to whether the website content itself is variable, can change based on conditions, or is fetched from some server or database. The essence of a dynamic website is that it's constructed at runtime, can change based on conditions, and is opposed to "static," meaning the contents are fixed and will not change unless someone modifies the source. In the case of the web, the source usually means the HTML itself.

Also, "serverless" doesn't mean there is no server. There must be a server machine hosted by someone somewhere to do the computation. Serverless means we, as the developers, do not maintain the server ourselves, and there is some service that hosts all the contents, including the web server we need. To give you an exaggerated example: technically speaking, if you hire a contractor for $10,000 to develop and maintain a website for you—so whenever you need to update or modify the website in any way, you go to the contractor and talk to them, and throughout this process you never manually oversee the website or the server—then it's also "serverless."

Quick Recap / TL;DR

Here are the key ideas:

Serverless may feel easy and highly scalable operationally, but there are so many service providers that picking the right one becomes a long process of trial and error. On the other hand, serverless has its pitfalls—it's not all that good.
Client-side rendering (and single-page applications) works very well with serverless—we can use a static server to host the initial webpage and JS, then use JS to fetch data from a CDN or other static sources, rendering everything in the user's browser. However, one must note that most web crawlers (including AI agents) simply do not play well with such dynamic/runtime-generated content—and it’s simply impossible to predict in most cases. So forget about SEO if we rely on serverless. Forget about path-based URLs or endpoints since they are not possible in client-side rendering. The only thing we might be able to do is use query parameters—but again, such URLs must first be exposed to the search engine at build time for them to be usable from a search engine perspective. Otherwise, their only use is mnemonic or for saving the address of specific things.
Dynamic Content: Besides client-side rendering, there are ways to implement dynamic content from a server/process perspective. On the server side, we can handle specific query parameters or URL paths and let the server return templated HTML directly. Or we could simply compile the entire website as static webpages.
Server-Side Rendering binds "endpoints" to specific HTTP methods and URLs (path and query parameter combinations). When a request is received, the server returns the completed response (e.g., in HTML form). This is very easy for search engines to index since it's "static" in the sense that each address uniquely maps to a specific response, and no JavaScript runtime is needed to understand what content is there.
Compiled Source: It's surprisingly easy to simply compile entire websites from some source, e.g., from a database or scattered files. And one certainly does not require advanced frameworks for that. In a nutshell, it's simply a string replacement script. Honestly, this is way more effective for pure content websites than using any of the common frameworks (e.g., ASP.NET Core, Flask, Ruby on Rails, etc.). Compiled sources are very friendly to serverless deployment since most static web hosts are willing to distribute them for free.

Overview

To really understand the problem, its causes, and the solutions, we must review some basic concepts. Since this is a dev log, I will just enumerate the topics without delving into much detail:

HTTP requests and responses. One doesn't need to understand the entire HTTP protocol, but being able to read raw requests and responses (which are usually in plain text format) is helpful, and one should understand the difference between headers and body content.
HTTP methods, URL paths, and query strings.
How very basic and barebones HTTP servers work—I happen to have a demo example here for C#.
How to do client-side rendering in JS with fetched content.

Remarks

We are in the 2020s, and AI/LLMs are ubiquitous, but it's surprising how little the HTML/webpage landscape has changed regarding static vs. dynamic websites. Search engines and AI agents simply can't and won't work well with client-side rendered content due to the fact that the outcomes of runtime JavaScript evaluation are unpredictable.

DevLog 20250520: Search Engine Architecture

Wed, 21 May 2025 01:14:38 +0000

Overview

For an internal database, we needed to implement some sort of text-based search capabilities to help users locate documents. One goal was to be lightweight, hassle-free, easy to set up, expand, and maintain—and ideally, to set it up once and forget about it. It's okay if it costs a bit of computing power, but agility, quick iterations, usability, and robustness are of top concern.

Summary

It's surprising how much we can do without a server. Most serverless or static webpage hosts are happy to serve anything from a few MB to a few GB for free, likely even distributed across the globe. For any particular individual, if the search experience is truly customized to their needs, then not much resource is needed to deliver a satisfying experience.

Let's illustrate this with an example. Suppose that, for any webpage, the information we care about includes the following:

Title
Abstract
URL
Keywords (weighted)
Tags
Last Modification Time
Webpage Language

If we assume the abstract is at most 200 words and the rest adds up to 50 bytes, then each website we index takes about 250 bytes to store. 1KB fits four entries, 1MB fits 4096 entries, and 3MB fits about 12K entries. If a blog updates daily, the most it can produce is 365 entries per year, so 3MB is enough to hold one year's worth of content for 40 blog-sized websites.

If we go a bit crazy, it's reasonable to expect a few hundred megabytes of memory consumption. Let’s say the payload is 300MB—that’s 40 × 100 = 4,000 blog-sized websites for one year. This means that, without a server, and with everything distributed as static content, a user's browser client can happily serve indexed website data locally for 4,000 websites. Considering that most of the time we dwell on just a few dozen sites, and that time relevance usually plays a big role, this leaves plenty of room for imagination.

The Indexing

Call it an indexer or web crawler. In this step, we try to gather information about all available websites and process them into condensed formats like a database, CSV, or JSON.

Conceptually, the job of a web crawler could be greatly simplified if every website simply reported what it had—that’s supposedly the purpose of a sitemap. But in practice, it's surprising how much effort people spend on so-called SEO optimization, simply because there's no straightforward way to submit such information. On the other hand, perhaps it’s a good thing—we don’t want website creators to fool the search engine by submitting dishonest summary information.

One major challenge with web crawling, besides parsing and potential blocking by website hosts, is rendering JavaScript. Nowadays, lots of content is dynamic, meaning it’s not directly available in plain HTML—it may not require a server per se, but much of the visible content is rendered only during runtime. For instance, a dictionary app may only display content once a search keyword is entered.

The Searching

Database search, plain/full-text search, fuzzy search, vector search, and advanced queries—there are so many possible ways to organize data and perform searches, and so many libraries to choose from, it’s surprising that some websites and software still have such poor built-in search functions.

At the current stage, we don’t have absolute recommendations yet, but suffice it to say: advanced platforms like ElasticSearch or even a traditional database are completely unnecessary for the majority of applications. A plain full-text search is efficient enough, and lots of lightweight JavaScript or WASM libraries exist to speed up search—such as Fuse, Lunr.js, and FlexSearch. There are databases built specifically for search (not to mention the many vector databases now available for AI embedding workloads), such as Manticore Search.

The Hosting

As mentioned, the simplest search engine frontend is just a plain website that fetches index data during loading and performs search in-memory on the client side, without any server other than static file hosting. Again, it's worth stressing how powerful this can be—not only versatile and easily deployed, but also sufficient for most small to medium workloads. For typical payloads in the 3–5MB range and up to 300MB at most, this is more than enough. We can fit a lot within such constraints.

The Ultimate Guide to Serverless Databases: How They Operate and Their Key Benefits

Tue, 08 Apr 2025 10:47:14 +0000

Serverless databases are revolutionizing the way applications handle data. By abstracting the need for server management and allowing resources to scale automatically, they provide a cost-efficient, highly scalable solution for modern cloud-based applications. But how do serverless databases operate, and what makes them so appealing for developers? In this guide, we’ll explore the core principles behind serverless databases, their benefits, popular options, and real-world use cases.

What Are Serverless Databases?

A serverless database is a cloud-based database service that eliminates the need for users to manage the infrastructure or worry about provisioning resources. Unlike traditional databases, where developers must allocate and manage servers, a serverless database automatically handles scaling and resource allocation. This means that when your database workload increases, the system automatically adjusts by adding resources, and when the workload decreases, it scales down.

The term "serverless" doesn’t mean there are no servers involved, but rather that the management of servers is abstracted away from the user. This allows developers to focus on building their applications instead of spending time on managing and maintaining database infrastructure. The serverless approach is especially useful for applications with variable or unpredictable workloads, where resource consumption fluctuates throughout the day.

How Do Serverless Databases Operate?

Serverless databases are designed to be scalable and efficient by automatically adjusting to the amount of traffic they handle. This is accomplished through event-driven architectures that respond to the workload by provisioning and de-provisioning resources in real time. Let’s break down the core operational elements of serverless databases:

1. Auto-Scaling

One of the most significant benefits of serverless databases is their auto-scaling capability. The database automatically adjusts resources such as CPU and memory based on incoming traffic. If your application experiences a sudden surge in users or queries, the database can scale up without requiring manual intervention. Conversely, during periods of inactivity, the database scales down, allowing you to save costs on unused resources.

2. Event-Driven Architecture

Serverless databases typically operate on an event-driven architecture. This means the system is designed to respond dynamically to events, such as incoming requests or data queries. For example, if there’s an influx of traffic to your application, the database will automatically scale to meet the demand. This elasticity ensures that the database is always optimized for performance, regardless of the workload.

3. Granular Resource Consumption

In a serverless setup, you only pay for the actual resources you consume, unlike traditional databases where you might pay for a fixed instance regardless of how much you use. Billing is typically based on the number of requests processed, the amount of storage used, or the query execution time. This "pay-as-you-go" model helps optimize costs, making serverless databases particularly attractive for small businesses and startups.

4. No Server Management

With serverless databases, the responsibility for server maintenance, patching, and updates is shifted to the cloud provider. This means developers no longer have to manage infrastructure or deal with issues such as hardware failures, backups, or scaling concerns. The provider automatically handles the underlying hardware and ensures the database remains available and performant.

Benefits of Serverless Databases

Serverless databases offer numerous advantages for modern cloud applications. Let’s explore some of the most significant benefits that make serverless databases an attractive choice:

1. Cost-Efficiency

Traditional databases require you to pay for a fixed amount of resources, regardless of whether you use them all or not. Serverless databases, however, only charge for the actual resources you consume. This can be a significant cost-saving, particularly for applications with irregular traffic patterns. You don't need to over-provision capacity to handle peak demand, reducing unnecessary costs during off-peak times.

2. Simplified Management

Serverless databases simplify the management of infrastructure, allowing developers to focus more on application logic rather than database maintenance. Since the cloud provider takes care of the scaling, backups, and updates, developers can spend their time building features and improving the user experience. This reduction in administrative overhead can significantly improve team productivity.

3. Scalability

Serverless databases are inherently scalable. Whether your application experiences a sudden surge in traffic or a slow, steady increase in demand, the database automatically adjusts to accommodate these changes. This flexibility makes serverless databases ideal for handling applications with unpredictable or fluctuating workloads, such as e-commerce sites, gaming platforms, and IoT applications.

4. Rapid Deployment

Serverless databases are quick to set up and deploy, making them ideal for rapidly developing and testing new applications. With minimal configuration required, developers can get started quickly without worrying about the underlying infrastructure. This is particularly beneficial in fast-paced development environments where time-to-market is critical.

Popular Serverless Database Options

Several serverless databases are available, each offering unique features suited to different use cases. Here are some of the most popular options:

1. Amazon Aurora Serverless

Amazon Aurora Serverless is a fully managed relational database service offered by AWS. It automatically adjusts database capacity based on your application’s needs, offering a high degree of flexibility. Aurora Serverless is compatible with MySQL and PostgreSQL, making it a great choice for applications built with those technologies. It’s particularly useful for applications with unpredictable or intermittent database workloads.

2. Google Cloud Firestore

Firestore is a serverless, NoSQL document database from Google Cloud. It’s designed for scalability and flexible data models, making it ideal for modern web and mobile applications. Firestore scales automatically and offers built-in high availability and security features, ensuring that your application can handle traffic spikes seamlessly. It’s particularly popular among developers building real-time applications.

3. Azure SQL Database Serverless

Microsoft Azure SQL Database Serverless offers automatic scaling and a low-cost, flexible pricing model. It pauses during periods of inactivity to save costs and resumes when needed. This makes it perfect for workloads that require variable compute and storage resources, such as applications with intermittent usage or varying demand.

4. FaunaDB

FaunaDB is a serverless, globally distributed NoSQL database that offers strong consistency and high availability. It’s designed for modern applications that need to scale effortlessly while maintaining ACID transactions. FaunaDB is particularly useful for applications with complex data models, providing low-latency access and strong consistency guarantees without the need for infrastructure management.

Use Cases for Serverless Databases

Serverless databases are well-suited for a wide variety of applications. Here are some common use cases:

1. Startups and Small Businesses

For startups and small businesses with limited resources, serverless databases provide an affordable way to scale as needed without large upfront costs. They can quickly respond to changing demands and avoid the hassle of managing servers.

2. Microservices Architecture

Serverless databases are ideal for microservices-based applications, where each service might require a different database configuration. The ability to scale automatically ensures that each microservice can function independently while maintaining overall system performance.

3. Real-Time Applications

Real-time applications, such as messaging platforms, gaming apps, or financial tracking systems, benefit from the scalability and low-latency access provided by serverless databases. They can automatically handle fluctuations in data volume and traffic, ensuring a seamless user experience.

Conclusion

Serverless databases represent a paradigm shift in how applications manage and interact with data. By automating scaling, management, and infrastructure, they enable developers to focus on application logic, improve cost efficiency, and ensure scalability without worrying about the underlying hardware. Whether you’re building a small startup or a large-scale application, serverless databases can provide the flexibility, efficiency, and cost savings you need to succeed in today’s cloud-driven world.

Simplifying Serverless Workflows with EventBridge Pipes

Tue, 18 Feb 2025 13:07:30 +0000

Introduction

Event-driven architectures (EDAs) are essential when creating scalable, decoupled systems. By allowing services to communicate asynchronously, they reduce bottlenecks and enable flexible scaling.
AWS offers many services for building EDAs, in this post I will focus on two different combinations:

The traditional SNS/SQS combo
The newer EventBridge Pipes.

We’ll explore how to build a serverless workflow using EventBridge Pipes (integrating SQS, DynamoDB Streams, and Lambda) and compare it to SNS/SQS setups.

What are eventBridge pipes?
EventBridge Pipes is a serverless integration service that connects AWS services like (SQS, DynamoDB Streams, and Lambda) without requiring intermediate code. It supports filtering, enrichment (we use Lambda to do it), and transformations, EventBridge Pipes simplifies creating event-driven workflows.

The diagram below illustrates the workflow of AWS EventBridge Pipes, which enables seamless integration between event sources and targets with optional processing steps.

Here's a breakdown of the components and their roles:

Event Source:
This is the origin of events, such as AWS services (e.g., S3, DynamoDB, Kinesis), SaaS applications, or custom applications. Events are ingested into the pipe from here.
Filter: Events are evaluated against predefined criteria (e.g., JSON-based rules). Only events that match the filter conditions proceed to the next step, reducing unnecessary processing.
Enrichment: Optional step to enhance the event data. For example, you might invoke an AWS Lambda function to fetch additional information from a database or transform the event payload before sending it to the target.
Target: The final destination where processed events are delivered. Targets can be AWS services (e.g., Lambda, SNS, SQS, EventBridge event buses), HTTP endpoints, or other resources.

How it works together:

The event source generates events (e.g., an S3 bucket upload).
The filter pick up relevant events.
The enrichment step adds/modifies data.
The target receives the refined event for further action (e.g., triggering a Lambda function to process the file).

Comparing EventBridge Pipes with SNS/SQS

Now as we know how it works let's learn by looking at a use case of E-Commerce Order Fulfillment system
In this e-commerce platform every time a customer places an order, the order details are stored in a DynamoDB table. Changes in the table (e.g., new orders) are captured by DynamoDB Streams. Instead of writing custom integration code to filter and route these events, you can use EventBridge Pipes. With Pipes, you configure the following workflow:

DynamoDB Streams as the Source:
When a new order is added to the DynamoDB table, the change event is captured.
EventBridge Pipes Filtering & Routing:
The Pipe ingests events from DynamoDB Streams and applies a filter to select only orders with a status of "NEW." It then routes these filtered events directly to a Lambda function.
Lambda Function for Order Processing:
The Lambda function processes the order (for example, updating inventory, initiating shipping, and sending notifications).

This approach is compared to the traditional setup where events are published to an SNS topic and then pushed to an SQS queue before being processed by a Lambda function.

Let's view both architectures and explain the advantages of each one, and when to use each one of them:

EventBridge Pipes approach

Diagram Explanation

Sources:
1. DynamoDB Streams: Capture changes from a table (for example, new or updated order records).
2. SQS Queue (if needed): Acts as an additional source that may buffer events from various systems.
EventBridge Pipes: Serves as the central hub that filters, transforms, and routes incoming events from both DynamoDB Streams and SQS.
Lambda Function: Consumes the refined events to process business logic (e.g., updating order status or triggering further workflows).

Traditional SNS-SQS approach

Diagram Explanation

SNS Topic: Typically used to broadcast events to multiple subscribers.
SQS Queue: Buffers messages and provides a pull-based delivery mechanism.
Lambda Function: Processes the events retrieved from the SQS queue.

Comparing EventBridge Pipes with SNS/SQS

EventBridge Pipes

Advantages:

Integrated Filtering & Transformation as you can set up rules directly within Pipes to process events without extra Lambda functions.
Simpler Configuration because it directly connects sources like DynamoDB Streams or SQS to Lambda with minimal configuration.
Reduced operational complexity by eliminating extra intermediary services.

When to Use:

When your workflow is relatively linear or requires simple filtering.
When you prefer a code-free, declarative integration between event sources and targets.
For pipelines where transformation and routing can be managed entirely within EventBridge Pipes.

Traditional SNS/SQS

Advantages:

Fan-Out Capabilities as SNS naturally supports broadcasting a single event to multiple subscribers.
Services reliability with fine-grained delivery and retry control.
Suitable for complex scenarios that require multiple consumer endpoints.

When to Use:

When your architecture needs to fan out events to multiple consumers.
If you require detailed control over message delivery policies and retry behaviors.
When your existing setup is already built around SNS/SQS patterns and migrating to Pipes isn’t practical.

Cost Components of Each Approach

EventBridge Pipes Approach

AWS Service	Cost Component
DynamoDB Streams	First 2.5M reads free per month, then $0.02 per 100,000 reads
EventBridge Pipes	$0.40 per 1M events processed
Lambda (Order Processing)	$0.20 per 1M requests + execution time cost (depends on memory & duration)

SNS/SQS Approach

AWS Service	Cost Component
DynamoDB Streams	$0.02 per 100,000 reads
SNS	$0.50 per 1M publishes
SQS (FIFO Queue)	$0.50 per 1M requests
Lambda (Order Processing)	$0.20 per 1M requests + execution time cost

Conclusion

EventBridge Pipes offer a modern, efficient way to build decoupled, event-driven workflows on AWS. By integrating sources like DynamoDB Streams and SQS directly with Lambda, you can streamline your architecture and reduce operational overhead. However, traditional SNS/SQS setups remain a good choice for scenarios that require broad event broadcasting and detailed control over message delivery.

The right choice depends on your specific use case: go for EventBridge Pipes when you seek simplicity and reduced code, or stick with SNS/SQS for more complex, fan-out scenarios. Either way, understanding these patterns is key to building scalable and resilient serverless applications.