Coder Legion - Recent posts tagged lambda

AWS Step Functions: Creating a ‘Busy Waiting’ flow to wait for successful lambda executions.

Wed, 14 Oct 2026 18:01:55 +0000

Hi everyone, welcome to this article. In the past few days, I had to create a logic in AWS Step functions which involved waiting for a lambda that queries athena before going forward in the flow.

When I was discussing the logic, it felt like an easy thing to accomplish in minutes but I was surprised upon knowing that Step Functions does not have its native technique/solve to wait for a state to succeed before starting another. This made me search for hours before coming up with a solution.

Here, I will share with you a technique which will help you in the future.

First of all, I will create a simple lambda function. All we need to know from the lambda is whether it did its job. So, this simple function will make sure to return its status.

Lambda function:

import json

def lambda_handler(event, context):
    response = {
        'status': 'success',
        'message': 'Hello, World!'
    }
    
    return response

You can copy both the function and the Step function definition which I will share below to replicate all of this in your own environment.

Now, lets go ahead and create a state machine in AWS Step Functions.

Click on ‘Create state machine’.

Choose the option to write your own definition and choose ‘standard’.

Now, in the definition section, copy and paste the following definition.

Step Functions definition:

{
  "Comment": "A Hello World example of the Amazon States Language using Pass states",
  "StartAt": "Main",
  "States": {
    "Main": {
      "Type": "Task",
      "Resource": "arn:aws:states:::lambda:invoke",
      "OutputPath": "$.Payload",
      "Parameters": {
        "Payload.$": "$",
        "FunctionName": "arn:aws:lambda:us-east-1:031342435657:function:stepfunctionlambda_test:$LATEST"
      },
      "Retry": [
        {
          "ErrorEquals": [
            "Lambda.ServiceException",
            "Lambda.AWSLambdaException",
            "Lambda.SdkClientException",
            "Lambda.TooManyRequestsException"
          ],
          "IntervalSeconds": 2,
          "MaxAttempts": 6,
          "BackoffRate": 2
        }
      ],
      "Next": "WaitforInvocation"
    },
    "WaitforInvocation": {
      "Type": "Wait",
      "Seconds": 2,
      "Next": "CheckInvocationsStatus"
    },
    "CheckInvocationsStatus": {
      "Type": "Choice",
      "Choices": [
        {
          "Variable": "$.status",
          "StringEquals": "success",
          "Next": "Final"
        }
      ],
      "Default": "WaitforInvocation"
    },
    "Final": {
      "Type": "Succeed"
    }
  }
}

Save the definition and click next.

Specify the name of your state machine and then leave everything else as default. A new execution role will be created if you don’t specify any.

Save it and then you will have created a state machine which looks like this:

Visual flow:

To explain the definition above, I have pointed out how this state machine works:

The state machine starts at the "Main" state.
The "Main" state is a Task state that invokes a Lambda function specified by the "FunctionName" parameter. The input to the Lambda function is the entire input passed to the state machine.
The "OutputPath" parameter is set to "$.Payload" to capture the output of the Lambda function.
If the Lambda function invocation fails with certain exceptions specified in the "Retry" section (Lambda.ServiceException, Lambda.AWSLambdaException, Lambda.SdkClientException, Lambda.TooManyRequestsException), the state machine will retry the invocation up to 6 times with a backoff interval of 2 seconds.
After invoking the Lambda function, the state machine proceeds to the "WaitforInvocation" state.
The "WaitforInvocation" state is a Wait state that pauses the execution for 2 seconds.
After the wait period, the state machine goes to the "CheckInvocationsStatus" state.
The "CheckInvocationsStatus" state is a Choice state that checks the value of the "$.status" field in the output of the Lambda function. If the status is "success", the state machine proceeds to the "Final" state. Otherwise, it goes back to the "WaitforInvocation" state.
The "Final" state is a Succeed state that indicates the successful completion of the state machine.

Now, lets go ahead and execute it!

Click on ‘Start execution’ and leave the input field empty.

Incase of a lambda that requires input, you can insert your input here as a Json

After you start your execution, each state will turn into different colours which indicates the following.

Green: Indicates successful completion of a state or step.
Red: Indicates a failure or error occurred during the execution of a state or step.
Yellow: Indicates a state is in a waiting or paused state.
Blue: Represents a decision or choice point where the state machine evaluates conditions to determine the next state.
Gray: Represents a passive state that simply passes input to output without performing significant work.
White: Represents transitions or connections between states, indicating the flow of execution from one state to another.

Finally this is how the execution will look like:

Thank you. Hope you can now go ahead and apply the “Busy Waiting” logic in your step functions.

Building a worker-evaluator retry loop in LangGraph (and where it bites you on Lambda)

Fri, 17 Jul 2026 05:44:06 +0000

Most agent demos stop at "the model called a tool and gave an answer." That answer is often wrong, and nothing in the loop notices. I wanted an agent that checks its own work against a success criteria before returning, and retries when it falls short. That is the whole idea behind Wingman: a worker that does the task, and a separate evaluator that decides whether the task is actually done.

Here is how the loop is built, the one decision that shaped everything, and the part that broke once it ran on Lambda behind API Gateway.

The problem

A single-pass agent has no idea when it failed. It calls a tool, produces text, and returns. If the output misses the point, the user finds out, not the system.

I wanted a gate. Something that reads the assistant's last response, compares it against a stated success criteria, and either accepts it or sends the worker back to try again with feedback. Two roles, not one. The worker produces. The evaluator judges.

The catch is that a retry loop with no ceiling is a bill waiting to happen. So the loop needs a hard stop, and it needs to know the difference between "the assistant needs to try again" and "the assistant is stuck and should ask the user."

The approach and the key decision

The graph has three nodes: worker, tools, and evaluator. The state carries the conversation, the success criteria, the latest feedback, two boolean flags, and a turn counter.

class State(TypedDict):
    messages: Annotated[List[Any], add_messages]
    success_criteria: str
    feedback_on_work: Optional[str]
    success_criteria_met: bool
    user_input_needed: bool
    turn_count: int

The worker runs first. If its last message contains tool calls, the router sends it to the tool node and back. If not, it goes to the evaluator.

def worker_router(self, state: State) -> str:
    last = state["messages"][-1]
    return "tools" if (hasattr(last, "tool_calls") and last.tool_calls) else "evaluator"

The key decision is making the evaluator a structured-output call, not a free-text one. It returns a typed object, so the routing logic reads clean booleans instead of parsing prose.

class EvaluatorOutput(BaseModel):
    feedback: str = Field(description="Feedback on the assistant's response")
    success_criteria_met: bool = Field(description="Whether the success criteria have been met")
    user_input_needed: bool = Field(
        description="True if more input is needed from the user, or the assistant is stuck"
    )

Both the worker and the evaluator run on gpt-4o-mini. The worker is bound to tools. The evaluator is bound to the schema above with .with_structured_output.

The routing after evaluation is where the loop's whole personality lives:

def route_based_on_evaluation(self, state: State) -> str:
    if state["success_criteria_met"] or state["user_input_needed"]:
        return "END"
    if state.get("turn_count", 0) >= self.MAX_TURNS:
        return "END"
    return "worker"

Three ways out. The criteria is met. The assistant is stuck and needs the user. Or the loop has spent its five turns. Otherwise it loops back to the worker, and this time the worker gets the feedback baked into its system prompt:

if state.get("feedback_on_work"):
    system_message += f"""
Previously you thought you completed the assignment, but your reply was rejected because the success criteria was not met.
Feedback:
{state["feedback_on_work"]}
Please continue the assignment..."""

That feedback injection is what makes the retry worth anything. The worker does not just try again blind. It tries again knowing why the last attempt was rejected.

One design detail I care about: the evaluator is told not to confuse "not done yet" with "stuck." A failed criteria means retry, not stop.

For user_input_needed: set True ONLY if the assistant explicitly asked the user a question or stated it cannot proceed.
Do NOT set user_input_needed=True just because the criteria was not met — the assistant should simply retry.

Without that line, the evaluator bails to the user the first time the worker misses, and the retry loop never actually retries.

What broke, and what I changed

The loop is stateless by design. Wingman reconstructs the full conversation from DynamoDB on every request and runs a fresh graph invocation with a throwaway thread ID, so there is no in-memory state to lose between Lambda cold starts. That part held up.

The timeout did not.

A worker-evaluator loop is slow. Five turns, each turn a worker call plus tool calls plus an evaluator call, adds up to real wall-clock time. I set the Lambda timeout to 300 seconds to give the loop room:

# LangGraph agent loops can take a while.
timeout     = 300  # 5 minutes (Lambda max is 15 min)
memory_size = 1024

That number is a lie if the request comes through API Gateway. The HTTP API has a hard 29-second integration cap that Lambda's 300 seconds cannot override. So a long agent run would keep working inside Lambda while the gateway had already given up on the caller. The function billed for the full run. The user got nothing.

The fix was to stop routing long tasks through the gateway at all. I added a Lambda Function URL, which is a direct HTTPS endpoint with no 29-second ceiling:

# Lambda function URL — direct HTTPS invoke, bypasses API Gateway 29s timeout.
resource "aws_lambda_function_url" "backend" {
  function_name      = aws_lambda_function.backend.function_name
  authorization_type = "NONE"
}

Now the gateway stays for the fast endpoints, and the agent loop can run against the Function URL without a proxy in the middle deciding it took too long.

There is a second sharp edge I have not fully closed. turn_count only increments inside the evaluator node:

"turn_count": state.get("turn_count", 0) + 1,

But the worker can loop worker -> tools -> worker -> tools several times before it ever hands off to the evaluator, and none of those hops count against MAX_TURNS. So the five-turn cap is really "five evaluator turns," not "five model calls." A task that keeps calling tools without producing a final answer can burn far more time than the cap implies. The honest fix is to count worker iterations too, or add a wall-clock budget inside the run, rather than trusting the evaluator counter to bound cost. That one is still on my list.

Takeaway

An evaluator that gates the worker's output is the cheap half of a self-correcting agent. The expensive half is making the retry ceiling account for every model call, not just the ones you happened to count.

The Truth About Serverless APIs Nobody Mentions

Wed, 27 May 2026 15:51:56 +0000

I've noticed serverless conversations tend to split into two camps:

People who think it solves everything, and people who think it's a mistake.

The reality is much more useful than either extreme.

Everyone who sells you on serverless leads with the same three points: no servers to manage, automatic scaling, and you only pay for what you use.

All of that is technically true.

None of it is the whole story.

I've shipped serverless APIs in production — Go Lambdas behind API Gateway, the kind that power real user-facing products. I'm not here to talk you out of it. For the right workload, it's genuinely excellent.

But the gaps between the pitch and the reality are specific, and knowing them in advance changes the decisions you make early, when they're still cheap to change.

Cold starts are real — just not how you think

Lambda cold starts happen when AWS needs to provision a new execution environment for your function. The runtime boots, your init code runs, and then the handler executes.

For a Go function on arm64, that's typically 50–150ms total. For Node.js or Python with heavy dependencies, it can be 500ms–2 seconds.

The part nobody leads with: cold starts are probabilistic, not guaranteed.

Under steady traffic, most invocations hit warm instances. You'll rarely see them in your P50 or even P95. Where they show up is P99 and beyond — the tail latency your best users experience right after a traffic spike, or the first request after a quiet weekend.

The practical advice: profile in production with real traffic before you add Provisioned Concurrency. It costs money and adds complexity.

For most APIs, the smarter move is optimizing your init code:

lazy-load dependencies
keep your binary small
initialize only what the first request actually needs

Fix the actual problem before reaching for the expensive solution.

The billing math will surprise you

Lambda pricing is simple on paper: you pay per request and per GB-second of execution time.

At low traffic, it's nearly free.

The surprise comes at scale — or in specific usage patterns.

The scenario that bites teams most often: a Lambda that does meaningful work per invocation. Parsing a large payload, doing in-memory transformation, making multiple downstream calls.

These functions run longer, use more memory, and the cost per invocation adds up faster than the pricing page suggests.

A few numbers from real workloads:

A lightweight REST Lambda (Go, 128MB, 20ms avg): ~$0.40 per million requests
A heavier processing Lambda (Node, 512MB, 800ms avg): ~$8.00 per million requests

That 20× difference in cost per million requests adds up quickly when you have real traffic.

It's still cheap compared to running servers — but the assumption that serverless is always the low-cost option needs to be verified, not assumed.

API Gateway pricing is also frequently missed. At $3.50 per million requests for HTTP APIs, a high-traffic public endpoint can make Gateway cost more than the Lambda it fronts.

Know the full cost model before you commit.

Debugging is a different skill

Local development for Lambda is awkward.

You can simulate the execution environment with tools like AWS SAM or custom test harnesses, but you're never truly running the same thing that runs in production.

The gap shows up in exactly the moments you can least afford it.

Observability requires deliberate setup.

A Lambda that exits silently leaves no trace unless you've instrumented it. CloudWatch Logs are the floor, not the ceiling — you need structured logging, correlation IDs, and ideally X-Ray tracing before you're in production, not after something breaks.

The failure modes are also harder to reproduce.

A Cognito authorizer rejecting a token before your code runs looks different from your code returning a 401. A DynamoDB throttle on a hot partition looks different from a Lambda timeout.

Getting fast at debugging serverless systems means learning the AWS console deeply:

CloudWatch Insights queries
X-Ray service maps
reading IAM policy errors without a stack trace to guide you

Statelessness is a constraint, not just a pattern

Every Lambda invocation starts with a clean slate.

That's the feature — it's what makes horizontal scaling effortless.

It's also the constraint that eliminates entire categories of architectural approaches.

No in-memory caches that persist across requests. No background jobs that run between invocations. No WebSocket connections that stay open. No files written to disk that the next request can read.

If your application assumes any of those things, serverless isn't wrong — but it's not your foundation. It's a piece of a larger architecture that needs something else alongside it.

This comes up most often when teams decide mid-project that they suddenly need a feature requiring persistent state.

Adding ElastiCache or SQS to a serverless architecture is straightforward — but it's a different architecture than what you started with, and the integration work is non-trivial.

The most dangerous part is that many of these problems don't show up until the system already matters.

Vendor lock-in is real, and that's okay

A Lambda function isn't a Docker container.

It doesn't run the same way everywhere else.

Your Go binary targeting provided.al2023 on arm64 can technically run in other environments, but your API Gateway routes, Cognito authorizers, DynamoDB single-table design, and SES integration are AWS-specific throughout.

I'm not arguing against it — I use all of those services in production.

But it's worth being honest:

If you go deep on serverless AWS, migrating later means a substantial rewrite.

Make that decision deliberately, not by default.

When not to use it

Serverless is the wrong tool when:

You have persistent connections — WebSocket servers, gRPC streams, database connection pools
You have long-running jobs — Lambda has a 15-minute max timeout
You have very high, sustained concurrency — account-level concurrency limits can affect multiple services simultaneously
You need low and predictable latency at P99 — cold-start tail latency is difficult to eliminate completely

When it's the right call

Event-driven workloads.

APIs with bursty or unpredictable traffic.

Background processing triggered by S3 uploads, SQS messages, or DynamoDB streams.

Any workload where you don't want to manage capacity and the per-request cost model fits your traffic profile.

The API powering this site is Go Lambdas behind API Gateway.

Contact form submissions, resume requests, visitor analytics, admin dashboard — all of it.

It handles the traffic I need, costs less than $1/month to run, and I've never paged for a server that went down.

For that workload, it's the right call.

The key is understanding what you're actually trading.

Serverless isn't simpler than traditional infrastructure.

It's differently complex.

The operational burden shifts from server management to observability, IAM, and distributed system debugging.

That trade is absolutely worth it in the right context.

Go in with your eyes open, and it's one of the most productive architectures available.

The right architecture should help your team maintain momentum as the system grows — not create friction six months later.

At M²S² Engineering Group, we help teams make these decisions before they become expensive to unwind.

Originally published at m2s2.io

Multi-Cloud Resilience: The Event-Driven Cellular Architecture Blueprint across AWS and Azure

Sat, 02 May 2026 12:42:02 +0000

1. Introduction

As enterprise systems reach massive scale, relying on a single cloud provider introduces systemic risk. Regional outages, vendor-specific API degradations, or severe misconfigurations can result in global downtime. Multi-cloud cellular architecture solves this by deploying isolated, self-contained failure domains (cells) across different cloud providers. In this tutorial, you will architect a unified, event-driven system where AWS and Azure serve as the physical hosts for identical logical cells.

Instead of building a "stretched" application that queries data across clouds—an anti-pattern that guarantees high latency and fragile dependencies—you will deploy complete, asynchronous processing pipelines on both AWS and Azure. A global, cloud-agnostic edge routing layer will inspect incoming traffic and forward it to either an AWS cell or an Azure cell based on a partition key (such as TenantId). By the end of this guide, you will understand how to orchestrate this ultimate fault-isolation boundary using Terraform, ensuring your application survives even a total cloud provider outage.

2. Prerequisites

To build a multi-cloud environment, your operational tooling must be strictly cloud-agnostic. You will need:

Active accounts on both Amazon Web Services (AWS) and Microsoft Azure with administrative privileges.
The AWS CLI and Azure CLI installed and authenticated on your local machine.
Terraform (version 1.3.0 or higher) installed. Terraform is crucial here, as it is the singular control plane capable of deploying resources to both clouds simultaneously.
Understanding of Global Server Load Balancing (GSLB) or Edge Compute platforms (like Cloudflare Workers or Fastly) to act as the multi-cloud router.
A Domain-Driven Design (DDD) approach to your data, ensuring that a specific tenant's data lives entirely within one cell and never needs to cross the multi-cloud boundary.

3. Step-by-Step

Step 1: Configuring Multi-Cloud Providers in Terraform

What to do: Create a root Terraform configuration that initializes providers for both AWS and Azure simultaneously.

Why do it: Infrastructure as Code (IaC) is the only sustainable way to manage multi-cloud environments. By declaring both providers in the same root module, you can orchestrate the deployment of AWS cells, Azure cells, and the DNS records that tie them together from a single pipeline, preventing configuration drift between your cloud environments.

Code Example (main.tf):

terraform {
  required_version = ">= 1.3.0"
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~> 3.0"
    }
  }
}

provider "aws" {
  region = "us-east-1"
}

provider "azurerm" {
  features {}
}

Step 2: Deploying the AWS and Azure Data Plane Cells

What to do: Utilize the modular approach to instantiate independent cells on both clouds. The AWS cell utilizes DynamoDB, SNS, and SQS. The Azure cell utilizes Cosmos DB, Service Bus Topics, and Subscriptions. Both rely on serverless compute (Lambda and Azure Functions).

Why do it: While the underlying managed services differ, the architectural pattern is identical: an HTTP entry point writes to a NoSQL database, which triggers a change stream, which is fanned out via a message bus to consumer workers. Encapsulating these into Terraform modules abstracts the cloud-specific complexities, allowing you to treat "AWS" and "Azure" simply as deployment targets for your business logic.

Code Example (data_plane.tf):

# AWS Cell (East Coast)
module "cell_aws_alpha" {
  source  = "./modules/aws_event_cell"
  cell_id = "aws-alpha"
  region  = "us-east-1"
}

# Azure Cell (East Coast)
module "cell_azure_beta" {
  source   = "./modules/azure_event_cell"
  cell_id  = "azure-beta"
  location = "East US"
}

# Azure Cell (Europe)
module "cell_azure_gamma" {
  source   = "./modules/azure_event_cell"
  cell_id  = "azure-gamma"
  location = "North Europe"
}

Step 3: Implementing the Cloud-Agnostic Edge Router

What to do: Deploy a global routing layer that sits entirely outside of AWS and Azure. This is typically achieved using an Edge Compute platform like Cloudflare Workers. The Worker intercepts the HTTP request, reads the TenantId, checks a globally distributed Key-Value store to find the assigned cell's API endpoint, and forwards the payload.

Why do it: If your global router is hosted on AWS (e.g., using API Gateway), and AWS experiences an outage, your Azure cells become unreachable, defeating the purpose of multi-cloud. The routing layer must be cloud-agnostic. Cloudflare Workers execute at the edge, providing sub-millisecond lookups for the routing map and directing traffic to the respective AWS API Gateway or Azure Function HTTP trigger.

Conceptual Edge Router Logic (JavaScript for Edge Worker):

export default {
  async fetch(request, env) {
    const url = new URL(request.url);
    const tenantId = request.headers.get("X-Tenant-ID");

    if (!tenantId) {
      return new Response("Missing Tenant ID", { status: 400 });
    }

    // Lookup cell assignment from global edge KV store
    const cellEndpoint = await env.ROUTING_MAP.get(tenantId);

    if (!cellEndpoint) {
      return new Response("Tenant mapping not found", { status: 404 });
    }

    // Proxy the request to the target cloud (AWS or Azure)
    const targetUrl = new URL(url.pathname, cellEndpoint);
    const modifiedRequest = new Request(targetUrl, request);
    
    return fetch(modifiedRequest);
  }
}

Step 4: Automating Disaster Recovery and Traffic Shifting

What to do: Establish a process to update the global routing map. If the AWS region hosting cell_aws_alpha goes offline, you update the Edge KV store to point those affected tenants to a standby cell on Azure.

Why do it: Cellular architecture drastically reduces the blast radius of an outage, but you still need a mechanism to restore service for the degraded cell. By decoupling the routing logic from the compute infrastructure, shifting traffic between clouds becomes a simple Key-Value update at the DNS/Edge layer, resulting in near-instantaneous failover.

4. Common Troubleshooting

The Multi-Cloud Data Split Brain: The biggest mistake in multi-cloud architecture is attempting synchronous database replication between AWS and Azure. This introduces severe latency and massive egress costs. Ensure strict cellular isolation: a tenant's data must live and be processed entirely within their assigned cell. Data should only cross clouds during a deliberate, asynchronous disaster recovery migration.
CI/CD Pipeline Complexity: Deploying to multiple clouds means your deployment pipelines must authenticate with two different IAM systems. Utilize OpenID Connect (OIDC) in platforms like GitHub Actions or GitLab CI. OIDC allows your pipelines to assume temporary, secure roles in both AWS (via IAM Identity Center) and Azure (via Microsoft Entra Workload ID) without storing long-lived, vulnerable access keys.
Observability Fragmentation: AWS logs live in CloudWatch; Azure logs live in Log Analytics. Troubleshooting a multi-cloud environment requires a unified view. You must export logs and metrics from both cloud providers into a centralized, agnostic observability platform (like Datadog, New Relic, or a self-hosted ELK stack) to effectively monitor the health of your global cells.

5. Conclusion

Building an event-driven cellular architecture across multiple clouds is the pinnacle of infrastructure resilience. By treating AWS and Azure as interchangeable hosts for isolated data planes, and governing them with a cloud-agnostic edge router and Terraform, you eliminate single points of failure at the vendor level. While this approach introduces operational complexity, it guarantees that a regional outage or provider degradation remains a contained incident rather than a business-ending catastrophe. As a next step, focus on standardizing your application packaging by utilizing containers (ECS on AWS, Container Apps on Azure) to ensure your worker code executes identically regardless of the host cloud.