Coder Legion - Recent posts tagged system-architecture

Why Your Tech Stack Doesn’t Matter

Wed, 06 May 2026 22:39:04 +0000

Architecting for Reliability in the Age of Multi-Agent Systems

We are currently over-indexing on "Model Orchestration."

Every week, a new library, a new vector database, or a new framework tops the GitHub trending charts.

This week it might be LangGraph. The next CrewAI. Something else right behind it.

Every week the same question shows up:

"Which stack should I use to build a reliable multi-agent system?"

It's the wrong question.

Because I've yet to see a system fail because to the wrong framework, language, or database.

I've seen them fail because they couldn't recover state, couldn't control context, and couldn't explain what they just did.

There’s a persistent belief that the logo on the documentation is the secret sauce for a production-ready system.

It isn’t. In fact, if you’re spending the majority of your time debating the stack, you’re missing the architectural patterns that actually determine whether your agents will succeed or hallucinate into oblivion.

The Illusion of the Framework

A Multi-Agent System (MAS) is not a library problem. It is a State Management problem disguised as an AI problem. Whether you use a graph-based logic or a role-based queue, the fundamental challenges and failure modes remain identical:

lost state
bloated context
untraceable decisions

The stack you choose is merely the syntax you use to solve universal engineering constraints.

The Core Thesis: Reliability in agentic workflows is derived from patterns, not packages. A secure, scalable system built in Python looks fundamentally the same as one built in Rust if the underlying system primitives are respected.

The Three Constants of Reliable Agents

Regardless of your tools, your architecture must solve for these three pillars to move from a "cool demo" to a production asset:

State is Sovereign
If an agentic loop fails at step 7 of 12, does your system restart from scratch? If so, your stack doesn't matter because your architecture is broken. A resilient system requires Deterministic Checkpointing:
- Capture the full thread state.
- Preserve intent, not just data.
- Resume execution without replaying the entire workflow.

Without this, your system is just a loop with amnesia.

The Context Tax
Context windows are not infinite. In reality, every token you give an agent is a tax on its reasoning. The "how" isn't about which LLM you use; it's about the Routing Layer:
- Classify intent
- Expose only relevant tools
- Minimize context surface area

Less context doesn’t limit the system—it sharpens it.

Governance as a First-Class Citizen
An agent is a service principal. If it cannot be audited, revoked, or sandboxed at the identity level, it shouldn't have access to your data or exist in production.

A reliable system enforces:

Least-Privilege Authorization, ensuring agents operate within a cryptographic "box" regardless of whether they are running in a Docker container or a serverless function.
Scoped tool usage
Traceable execution

Example

Consider a simple multi-agent workflow:

If your system can't resume from that point with the same context and intent, you don't have a system.

You have a demo.

A reliable system looks different.

The Framework-Agnostic Checklist

Pillar	The Real Question
Coordination	How do agents hand off work without bloating context or losing intent?
Observability	Can we trace every decision back to inputs and reasoning steps?
Resilience	What happens when a model fails mid-workflow? Can we resume without replaying?
Sovereignty	Who owns the data and execution environment—us or the platform?

Closing Thoughts

These are not new problems. They're just showing up in a new layer.

Stop chasing the framework. A system built in Python and one built in Rust will fail in exactly the same ways if the architecture is wrong.

The difference isn't the stack. It's whether you've designed for:

State
Context
Control

The tools are interchangeable. The architecture is not.

This is the foundation for the upcoming Sovereign Synapse series—where we move from theory to a local-first system that treats memory, context, and ownership as first-class concerns.

8 Most Important System Design Concepts You Should Know

Sun, 29 Jun 2025 08:22:17 +0000

Get a Free System Design PDF with 158 pages by subscribing to our weekly newsletter: https://bit.ly/bbg-social

Animation tools: Adobe Illustrator and After Effects.

Checkout our bestselling System Design Interview books:
Volume 1: https://amzn.to/3Ou7gkd
Volume 2: https://amzn.to/3HqGozy

The digital version of System Design Interview books: https://bit.ly/3mlDSk9

ABOUT US:
Covering topics and trends in large-scale system design, from the authors of the best-selling System Design Interview series.

The subtle art of waiting

Wed, 23 Apr 2025 21:40:24 +0000

Recently, while working on a workshop titled Testing Your Pull Request on Kubernetes with GKE, and GitHub Actions, I faced twice the same issue: service A needs service B, but service A starts faster than service B, and the system fails. In this post, I want to describe the context of these issues and how I solved them both with the same tool.

Waiting in Kubernetes

It might sound strange to wait in Kubernetes. The self-healing nature of the Kubernetes platform is one of its biggest benefits. Let's consider two pods: a Python application and a PostgreSQL database.

The application starts very fast and eagerly tries to establish a connection to the database. Meanwhile, the database is initializing itself with the provided data; the connection fails. The pod ends up in the Failed state.

After a while, Kubernetes requests the application pod's state. Because it's failed, it terminates it and starts a new pod. At this point, two things can happen: the database pod isn't ready yet, and it's back to square one or it's ready, and the application finally connects.

To speed up the process, Kubernetes offers startup probes:

startupProbe:
  httpGet:
    path: /health
    port: 8080
  failureThreshold: 30
  periodSeconds: 10

With the above probe, Kubernetes waits for an initial ten seconds before requesting the pod's status. If it fails, it waits for another ten seconds. Rinse and repeat 30 times before it fails definitely.

You may have noticed the HTTP /healthz endpoint above. Kubernetes offers two exclusive Probe configuration settings: httpGet or exec. The former is suitable for web applications, while the latter is for other applications. It implies we need to know which kind of container the pod contains and how to check its status, provided it can. I'm no PostgreSQL expert, so I searched for a status check command. The Bitnami Helm Chart looks like the following when applied:

startupProbe:
  exec:
    command:
      - /bin/sh
      - -c
      - -e
      - exec pg_isready -U $PG_USER -h $PG_HOST -p $PG_PORT

Note that the above is a simplification, as it gladly ignores the database name and an SSL certificate.

The startup probe speeds things up compared to the default situation if you configure it properly. You can set a long initial delay, and then shorter increments. Yet, the more diverse the containers, the harder it gets to configure, as you need to be an expert in each of the underlying containers.

It would be beneficial to look for alternatives.

Wait4x

Alternatives are tools whose focus is on waiting. A long time ago, I found the wait-for script for this. The idea is straightforward:

>./wait-for is a script designed to synchronize services like docker containers. It is sh and alpine compatible.

Here's how to wait for an HTTP API:

sh -c './wait-for http://my.api/health -- echo "The api is up! Let's use it"'

It got the job done, but at the time, you had to copy the script and manually check for updates. I've checked, and the project now provides a regular container.

wait4x plays the same role, but is available as a versioned container and provides more services to wait for: HTTP, DNS, databases, and message queues. That's my current choice.

Whatever tool you use, you can use it inside an https://kubernetes.io/docs/concepts/workloads/pods/init-containers/[init container]:

>A Pod can have multiple containers running apps within it, but it can also have one or more init containers, which are run before the app containers are started.

>Init containers are regular containers, except:

> Init containers always run to completion.
> Each init container must complete successfully before the next one starts.

Imagine the following Pod that depends on a PostgreSQL Deployment:

apiVersion: v1
kind: Pod
metadata:
  labels:
    type: app
    app: recommandations
spec:
  containers:
    - name: recommandations
      image: recommandations:latest
      envFrom:
        - configMapRef:
            name: postgres-config

The application is Python and starts quite fast. It attempts to connect to the PostgreSQL database. Unfortunately, the database hasn't finished initializing, so the connection fails, and Kubernetes restarts the pod.

We can fix it with an initContainer and a waiting container:

apiVersion: v1
kind: Pod
metadata:
  labels:
    type: app
    app: recommandations
spec:
  initContainers:
    - name: wait-for-postgres
      image: atkrad/wait4x:3.1
      command:
        - wait4x
        - postgresql
        - postgres://$(DATABASE_URL)?sslmode=disable
      envFrom:
        - configMapRef:
            name: postgres-config
  containers:
    - name: recommandations
      image: recommandations:latest
      envFrom:
        - configMapRef:
            name: postgres-config

In the above setup, the initContainer doesn't stop until the database accepts connections. When it does, it terminates, and the recommandations container can start. Kubernetes doesn't need to terminate the Pod as in the previous setup! It entails fewer logs and potentially fewer alerts.

When waiting becomes mandatory

The above is a slight improvement, but you can do without it. In other cases, waiting becomes mandatory. I experienced it recently when preparing for the workshop mentioned above. The scenario is the following:

The pipeline applies a manifest on the Kubernetes side
In the next step, it runs the test
As the test starts before the application is read, it fails.

We must wait until the backend is ready before we test. Let's use wait4x to wait for the Pod to accept requests before we launch the tests:

- name: Wait until the application has started
  uses: addnab/docker-run-action@v3                                       #1
  with:
    image: atkrad/wait4x:latest
    run: wait4x http ${{ env.BASE_URL }}/health --expect-status-code 200  #2

The GitHub Action allows running a container. I could have downloaded the Go binary instead.
Wait until the /health endpoint returns a 200 response code.

Conclusion

Kubernetes startup probes are a great way to avoid unnecessary restarts when you start services that depend on each other. The alternative is an external waiting tool configured in an initContainer. wait4x is a tool that can be used in other contexts. It's now part of my toolbelt.

To go further:

Originally published at A Java Geek on April 20^th, 2025