Coder Legion - Recent posts tagged backend-development

The Semicolon That Stole 5 Days of My Life

Sat, 31 Oct 2026 08:21:59 +0000

I have never been this frustrated with a single bug in my life.

And the culprit? The classic, infamous semicolon.

I recently started exploring Express.js and wired up a MongoDB database for an assignment. Connecting the database was a small hurdle, but that wasn’t the real fight.
The terminal said the database was connected. But Postman and the browser? Total silence. No response, no error.

I created a fresh test database. Nothing. Then I disconnected the database entirely and bypassed my .env file, and suddenly the routes worked.

So my brain locked onto the wrong villain: “The URI must be broken.” “Maybe it’s the network whitelist.” “Maybe the connection string is formatted wrong.”

I spent 5 days down that rabbit hole.

This morning, with the submission deadline today, I told myself: if I don’t fix this now, I’m deleting the whole folder and starting over. So I opened every file one last time.
And there it was, hiding in plain sight in my .env file:

PORT=3000;

In JavaScript, a semicolon ends a statement. In a .env file, it’s just part of the value. My port wasn’t 3000, it was “3000;”, so the server never listened where I was sending requests.

Five days. One character.

I still can’t decide whether to be relieved that it works or annoyed that a single semicolon stole my peace for a week. But this is part of what we signed up for as developers. The wins are great, and the silent syntax traps are real.

No time to celebrate though. The deadline is today.

What’s the most ridiculous bug that ate days of your life for no good reason? I want to hear your horror stories.

Adiós.

Moving from Frontend JS to Node.js

Fri, 23 Oct 2026 16:16:26 +0000

For a while now, frontend JavaScript has been my comfort zone. I like the immediate feedback: you tweak state, manipulate the DOM, fix a UI interaction, and see the results instantly on screen.

Recently, though, I shipped the live frontend build of Sovari (my reading and jotting app). Using it daily made one thing clear very fast: browser storage can only get you so far. If I want real user accounts, persistent notes that don't vanish when a cache clears, and actual data flow, I have to step into backend development.

The natural choice for me was Node.js.

Since JavaScript was already my daily driver, I didn't have to battle an entirely new syntax. But running JS outside the browser? That completely changed how I had to think about my code.

Here’s a quick look at what I’ve been wrapping my head around so far, what clicked, and where I’m taking this next.

What I have notices is that there is no longer window, but there is more responsibility.
In the browser, JavaScript is tucked inside a sandbox. If something throws an error, you break a component or crash one user's tab. Refresh, and you’re usually back.

On the server, it’s a whole different game. Node gives you direct access to the machine: the file system, network streams, and environment variables. The flip side? An unhandled error or a blocked loop doesn't just affect one person; it can bring the entire service down.

Same language, completely different weight.

The Basics I've Been Digging Into

Before rushing to install heavy frameworks or abstractions, I wanted to understand the actual plumbing:

The Client-Server Module: Getting past the idea of UI state and actually thinking about stateless HTTP requests and responses. The server doesn’t know who you are until you inspect the headers, parse the body, and send something back.

HTTP Methods & Codes: Actually being intentional with REST conventions. Using GET just for reading data, POST to create resources, and PUT/PATCH to update notes. More importantly, getting comfortable with status codes beyond just 200—like knowing when to send 201 Created, handling defensive 400 vs 404 responses, and dreading the dreaded 500.

CommonJS vs. ES Modules: In frontend React land with Vite, you just import and export without thinking much. In Node, I had to understand how require() works, how module caching behaves, and how to configure "type": "module" in package.json to keep my modern import syntax.

Environment Variables (process.env): In frontend builds, bundled environment variables can easily leak into the client if you aren't careful. On the backend, setting up .env files and reading through process.env is essential for keeping ports and sensitive keys tucked safely away from the public eye.

Spinning Up My First Raw Node Server

Instead of immediately reaching for a framework, I wanted to see what vanilla Node does with its built-in http module.

Writing the hand was an eye-opener. You actually see the incoming stream, inspect the URL string, manually set status headers, serialize your JSON, and close the response stream with res.end().

It demystifies backends a bit: frameworks aren't doing black magic, they're just wrapping these exact streams so you don't have to write fifty if statements.

What’s Next: Express and Sovari

Writing raw routing logic by hand is a great learning exercise, but you quickly realize why nobody does it for production. Once you need URL params, request body parsing, and route grouping, vanilla Node gets messy fast.

The next step on my roadmap is Express.js:

Figuring out middleware pipelines.

Splitting routes cleanly with express.Router (keeping books and notes organized).

Setting up proper global error handling.

The end goal hasn't changed: giving Sovari the production backend it needs. The client works and I'm actively using it, but hooking it up to a real database and a clean REST API is what will take it from a side project prototype to a reliable tool.

If you made the jump from frontend to backend with Node, what was the first concept that really clicked for you? Drop your take in the comments.

Adiós.

Building Your First Production-Ready API with Python: A Complete Guide for Full Stack Developers

Sun, 11 Oct 2026 17:37:55 +0000

Application Programming Interfaces (APIs) are the backbone of modern software development. They allow different applications, services, and systems to communicate with each other. Whether you are building a web application, a mobile app, an e-commerce platform, or an AI-powered SaaS product, APIs play a critical role in delivering reliable and scalable functionality.

Python has become one of the most popular programming languages for API development because of its clean syntax, extensive ecosystem, and powerful frameworks. Developers can use Python to build everything from simple REST APIs to complex microservices that support thousands of users.

However, creating an API that works on your local computer is only the first step. A production-ready API must be secure, maintainable, well-documented, properly tested, observable, and capable of handling real-world traffic.

In this article, we will explore how to build your first production-ready API with Python. We will use FastAPI as our primary framework and examine essential concepts such as project structure, request validation, database integration, authentication, error handling, automated testing, logging, performance, and deployment.

The goal is not just to build an endpoint that returns data. It is to understand how professional developers design APIs that can be maintained and improved over time.

1. Understanding What Makes an API Production-Ready

Before writing code, it is important to understand the difference between a basic API and a production-ready API.

A basic API might accept a request, perform an operation, and return a response. That is sufficient for learning, but production environments introduce additional challenges.

For example, users may send invalid input, databases may become temporarily unavailable, attackers may attempt unauthorized access, and traffic may increase unexpectedly. Without proper safeguards, these situations can cause failures or expose sensitive information.

A production-ready API should include several essential characteristics.

Reliability: The API should handle expected failures gracefully and return consistent responses.

Security: Sensitive resources must be protected through authentication, authorization, input validation, and secure configuration.

Maintainability: The codebase should have a clear structure that makes it easy to understand, test, and extend.

Performance: Database queries, network operations, and request processing should be efficient.

Observability: Logs, metrics, and error reports should help developers understand what happens inside the application.

Documentation: Developers should be able to understand endpoints, request formats, response schemas, and authentication requirements.

Testing: Automated tests should verify that important functionality continues working after changes.

These qualities should influence the design from the beginning rather than being added as an afterthought.

2. Choosing Python and FastAPI

Python offers several frameworks for API development, including Flask, Django REST Framework, and FastAPI.

Flask is lightweight and flexible, making it suitable for small applications and projects that need a minimal framework. Django REST Framework is particularly useful when building APIs around Django applications that require an integrated ORM, administrative interface, and established authentication features.

FastAPI is an excellent choice for modern API projects because it supports asynchronous programming, automatic request validation, Python type hints, and automatically generated OpenAPI documentation.

It also works well with Pydantic, which provides structured data validation and serialization.

Consider a simple example:

from fastapi import FastAPI

app = FastAPI()

@app.get("/")
def home():
    return {"message": "Welcome to my API"}

When the application receives a GET request at the root endpoint, it returns a JSON response.

Although this example is small, it demonstrates the basic structure of a FastAPI application. The framework handles routing and converts the returned Python dictionary into a JSON response.

For a production project, we will expand this foundation with validation, persistence, testing, and security.

3. Setting Up the Development Environment

A clean development environment helps prevent dependency conflicts and makes a project easier to reproduce.

First, install Python and create a directory for the project.

mkdir production-api
cd production-api

Create a virtual environment:

python -m venv .venv

Activate it on Windows:

.venv\Scripts\activate

On macOS or Linux:

source .venv/bin/activate

Next, install the primary dependencies:

pip install fastapi "uvicorn[standard]" sqlalchemy \
  psycopg[binary] pydantic-settings \
  pyjwt "pwdlib[argon2]" pytest httpx

These packages serve different purposes:

FastAPI provides API routing and request handling.
Uvicorn runs the application as an ASGI server.
SQLAlchemy provides database access and ORM functionality.
Psycopg connects Python applications to PostgreSQL.
Pydantic Settings manages configuration.
PyJWT supports JSON Web Token operations.
pwdlib with Argon2 supports secure password hashing.
Pytest and HTTPX support automated API testing.

For a real project, record the exact tested dependency versions in a lockfile or another reproducible dependency-management workflow. Keep development tools separate from production dependencies where appropriate.

4. Designing a Maintainable Project Structure

As an application grows, placing every endpoint, database operation, and validation rule inside one Python file becomes difficult to maintain.

A modular structure separates responsibilities and improves readability.

One possible project layout is:

production-api/
├── app/
│   ├── __init__.py
│   ├── main.py
│   ├── config.py
│   ├── database.py
│   ├── models.py
│   ├── schemas.py
│   ├── security.py
│   └── routers/
│       ├── __init__.py
│       └── tasks.py
├── tests/
│   └── test_tasks.py
├── .env
├── .gitignore
└── requirements.txt

Each module has a specific responsibility.

The main application initializes FastAPI and registers routers. Configuration manages environment-specific settings. Database code creates database connections and sessions. Models describe database tables, while schemas define the structure of incoming and outgoing data.

Routers group related endpoints, and tests verify application behavior.

This organization makes it easier to add features without turning the codebase into a tightly connected collection of functions.

The ideal structure depends on the size of the application. A small API does not need dozens of folders, but even a small project benefits from clear separation between business logic, persistence, and HTTP handling.

5. Building the Main Application

Create app/main.py:

from fastapi import FastAPI
from app.routers import tasks

app = FastAPI(
    title="Production Tasks API",
    version="1.0.0",
    description="A practical task management API"
)

app.include_router(tasks.router, prefix="/api/v1")

@app.get("/health/live", tags=["Health"])
def liveness():
    return {"status": "alive"}

This creates the application and registers a router under the /api/v1 prefix.

Versioning endpoints helps developers evolve an API without unexpectedly breaking existing clients. A future version can introduce different response formats or behavior while the previous version remains available during migration.

The liveness endpoint provides a simple indication that the application process can respond. It does not prove that PostgreSQL or every external dependency is available. A separate readiness check can verify whether the service is ready to receive traffic.

FastAPI also generates interactive API documentation by default at /docs and an OpenAPI schema at /openapi.json.

These features are particularly helpful when frontend developers, mobile developers, and third-party integrators need to understand how the API works.

6. Managing Configuration Safely

Production applications should not depend on hardcoded database passwords, secret keys, or environment-specific URLs.

Instead, use environment variables or a dedicated secrets manager.

Create app/config.py:

from pydantic_settings import BaseSettings, SettingsConfigDict

class Settings(BaseSettings):
    database_url: str
    jwt_secret: str

    model_config = SettingsConfigDict(
        env_file=".env",
        extra="ignore"
    )

settings = Settings()

A local .env file might contain:

DATABASE_URL=postgresql+psycopg://user:password@localhost/tasks
JWT_SECRET=replace-with-a-long-random-secret

The example values are placeholders. Replace them with actual credentials for your development environment.

Add .env to .gitignore so credentials are not accidentally committed to Git.

In production, inject secrets through the deployment platform or a secrets manager instead of distributing a local .env file. Use a strong, randomly generated signing key, rotate credentials when necessary, and never expose them in API responses or logs.

Configuration validation also helps the application fail early when required settings are missing.

7. Connecting to PostgreSQL

Many APIs need persistent storage. PostgreSQL is a strong choice because it supports relational data, transactions, indexes, and advanced querying capabilities.

Create app/database.py:

from sqlalchemy import create_engine
from sqlalchemy.orm import DeclarativeBase, sessionmaker

from app.config import settings

engine = create_engine(
    settings.database_url,
    pool_pre_ping=True
)

SessionLocal = sessionmaker(
    bind=engine,
    autoflush=False,
    expire_on_commit=False
)

class Base(DeclarativeBase):
    pass

The engine manages connections to the database, while SessionLocal creates database sessions.

The pool_pre_ping=True option checks pooled connections before using them, helping detect connections that have become invalid.

Next, define a database model in app/models.py:

from sqlalchemy import Boolean, String, Text
from sqlalchemy.orm import Mapped, mapped_column

from app.database import Base

class Task(Base):
    __tablename__ = "tasks"

    id: Mapped[int] = mapped_column(
        primary_key=True
    )
    title: Mapped[str] = mapped_column(
        String(200),
        index=True
    )
    description: Mapped[str | None] = mapped_column(
        Text,
        nullable=True
    )
    completed: Mapped[bool] = mapped_column(
        Boolean,
        default=False
    )

The model defines a task with an identifier, title, optional description, and completion status.

In a production system, database schema changes should be managed with migrations, such as Alembic, instead of relying on automatic table creation during application startup.

Migrations provide a controlled way to introduce columns, indexes, constraints, and other structural changes without losing existing data.

8. Validating Incoming and Outgoing Data

Input validation is one of the most important responsibilities of an API.

Without validation, clients may submit missing fields, excessively long strings, unexpected values, or data in the wrong format.

Create app/schemas.py:

from pydantic import BaseModel, ConfigDict, Field

class TaskCreate(BaseModel):
    title: str = Field(
        min_length=1,
        max_length=200
    )
    description: str | None = Field(
        default=None,
        max_length=2000
    )

class TaskUpdate(BaseModel):
    title: str | None = Field(
        default=None,
        min_length=1,
        max_length=200
    )
    description: str | None = Field(
        default=None,
        max_length=2000
    )
    completed: bool | None = None

class TaskRead(BaseModel):
    id: int
    title: str
    description: str | None
    completed: bool

    model_config = ConfigDict(
        from_attributes=True
    )

These schemas define the expected shape of task data.

TaskCreate validates new tasks. TaskUpdate supports partial updates, while TaskRead defines the public representation returned by the API.

Using separate schemas prevents clients from directly controlling internal database fields.

For example, a client should not be able to set an administrative flag or assign a different user's identifier simply by including extra JSON properties in a request.

Validation does not replace authorization, but it reduces the amount of unexpected data that reaches the application's business logic.

9. Implementing CRUD Endpoints

CRUD stands for Create, Read, Update, and Delete. These operations form the foundation of many business APIs.

First, create a database-session dependency in app/database.py:

from collections.abc import Generator

def get_db() -> Generator:
    db = SessionLocal()
    try:
        yield db
    finally:
        db.close()

The dependency creates a session for each request and closes it when the request finishes.

Now create app/routers/tasks.py:

from fastapi import APIRouter, Depends, HTTPException, Response
from sqlalchemy import select
from sqlalchemy.orm import Session

from app.database import get_db
from app.models import Task
from app.schemas import TaskCreate, TaskRead, TaskUpdate

router = APIRouter(
    prefix="/tasks",
    tags=["Tasks"]
)

@router.post("/", response_model=TaskRead, status_code=201)
def create_task(
    payload: TaskCreate,
    db: Session = Depends(get_db)
):
    task = Task(**payload.model_dump())
    db.add(task)
    db.commit()
    db.refresh(task)
    return task

@router.get("/", response_model=list[TaskRead])
def list_tasks(
    db: Session = Depends(get_db)
):
    return db.scalars(
        select(Task).order_by(Task.id)
    ).all()

@router.get("/{task_id}", response_model=TaskRead)
def get_task(
    task_id: int,
    db: Session = Depends(get_db)
):
    task = db.get(Task, task_id)

    if task is None:
        raise HTTPException(
            status_code=404,
            detail="Task not found"
        )

    return task

@router.patch("/{task_id}", response_model=TaskRead)
def update_task(
    task_id: int,
    payload: TaskUpdate,
    db: Session = Depends(get_db)
):
    task = db.get(Task, task_id)

    if task is None:
        raise HTTPException(
            status_code=404,
            detail="Task not found"
        )

    changes = payload.model_dump(exclude_unset=True)

    for field, value in changes.items():
        setattr(task, field, value)

    db.commit()
    db.refresh(task)
    return task

@router.delete("/{task_id}", status_code=204)
def delete_task(
    task_id: int,
    db: Session = Depends(get_db)
):
    task = db.get(Task, task_id)

    if task is None:
        raise HTTPException(
            status_code=404,
            detail="Task not found"
        )

    db.delete(task)
    db.commit()
    return Response(status_code=204)

This router provides a basic task management API.

The POST endpoint creates a task. GET endpoints retrieve all tasks or a single task. PATCH updates selected fields, and DELETE removes a task.

Returning appropriate HTTP status codes improves consistency. For example, 201 Created indicates successful resource creation, 404 Not Found indicates a missing resource, and 204 No Content indicates successful deletion without a response body.

This implementation is a useful learning foundation, but it is not yet suitable for unrestricted public deployment. The list endpoint needs pagination, database errors need appropriate handling, and every task operation must enforce authorization once users are introduced.

10. Adding Authentication and Authorization

Authentication answers the question, "Who is making this request?" Authorization answers, "What is this user allowed to access?"

These are different responsibilities.

An API that allows users to manage private tasks must ensure that one user cannot read, update, or delete another user's data.

A common approach is to authenticate users with securely hashed passwords and issue access tokens after successful login.

Password hashing should use a dedicated password-hashing algorithm such as Argon2. Passwords must never be stored in plain text or encrypted with a reversible method when hashing is appropriate.

JSON Web Tokens (JWTs) are often used to represent authenticated sessions. A signed access token may contain a subject identifying the user and an expiration timestamp.

However, a signed token is not automatically encrypted. Its contents should not include passwords, private secrets, or unnecessary sensitive information.

A production authentication flow should generally include:

User registration with strict input validation.
Password hashing using an appropriate password-hashing algorithm.
Login verification and controlled token issuance.
Short-lived access tokens with validated expiration and signing algorithms.
Authorization checks on protected resources.
A secure token renewal or reauthentication strategy.
Rate limiting and monitoring for repeated failed login attempts.

When using OAuth2 or JWT bearer tokens with FastAPI, developers can use the framework's security utilities and validate tokens centrally.

Most importantly, authentication must be connected to data ownership. Adding a login endpoint alone does not protect task records.

For example, a task model might include an owner_id foreign key. A database query for a task should then verify that both the task identifier and the authenticated user's identifier match.

Conceptually:

task = db.scalar(
    select(Task).where(
        Task.id == task_id,
        Task.owner_id == current_user.id
    )
)

This query illustrates the ownership check; it assumes that the model contains owner_id and that current_user comes from a correctly implemented authentication dependency.

For a private resource, returning a not-found response for records the user cannot access can also reduce unintended disclosure of resource existence.

11. Handling Errors Consistently

Errors are inevitable in production systems. A client may send invalid data, a requested record may not exist, or a database operation may fail.

The API should return understandable errors without exposing internal implementation details.

FastAPI's HTTPException is useful for expected HTTP failures.

For example:

from fastapi import HTTPException

def require_task(task):
    if task is None:
        raise HTTPException(
            status_code=404,
            detail="Task not found"
        )

FastAPI automatically converts this exception into an appropriate HTTP response.

Validation errors are also handled by the framework. Developers should avoid catching every exception and returning a successful response because this hides failures and makes debugging difficult.

Unexpected exceptions should be logged internally and translated into safe client-facing error responses. The response should not contain database connection strings, stack traces, secret keys, or sensitive user information.

A consistent error format helps frontend applications display useful messages and handle failures predictably.

For example, a team may adopt a standard structure containing an error code, a human-readable message, and an optional request identifier.

The exact format matters less than applying it consistently across the application.

12. Improving Performance with Pagination and Indexing

An API that works well with ten records may struggle when a database contains millions of records.

Returning every row from a table is rarely a good default for a public endpoint. It can consume memory, increase response times, and place unnecessary pressure on the database.

Pagination allows clients to request a manageable subset of records.

For example, a list endpoint might accept a page size and a cursor or offset. The API should enforce a maximum page size to prevent clients from requesting excessively large responses.

Cursor-based pagination is often preferable for large or frequently changing datasets because it can provide more stable navigation when new records are added.

Database indexes also improve performance when queries repeatedly filter or sort by particular fields. However, indexes require additional storage and can slow down writes, so they should be selected based on actual query patterns.

Other performance improvements include:

Selecting only the columns the application needs.
Avoiding unnecessary database queries.
Using appropriate connection-pool settings.
Applying caching to suitable read-heavy endpoints.
Moving long-running jobs to background workers.
Profiling slow endpoints before attempting complex optimizations.

Asynchronous endpoints can improve concurrency for I/O-bound workloads, but they are not automatically faster in every situation. Blocking operations inside an asynchronous handler can undermine the expected benefits.

Use asynchronous database drivers and compatible libraries when an asynchronous architecture is appropriate, and measure the result rather than relying on assumptions.

13. Testing the API Before Deployment

Testing helps developers identify defects before users encounter them.

A production-ready API should have tests for normal behavior, invalid input, missing resources, permissions, and important failure scenarios.

Create tests/test_tasks.py:

from fastapi.testclient import TestClient

from app.main import app

client = TestClient(app)

def test_liveness_endpoint():
    response = client.get("/health/live")

    assert response.status_code == 200
    assert response.json() == {"status": "alive"}

Run the test with:

pytest

This test verifies that the application responds correctly to a liveness request.

For the task endpoints, additional tests should verify that valid requests create records, invalid requests return validation errors, nonexistent tasks return 404, and updates change only the intended fields.

Database tests should use an isolated test database or a controlled test fixture rather than modifying production data.

Authentication tests should also confirm that anonymous users cannot access protected resources and that one user cannot access another user's records.

Integration tests can verify interactions between FastAPI, PostgreSQL, and other services. End-to-end tests can validate complete workflows from the client's perspective.

A useful testing strategy combines fast unit tests with integration tests that cover the boundaries where failures are most likely.

14. Logging and Monitoring in Production

Once an API is deployed, developers need visibility into its behavior.

Logging provides a record of important events, while monitoring helps teams identify trends and emerging problems.

Useful production metrics include request volume, latency, error rates, database connection usage, and resource consumption.

Structured logging makes events easier to search and analyze. For example, a log entry may include the endpoint, HTTP status code, duration, and a request identifier.

However, logs should not contain passwords, access tokens, sensitive request bodies, or unnecessary personal information.

A request identifier can help developers trace an error across several services without exposing user credentials.

Monitoring alerts should focus on meaningful conditions, such as a sustained increase in server errors, a sharp rise in response times, or database connection exhaustion.

Health checks should be designed carefully. A liveness check should determine whether the process is functioning, while a readiness check can indicate whether the service can accept traffic based on critical dependencies.

Logging and monitoring are not simply operational extras. They help teams discover issues before those issues affect a large number of users.

15. Protecting the API Against Common Security Risks

Security needs to be considered throughout the development process.

Input validation: Validate request bodies, query parameters, path parameters, and file uploads. Set reasonable limits on field lengths and payload sizes.

Authentication: Require valid credentials for protected endpoints and validate token expiration, signatures, and expected claims.

Authorization: Check permissions for every protected operation, including access to individual records.

Rate limiting: Limit excessive requests, especially for login, password recovery, expensive searches, and other abuse-prone operations.

Transport security: Use HTTPS in production to protect information transmitted between clients and the API.

CORS configuration: Allow only the browser origins that need access. CORS is not a substitute for authentication or authorization.

Secret management: Keep credentials outside source control and restrict access to production secrets.

Dependency management: Regularly update dependencies, review security advisories, and remove unnecessary packages.

Database security: Use a dedicated database account with only the permissions required by the application.

Resource limits: Restrict request sizes, page sizes, execution times, and other resource-intensive operations.

Safe error responses: Avoid returning stack traces or internal system details to clients.

Security testing should include attempts to access resources without permission and attempts to bypass validation rules.

A secure API is not defined by the number of security libraries it uses. It is defined by whether its controls correctly protect data and enforce the application's rules.

16. Documenting the API for Other Developers

Good documentation reduces confusion and speeds up integration.

FastAPI generates interactive documentation based on route definitions, type hints, and Pydantic schemas.

Developers can visit /docs to inspect endpoints, review request and response formats, and try API calls.

For a production API, documentation should also explain authentication requirements, pagination behavior, error codes, rate limits, and any important business rules.

Endpoint descriptions should be clear enough that another developer can understand how to use the API without reading its internal implementation.

When an API changes, its documentation and tests should be updated together.

For teams maintaining multiple API versions, a documented deprecation policy helps clients migrate without unexpected interruptions.

Clear documentation is part of the API's developer experience, not merely a feature of the framework.

17. Preparing the API for Deployment

After implementation and testing, the API needs a reliable deployment process.

A typical deployment pipeline includes installing dependencies, running automated tests, checking configuration, applying database migrations, and deploying the application to a server or container platform.

For a basic local run, use:

uvicorn app.main:app --reload

The reload option is intended for development because it automatically restarts the application when files change.

For production, run the application using a suitable ASGI server configuration and a managed process or container platform. Configure worker processes according to the platform, available resources, and workload.

A Docker image can package the application and its dependencies into a reproducible deployment unit. PostgreSQL should generally run as a separate service rather than being embedded in the API container.

Production configuration should include the database connection string, signing keys, allowed origins, logging settings, and other environment-specific values.

Database migrations should run through a controlled release process. The application should not unexpectedly change the database schema every time a worker starts.

A mature deployment process also needs automated rollback or recovery procedures, database backups, and a strategy for handling incompatible schema changes.

Deploying the code is only one part of releasing a reliable service. The surrounding operational process matters just as much.

18. Using Docker for Consistent Environments

Docker helps teams reduce differences between development, testing, and production environments.

A simplified Dockerfile might look like this:

FROM python:3.12-slim

ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1

WORKDIR /app

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY app ./app

RUN useradd --create-home appuser
USER appuser

EXPOSE 8000

CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]

This example uses a slim Python image, installs dependencies, copies the application code, and runs the server as a non-root user.

A complete production setup should pin and verify dependency versions, use a maintained base image, scan images for vulnerabilities, and configure appropriate resource limits.

The application also needs valid environment variables and access to its database when the container starts.

Do not include .env files, passwords, or private keys in the Docker image. Inject sensitive configuration securely at runtime.

For a real deployment, add a suitable readiness check and coordinate application startup with database availability. Container orchestration platforms can manage restarts and scaling, but they cannot compensate for incorrect application logic or unsafe database migrations.

19. Automating Quality Checks with CI/CD

Continuous Integration and Continuous Deployment (CI/CD) help teams deliver changes more reliably.

A CI pipeline can run automatically whenever a developer pushes code or opens a pull request.

Typical pipeline steps include:

Installing pinned dependencies.
Checking code formatting and style.
Running static analysis and type checks.
Running unit tests.
Running integration tests against a test database.
Scanning dependencies for known vulnerabilities.
Building the deployment image.
Publishing and deploying an approved release.

Automation catches problems earlier and reduces the risk of deploying code that has not been tested.

Secrets should be stored in the CI/CD platform's protected secret-management facilities, not committed to the repository.

Production deployments should also include approval controls where appropriate, health verification after release, and a rollback strategy if the new version fails.

Even a small personal project can benefit from a simple pipeline that runs tests and checks before each release.

20. Common Mistakes Developers Should Avoid

Many API problems result from shortcuts that appear harmless during early development.

One common mistake is putting all application logic in a single file. This makes it difficult to isolate responsibilities as the codebase grows.

Another is returning every database record without pagination. Large responses can degrade both server and client performance.

Developers also sometimes assume that hidden buttons or frontend checks provide security. They do not. Authorization must be enforced by the server.

Hardcoding secrets is another dangerous practice because credentials can leak through repositories, logs, and deployment artifacts.

Poor error handling creates additional problems. Returning raw exception messages may reveal sensitive implementation details, while silently ignoring exceptions makes failures difficult to diagnose.

Skipping automated tests increases the chance that a small code change will break an existing feature.

Finally, many developers optimize too early. Introducing complex caching, asynchronous patterns, or microservices before measuring actual bottlenecks can increase complexity without improving performance.

The better approach is to establish clear requirements, build a maintainable foundation, test critical behavior, and use production measurements to guide improvements.

21. A Practical Production-Readiness Checklist

Before releasing your Python API, review the following areas.

Application design

Clear project structure and separation of responsibilities.
Versioned endpoints and documented request/response schemas.
Consistent HTTP status codes and error handling.

Database

Controlled schema migrations.
Proper indexes and efficient queries.
Connection-pool configuration and backup procedures.

Security

Authentication and resource-level authorization.
Secure password hashing and token validation.
HTTPS, safe CORS settings, rate limiting, and secret management.

Testing

Unit and integration tests.
Validation and error-handling tests.
Permission and cross-user access tests.

Operations

Structured logs and useful metrics.
Liveness and readiness checks.
Reproducible deployments and recovery procedures.

Maintenance

Dependency updates and security reviews.
Automated CI checks.
Current documentation and a plan for API version changes.

Not every project requires a complex infrastructure from day one. The objective is to identify the risks that matter for the application's users and address them proportionately.

Conclusion

Building your first production-ready API with Python is an important milestone for any full stack developer. It teaches much more than defining routes and returning JSON responses. It requires thinking about data integrity, authentication, authorization, error handling, performance, testing, deployment, and long-term maintenance.

FastAPI provides a powerful foundation for this work through type-driven validation, automatic documentation, and support for modern Python application patterns. Combined with PostgreSQL, SQLAlchemy, automated testing, and a secure deployment workflow, it can support applications ranging from small business tools to sophisticated backend services.

The most important lesson is that production readiness is not a single feature or final development step. It is the result of many small engineering decisions made throughout the project.

Start with a clear architecture, implement one feature at a time, validate inputs, protect user data, and test the behavior that matters most. Add monitoring and automation before real users depend on the service.

Once you understand these fundamentals, you will be better prepared to build reliable APIs for web applications, mobile platforms, e-commerce systems, and AI-powered products.

A successful API is not simply one that works when everything goes right. It is one that behaves predictably, protects its users, and remains maintainable when real-world conditions become complicated.

My Journey as a PHP Developer: Building Solutions Through Code

Fri, 09 Oct 2026 12:40:54 +0000

As a PHP developer, I enjoy turning ideas into functional websites and building backend solutions that solve real-world problems. For me, programming is not just about writing code; it’s about understanding challenges, finding practical solutions, and creating applications that make everyday tasks easier.

I enjoy working on dynamic websites, developing backend functionality, and exploring ways to make applications more efficient, reliable, and user friendly. I believe that clean, maintainable code is the foundation of good software development, and I always try to improve the way I approach problems and build solutions.

Technology is constantly evolving, which makes software development an exciting journey. I enjoy learning new tools, exploring different approaches, debugging problems, and discovering better ways to write code. Every challenge is an opportunity to learn something new and become a better developer.

Beyond coding, I value sharing knowledge, exchanging ideas, and connecting with other developers. I believe we can grow faster when we learn from each other's experiences, discuss different perspectives, and support the developer community.

My goal is to keep improving my PHP development skills, build meaningful projects, and create solutions that deliver real value.

I’m always learning, always building, and always looking for the next problem to solve.

I'd love to connect with fellow developers, hear about your projects, and learn from your experiences. Let's keep building, sharing, and growing together!

How Java Developers Build Scalable Applications and Backend Systems

Fri, 02 Oct 2026 04:00:39 +0000

As a Java developer, I enjoy building applications that solve real problems, handling complex logic, and exploring how software works behind the scenes. Java is a powerful programming language that helps developers create everything from small applications to large backend systems used by businesses every day.

But how does the work of a Java developer actually happen? It involves much more than writing code. It requires understanding problems, designing solutions, managing data, testing functionality, and making sure applications work reliably as they grow.

Understanding the Problem Before Writing Code

Every application starts with a problem or an idea. Before writing Java code, I need to understand what the application should do, who will use it, and what features are required.

For example, if I am building an online shopping application, I need to think about user registration, product management, shopping carts, payments, and order tracking.

Breaking a large problem into smaller tasks makes development easier and helps create a clear plan for implementation.

Building the Backend with Java

The backend is the part of an application that handles business logic, processes requests, communicates with databases, and returns responses to users.

Java is useful for backend development because it supports object-oriented programming, strong type checking, and a large ecosystem of development tools and libraries.

For example, when a user logs into an application, the backend receives the login request, validates the submitted information, checks the stored user data, and returns an appropriate response.

Frameworks such as Spring Boot help Java developers build REST APIs, organize application logic, manage dependencies, and connect different parts of a system.

Connecting Applications to Databases

Most applications need to store and retrieve information. This is where databases become important.

A Java application might use MySQL or PostgreSQL to store user accounts, product information, transactions, or other business data.

Using tools such as Spring Data JPA and Hibernate, developers can work with database records through Java objects while still understanding the SQL queries and database operations involved.

For example, when a customer places an order, the application may validate the order, save its details, update inventory, and return a confirmation.

Handling these operations correctly is important because a small database error can affect the entire application.

Designing Scalable Backend Systems

Scalability means an application can handle increasing workloads as the number of users or requests grows.

Imagine a small application that works perfectly when 100 people use it. What happens when thousands of users start accessing it simultaneously?

Developers need to consider database performance, efficient algorithms, caching, connection management, and application architecture.

Depending on the requirements, a Java application might use Redis for caching, message queues for background processing, or multiple application instances to handle incoming requests.

Scalability does not always mean using microservices or adding more servers. Sometimes, improving a database query or removing an unnecessary operation is enough to make a significant difference.

Solving Complex Problems Through Debugging

One of the most important parts of software development is debugging.

An application may fail because of incorrect business logic, unexpected input, database connection problems, or communication failures between services.

Instead of fixing errors randomly, I try to understand what caused the problem, reproduce it, examine logs, and test possible solutions.

Writing unit tests and integration tests also helps identify problems early and reduces the risk of breaking existing functionality when new features are added.

Every bug provides an opportunity to understand the system more deeply and improve the quality of the code.

Learning and Exploring Software Architecture

Technology continues to evolve, so learning is an important part of being a Java developer.

I enjoy exploring software architecture, design patterns, REST APIs, performance optimization, and tools that make applications easier to maintain.

For example, separating an application into controller, service, and repository layers can help organize responsibilities and make the code easier to test.

As projects become more complex, understanding how different components communicate becomes just as important as knowing the programming language itself.

What Makes Development Meaningful?

For me, software development is about turning ideas into working solutions. Writing code is only one part of the process. Understanding the problem, making thoughtful technical decisions, testing the result, and improving the system are equally important.

I believe that becoming a better Java developer requires consistent practice, curiosity, and a willingness to learn from mistakes.

My goal is to keep improving my Java development skills, build reliable and scalable backend applications, and explore better ways to solve real-world problems through software.

Every project brings a new challenge, and every challenge offers an opportunity to learn something valuable.

Building a URL Shortener with FastAPI and SQLite — V1 Complete

Mon, 28 Sep 2026 16:49:18 +0000

After completing my Personal Expense Tracker, I started my fourth project: a URL Shortener. This was also my first web backend project, so the goal was to move from CLI applications into building a small web application and get more practical experience with FastAPI, SQLite, HTML, CSS, and Jinja2.

The result is a simple URL Shortener that takes a long URL, generates a short code, stores the mapping in SQLite, and redirects users to the original URL.

What Is a URL Shortener?

A URL shortener converts a long URL into a shorter link.

For example:

https://example.com/some/very/long/path/to/a/page

can become something like:

http://localhost:8000/aB72xK

When someone opens the shortened URL, the application looks up the short code in the database and redirects them to the original URL.

So the basic flow is:

Long URL
   ↓
Generate short code
   ↓
Store both in SQLite
   ↓
Return short URL
   ↓
User opens short URL
   ↓
Look up short code
   ↓
Redirect to original URL

Project Architecture

I kept the architecture intentionally simple for V1.

There are two main backend responsibilities:

FastAPI application

Handles the web routes, receives the submitted URL, generates the short code, displays the result, and performs the redirect.

SQLite database layer

Handles storing and retrieving the relationship between the short code and the original URL.

The database table contains:

urls
├── short_code
└── original_url

The short_code is the primary key because each shortened URL needs its own unique identifier.

Moving From API-Style Responses to a Web Interface

One of the interesting parts of this project was learning that FastAPI doesn't have to be used only for JSON APIs.

For the first stage of learning FastAPI, I worked with API-style routes and responses.

For this project, I wanted a simple user-facing interface instead.

So I used:

HTML for the page structure
CSS for basic styling
Jinja2 for server-side HTML rendering
FastAPI as the backend

The user submits a URL through the HTML form, FastAPI processes it, and Jinja2 renders the result back into the page.

I intentionally did not add JavaScript for V1. The goal was to understand the traditional server-rendered web application flow first.

Generating the Short Code

For the current version, the application generates a 6-character short code using letters and numbers.

The generated code is then stored together with the original URL.

For example:

short_code:   aB72xK
original_url: https://example.com/some/long/url

When /aB72xK is requested, the application searches for that short code and retrieves the original URL.

Redirecting the User

The redirect is the part that makes the application actually function as a URL shortener.

The application receives a request such as:

/aB72xK

It looks up aB72xK in SQLite.

If the code exists, FastAPI returns a redirect response to the stored original URL.

If the code doesn't exist, the application returns a not-found response.

This helped me understand that a shortened URL isn't just a smaller version of the original URL. It is an identifier that points to data stored by the application.

Keeping V1 Simple

One thing I wanted to maintain throughout this project was a limited scope.

The purpose of V1 was to learn the fundamental workflow, not to build a production-scale URL-shortening service.

Current V1 Scope

URL shortening
6-character short code generation
SQLite storage
URL redirection
Simple HTML + CSS interface
Jinja2 server-side rendering
Empty-input handling

That's where I decided to stop V1.

There are many features that could be added later, such as URL expiration, analytics, custom short codes, URL management, authentication, and more.

But adding all of them now would move the project away from its original learning goal.

What I Learned

This project helped me connect several concepts that I had previously learned separately.

I practiced:

Building a web application with FastAPI
Connecting FastAPI with SQLite
Designing a simple database structure
Using a database class to separate database operations
Generating unique identifiers
Handling redirects
Using Jinja2 templates
Connecting HTML forms with backend routes
Serving CSS through FastAPI
Organizing a small web application
Deploying the application as a web service

More importantly, I got a better understanding of how the frontend, backend, and database communicate with each other.

Project Structure

url-shortener/
├── main.py
├── database.py
├── requirements.txt
├── templates/
│   └── index.html
├── static/
│   └── style.css
├── .gitignore
├── LICENSE
└── README.md

What's Next?

V1 is now complete.

I want to keep the project available as a small working example of a FastAPI web application rather than continuously adding features without a specific learning goal.

Future versions could explore more advanced concepts when they become relevant to my learning path.

Final Thoughts

This project was a useful step from building command-line applications toward building web applications.

My previous projects helped me practice Python, OOP, CRUD operations, SQLite, and application structure. With the URL Shortener, I was able to bring those concepts together with FastAPI and a browser-based interface.

It's a small application, but it gave me practical experience with the complete basic flow:

Browser → FastAPI → SQLite → FastAPI → Browser

And that's exactly what I wanted from Project #4.

Project #4 — URL Shortener: V1 Complete.

Live Demo: https://url-shortener-k5ed.onrender.com/

Source Code: https://github.com/xDK0d3r/url-shortener

Continue Reading

I also wrote about this project on Dev.to, where I shared more about my learning experience, what I learned while building my first web backend project, and how this project fits into my overall development journey.

If you'd like to read that perspective, check it out here:

Read the Dev.to article →

A Tiny Nim Microservice, Three Real Bugs, and What They Taught Me About My Own Mummy Fork

Tue, 15 Sep 2026 00:19:50 +0000

I recently needed something small: a POST /api/contact endpoint for my portfolio site at https://www.isaiah.name.ng that takes a name, email, and message, validates it, and emails it to me. The kind of thing you'd normally knock out without thinking twice.

I decided to build it directly on my own Mummy fork — the one with OpenAPI schema generation, typed validation, and composable middleware — partly to actually dogfood it on something real, and partly because I assumed a single-endpoint service would be trivial. It mostly was. But three specific bugs along the way taught me more about the fork's internals than building the fork itself did.

The Shape of the Service

Nothing exotic: a ContactBody type with name, email, and message, all required strings. Validation happens in two layers — parseValidatedBody[ContactBody] checks presence and type against the fork's schemaOf-generated schema, and then explicit checks enforce non-empty fields, a 2000-character cap, and a hand-rolled email format check. Either layer raising the fork's ValidationError gets mapped to a clean 400 {"error": ...} by installDefaultErrorHandler. A send failure is logged server-side and answered with a generic 500 — the caller never sees why an email failed to send, only that it did.

On top of that: CORS middleware that rejects disallowed origins with 403 and answers preflight requests properly, an in-memory rate limiter capping submissions at 5 per IP per 10 minutes, and the fork's loggingMiddleware as the outermost layer so every request logs method, path, and timing. The OpenAPI docs come free from the same schema the validator uses, served at /docs with the spec at /openapi.json — the whole reason I built the fork in the first place.

The Pivot Nobody Plans For: SMTP → Resend

The original plan was plain SMTP via std/smtp. It compiled fine against the fork. Then I deployed to Render's free tier and every send just hung until timeout — Render blocks outbound SMTP ports (25/465/587) on that tier. Resend's REST API runs over HTTPS:443, which isn't blocked, and needs just one API key. As a bonus, its failures come back as ordinary HTTP errors the handler already catches and logs, instead of the occasional AssertionDefect that a raw SMTP library can throw and potentially take a worker down with it.

This is the kind of constraint you only discover by actually deploying, not by reading docs — worth remembering if you're building anything Render-hosted that needs to send mail.

Three Bugs That Taught Me More Than the Feature Did

1. Two HttpHeaders types that look identical and aren't.
Mummy imports std/httpcore aliased internally, and Request.headers is typed against that aliased import. My module also did a plain import std/httpcore for the Resend payload types — and suddenly every header operation threw a "type mismatch" between what looked like the same type. Nim's module system treats the two imports as distinct type identities even though they're structurally the same. The fix was narrow: import std/httpcore except HttpHeaders, so my module only pulled in the pieces it needed for the Resend payload and left the header type alone. This was, by a wide margin, the buggiest hour of the whole build — and a good reminder that "same type, different import path" is a real failure mode in Nim, not a theoretical one.

2. gcsafe doesn't care that your global never actually changes.
The fork's Middleware and RequestHandler types are marked {.gcsafe.}, which means the compiler's data-race checker gets strict about any global state touched inside a handler — because Mummy dispatches requests across a worker-thread pool, not a single event loop. My CORS middleware read a var global holding the allowed-origins list, set once at startup and never touched again — and the compiler didn't care that it was logically immutable after init. It only sees a mutable global read from a gcsafe context. The fix was a {.cast(gcsafe).} block around that specific read, while the genuinely-shared rate-limit table stayed behind a real Lock — the same pattern the fork already uses internally for bearerAuthMiddleware. It's a useful distinction to internalize on a threaded server: "never mutated after startup" and "thread-safe" are not the same claim, and the compiler will only ever check for the second one.

3. An unregistered route means your middleware never runs — including for the request that most needs it.
Browsers send an OPTIONS preflight before certain cross-origin POST requests. Mummy only runs its middleware chain for methods that have a registered route — so with no OPTIONS /api/contact route, Mummy answered preflight with its own 405 before corsMiddleware ever got a chance to run. The browser never saw the Access-Control-Allow-Origin header it needed, so the real POST was blocked client-side despite the server being otherwise correctly configured. The fix was almost comically small: register an OPTIONS /api/contact route whose handler does nothing but exist, which is enough to route the request into the middleware chain where corsMiddleware can actually answer it with a 204. Easy to fix once you know it — genuinely confusing the first time a working-looking CORS setup silently fails only on the preflight leg.

What Actually Got Verified

Before calling it done, I ran it locally and checked the specific paths that tend to hide bugs rather than just the happy path: a disallowed origin (403), a malformed email (400 with the exact field named), a name at the 2000-character boundary (passes) versus one character over (rejected), six rapid submissions from one IP (five succeed, the sixth comes back 429 with Retry-After), and that /health and OPTIONS both stay outside the rate limiter so Render's health checks and CORS preflight never get throttled.

Why This Was Worth Writing Up

None of these three bugs were about the feature — a contact form endpoint is about as unglamorous as backend work gets. They were about the specific intersection of Nim's module aliasing, gcsafe's literal (not logical) read of what's safe, and Mummy's routing-before-middleware order. If you're building anything on Mummy — my fork or the original — that touches CORS, thread-shared state, or has more than one import path into std/httpcore, these three are worth checking for before you hit them the hard way.

The full service, Dockerfile, and a drop-in script.js for wiring this into any static contact form are in the repo if you want to see the whole thing end to end.

🚀 Introducing ReplyPilot | AI Customer Support for Growing Businesses

Tue, 08 Sep 2026 11:46:47 +0000

I’m excited to make ReplyPilot publicly available.

Most businesses don’t need another generic chatbot.

They need something that can actually handle customer conversations, answer questions using their real business information, and know when it should stop talking and bring in a human.

That’s what I built ReplyPilot for.

🤖 ReplyPilot helps businesses automate customer support across:

• WhatsApp
• Instagram
• AI-powered conversations
• Human agent handoff
• Business FAQs
• Products & pricing
• Business policies
• Team based support
• Conversation history & audit trails

The important part: AI that knows its limits.

ReplyPilot doesn't simply generate an answer and hope it's correct.

You provide your business knowledge like FAQs, products, prices, policies, and other information and the AI uses that information to respond.

When it isn't confident enough, it can hand the conversation over to a real person instead of confidently making something up.

You can even configure the confidence threshold and escalation behaviour.

Built for real businesses

A customer shouldn't have to wait hours just because nobody is available to answer WhatsApp.

ReplyPilot can handle the routine questions instantly and bring your team in when human intervention is actually needed.

And your team gets their own accounts and roles, so support can scale beyond one person.

💰 Starting at just $3.50/month

No complicated enterprise pricing.

No setup fees.

Flat pricing.

Cancel anytime.

Additional agents are just $2/month each.

You also get a 5-day free trial with no credit card required.

🔗 Try ReplyPilot:
https://replypilot.royalbv.name.ng

📚 Read the documentation:
https://replypilot.royalbv.name.ng/docs

This is now live and publicly available.

If you run a business that receives a lot of customer questions through WhatsApp or Instagram, ReplyPilot was built for you.

I'd love to get your feedback, bug reports, feature requests, and honest opinions.

AI should not just reply. It should know when to reply, what to say, and when to get out of the way.

Foundations of System Design: Learn It the Way I Wish Someone Taught Me

Sun, 06 Sep 2026 13:32:05 +0000

When I first started learning system design, I felt overwhelmed.

Load balancers. Caches. Message queues. Replication. Sharding. Distributed systems.

There are a lot of terms. And most guides throw them at you like a vocabulary test.

But here is the thing: I don't think the best way to learn system design is to memorize diagrams or vocabulary.

I prefer starting with something much simpler:

Start with one request. Then introduce a new concept only when the system actually needs it.

That is what we are going to do in this guide.

We will start with a very small application and gradually make it capable of handling more traffic and more failures. Every new concept we introduce will be because the system actually broke and needed it.

If you have ever felt like system design is "too advanced" or "not for you" yet, I promise you it is. You just need the right entry point.

Let's build it together.

What We Will Cover

What system design actually means (and why it is not just for senior engineers)
How a browser request travels to your server
Where the database fits and why relational databases are useful
Transactions and why they matter
Scaling: vertical and horizontal
Load balancers and how they distribute traffic
Health checks and detecting failed servers
Redundancy and why it matters
The architecture we built and the problems we have not solved yet
The mental model for approaching any system design problem

Let's start.

What Is System Design?

In simple terms, system design is about deciding how the different parts of a software system should work together as the system grows.

When your application has 10 users, almost any reasonable architecture can work.

When it has 100,000 users, things start getting interesting.

Your server has limited CPU and memory.

Your database has limits.

Networks fail.

Servers crash.

Requests take longer than expected.

Two users can try to change the same data at the same time.

So system design is not just:

"How do I make this application work?"

It becomes:

"How do I make this application continue working as traffic, data, and failures increase?"

If that question excites you, you are in the right place.

A Request From the Browser

Imagine you have a frontend application running in the browser.

The browser needs a list of products.

It sends a request:

Browser
   |
   | GET /products
   ↓
api.example.com

But api.example.com is not the server's IP address.

It is a domain name.

The browser needs to find the IP address associated with that domain.

This is where DNS (Domain Name System) comes in.

Very roughly:

Browser
   |
   | "Where is api.example.com?"
   ↓
DNS
   |
   | "It is available at this IP"
   ↓
IP address
   |
   ↓
Server

DNS is basically the system that helps translate human-readable domain names into network addresses.

You do not need to remember the entire DNS process yet.

For now, just remember:

Domain name → DNS resolution → IP address → server

The Server Processes the Request

Now the request reaches our server.

The server receives:

GET /products

It needs to retrieve the products from the database.

So our system currently looks like this:

Client
   |
   | Request
   ↓
Server
   |
   | Query
   ↓
Database

The database returns the data:

Database
   |
   | Products
   ↓
Server
   |
   | JSON response
   ↓
Client

The complete flow is:

Client
   ↓
Server
   ↓
Database
   ↓
Server
   ↓
Client

At this point, everything looks simple.

And that is exactly what we want.

Do not introduce a load balancer, Redis, Kafka, Kubernetes and 15 other things when one server is perfectly capable of handling the application.

Where Does the Database Fit?

Let us say our application has these entities:

Users
Products
Orders
Accounts
Sessions

This data has structure.

A user has an ID, name and email.

A product has an ID, name and price.

An order belongs to a user and contains products.

This is where a relational database can be a very good fit.

Examples include:

PostgreSQL
MySQL
Oracle
SQLite

Relational databases organize data into tables, which contain rows and columns.

For example:

users

id: 1
name: Noor
email: Emails are not allowed
id: 2
name: Ali
email: Emails are not allowed

And:

products

id: 1
name: Keyboard
price: 100
id: 2
name: Mouse
price: 50

The important thing is not memorizing the names of databases.

The important question is:

What kind of data and guarantees does my application need?

Why Relational Databases Are Useful

One major advantage of relational databases is that they let us model relationships between data.

Suppose Noor buys a keyboard.

We might have:

users
  ↓
orders
  ↓
order_items
  ↓
products

The database can represent those relationships explicitly.

We can then query information such as:

Which products did this user purchase?

This is where SQL and operations such as JOIN become useful.

For example, conceptually:

SELECT users.name, products.name
FROM users
JOIN orders ON orders.user_id = users.id
JOIN order_items ON order_items.order_id = orders.id
JOIN products ON products.id = order_items.product_id;

You do not need to memorize this query.

The important idea is that relational databases are very good when your data has clear relationships and you need strong consistency around those relationships.

Transactions

Now we have reached our first really important system-design problem.

Imagine a user places an order.

Two things need to happen:

1. Create the order
2. Deduct the user's balance

What happens if step 1 succeeds but step 2 fails?

We could end up with:

Order created
Money not deducted

That is an invalid state for many applications.

This is where transactions become important.

A transaction allows us to group multiple database operations into one logical operation.

Conceptually:

BEGIN TRANSACTION

Create order
Deduct balance

COMMIT

If something goes wrong:

BEGIN TRANSACTION

Create order
Deduct balance ← FAIL

ROLLBACK

The database can roll back the changes made by the transaction.

The simple mental model is:

Either the required operations succeed together, or the transaction does not commit their changes.

Transactions are built around the ACID properties:

A → Atomicity
C → Consistency
I → Isolation
D → Durability

You do not need to master ACID right now.

Just understand why transactions exist:

They help us keep related database operations in a valid state when something fails or when multiple operations happen concurrently.

Our First Bottleneck

So far we have:

                  ┌──────────┐
Client ────────→ │  Server  │
                  └────┬─────┘
                       │
                       ↓
                  ┌──────────┐
                  │ Database │
                  └──────────┘

Now imagine our application becomes popular.

We start getting:

10 requests/second

Then:

100 requests/second

Then:

1,000 requests/second

Our server has limited resources.

It has:

CPU
RAM
Network
Disk

At some point, one machine may no longer be enough.

Now we have to think about scaling.

This is where system design starts becoming interesting.

Vertical Scaling

The first thing we can do is make the existing server stronger.

Maybe we currently have:

4 CPU cores
8 GB RAM

We could upgrade it to:

16 CPU cores
64 GB RAM

This is called vertical scaling.

In simple terms:

Make the machine bigger.

It is easy to understand and often the simplest solution.

But it has limits.

Problem 1: Hardware Has Limits

You cannot infinitely increase the CPU and RAM of one machine.

Eventually you reach the limits of the available hardware or the cost becomes unreasonable.

Problem 2: Single Point of Failure

We still have one server.

If that server goes down:

Server crashes
   ↓
Application unavailable

This is a single point of failure.

A single point of failure is a component whose failure can bring down an important part of the system.

So we have solved one problem but created another question:

Can we run multiple servers?

Horizontal Scaling

Instead of making one server bigger, we can add more servers.

For example:

         Server 1
       /
Client ── Server 2
       \
         Server 3

This is called horizontal scaling.

Instead of:

1 very powerful server

we have:

3 servers

Now if one server fails, the other servers may still be able to handle requests.

But we have created a new problem.

How does the client know which server to talk to?

We do not want clients manually choosing:

server-1.example.com
server-2.example.com
server-3.example.com

We need something in front of them.

Load Balancer

This is where a load balancer comes in.

A load balancer receives incoming requests and decides which server should handle each request.

Our architecture becomes:

                    ┌──────────┐
                    │ Server 1 │
                  /
Client → Load Balancer → Server 2
                  \
                    └──────────┘
                         Server 3

The client does not need to know about all the application servers.

It talks to the load balancer.

The load balancer distributes traffic between available servers.

Now we have:

Client
   ↓
Load Balancer
   ↓
┌────────┬────────┬────────┐
│Server 1│Server 2│Server 3│
└────────┴────────┴────────┘

This gives us more capacity and can improve availability.

But now we need to decide:

How does the load balancer choose a server?

Load-Balancing Algorithms

There is not one universal algorithm.

Different algorithms work better for different workloads.

Round Robin

The load balancer sends requests in sequence:

Request 1 → Server 1
Request 2 → Server 2
Request 3 → Server 3
Request 4 → Server 1

Simple and useful when servers have similar capacity and requests have roughly similar cost.

Weighted Round Robin

Maybe our servers are not equally powerful.

Server 1 → weight 1
Server 2 → weight 2
Server 3 → weight 3

The stronger server receives more traffic.

Least Connections

The load balancer sends a new request to the server currently handling the fewest active connections.

Server 1 → 30 connections
Server 2 → 12 connections
Server 3 → 21 connections

New request → Server 2

This can be useful when requests have different durations.

IP Hash

The load balancer hashes the client's IP address and uses the result to choose a server.

Conceptually:

Client IP
   ↓
Hash
   ↓
Server selection

This can provide a form of request affinity, although relying on IP alone has limitations.

Random

The load balancer randomly chooses an available server.

Simple, but whether it is appropriate depends on the workload and implementation.

Least Response Time

The load balancer can consider response times and direct traffic toward servers that are responding faster.

This can be useful when server load is not evenly distributed.

Consistent Hashing

Consistent hashing is useful in systems where we want related keys to map predictably to servers while minimizing how much mapping changes when servers are added or removed.

It becomes especially useful in distributed caches and partitioned systems.

Do not worry if that sounds complicated.

We will come back to it when we actually need it.

But What If a Server Is Dead?

Imagine we have:

Server 1 → healthy
Server 2 → healthy
Server 3 → crashed

We do not want the load balancer to continue sending traffic to Server 3.

This is where health checks come in.

The load balancer periodically checks whether a server is healthy.

For example:

GET /health

The server might respond:

{
  "status": "ok"
}

If the server stops responding or reports that it is not ready, the load balancer can stop sending new traffic to it.

So now:

              Load Balancer
               /         \
              ↓           ↓
         Server 1      Server 2
         healthy       healthy

         Server 3
        unhealthy
            X

This is much better than blindly trusting that every server is alive.

If you want a ready-made health check implementation you can drop into an Express, Fastify or Hono app, I built one for Blockend. It handles liveness and readiness checks out of the box:

https://blockend.noorulhassan.com/docs/02-blocks/07-health-check

Health Checks

A load balancer should not blindly send traffic to every server.

Instead, it should periodically check whether each server is healthy.

For example:

Load Balancer
       /         \
      ↓           ↓
 Server 1      Server 2
 healthy       healthy

 Server 3
unhealthy
    X

This is much better than blindly trusting that every server is alive.

If you want a ready-made health check implementation you can drop into an Express, Fastify, or Hono app, I built one for Blockend. It handles liveness and readiness checks out of the box:

https://blockend.noorulhassan.com/docs/02-blocks/07-health-check

Health Checks Are More Than "Is the Process Running?"

There is an important distinction here.

A server process can be running while the application is not actually ready to serve traffic.

For example:

Node process → running
Database     → unavailable

The process itself is alive.

But the application might not be able to perform useful work.

This is why production systems often distinguish between two different concepts:

Liveness
Readiness

Liveness

Liveness asks something like:

Is the application process alive?

Readiness

Readiness asks:

Is the application ready to receive traffic?

This distinction becomes important when deploying applications, restarting services, and handling failures.

Redundancy

We have now introduced another important idea:

Redundancy.

Instead of depending on one component:

Server

we have:

Server 1
Server 2
Server 3

If one fails, the others can continue serving traffic.

The same principle can apply to other parts of the system.

For example:

Multiple application servers
Multiple database replicas
Multiple cache nodes
Multiple availability zones

The goal is to avoid unnecessary single points of failure.

But redundancy creates another problem.

If we have three application servers, where is the user's session stored?

And if all three servers need the same data, how do they share it?

This is where our simple architecture starts becoming a distributed system.

The Architecture We Have Built

Let us stop here and look at how far we have come.

We started with:

Client → Server → Database

Then traffic increased.

We added vertical scaling.

Then we reached the limits of one machine.

We added horizontal scaling.

Then we needed something to distribute traffic.

We added a load balancer.

Then we needed to detect failed servers.

We added health checks.

Our architecture now looks roughly like:

                            ┌───────────┐
                            │ Database  │
                            └─────▲─────┘
                                  │
                                  │
                            ┌─────┴─────┐
                            │   Load    │
Client ─────────────────→ │ Balancer  │
                            └─────┬─────┘
                            ┌─────┼─────┐
                            ↓     ↓     ↓
                        Server Server Server
                           1     2     3

Notice something important.

We did not start with all of these components.

We earned each component by encountering a problem.

That is how I think system design should be learned.

The Problems We Have Not Solved Yet

Our system is better, but we are not finished.

We still have several problems.

Database Bottleneck

All servers are talking to one database.

What happens when database traffic becomes too high?

We might need:

Indexes
Query optimization
Connection pooling
Caching
Read replicas
Partitioning
Sharding

But we should not add these just because they exist.

We add them when the problem requires them.

Repeated Expensive Requests

Suppose thousands of users request the same product:

GET /products/123

Every request goes:

Server → Database

Why query the database 10,000 times if the data does not change frequently?

This is where caching becomes useful.

Background Work

Suppose creating an account also requires:

Send email
Generate report
Process image
Notify another service

Do we really want the HTTP request to wait for all of that?

Probably not.

This leads us toward:

Queues
Background workers
Asynchronous processing

Service Failures

What happens if another service is temporarily unavailable?

Do we retry?

How many times?

How quickly?

What if retries make the problem worse?

This leads us toward:

Timeouts
Retries
Backoff
Circuit breakers
Idempotency

Data Growth

What happens when our database grows from:

1 GB

to:

1 TB

Eventually we may need to think about:

Partitioning
Archiving
Replication
Sharding

And this is where system design starts becoming much deeper.

The Mental Model I Want You to Keep

If you are learning system design, do not try to memorize diagrams.

Instead, keep asking:

What is the bottleneck?

Then:

What failure can happen here?

Then:

What happens when traffic increases?

Then:

What happens when this component goes down?

Then:

What consistency does the application actually need?

Then:

What is the simplest solution that solves the problem?

That last question is important.

A system with:

Client
  ↓
Server
  ↓
PostgreSQL

is not a bad architecture simply because it does not have Kafka, Redis, Kubernetes, and 20 microservices.

If that system handles the workload reliably, it is a good system.

Good system design is not about adding more components.

It is about making the right tradeoffs for the problem you actually have.

Where We Go From Here

We started with one request:

Client
  ↓
Server
  ↓
Database

Then we introduced concepts only when we needed them:

More traffic
     ↓
Scaling
     ↓
Horizontal scaling
     ↓
Load balancer
     ↓
Health checks
     ↓
Redundancy

And this is only the beginning.

From here, we can start introducing:

Caching
   ↓
Database scaling
   ↓
Read replicas
   ↓
Queues
   ↓
Background workers
   ↓
Retries and backoff
   ↓
Idempotency
   ↓
Consistency
   ↓
Replication
   ↓
Partitioning
   ↓
Sharding
   ↓
Distributed systems

We will introduce each one when there is a real problem that requires it.

Because that is the way I wish someone had taught me system design.

Not:

"Here are 50 components. Memorize them."

But:

"Here is a system. Let's see what breaks when we push it."

That is where system design actually starts.

What You Learned

Let us take a step back.

You started with a single browser request and ended up with:

A server that processes requests
A relational database that stores structured data
Transactions that keep related operations consistent
Vertical scaling to handle more load
Horizontal scaling to handle even more
A load balancer that distributes traffic across servers
Health checks that detect failed servers
Redundancy that avoids single points of failure

And you did not just memorize these concepts.

You encountered each problem and solved it with the right tool.

That is the mental model. That is what matters.

The next time someone says "you need a load balancer," you will not just know what it is.

You will know why it exists.

And that makes all the difference.

StreetJS Is Open Source — And I’m Looking for Developers to Build It With Me

Fri, 04 Sep 2026 07:59:02 +0000

Building a backend framework alone teaches you a lot.

It also makes one thing very clear:

A framework becomes better when more developers challenge its ideas.

That’s why I’m opening the door wider for developers who want to contribute to StreetJS.

StreetJS is an open-source, TypeScript-first backend framework for Node.js.

The goal is to provide a modern foundation for building APIs and backend applications while keeping the architecture predictable, strongly typed, and extensible.

But I don't want StreetJS to be shaped by only one developer.

What can you contribute?

You don't have to be a framework expert.

There are many ways to contribute:

🐛 Find and fix bugs

Use StreetJS, break things, report what you find, and help fix them.

Real-world usage is one of the best ways to discover problems that tests don't catch.

📚 Improve documentation

Good documentation can be more valuable than another feature.

If something is confusing, difficult to discover, or poorly explained, improving it is a meaningful contribution.

🧪 Write tests

Frameworks need strong test coverage.

You can contribute:

Unit tests
Integration tests
Edge-case tests
Regression tests
Property-based tests

⚡ Improve performance

If you enjoy profiling Node.js applications, there are opportunities to investigate:

Request performance
Middleware execution
Dependency injection
Database operations
Memory usage
Startup time

🏗️ Work on the architecture

This is especially interesting to me.

StreetJS has several core subsystems, and I'm interested in developers who want to think deeply about their boundaries and long-term design.

For example:

What should StreetJS own?

What should remain replaceable?

Where should plugins begin and framework responsibilities end?

These are architectural questions—not just coding tasks.

🔌 Build plugins

One of the directions I'm particularly interested in is growing the StreetJS ecosystem.

Plugins can provide integrations without forcing every capability into the framework core.

That could include integrations for:

Payments
Storage
Messaging
Authentication
Databases
Third-party APIs
Developer tooling

You can contribute without writing code

Open source isn't only about pull requests.

You can help by:

Trying StreetJS
Opening issues
Reviewing PRs
Improving examples
Writing tutorials
Sharing feedback
Finding confusing APIs
Suggesting architectural improvements
Building projects with it

Even saying "I tried this and here's what confused me" can be extremely valuable.

What I'm looking for

I'm especially interested in developers who enjoy:

TypeScript

Node.js

Backend architecture

Open-source development

Performance

Testing

Developer tooling

But you don't need to know all of these.

If you're curious, that's enough to start.

One important thing

I'm not looking for contributors who will simply agree with every decision I've made.

I'd rather have someone say:

"I don't think this abstraction is going to scale."

and explain why.

Good open source needs disagreement.

It needs code reviews.

It needs alternative approaches.

It needs people willing to ask uncomfortable architectural questions.

That's how projects get better.

If you want to contribute

Start by exploring the project:

GitHub: https://github.com/hassanmubiru/StreetJS

Documentation: https://hassanmubiru.github.io/StreetJS/

npm: https://www.npmjs.com/package/streetjs

If you're a developer interested in TypeScript, Node.js, backend frameworks, or open source, I'd love to have you take a look.

You don't need to be an expert.

You don't need to make a huge contribution.

Pick an issue.

Try the framework.

Open a discussion.

Tell me what you think.

Let's build StreetJS as a community, not just a project.

I Enjoy Software Development Even More With AI

Thu, 20 Aug 2026 06:49:20 +0000

I keep reading that AI turns an engineer into a reviewer of generated code and removes the enjoyable part of the work. My experience with Codex is quite opposite (will try Claude one day). I have always enjoyed software engineering, and now I enjoy it even more.

Before I get to TDD and implementation, I use the agent to analyse the problem and technical domains and model the system in detail. Most of my design decisions happen there.

I can examine more alternatives and find missing decisions earlier. However, to judge the output, I have to know beforehand how and why the system should work, and how the test and code should look like. The agent makes the analysis and modelling many times faster and helps me update the entire model while the design changes.

The largest difference appears after a redesign. Updating diagrams and specifications and then reworking the code and tests used to consume a lot of time. Now it happens like 50 times faster (gut feeling). So I can redesign more freely and do not have to keep a weaker decision just because changing everything around it would take too long.

The agent also does much of the routine work of propagating my decisions through specifications, code and tests. This allows me to spend much more time analyzing the domain and designing the system.

Pure joy.

Why API Architecture Is a Business Decision, Not a Technical One

Mon, 17 Aug 2026 14:45:57 +0000

Modern software products are API-driven businesses.

Whether building SaaS platforms, ERP systems, marketplaces, mobile applications, or internal enterprise software, nearly every business operation eventually becomes an API request.

Customers place orders through APIs.

Mobile applications consume APIs.

Partners integrate through APIs.

Automation workflows execute through APIs.

Future products are built on top of APIs.

As a result, API architecture is no longer just a backend implementation detail. It becomes one of the most important strategic decisions in software systems because it directly influences growth, scalability, operational efficiency, product velocity, and long-term maintenance costs.

The quality of an organization's API architecture often determines whether the platform becomes easier to evolve over time or increasingly difficult to change.

Executive Summary

Most discussions about APIs focus on endpoints, authentication mechanisms, request formats, or framework choices.

These topics are important.

However, they miss the larger picture.

API architecture defines how business capabilities are exposed, consumed, reused, secured, and evolved.

A well-designed API architecture creates:

Faster product delivery
Easier integrations
Better customer experience
Lower operational costs
Improved scalability
Stronger security
Better developer experience
Greater long-term flexibility

A poorly designed API architecture creates:

Feature bottlenecks
Integration friction
Engineering complexity
Repeated implementations
Platform inconsistency
Expensive rewrites
Operational inefficiencies

The difference between a software product and a software platform is often the quality of its API architecture.

The Real Problem

Most systems do not fail because of insufficient features.

They fail because the architecture cannot support growth.

Consider a business platform that contains:

User Management
Products
Inventory
Orders
Billing
Reporting
Notifications
Automation

Initially, development feels simple.

A frontend sends requests.

The backend processes them.

The database stores data.

Everything appears manageable.

Over time, however, new requirements emerge:

Mobile applications
Third-party integrations
Internal automation
Public APIs
Partner portals
Analytics systems
External marketplaces

Without a structured API architecture, complexity begins growing faster than business value.

Common symptoms appear:

Business logic duplication
Inconsistent interfaces
Difficult integrations
Expensive feature development
Slower releases
Rising maintenance costs

Eventually, the platform becomes harder to maintain than the business itself.

At this point, architecture becomes a business constraint.

Understanding APIs as Business Capabilities

Many systems are designed around screens.

Mature systems are designed around capabilities.

This distinction is critical.

Screen-oriented thinking often produces APIs like:

POST /create-product
GET /get-user-orders
POST /submit-payment

Capability-oriented thinking produces APIs like:

POST /products
GET /orders
POST /payments

The difference appears small.

The impact is enormous.

Capabilities can be reused.

Screens cannot.

A capability-oriented API allows multiple consumers to share the same business logic:

                Products API
                       │
      ┌────────────────┼────────────────┐
      ▼                ▼                ▼
   Web App        Mobile App      Partner API
      ▼                ▼                ▼
      └──────────── Same Capability ───┘

Instead of rebuilding functionality repeatedly, the system exposes reusable business capabilities that can support current and future products.

This transforms APIs from implementation details into long-term business assets.

The Cost of Architectural Debt

Most technical debt is visible.

API debt often remains hidden until scale arrives.

Stage 1: Initial Product

Frontend
    │
    ▼
Backend
    │
    ▼
Database

Simple.

Fast.

Easy to understand.

Stage 2: Additional Clients

Web
 │
 ▼

Backend
 │
 ▼
Database

Mobile
 │
 ▼
Backend

API quality now matters.

Multiple consumers depend on the same contract.

Stage 3: Integrations

Web App
Mobile App
Partner Systems
Internal Tools
Automation
       │
       ▼
    Backend
       │
       ▼
    Database

Inconsistencies become expensive.

Every integration requires additional effort.

Every change introduces risk.

Stage 4: Platform Scale

Customers
Partners
Vendors
Automation
Analytics
Future Products
        │
        ▼
     Platform

The architecture now influences every business initiative.

The cost of poor decisions compounds.

The value of good decisions compounds as well.

Architecture as an Execution Layer

Every business strategy eventually becomes software behavior.

Every software behavior eventually becomes an API interaction.

This makes APIs the execution layer between business goals and technical systems.

Business Objectives
         │
         ▼
Business Capabilities
         │
         ▼
API Contracts
         │
         ▼
Application Services
         │
         ▼
Infrastructure

When API architecture is designed well, new initiatives move quickly through this chain.

When API architecture is designed poorly, every initiative becomes slower, more expensive, and riskier.

High-Level Architecture Blueprint

A scalable API platform should separate concerns into distinct layers.

                    ┌─────────────────────┐
                    │    Client Layer     │
                    │ Web, Mobile, APIs   │
                    └──────────┬──────────┘
                               │
                               ▼
                    ┌─────────────────────┐
                    │     API Gateway     │
                    └──────────┬──────────┘
                               │
      ┌────────────────────────┼────────────────────────┐
      ▼                        ▼                        ▼

┌──────────────┐      ┌──────────────┐      ┌──────────────┐
│Authentication│      │ Authorization│      │ Rate Limiting│
└──────┬───────┘      └──────┬───────┘      └──────┬───────┘
       └─────────────────────┼─────────────────────┘
                             ▼

                 ┌─────────────────────────┐
                 │      Service Layer      │
                 │ Business Capabilities   │
                 └───────────┬─────────────┘
                             │
       ┌─────────────────────┼─────────────────────┐
       ▼                     ▼                     ▼

 ┌────────────┐      ┌────────────┐      ┌────────────┐
 │ PostgreSQL │      │ Redis      │      │ Message Bus│
 │ Source     │      │ Cache      │      │ Async Work │
 │ of Truth   │      │ Layer      │      │ Processing │
 └────────────┘      └────────────┘      └────────────┘

Each layer exists for a specific reason.

Authentication establishes identity.

Authorization enforces business rules.

Rate limiting protects resources.

Services execute business capabilities.

Databases persist information.

Caches improve performance.

Message queues enable scale.

The architecture remains understandable because responsibilities remain separated.

The Request Lifecycle

Every request should follow a predictable journey.

Client Request
      │
      ▼
API Gateway
      │
      ▼
Authentication
      │
      ▼
Authorization
      │
      ▼
Input Validation
      │
      ▼
Business Logic
      │
      ▼
Data Persistence
      │
      ▼
Response

A consistent lifecycle improves:

Security
Testing
Debugging
Monitoring
Maintainability
Reliability

Predictability becomes increasingly valuable as systems grow.

The Four Architectural Decisions That Shape Growth

1. API-First vs Interface-First

Many systems are designed around screens.

Scalable systems are designed around capabilities.

Interface-First

Screen
   │
   ▼
Backend Logic

API-First

Business Capability
         │
         ▼
         API
         │
 ┌───────┼────────┐
 ▼       ▼        ▼

Web   Mobile   Partner

The API-first approach creates reusable building blocks rather than isolated implementations.

2. Synchronous vs Event-Driven Processing

Not every operation should complete during a request.

Synchronous

Request
   │
   ▼
Process
   │
   ▼
Response

Simple.

But limited.

Event-Driven

Request
   │
   ▼
Queue Message
   │
   ▼
Worker
   │
   ▼
Background Processing

Ideal for:

Notifications
Emails
Reporting
Exports
Integrations
Audit Logs

This approach improves responsiveness while supporting growth.

3. Modular Monolith vs Distributed Systems

One of the most common mistakes is introducing complexity too early.

A well-structured modular monolith often provides:

Faster development
Easier debugging
Lower operational costs
Simpler deployments
Better maintainability

Application
│
├── Users Module
├── Orders Module
├── Inventory Module
├── Billing Module
└── Reporting Module

Strong internal boundaries usually provide more value than premature distribution.

4. APIs as Products

Many systems expose APIs.

Few treat them as products.

A product-quality API requires:

Consistency
Documentation
Reliability
Versioning
Security
Developer Experience

The API itself becomes part of the customer experience.

Versioning Strategy

Change is inevitable.

Breaking consumers is optional.

A versioning strategy allows systems to evolve safely.

/api/v1/products

/api/v2/products

Benefits include:

Backward compatibility
Safer deployments
Controlled migrations
Reduced disruption

Versioning is not merely a technical mechanism.

It is an operational stability strategy.

Authentication and Authorization

Authentication answers:

Who are you?

Authorization answers:

What are you allowed to do?

These concerns should remain separate.

A common authorization model is RBAC:

User
 │
 ▼
Role
 │
 ▼
Permissions

Example permissions:

invoice.create
invoice.view
inventory.adjust
user.invite

This structure scales naturally as systems become more complex.

Scalability Beyond Infrastructure

Scalability is often misunderstood.

Adding servers is infrastructure scaling.

Reducing architectural friction is system scaling.

Important scalability mechanisms include:

Pagination
Filtering
Search
Caching
Background processing
Rate limiting

For example:

GET /orders?page=1&page_size=50

is significantly more scalable than returning hundreds of thousands of records in a single response.

Architecture determines efficiency long before infrastructure becomes a problem.

Observability as a Platform Requirement

Visibility is not optional.

Without observability, systems become difficult to understand and improve.

Three pillars are essential:

Logs
Metrics
Traces

Together they answer:

What happened?
How often?
Where did it happen?
Why did it happen?

An observable platform is easier to operate, maintain, and evolve.

Common Architectural Mistakes

Breaking Existing Contracts

Changing APIs without versioning creates instability.

Business Logic Inside Controllers

Bad:

Controller = Business Logic

Better:

Controller
    │
    ▼
Service Layer
    │
    ▼
Repository

Returning Excessive Data

Large responses increase:

Memory consumption
Query costs
Response times

Use pagination.

Missing Rate Limiting

Without protection, systems become vulnerable to:

Abuse
Traffic spikes
Resource exhaustion
Cost increases

Rate limiting should be considered a foundational capability.

Architecture Evolution Roadmap

Most successful platforms evolve gradually.

Simple CRUD APIs
         │
         ▼
Authentication
         │
         ▼
Authorization
         │
         ▼
Caching
         │
         ▼
Background Jobs
         │
         ▼
API Gateway
         │
         ▼
Event-Driven Architecture

The goal is not maximum complexity.

The goal is appropriate complexity.

Systems should evolve in response to real requirements rather than assumptions.

Conclusion

API architecture is not about endpoints.

It is about designing how capabilities are exposed, consumed, secured, reused, and evolved.

A well-designed API architecture creates leverage.

It allows multiple products, integrations, automations, and future initiatives to operate on a shared foundation.

The result is:

Faster development
Better scalability
Easier integrations
Lower operational costs
Stronger security
Improved maintainability
Better developer experience

The most valuable APIs are not simply interfaces.

They are the contracts that define how an entire software platform operates, grows, and adapts over time.

That is why API architecture is not merely a technical concern.

It is a fundamental platform design decision with long-term business consequences.

Production Health Checks & Graceful Shutdown - Complete Guide

Fri, 07 Aug 2026 03:32:25 +0000

Production Health Checks & Graceful Shutdown

Most backend tutorials teach you how to start a server.

Very few teach you how a server should stop.

That might sound strange at first.

If your application starts successfully, serves requests, and all your tests pass, isn't that enough?

Not in production.

Production has a completely different set of problems.

Your application isn't running on your laptop anymore. It's behind load balancers, inside containers, managed by Kubernetes or another orchestrator. Servers are restarted, deployments happen every day, machines fail, and containers are replaced all the time.

Eventually, your application will receive a SIGTERM.

The question isn't if it happens.

The question is what your application does next.

SIGTERM
   │
   ▼
❓ Exit immediately?
❓ Finish active requests?
❓ Accept new traffic?
❓ Close database connections?
❓ Flush logs?

These aren't business logic problems.

They're production problems.

Two concepts solve most of them:

Concept	Responsibility
❤️ Health Checks	Tell infrastructure whether the application should receive traffic
🛑 Graceful Shutdown	Stop the application safely without interrupting users

Although they solve different problems, they're designed to work together.

Why should you care?

Imagine you're deploying a new version of your API.

Kubernetes starts a new pod and sends a SIGTERM to the old one.

At the exact same moment, a customer submits an order.

If your application exits immediately:

❌ The request never finishes.

❌ The client receives an error.

❌ Maybe payment succeeded.

❌ Maybe the database transaction didn't.

❌ Maybe logs never got flushed.

Nothing was wrong with your business logic.

Your application simply wasn't prepared to stop.

Production reliability isn't just about writing correct code.

It's also about handling your application's lifecycle correctly.

What exactly is a health check?

Before Kubernetes or a load balancer sends traffic to your application, it needs to answer one simple question:

"Can I trust this instance to handle another request?"

It can't inspect your code.

It doesn't know whether PostgreSQL is available.

It doesn't know whether Redis is responding.

It can't see whether your application is shutting down.

The only thing it can do is ask your application.

That's exactly what a Health Check is.

Load Balancer
      │
      ▼
 GET /health/ready
      │
      ▼
Application
      │
      ▼
"Yes, I'm ready." ✅

or

"Don't send me traffic." ❌

Health checks are simply your application describing its own state over HTTP.

Why isn't one endpoint enough?

One of the biggest misconceptions is exposing a single /health endpoint.

Different systems ask different questions.

Consumer	Question
Kubernetes	Is the process alive?
Load Balancer	Should this instance receive traffic?
Monitoring	What's wrong?
Developers	Which dependency failed?

Trying to answer all of those questions with one endpoint usually causes unnecessary restarts or incorrect routing decisions.

That's why production applications expose multiple endpoints.

`/health/live`

Is the process alive?

That's the only responsibility of this endpoint.

Don't:

Connect to PostgreSQL
Ping Redis
Check message queues
Call external APIs

If Node.js is running:

HTTP/1.1 200 OK

That's enough.

Why?

Imagine Redis has a temporary outage.

If your liveness endpoint depends on Redis, Kubernetes may restart perfectly healthy pods because of a dependency that would've recovered on its own.

Liveness should answer only one question:

Is the process alive?

`/health/ready`

Should this instance receive new traffic?

This endpoint is much more important.

Return 503 whenever:

The application is shutting down
A critical dependency has failed
Startup hasn't completed yet

Readiness = 200
       │
       ▼
Load Balancer
continues sending traffic

──────────────

Readiness = 503
       │
       ▼
Remove instance
from service discovery

Notice something important.

The process doesn't stop.

It simply stops receiving new work.

That's exactly what you want during deployments.

`/health`

Unlike the previous endpoints, this one is for humans and monitoring systems.

Instead of simply saying healthy or unhealthy, it provides a complete report.

For example:

{
  "status": "degraded",
  "checks": {
    "database": "healthy",
    "redis": "unhealthy"
  },
  "uptime": 12543,
  "timestamp": "2026-08-07T12:00:00Z"
}

Perfect for:

Dashboards
Monitoring
Debugging
Incident response

Not every dependency should be treated the same

This is where many applications get it wrong.

Imagine Redis is only used for caching.

If Redis becomes unavailable:

✅ Users can still log in
✅ Orders still work
✅ Database is healthy

The application is slower.

It isn't broken.

That's a Degraded state.

Now imagine PostgreSQL goes down.

Nothing works anymore.

That's Unhealthy.

Visualizing it makes the difference obvious.

Dependency	Critical?	Status
PostgreSQL	✅ Yes	🔴 Unhealthy
Redis Cache	❌ No	🟡 Degraded
External Analytics	❌ No	🟡 Degraded

Health checks shouldn't simply answer:

Working?

Broken?

They should accurately describe reality.

What is Graceful Shutdown?

Every server eventually stops.

Deployments
Scaling down
Node maintenance
Container replacement
Machine failure

Shutdown isn't optional.

The only question is how your application behaves when it happens.

Without graceful shutdown:

SIGTERM
   │
   ▼
Process exits ❌

With graceful shutdown:

SIGTERM
   │
   ▼
Stop new traffic
   │
   ▼
Finish active requests
   │
   ▼
Close resources
   │
   ▼
Exit

What should happen during shutdown?

A production shutdown generally looks like this.

SIGTERM
    │
    ▼
Application starts shutdown
    │
    ▼
Readiness → 503
    │
    ▼
Load balancer removes instance
    │
    ▼
Existing requests finish
    │
    ▼
Cleanup resources
    │
    ▼
Exit

Every step exists for a reason.

Skipping any of them makes deployments less reliable.

Why request draining matters

Imagine one request takes five seconds.

Kubernetes begins shutting down the pod after two seconds.

Without request draining:

Request Started
       │
       ▼
SIGTERM
       │
       ▼
Process exits ❌
       │
       ▼
Client receives error

With request draining:

Request Started
       │
       ▼
SIGTERM
       │
       ▼
Wait...
       │
       ▼
Request finishes ✅
       │
       ▼
Shutdown continues

Users never notice the deployment.

That's the goal.

Why shutdown order matters

Resources depend on each other.

Closing them in the wrong order creates failures.

A common production sequence looks like this:

1. Stop HTTP traffic
        │
        ▼
2. Stop queue workers
        │
        ▼
3. Finish active requests
        │
        ▼
4. Close database pools
        │
        ▼
5. Close Redis
        │
        ▼
6. Flush logs
        │
        ▼
7. Exit

Order matters.

Imagine closing your database before active requests finish.

Every request immediately starts failing.

Why Health Checks and Graceful Shutdown belong together

These concepts solve two halves of the same problem.

Health Checks without Graceful Shutdown

Healthy ✅
      │
Deployment
      │
Process exits
      │
Interrupted requests ❌

Graceful Shutdown without Readiness

Shutdown starts
       │
Readiness still 200
       │
Load Balancer
keeps sending traffic
       │
New requests fail ❌

One endpoint for everything

Using a single /health endpoint for:

Liveness
Readiness
Monitoring

usually causes:

Unnecessary restarts
Incorrect routing
Poor observability

Each endpoint exists for a different reason.

Keep them separate.

Why I built these Blockend blocks

After building enough backend projects, I realized I kept writing the same production infrastructure over and over again.

Every project needed:

Health endpoints
Graceful shutdown
Request draining
Cleanup logic
Signal handling

None of it was unique to the application.

It was production plumbing.

That's why I built these Blockend blocks.

Not to create another framework.

Not to hide how things work.

Simply to package production patterns I kept rewriting into reusable building blocks that can be copied into any Node.js project.

What does the complete production lifecycle look like?

                Deployment
                     │
                     ▼
                 SIGTERM
                     │
                     ▼
        Application enters shutdown
                     │
                     ▼
          Readiness returns 503
                     │
                     ▼
     Load Balancer stops new traffic
                     │
                     ▼
       Existing requests complete
                     │
                     ▼
       Cleanup tasks execute safely
                     │
                     ▼
          Process exits cleanly

No interrupted requests.

No leaked connections.

No partially completed cleanup.

Just a predictable, reliable deployment.

Final thoughts

Health checks aren't just monitoring endpoints.

Graceful shutdown isn't just listening for SIGTERM.

Together, they define how your application behaves in production.

Most tutorials stop after teaching you how to start a server.

Production systems also need to know how to stop.

If you're building backend services, these patterns aren't optional.

They're part of building reliable software.

Continue Reading

If you'd like to implement everything covered in this guide—including framework adapters, configuration, and production-ready examples—you can follow the complete Blockend guide:

📖 Production Health Checks & Graceful Shutdown Guide

https://blockend.noorulhassan.com/docs/03-guides/01-production-health-checks-and-graceful-shutdown

🚀 One of the most interesting things I discovered about Node.js is its asynchronous nature.

Wed, 22 Jul 2026 08:21:47 +0000

"How can a single-threaded runtime handle so many requests efficiently?"

The answer lies in asynchronous programming.

Imagine you're at a library. You ask the librarian to find a book that's stored in the archive.
Instead of standing at the desk doing nothing until the book is found, you walk around the library, read another book, or continue your research. When the librarian finds your book, they call you back to collect it.

That's similar to how Node.js works.
When it needs to perform a task that might take some time—like reading a file, querying a database, or calling an external API—it doesn't sit idle waiting for the task to finish. Instead, it continues handling other work. Once the operation is complete, Node.js processes the result.
This non-blocking approach is one of the reasons Node.js is well suited for applications that serve many users at the same time, such as APIs, chat applications, and real-time services.

💡 My takeaway: Understanding the asynchronous nature of Node.js helped me realize that efficient software isn't just about getting the right result—it's also about making the best use of time while waiting for long-running tasks to complete.
I'd love to hear from you: Which Node.js concept has really stood out to you?

Go Slice Initialization Gotchas: The Hole-Creating Bug I Keep Seeing (And a Linter That Catches It)

Tue, 21 Jul 2026 16:15:52 +0000

Go slices are simple on the surface. Under the hood, the interaction between length, capacity, and append has a specific failure mode I've been bitten by more than once — and seen in enough code reviews to write about. The bug is invisible at compile time, often invisible in tests, and can corrupt data silently.

The three ways to initialize a slice

Before the bug, the mechanics. make for slices takes up to three arguments:

make([]T, length)
make([]T, length, capacity)

These behave very differently:

// Length 0, capacity 100
// Content: []
// append() fills from index 0
a := make([]int, 0, 100)

// Length 5, capacity 5
// Content: [0 0 0 0 0]
// Indexed assignment fills existing slots
b := make([]int, 5)

// Length 5, capacity 10
// Content: [0 0 0 0 0]
// Indexed assignment fills existing slots 0–4
// append() fills from index 5 onward
c := make([]int, 5, 10)

The key rule: append always starts after the last element, at index len(slice). It does not fill from the beginning. It does not know or care what's in the existing slots.

s := make([]int, 5, 10)
fmt.Println(s)       // [0 0 0 0 0]
fmt.Println(len(s))  // 5
fmt.Println(cap(s))  // 10

s = append(s, 42)
fmt.Println(s)       // [0 0 0 0 0 42]  ← 42 is at index 5, not 0

This is correct, expected behavior. The problem arises when you mix the two initialization patterns during a refactor.

The bug: optimization that creates holes

Here's the sequence of events. You have code that initializes a slice with a fixed length and fills it by index:

// Version 1: works correctly
results := make([]int, 5)

for i, job := range jobs {
    results[i] = process(job)
}
// results: [r0 r1 r2 r3 r4]

This is fine. Five jobs, five slots, indexed assignment, everything lines up.

Later, someone (possibly future-you) decides to optimize memory allocation by pre-allocating capacity for growth:

// Version 2: "optimization"
results := make([]int, 5, 10)  // added capacity

The loop still works. Indexed assignment into results[i] fills positions 0–4. No change in behavior yet.

Now, a separate change: the indexed loop gets refactored. Maybe the loop structure changes, maybe someone decides append is more idiomatic, maybe the jobs slice changes to a channel. The replacement looks like this:

// Version 3: refactored loop using append
results := make([]int, 5, 10)

for _, job := range jobs {
    results = append(results, process(job))
}

This compiles. This runs. And this produces:

// results: [0 0 0 0 0 r0 r1 r2 r3 r4]

Five zero-value holes at the front, followed by the actual results. The slice now has length 10 instead of 5. Any code downstream that iterates results processes 10 elements instead of 5, with the first 5 being garbage zeros.

If the downstream code is a JSON response, you send extra zeros to clients. If it's an aggregation, your totals are wrong. If it's a database write, you insert rows you didn't intend to.

Why this is hard to catch

No compile error. Both indexed assignment and append are valid operations on a []int. The compiler has nothing to complain about.

Tests often miss it. If your test asserts len(results) == 5, it fails — but if it only checks specific indices or iterates the whole slice, the zeros look like valid data. A test against a jobs list where process legitimately returns 0 for some inputs will pass completely.

The change looks like a safe refactor. Changing results[i] = x to results = append(results, x) seems equivalent to anyone who hasn't internalized how append interacts with a non-zero initial length. The diff is small and the intent looks the same.

The capacity change seems unrelated. The make([]int, 5) to make([]int, 5, 10) change is in a different commit, possibly by a different person. The connection between that change and the later append refactor isn't obvious.

The three clean patterns — pick one and be consistent

The root cause is mixing patterns that don't compose. Each of these is correct on its own:

Pattern 1: Length 0, append only

// Correct: length 0, append fills from the start
results := make([]int, 0, len(jobs))

for _, job := range jobs {
    results = append(results, process(job))
}
// results: [r0 r1 r2 r3 r4]

Use this when you're building a slice incrementally and don't need indexed access during construction. The capacity hint len(jobs) avoids reallocations without creating holes.

Pattern 2: Length N, indexed assignment only

// Correct: length N, indexed assignment fills all slots
results := make([]int, len(jobs))

for i, job := range jobs {
    results[i] = process(job)
}
// results: [r0 r1 r2 r3 r4]

Use this when you're processing a collection with known size and need random-access writes (out-of-order filling, parallel writes with sync, etc.).

Pattern 3: Length N with capacity, indexed assignment only

// Correct: length N, capacity M, indexed assignment for first N slots
// append for subsequent elements
results := make([]int, len(jobs), len(jobs)*2)

for i, job := range jobs {
    results[i] = process(job)
}

// Later appends go to indices N onward — intentional
results = append(results, extraResults...)

Use this only when you genuinely need both: a pre-filled region AND growth capacity. This is the rarest case and the one that most needs a comment explaining the intent.

The linter: makezero

The makezero linter catches exactly this class of mistake. It flags append calls on slices that were initialized with non-zero length, giving you a chance to examine whether the initialization and the append are both intentional.

go install github.com/ashanbrown/makezero@latest
makezero ./...

Or via golangci-lint (recommended — runs alongside your other linters):

# .golangci.yml
linters:
  enable:
    - makezero

What it flags:

results := make([]int, 5, 10)
results = append(results, 42)
// makezero: append to slice results initialized with non-zero length at line N

What it does NOT flag (both are intentional patterns):

// Pattern 1: length 0, append is fine
a := make([]int, 0, 10)
a = append(a, 42)  // ✅ no warning

// Pattern 2: length N, no append
b := make([]int, 5)
b[0] = 42  // ✅ no warning

The linter doesn't tell you your code is wrong — it tells you that you have a combination that might be wrong and deserves a look. That's the right level of signal. The version 3 code from the bug scenario above would have triggered this warning, prompting a review of the initialization before the append refactor shipped.

Adding an explicit comment when you genuinely mix patterns

If you have a legitimate reason to initialize with non-zero length and later append, say so:

// Pre-fill the first slot with a sentinel value.
// Actual results are appended starting at index 1.
results := make([]Result, 1, len(jobs)+1)
results[0] = Result{sentinel: true}

for _, job := range jobs {
    results = append(results, process(job)) //nolint:makezero // intentional: sentinel at index 0
}

A comment here, paired with a linter suppression (//nolint:makezero), tells reviewers and future maintainers that the mixed pattern is deliberate. Without it, the next person to read the code — or the next refactor — will make the same assumption the bug relied on.

Quick mental checklist for slice initialization

Before writing make([]T, n, m) ask:

Will I use indexed assignment or append? Pick one. If indexed, length = final size. If append, length = 0.
Do I need pre-filled slots AND append? Only then use make([]T, n, m) with n > 0. Add a comment.
Is the capacity hint right? make([]T, 0, len(input)) is almost always the right pattern for transforming one slice into another.

The simplest version of this rule: if you're using append, start with length 0.

Summary

make([]T, 0, cap) — Use with: append only. Risk if mixed: none — this is the safe default.

make([]T, n) — Use with: indexed assignment only. Risk if mixed: none if consistent.

make([]T, n, m) — Use with: indexed assignment for the first n, append after. Risk if mixed: high — easy to introduce holes during a refactor.

The bug isn't exotic. It's the interaction of two individually correct operations that become incorrect when combined without awareness of how append uses len, not cap, to decide where to write. The makezero linter closes the gap that the compiler leaves open.

This is part of a series on Go patterns and production gotchas. Previous posts cover concurrency mistakes, typed constants, context propagation, and profiling.

Profiling a Go Service in Production: pprof in 10 Minutes

Mon, 20 Jul 2026 19:45:14 +0000

Go ships with a profiler built in. No third-party tools, no agents to install, no instrumentation to add upfront. If your service imports net/http/pprof, you can profile it live in production right now — CPU usage, memory allocations, goroutine counts, blocking operations. The data is available over HTTP, readable with standard Go tooling.

Most engineers know pprof exists. Fewer have actually used it under pressure, on a real service, to find a real problem. This post walks through the mechanics — how to enable it, how to collect a profile, how to read a flame graph — and then shows the three classes of problems it catches most often.

Enabling pprof

For an HTTP service, one import is all it takes:

import (
    "log"
    "net/http"

    _ "net/http/pprof" // registers handlers on DefaultServeMux
)

func main() {
    // Your service setup...

    // pprof endpoints are now available on DefaultServeMux
    go func() {
        log.Println(http.ListenAndServe("localhost:6060", nil))
    }()

    // ... rest of main
}

The blank import registers the pprof HTTP handlers automatically. They’re available at:

GET /debug/pprof/ — index of available profiles
GET /debug/pprof/goroutine — all current goroutines
GET /debug/pprof/heap — memory allocations
GET /debug/pprof/profile?seconds=30 — CPU profile (30-second sample)
GET /debug/pprof/block — goroutine blocking events
GET /debug/pprof/mutex — mutex contention

Security note: bind pprof to localhost only, or a private interface. Never expose it on your public port. If your service runs in Kubernetes, use kubectl port-forward to reach it.

For services not using DefaultServeMux, register the handlers explicitly:

mux := http.NewServeMux()
mux.HandleFunc("/debug/pprof/", pprof.Index)
mux.HandleFunc("/debug/pprof/cmdline", pprof.Cmdline)
mux.HandleFunc("/debug/pprof/profile", pprof.Profile)
mux.HandleFunc("/debug/pprof/symbol", pprof.Symbol)
mux.HandleFunc("/debug/pprof/trace", pprof.Trace)

Collecting a CPU profile

With the service running under load (profiling a quiet service tells you nothing useful), collect a 30-second CPU sample:

go tool pprof http://localhost:6060/debug/pprof/profile?seconds=30

This downloads the profile and drops you into an interactive shell. Or skip the shell and go straight to a flame graph:

# Collect the profile to a file
curl -o cpu.prof "http://localhost:6060/debug/pprof/profile?seconds=30"

# Open the flame graph in a browser
go tool pprof -http=:8080 cpu.prof

The -http flag serves a web UI with multiple views. The flame graph is the most useful starting point.

Reading a flame graph

A flame graph shows where your program spends time. Each horizontal bar is a function. Width represents time — wider means more CPU. The vertical axis is the call stack — functions lower down called the ones above them.

|          processJob          |     |  encode  |
|       fetchFromDB      |other|
|         sql.Query            |
|          net.Read            |

You’re looking for wide bars near the top — those are the functions consuming the most CPU. If json.Marshal is unexpectedly wide, you’re spending a lot of time serializing. If runtime.mallocgc is wide, you’re allocating heavily. If sync.Mutex.Lock is wide, you have lock contention.

The three most common findings in worker pool services:

Problem 1: Excessive allocations

Symptom: runtime.mallocgc is prominent in the CPU profile. Heap profile shows high allocation rate with short-lived objects.

# Collect a heap profile
curl -o heap.prof "http://localhost:6060/debug/pprof/heap"
go tool pprof -http=:8080 heap.prof

In the heap profile, switch to “alloc_objects” or “alloc_space” view (not “inuse” — that shows what’s live, not what’s being allocated). Look for hot allocation sites.

Common cause in worker pools: allocating a new buffer or slice on every job iteration.

// ❌ Allocates a new byte slice for every job
func processJob(ctx context.Context, job Job) error {
    buf := make([]byte, 4096)
    // use buf...
    return nil
}

// ✅ Reuse buffers with sync.Pool
var bufPool = sync.Pool{
    New: func() interface{} {
        b := make([]byte, 4096)
        return &b
    },
}

func processJob(ctx context.Context, job Job) error {
    bufPtr := bufPool.Get().(*[]byte)
    defer bufPool.Put(bufPtr)
    buf := *bufPtr
    // use buf...
    return nil
}

sync.Pool maintains a pool of reusable objects. Objects not retrieved before the next GC cycle are collected, so it doesn’t prevent garbage collection — it reduces the rate of allocation, which reduces GC pressure.

Problem 2: Goroutine leak

Symptom: goroutine count grows over time and never decreases.

# Check current goroutine count
curl "http://localhost:6060/debug/pprof/goroutine?debug=1"

The debug=1 parameter returns a text listing of all goroutines with stack traces. Look for hundreds or thousands of goroutines all blocked at the same location — that’s your leak.

goroutine 1042 [chan receive, 3 minutes]:
main.worker()
    /app/worker.go:45 +0x68
created by main.startWorkers
    /app/pool.go:23 +0x4c

goroutine 1043 [chan receive, 3 minutes]:
...
(997 more like this)

A thousand goroutines blocked on chan receive for 3 minutes means the jobs channel was never closed. The fix is covered in the concurrency mistakes post in this series — the producer must own close(jobs) via defer.

You can also check the goroutine count programmatically:

// In your health check endpoint
func healthHandler(w http.ResponseWriter, r *http.Request) {
    fmt.Fprintf(w, "goroutines: %d\n", runtime.NumGoroutine())
}

If that number grows continuously over hours of runtime and never stabilizes, you have a leak.

Problem 3: Lock contention

Symptom: throughput is lower than CPU utilization suggests it should be. Workers are running but not making progress.

Mutex profiling is off by default because it has overhead. Enable it from inside the process only while investigating contention:

import "runtime"

runtime.SetMutexProfileFraction(1)
defer runtime.SetMutexProfileFraction(0)

Then collect the mutex profile:

curl -o mutex.prof "http://localhost:6060/debug/pprof/mutex"
go tool pprof -http=:8080 mutex.prof

Common cause in worker pools: a shared results map protected by a single mutex, written to by all workers simultaneously.

// ❌ All workers contend on one mutex
type ResultCollector struct {
    mu      sync.Mutex
    results map[string]Result
}

func (c *ResultCollector) Add(id string, result Result) {
    c.mu.Lock()
    c.results[id] = result
    c.mu.Unlock()
}

Solutions, in order of preference:

// ✅ Option 1: Use a results channel instead of shared state
results := make(chan Result, 1000)

// Workers send to channel (no lock)
go func() {
    for job := range jobs {
        result := processJob(ctx, job)
        results <- result
    }
}()

// Single collector goroutine reads from channel (no contention)
go func() {
    for result := range results {
        collected[result.ID] = result
    }
}()

// ✅ Option 2: sync.Map for concurrent read-heavy workloads
var results sync.Map
results.Store(id, result)
val, ok := results.Load(id)

The channel approach is idiomatic Go — share memory by communicating rather than communicating by sharing memory. The sync.Map approach is better when you need random access to results across goroutines.

A quick reference for which profile to use

High CPU, unclear where — CPU profile: pprof .../profile?seconds=30.

Memory growing over time — heap profile (inuse_space): pprof .../heap.

High GC pressure — heap profile (alloc_objects): pprof .../heap.

Goroutine count growing — goroutine profile: pprof .../goroutine?debug=1.

Low throughput despite low CPU — mutex or block profile: pprof .../mutex or .../block. Enable mutex profiling with runtime.SetMutexProfileFraction; enable block profiling with runtime.SetBlockProfileRate.

One workflow for production incidents

When something is wrong and you need to understand it quickly:

# 1. Check goroutine count (cheap, immediate)
curl localhost:6060/debug/pprof/goroutine?debug=2 | head -50

# 2. If CPU is high, get a 30s CPU profile
curl -o cpu.prof localhost:6060/debug/pprof/profile?seconds=30

# 3. If memory is growing, get a heap snapshot
curl -o heap.prof localhost:6060/debug/pprof/heap

# 4. Open whichever profile is relevant
go tool pprof -http=:8080 cpu.prof

The whole process takes under 5 minutes. You don’t need to reproduce the problem locally, you don’t need to redeploy with extra instrumentation, and you don’t need to restart the service. The profiler runs against the live process.

Summary

pprof is built into Go’s standard library and available with a single import. The three problems it catches most reliably in worker pool services are excessive allocations (fix: sync.Pool), goroutine leaks (fix: close channels, use context), and mutex contention (fix: channel-based result collection or sync.Map). The flame graph view in go tool pprof -http is the fastest path from profile to finding.

The only prerequisite is running the service under realistic load when you collect the profile. A profile from a quiet service tells you where your code could spend time, not where it does.

Previous in this series: context.Context Is Not Optional
Next: Container-aware resource management in Go.

context.Context Is Not Optional: A Practical Guide to Cancellation in Go Services

Mon, 20 Jul 2026 16:58:42 +0000

Every Go service that does I/O — database calls, HTTP requests, queue polling, file reads — should be passing a context.Context through every layer. In practice, a large number of codebases treat context.Context as optional plumbing that gets added later, or as something only the HTTP handler layer needs to worry about.

That’s the mistake. By the time you realize you need cancellation deep in a worker pool, retrofitting it is painful. This post shows what context propagation looks like in a real worker pool, what the failure modes are when you skip it, and how to do it correctly from the start.

What context.Context actually does

A context.Context carries three things:

A cancellation signal — a channel that’s closed when the work should stop
A deadline or timeout — an absolute time after which the context is automatically cancelled
Key-value pairs — request-scoped values like trace IDs (use sparingly)

When a context is cancelled, any blocking operation that’s watching that context should stop and return an error. The context propagates down the call stack — a parent context cancelled cancels all children. It never propagates up.

The critical point: the context has to be passed all the way down to the code that actually blocks. Holding it at the HTTP handler layer and never passing it to your database call means cancellation never reaches the work.

What happens without context

Here is a worker pool that ignores context:

func runWorkers(jobs <-chan Job) {
    var wg sync.WaitGroup
    for i := 0; i < 10; i++ {
        wg.Add(1)
        go func() {
            defer wg.Done()
            for job := range jobs {
                processJob(job) // no context
            }
        }()
    }
    wg.Wait()
}

func processJob(job Job) error {
    result, err := fetchFromDatabase(job.ID) // no context
    if err != nil {
        return err
    }
    return sendToDownstream(result) // no context
}

Now consider what happens when:

The calling service shuts down
A client disconnects
A deadline passes
An operator sends SIGTERM

The workers keep running. fetchFromDatabase keeps waiting. sendToDownstream keeps trying. The process can’t exit cleanly because goroutines are blocked on operations that have no way of knowing they should stop.

In the best case, the process takes 30 seconds to shut down waiting for goroutines to unblock naturally. In the worst case, it never terminates and has to be killed, potentially leaving in-flight work in an inconsistent state.

The correct structure: context flows in, errors flow out

The pattern is simple. Every function that does I/O or blocks takes a context as its first parameter:

func runWorkers(ctx context.Context, jobs <-chan Job) error {
    var wg sync.WaitGroup
    errs := make(chan error, 10)

    for i := 0; i < 10; i++ {
        wg.Add(1)
        go func() {
            defer wg.Done()
            for {
                select {
                case <-ctx.Done():
                    return // context cancelled, stop cleanly
                case job, ok := <-jobs:
                    if !ok {
                        return // channel closed, no more jobs
                    }
                    if err := processJob(ctx, job); err != nil {
                        select {
                        case errs <- err:
                        default:
                        }
                    }
                }
            }
        }()
    }

    wg.Wait()
    close(errs)

    // Return first error if any
    return <-errs
}

func processJob(ctx context.Context, job Job) error {
    result, err := fetchFromDatabase(ctx, job.ID)
    if err != nil {
        return err
    }
    return sendToDownstream(ctx, result)
}

Now when the context is cancelled — whether by timeout, operator signal, or parent cancellation — each worker’s select unblocks on ctx.Done() and returns cleanly. fetchFromDatabase and sendToDownstream propagate the context to their underlying I/O calls (database drivers, HTTP clients), which respect it.

Wiring context to OS signals for clean shutdown

The context that flows into your worker pool should ultimately be rooted at main, wired to OS signals:

func main() {
    ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
    defer stop()

    jobs := make(chan Job, 100)

    // Producer
    go func() {
        defer close(jobs)
        if err := loadJobs(ctx, jobs); err != nil {
            log.Printf("producer stopped: %v", err)
        }
    }()

    // Workers
    if err := runWorkers(ctx, jobs); err != nil {
        log.Printf("workers stopped with error: %v", err)
        os.Exit(1)
    }

    log.Println("shutdown complete")
}

signal.NotifyContext (added in Go 1.16) returns a context that’s cancelled when SIGINT or SIGTERM arrives. That cancellation propagates through loadJobs and runWorkers automatically. When the signal arrives, the producer stops generating new jobs, in-flight I/O observes cancellation through the propagated context, workers exit, and the process shuts down cleanly.

Timeouts: per-job vs. pool-wide

There are two distinct timeout requirements in a worker pool, and they need separate contexts:

Pool-wide timeout: how long the entire batch is allowed to run. Use context.WithTimeout or context.WithDeadline on the root context.

Per-job timeout: how long a single job is allowed to take. Use a derived context inside the worker loop.

func runWorkers(ctx context.Context, jobs <-chan Job) error {
    // ctx already carries the pool-wide deadline from the caller

    var wg sync.WaitGroup
    for i := 0; i < 10; i++ {
        wg.Add(1)
        go func() {
            defer wg.Done()
            for {
                select {
                case <-ctx.Done():
                    return
                case job, ok := <-jobs:
                    if !ok {
                        return
                    }

                    // Per-job timeout: 30 seconds max per job
                    jobCtx, cancel := context.WithTimeout(ctx, 30*time.Second)
                    err := processJob(jobCtx, job)
                    cancel() // always call cancel to release resources

                    if err != nil {
                        log.Printf("job %s failed: %v", job.ID, err)
                    }
                }
            }
        }()
    }

    wg.Wait()
    return ctx.Err()
}

Two things to note. First, jobCtx is derived from ctx — if the pool-wide context is cancelled, the per-job context is also cancelled automatically. The hierarchy works in your favor. Second, cancel() is called immediately after processJob returns, not deferred. Deferring a context cancel inside a loop means you accumulate uncancelled contexts until the goroutine exits. Call cancel immediately after the work is done.

Checking context in CPU-bound work

The ctx.Done() pattern works for I/O that blocks, because the underlying library propagates the context. But what about CPU-bound work — a job that does heavy computation without any I/O?

func processJob(ctx context.Context, job Job) error {
    chunks := split(job.Data)

    for i, chunk := range chunks {
        // Check context periodically in long-running CPU work
        if i%100 == 0 {
            select {
            case <-ctx.Done():
                return ctx.Err()
            default:
            }
        }

        if err := processChunk(chunk); err != nil {
            return err
        }
    }

    return nil
}

The select with a default branch is non-blocking — it checks whether the context is done and moves on immediately if not. Checking every 100 iterations (or every N milliseconds using a ticker) is a reasonable balance between responsiveness and overhead.

The three rules

1. Context is always the first parameter.

func doWork(ctx context.Context, ...) error

Not a struct field, not a global, not injected via closure. First parameter, every time. This is a Go convention strong enough to be in the standard library style guide.

2. Never store context in a struct.

// ❌
type Worker struct {
    ctx context.Context
}

// ✅
func (w *Worker) Process(ctx context.Context, job Job) error

Contexts are request-scoped. A struct outlives any single request. Storing context in a struct means you’re using the wrong context for future requests, or holding a cancelled context longer than you should.

3. Always call cancel.

ctx, cancel := context.WithTimeout(parent, 5*time.Second)
defer cancel() // even if the timeout fires first

If the timeout fires before you call cancel, calling cancel is a no-op. But if the work finishes before the timeout, calling cancel releases the timer resources immediately rather than waiting for the timeout to expire. It costs nothing and prevents a resource leak.

Summary

Clean shutdown on SIGTERM — signal.NotifyContext at main, propagate down.

Stop workers on cancellation — select { case <-ctx.Done(): return } in the worker loop.

Per-job timeout — context.WithTimeout(ctx, duration) inside the loop, cancel immediately after.

CPU-bound work — periodic select { case <-ctx.Done(): return ctx.Err(); default: }.

Function signatures — ctx context.Context is always the first parameter.

Context propagation is not boilerplate. It is the mechanism that lets your service behave correctly under pressure — when deadlines pass, when clients disconnect, when operators need to restart the service without waiting for goroutines to unblock naturally. The cost of adding it from the start is a few extra parameters. The cost of retrofitting it later is much higher.

Previous in this series: Three Go Concurrency Mistakes I See in Almost Every Worker Pool
Next: Profiling a Go service in production with pprof.

Dealing with Feign Decode Exception: Error While Extracting Response

Fri, 17 Jul 2026 03:41:43 +0000

In a microservice ecosystem, Spring Cloud OpenFeign makes synchronous HTTP requests look like simple, clean Java interface calls. It handles everything under the hood—until the data structure changes, a service returns a dynamic error payload, or a mapping framework trips over an unexpected data format.

When things break during data parsing, you get the classic runtime headache: feign.codec.DecodeException: Error while extracting response...

How to Make It (Replicating the Error)

Triggering a decoding exception is incredibly easy. OpenFeign uses a decoder (usually backed by Jackson) to map incoming JSON text directly into a Java object.

To forcefully break this mechanism, you can set up any of these scenarios:

Mismatched Data Types: Define your Java record or DTO with an Integer id, but have the producer microservice send back a UUID string like "id": "abc-123-xyz".

Missing Default Constructors/Getters: Use an immutable target object that lacks proper Jackson annotations (@JsonProperty) or proper creator configurations, preventing the deserializer from instantiating it.

Format Deviations: Expect a strict LocalDate or LocalDateTime in the format yyyy-MM-dd, while the external service sends a completely different ISO timestamp format or a UNIX epoch millisecond timestamp.

Dynamic Error Payloads: Set up your Feign client to map standard execution into a SuccessDTO. If the downstream service runs into an unhandled exception and spits out a completely different error JSON block while returning an HTTP 200 OK, the decoder will fail instantly trying to force that error data into the success object.

The Real-World Problems It Causes

When this exception pops up in production, it rarely behaves nicely. It creates secondary ripples across your system that can degrade the user experience and complicate debugging:

Obfuscated Root Causes: The DecodeException effectively wraps the original parsing error (like MismatchedInputException or InvalidFormatException), masking what actually went wrong unless you meticulously dig deep into the nested stack trace.

Broken Circuit Breakers: If you use Resilience4j or a similar framework, unhandled decoding issues look like sudden service failures. A wave of mapping errors can accidentally trip your circuit breaker, completely cutting off a healthy microservice.

Corrupted Downstream Data: If partial deserialization occurs or defaults are filled in blindly, your application might continue processing corrupted, incomplete data downstream, leading to unpredictable business logic bugs.

Cascading UI Failures: Instead of receiving a graceful fallback error message, front-end layers often crash out with generic 500 Internal Server Errors because the gateway or orchestration service couldn't gracefully intercept the failure.

How We Solve This Problem

Fixing this permanently requires moving away from default configurations and building resilience directly into the decoding pipeline.

Implement a Custom ErrorDecoder: Do not rely on standard object mapping for bad responses. Build a custom implementation of feign.codec.ErrorDecoder to explicitly handle HTTP 4xx and 5xx status codes, extracting error payloads before they ever reach the standard object decoder.

Relax Jackson Deserialization Rules: Configure your ObjectMapper bean inside the Feign configuration to be more forgiving. Specifically, toggle DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES to false so your client doesn't explode if a downstream service adds a new field to its payload.

Register Java Time Modules Explicitly: Ensure your Jackson mapper is armed with the JavaTimeModule to natively handle modern java.time types without barfing on string-to-date conversions.

Leverage Native ResponseEntity for Dynamic APIs: If you are dealing with a highly unstable external API with unstable data types, map the Feign method response directly to a raw String or JsonNode. You can then manually parse it using a custom try-catch block to handle layout variations gracefully.

Sumita
Web Developer

I Built a Full POS & Inventory System with Laravel (Shop-Control)

Sat, 30 May 2026 07:27:21 +0000

A few days ago, I decided to go beyond tutorials and build something that reflects how real businesses actually operate.

That’s how Shop-Control was born — a full-stack Point of Sale (POS) and inventory management system built with Laravel.

This project helped me deeply understand how backend systems connect real business workflows like sales, inventory tracking, and customer management.

Why I Built This Project

Most beginner Laravel projects stop at CRUD apps.

I wanted to challenge myself by building something closer to a real production system — something that handles:

Sales processing
Inventory updates in real-time
Role-based access control
Business reporting logic

Instead of isolated features, I focused on how everything connects together as a system.

What Shop-Control Does

Shop-Control simulates a real retail environment where different users interact with the system:

1. Admins & Staff

Manage users and roles
Monitor business performance

2. Products & Inventory

Add, update, and track stock
Organize products by categories

3. Point of Sale (POS)

Add items to cart
Calculate totals dynamically
Process sales transactions

4. Customers

Store customer data
Track purchase history

5. Analytics Dashboard

View sales summaries
Track inventory changes
Monitor business activity

⚙️ Tech Stack

Laravel – Backend framework and business logic
Blade Templates – Server-side rendering
SQLite – Lightweight database for local development
Tailwind CSS / Bootstrap – UI styling

What I Learned Building This

This project taught me more than any tutorial ever did:

Database Design Matters More Than Code

Poor relationships = broken system logic.

I had to carefully design relationships between:

products
sales
customers
cart sessions

2. Business Logic Lives in the Backend

A POS system is not just UI — it’s rules:

stock must update after every sale
carts must calculate totals correctly
users must only access allowed features

3. Real Systems Are About Flow, Not Features

The hardest part wasn’t building features — it was making them work together smoothly.

Preview

Dashboard Overview:

2. POS / Checkout Screen

3. Product & Inventory Management

4. Sales History / Transactions

What’s Next

I plan to improve Shop-Control by adding:

REST API version for mobile apps
PDF invoice generation
Advanced analytics dashboard
Barcode scanning support

Final Thoughts

Building Shop-Control changed how I see Laravel.

It stopped being just a framework for small apps — and became a tool for building real business systems.

If you're learning Laravel, my advice is simple:

Stop only building tutorials. Start building systems.

That’s where real growth happens.

I’d love to hear from other developers:
What was the first project that made Laravel “click” for you?

Multi-Tenant Security in SaaS: Data Isolation Patterns That Actually Work

Mon, 04 May 2026 11:15:43 +0000

Multi-tenancy is the economic engine of SaaS. Sharing infrastructure across customers reduces cost and simplifies operations. But it introduces a risk that can end your business overnight: tenant data leakage.

When one customer can see another customer's data — even accidentally — the consequences are severe. Regulatory fines, contract termination, public disclosure requirements, and irreparable trust damage. I have worked on multi-tenant platforms serving hundreds of organisations, and I have learned that data isolation is not something you bolt on later. It is a foundational architectural decision that shapes everything from database design to query patterns to testing strategy.

This article covers the three main tenancy models, practical implementation patterns in Laravel and Node.js, and the security controls that prevent data leakage in production.

The Three Tenancy Models

1. Database-per-Tenant

Each tenant gets a completely separate database. Maximum isolation, maximum operational complexity.

Pros: Strongest isolation. Easy compliance with data residency requirements. Independent backup and restore per tenant.

Cons: Connection management overhead. Schema migrations must run across all databases. Does not scale well beyond a few hundred tenants.

Best for: Regulated industries (healthcare, finance), enterprise customers with strict data sovereignty requirements.

2. Schema-per-Tenant

All tenants share a database server, but each gets a separate schema (or namespace). Moderate isolation with lower overhead than separate databases.

Pros: Good isolation without connection management complexity. PostgreSQL handles this natively with schemas.

Cons: Not supported equally across all databases. MySQL schema separation is effectively database separation. Migrations still need to run per schema.

Best for: Mid-market SaaS with tens to low hundreds of tenants.

3. Shared Database with Row-Level Isolation

All tenants share the same tables. Every row includes a tenant_id column. Isolation is enforced at the application and query level.

Pros: Simplest to operate. Migrations run once. Scales to thousands of tenants. Lowest infrastructure cost.

Cons: One missed WHERE clause leaks data. Requires rigorous enforcement at every layer.

Best for: High-volume SaaS, platforms with many small tenants, startups optimising for speed.

For most SaaS applications, shared database with row-level isolation is the right starting point. It is also the hardest to get right, because isolation depends entirely on your code never forgetting to filter by tenant. The rest of this article focuses on making that bulletproof.

Tenant Context Management

The foundation of row-level tenancy is a reliable tenant context — a mechanism that ensures every database query automatically includes the correct tenant filter.

Laravel: Global Scopes and Middleware

namespace App\Http\Middleware;

use App\Services\TenantContext;
use Closure;
use Illuminate\Http\Request;

class ResolveTenant
{
    public function __construct(private TenantContext $tenantContext) {}

    public function handle(Request $request, Closure $next)
    {
        $tenantId = $this->resolveTenantId($request);

        if (!$tenantId) {
            return response()->json(['error' => 'Tenant could not be resolved.'], 403);
        }

        $this->tenantContext->set($tenantId);

        $response = $next($request);

        $this->tenantContext->clear();

        return $response;
    }

    private function resolveTenantId(Request $request): ?string
    {
        if ($request->hasHeader('X-Tenant-ID')) {
            return $this->validateTenantAccess(
                $request->user(),
                $request->header('X-Tenant-ID')
            );
        }

        $host = $request->getHost();
        $subdomain = explode('.', $host)[0] ?? null;

        if ($subdomain) {
            return $this->resolveTenantBySubdomain($subdomain);
        }

        return $request->user()?->default_tenant_id;
    }

    private function validateTenantAccess($user, string $tenantId): ?string
    {
        if (!$user) return null;

        $hasAccess = $user->tenants()->where('tenant_id', $tenantId)->exists();
        return $hasAccess ? $tenantId : null;
    }

    private function resolveTenantBySubdomain(string $subdomain): ?string
    {
        return cache()->remember(
            "tenant:subdomain:{$subdomain}",
            now()->addHours(1),
            fn () => \App\Models\Tenant::where('subdomain', $subdomain)->value('id')
        );
    }
}

namespace App\Services;

class TenantContext
{
    private ?string $tenantId = null;

    public function set(string $tenantId): void
    {
        $this->tenantId = $tenantId;
    }

    public function get(): string
    {
        if ($this->tenantId === null) {
            throw new \RuntimeException('Tenant context not set. This is a critical isolation failure.');
        }

        return $this->tenantId;
    }

    public function clear(): void
    {
        $this->tenantId = null;
    }

    public function isSet(): bool
    {
        return $this->tenantId !== null;
    }
}

Automatic Query Scoping with Eloquent

The most dangerous aspect of row-level tenancy is forgetting to add WHERE tenant_id = ? to a query. Eloquent global scopes eliminate this risk:

namespace App\Models\Traits;

use App\Models\Scopes\TenantScope;
use App\Services\TenantContext;

trait BelongsToTenant
{
    protected static function bootBelongsToTenant(): void
    {
        static::addGlobalScope(new TenantScope());

        static::creating(function ($model) {
            if (!$model->tenant_id) {
                $model->tenant_id = app(TenantContext::class)->get();
            }
        });
    }
}

namespace App\Models\Scopes;

use App\Services\TenantContext;
use Illuminate\Database\Eloquent\Builder;
use Illuminate\Database\Eloquent\Model;
use Illuminate\Database\Eloquent\Scope;

class TenantScope implements Scope
{
    public function apply(Builder $builder, Model $model): void
    {
        $tenantContext = app(TenantContext::class);

        if ($tenantContext->isSet()) {
            $builder->where($model->getTable() . '.tenant_id', $tenantContext->get());
        }
    }
}

Now every model that uses the BelongsToTenant trait automatically filters by tenant:

class Invoice extends Model
{
    use BelongsToTenant;

    protected $fillable = ['tenant_id', 'number', 'amount', 'status'];
}

// These queries automatically include WHERE tenant_id = ?
$invoices = Invoice::where('status', 'pending')->get();
$invoice = Invoice::findOrFail($id);
$count = Invoice::count();

Node.js: Middleware and Query Builder Integration

import { Request, Response, NextFunction } from 'express';
import { AsyncLocalStorage } from 'async_hooks';

interface TenantContextData {
  tenantId: string;
}

export const tenantStorage = new AsyncLocalStorage<TenantContextData>();

export function getTenantId(): string {
  const context = tenantStorage.getStore();
  if (!context?.tenantId) {
    throw new Error('Tenant context not set. This is a critical isolation failure.');
  }
  return context.tenantId;
}

export function tenantMiddleware(
  req: Request,
  res: Response,
  next: NextFunction
): void {
  const tenantId = resolveTenantId(req);

  if (!tenantId) {
    res.status(403).json({ error: 'Tenant could not be resolved.' });
    return;
  }

  tenantStorage.run({ tenantId }, () => next());
}

function resolveTenantId(req: Request): string | null {
  const headerTenant = req.headers['x-tenant-id'] as string;
  if (headerTenant) return headerTenant;

  const host = req.hostname;
  const subdomain = host.split('.')[0];
  if (subdomain) return subdomain;

  return (req as any).user?.defaultTenantId ?? null;
}

Prisma Extension for Automatic Tenant Filtering

import { PrismaClient, Prisma } from '@prisma/client';

function createTenantPrisma() {
  const prisma = new PrismaClient();

  return prisma.$extends({
    query: {
      $allOperations({ model, operation, args, query }) {
        const tenantModels = ['Invoice', 'Payment', 'Project', 'Document'];

        if (!model || !tenantModels.includes(model)) {
          return query(args);
        }

        const tenantId = getTenantId();

        if (['findMany', 'findFirst', 'findUnique', 'count', 'aggregate'].includes(operation)) {
          args.where = { ...args.where, tenantId };
        }

        if (['create', 'createMany'].includes(operation)) {
          if (operation === 'create') {
            args.data = { ...args.data, tenantId };
          }
        }

        if (['update', 'updateMany', 'delete', 'deleteMany'].includes(operation)) {
          args.where = { ...args.where, tenantId };
        }

        return query(args);
      },
    },
  });
}

export const prisma = createTenantPrisma();

Row-Level Security in PostgreSQL

For an additional layer of defence, PostgreSQL's Row-Level Security (RLS) enforces tenant isolation at the database level — even if the application code has a bug:

ALTER TABLE invoices ENABLE ROW LEVEL SECURITY;

CREATE POLICY tenant_isolation ON invoices
  USING (tenant_id = current_setting('app.current_tenant_id')::uuid);

ALTER TABLE invoices FORCE ROW LEVEL SECURITY;

Set the tenant context at the beginning of each database connection:

Laravel with RLS

class SetTenantOnConnection
{
    public function handle(Request $request, Closure $next)
    {
        $tenantId = app(TenantContext::class)->get();

        DB::statement("SET app.current_tenant_id = '{$tenantId}'");

        return $next($request);
    }
}

Node.js with RLS

async function withTenantRLS<T>(
  callback: (tx: PrismaClient) => Promise<T>
): Promise<T> {
  const tenantId = getTenantId();

  return prisma.$transaction(async (tx) => {
    await tx.$executeRawUnsafe(
      `SET LOCAL app.current_tenant_id = '${tenantId}'`
    );
    return callback(tx as any);
  });
}

RLS acts as a safety net. Even if a developer writes a raw query that omits the tenant filter, PostgreSQL rejects rows that do not belong to the current tenant.

Testing for Tenant Leakage

The most critical test you can write for a multi-tenant system verifies that Tenant A cannot access Tenant B's data:

Laravel Test

class TenantIsolationTest extends TestCase
{
    public function test_tenant_cannot_access_other_tenant_data(): void
    {
        $tenantA = Tenant::factory()->create();
        $tenantB = Tenant::factory()->create();

        $invoiceA = Invoice::factory()->create(['tenant_id' => $tenantA->id]);
        $invoiceB = Invoice::factory()->create(['tenant_id' => $tenantB->id]);

        app(TenantContext::class)->set($tenantA->id);

        $visibleInvoices = Invoice::all();

        $this->assertTrue($visibleInvoices->contains($invoiceA));
        $this->assertFalse($visibleInvoices->contains($invoiceB));
    }

    public function test_tenant_cannot_access_other_tenant_by_id(): void
    {
        $tenantA = Tenant::factory()->create();
        $tenantB = Tenant::factory()->create();

        $invoiceB = Invoice::factory()->create(['tenant_id' => $tenantB->id]);

        app(TenantContext::class)->set($tenantA->id);

        $this->expectException(ModelNotFoundException::class);
        Invoice::findOrFail($invoiceB->id);
    }

    public function test_queries_without_tenant_context_throw(): void
    {
        $this->expectException(\RuntimeException::class);
        Invoice::all();
    }
}

Node.js Test

describe('Tenant Isolation', () => {
  it('should not return data from another tenant', async () => {
    const tenantA = await createTenant();
    const tenantB = await createTenant();

    await createInvoice({ tenantId: tenantA.id, amount: 100 });
    await createInvoice({ tenantId: tenantB.id, amount: 200 });

    const invoices = await tenantStorage.run(
      { tenantId: tenantA.id },
      () => prisma.invoice.findMany()
    );

    expect(invoices).toHaveLength(1);
    expect(invoices[0].tenantId).toBe(tenantA.id);
  });

  it('should throw when tenant context is missing', async () => {
    await expect(prisma.invoice.findMany()).rejects.toThrow(
      'Tenant context not set'
    );
  });
});

Run these tests on every deployment. A failing tenant isolation test should block the release.

Common Tenant Leakage Vectors

Even with global scopes and RLS, leakage can happen through:

Raw queries: Any DB::select() or prisma.$queryRaw() bypasses ORM scoping. Audit every raw query for tenant filtering.
Background jobs: Queue workers do not have HTTP request context. Tenant ID must be serialised into the job payload and restored before execution.
Cache keys: If cache keys do not include the tenant ID, one tenant's cached data can be served to another.
File storage paths: Uploaded files must be stored under tenant-specific prefixes. A flat storage structure risks cross-tenant file access.
Admin endpoints: Internal tools that query across tenants must be explicitly authorised and logged.

Laravel: Tenant-Aware Job Example

class ProcessInvoice implements ShouldQueue
{
    use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;

    public function __construct(
        private string $tenantId,
        private int $invoiceId,
    ) {}

    public function handle(TenantContext $tenantContext): void
    {
        $tenantContext->set($this->tenantId);

        $invoice = Invoice::findOrFail($this->invoiceId);
        // Process invoice within correct tenant context
    }
}

Production Results

After implementing comprehensive tenant isolation across multi-tenant platforms:

Metric	Before	After
Tenant data leakage incidents	2–3 per year	0
Security audit findings (critical)	4	0
Time to onboard new tenant	2 days	15 minutes
Cross-tenant query bugs caught in CI	N/A	12 (prevented from reaching production)
Customer trust incidents	1 per year	0

Key Takeaways

Start with shared database + row-level isolation. It scales to thousands of tenants and has the lowest operational overhead. Move to database-per-tenant only when regulation demands it.
Never rely on developers remembering to filter. Use global scopes (Laravel) or Prisma extensions (Node.js) to make tenant filtering automatic and invisible.
Add PostgreSQL RLS as a safety net. Application-level filtering is necessary. Database-level filtering is insurance against application bugs.
Test for leakage explicitly. Write tests that create data in Tenant A, set context to Tenant B, and verify the data is invisible. Run these tests on every deployment.
Audit raw queries, background jobs, cache keys, and file paths. These are the most common vectors for accidental cross-tenant data exposure.
Serialise tenant context into background jobs. Queue workers do not inherit HTTP request context. The tenant ID must travel with the job.

Conclusion

Multi-tenant security is not a feature you add — it is a design constraint that shapes your entire architecture. The patterns in this article are not theoretical. They come from building platforms where a tenant leakage incident would mean regulatory consequences and broken customer relationships.

Whether you build in Laravel or Node.js, the principles are the same: automatic query scoping, database-level enforcement, explicit testing, and paranoid attention to the edge cases where isolation breaks down. Get this right, and your multi-tenant platform can scale with confidence.

now ribbon support launching by bun and npx!

Wed, 29 Apr 2026 08:57:26 +0000

Main changes

now we support two different ways to launch ribbon (bun and npx-tsx)
you can open alias.json and config.json by rib open

Turning a Research Idea into a Real Backend System

Fri, 17 Apr 2026 14:11:16 +0000

Recently got featured as an early contributor on CoderLegion — really appreciate being part of this growing community.

But honestly, what matters more is what I’m building right now.

I’ve been working on turning a research idea into something real.

The idea is simple:
Using machine learning to understand and optimize how backend systems behave under load — balancing performance (latency) and resource usage (CPU, memory).

Instead of keeping it theoretical, I’ve started applying this to a real backend use case.

Starting small:
A simple file server with basic profiling (CPU, memory, request timing).

The goal isn’t to overengineer things, but to build a small working loop:
→ collect real system metrics
→ analyze behavior
→ suggest simple optimizations

Not just building systems…
but understanding and improving how they actually run.

Still early, but I’ll be sharing the progress.

If you’ve worked on backend performance or system optimization before, I’d love to hear your thoughts.

I Wasted 10 Days on Backend Setup. Here’s What I’d Do in 2 Hours Now

Fri, 17 Apr 2026 13:50:42 +0000

Every new idea feels exciting.

You open your editor.
You’re ready to build something real.

Then it starts…

setting up auth
defining roles
handling tenants
creating base APIs

Day 1 goes by.
Then day 3.
Then day 10.

And you still haven’t built the actual product.

What I did wrong

On one project, I spent around 10 days just on backend setup.

I told myself:

“This needs to be clean”
“I’ll reuse this later”
“Better to do it properly now”

After all that work, I had:

no users
no feedback
nothing shipped

That’s when it hit me.

I wasn’t building a product.

I was rebuilding the same backend again.

The real problem

This work isn’t hard.

It’s repeated.

Every project starts from zero:

same auth flow
same role checks
same tenant logic
same base structure

You’re not solving new problems.

You’re rewriting solved ones.

What I do now (2-hour approach)

Now, I don’t start from scratch.

I start with a base that already has:

authentication ready
roles and permissions wired
tenant handling in place
basic structure set

From there, I focus on:

the core feature
the actual user problem

Not the setup.

What actually changed

Nothing about my skill.

Only this:

I stopped treating backend setup as “part of the product”.

It’s not.

It’s just the cost of starting.

And reducing that cost changes everything.

A simple example

Earlier:

create user system
design role model
write access checks everywhere
debug edge cases

Now:

plug into existing auth
assign roles
enforce rules once
move on

Same outcome.
Way less time.

The shift that matters

Early stage is simple:

ship fast
get feedback
improve

If setup takes 10 days, you break that loop.

If setup takes 2 hours, you stay in it.

That’s the difference.

Final thought

Your first version doesn’t need a perfect backend.

It needs:

something that works
something users can try
something you can improve

Next time you start a project, don’t ask:

“What’s the best way to set this up?”

Ask:

“How fast can I stop setting up and start building?”

If you want to skip the repeated backend work and start faster, you can check this out:

https://buildbasekit.com/boilerplates/authkit-lite/

Why Your Backend, Frontend, and Database “Don’t Talk”

Fri, 17 Apr 2026 11:13:45 +0000

I expected my first ever full-stack project to end with a sense of completion. my plan was to build the frontend, set up the backend, connect a database, then watch everything come together into a finished product.

Instead, I ran into something far more frustrating than your average bug: nothing was connected properly, and i had no idea why. The strangest part was that, on the surface, everything looked completely fine. The frontend loaded without issues, the backend server ran, and the database connection didn’t show any errors, nothing was fundamentally wrong. But the moment I tried to use the system as a whole, everything collapsed in a way that didn’t make sense at fisrt.

Requests didn’t behave correctly, data didn’t flow properly between layers, and responses that I did manage to recieve didn’t match what I was expecting at all. It felt less like a broken program, and more like 3 separate systems pretending to. be connected, but never actually communicating in any meaningful way.

Naturally, I fell into what i now call the debugging spiral. I started guessing, checking to see if the frontend was sending the right requests. When I failed to find anything, i moved to the backend routes, assuming something wasn’t being handled correctly. I went back and forth, checking the database connections, back to the front end, repeating the same cycle slightly differently each time.

But there was no single obvious error that I could fix. And this was where the real frustration came in, because debugging becomes almost disorientating when you have nothing concrete to grasp onto. You move endlessly between parts of the system, hoping if you look at it from a slightly different angle, something will eventually look wrong. But everything remains to appear technically correct, and the system remains to behave incorrectly.

The real issue, which I didn’t realise at the time, was that I was treating the app as separate components instead of a single, connected system. In my head, the frontend was one thing, the backend was another, and the database was another independent piece. So, I assumed debugging meant finding which one of them was broken and fixing it in isolation. But, when dealing with full-stack systems, things don’t usually fail in a clean, isolated way. They fail in the spaces between components, the communication layers that tie everything together. But at that point in my learning, I didn’t really understand those connections properly, and understand how fragile they really are.

What I eventually started to realise is that ‘connection’ in full-stack development isn’t just an abstract idea that magically links everything together. It consists of a series of very explicit interactions that all have t line up perfectly for a system to work.

You define routes on the backend that expect specific requests, which you have to send from the frontend in a particular format. The data has to be structured using formats like JSON so both sides can understand each other. You handle responses manually and then update the UI based off of that data. If even one of these steps fails, the system doesn’t only partially fail, or break a little bit, it quietly stops behaving in the way you would expect, without always sowing you a clear error message explaining why.

That is part of what made the whole experience so confusing. The most deceptive part for me wasn’t that things were breaking, it was that everything looked like it was working. No console errors were being logged, no visible crashes, which lead me to a dangerous assumption: if nothing is visibly wrong, then everything must be fine. But when dealing with full-stack development, that assumption falls apart incredibly quickly.

You can have a system that is completely broken in terms of logic and data flow, while showing no obvious signs of failure in wither the logs or the interface. The system doesn’t always announce it’s problems, it expects you to connect the dots yourself.

Looking back, the real lesson I learnt wasn’t about fixing one specific bug in my code. It was about changing how I think about a system as a whole. I spent hours looking for broken parts, when I should’ve been looking for broken communication instead. And that difference matters more than I realised at the time, because debugging isn’t all about understanding individual components, you have to also understand how those components interact with one another, and where that interaction has the possibility of falling apart.

My project didn’t fail because I wrote bad code, it failed because I failed to understand that debugging a system is a completely different skill to debugging a single piece of code.

You’re Not Building a Product. You’re Rebuilding the Same Backend Again.

Mon, 13 Apr 2026 16:56:23 +0000

Every new idea starts the same way.

You open your editor.

You’re ready to build.

Then you realize…

You need to set up everything again:

Auth
Roles and permissions
Multi-tenancy
Database setup
Basic APIs

A few days later, nothing is shipped.

Just setup.

The real issue

This work isn’t difficult.

It’s repeated work.

You are solving the same problems again and again instead of building something users care about.

Why this slows everything down

Early stage should be simple:

ship fast
get feedback
improve

But instead, most of the time goes into backend setup.

And that kills momentum.

A mistake I kept making

On one project, I spent around 10 days setting up:

authentication
roles
tenant handling
base APIs

After all that, I had:

no users
no feedback
no product

That’s when I realized something.

I wasn’t building a product.

I was rebuilding infrastructure.

What actually matters

You don’t need a perfect backend.

You need something that:

works
supports real users
helps you move fast

That’s it.

A better way to approach this

Instead of starting from scratch every time, start with a base that already handles:

auth
roles
structure

Then spend your time on what actually matters.

Your product.

Reality check

Most developers don’t fail because of bad architecture.

They fail because they never ship.

Final thought

Next time you start something new, don’t ask:

“What’s the best setup?”

Ask:

“How fast can I ship something useful?”

If you want to skip backend setup and start faster, try this free:
https://buildbasekit.com/boilerplates/authkit-lite/

The 2026 Software Budget Oversight: Optimizing API Architectures to Prevent Billing Spikes

Thu, 19 Mar 2026 14:30:00 +0000

As engineers, we love plug-and-play solutions. Need authentication? Auth0. Need an email receipt? SendGrid. Need a conversational interface? OpenAI. It drastically reduces our time-to-market. But building a product entirely out of third-party microservices creates a fragile financial ecosystem.

When an application transitions from a closed beta to a public launch, server pricing tiers move exponentially. Here is how to refactor your architecture to prevent a sudden spike in traffic from exhausting your operational budget.

1. Kill the Long Polling
Never use long polling where your app constantly asks a server "Is the data ready yet?" every few seconds. This racks up millions of useless backend server hits, inflating your EC2 and database hosting costs. Instead, rely entirely on Webhooks. The third-party API will independently notify your server the exact millisecond an event occurs (e.g., a payment clearing).

2. Cap Your AI Context Windows
Integrating AI models relies on token-based pricing. The hidden cost is the context window. If your app allows users to chat with an AI, the app must re-send the entire conversation history with every new message. This causes input token consumption to double and triple exponentially with long chats. Hardcode an absolute limit on user queries per day, and aggressively summarize past context windows before sending them back to the LLM.

Choosing who builds and manages these optimizations is a critical decision. Should you hire a full-time CTO, an agency, or freelancers? We detail the exact pros and cons in our guide: In-House vs Agency vs Freelancers.

By taking a microscopic look at how your code interacts with external servers, you can cut your monthly cloud infrastructure bill by a significant margin.

Merancang Backend Bisnis ISP: API Pelanggan, Paket Internet, Invoice, dan Tiket Support

Fri, 13 Mar 2026 17:00:00 +0000

Di industri konektivitas, tantangan teknis tidak lagi berhenti di pemasangan jaringan atau uptime perangkat. Hari ini, operator internet juga dituntut punya sistem internal yang rapi: data pelanggan sinkron, paket layanan mudah dikelola, invoice tercatat otomatis, dan tiket gangguan bisa dipantau real-time. Bahkan lanskap telco global mulai bergerak ke model yang makin API-driven, sebagaimana terlihat dalam laporan Reuters tentang kolaborasi operator dan network APIs yang menunjukkan bagaimana API kini menjadi bagian penting dari masa depan layanan digital. Di level implementasi, tantangannya bermuara pada satu hal: backend bisnis isp.

Dari sisi arsitektur perangkat lunak, tantangan itu tidak bisa dijawab dengan CRUD seadanya. Sistem untuk ISP perlu memikirkan autentikasi, otorisasi, integrasi billing, audit trail, hingga komunikasi antarlayanan yang aman. Ini sejalan dengan temuan dalam studi ilmiah tentang authentication dan authorization pada arsitektur microservices, yang menekankan pentingnya API gateway, OAuth 2.0, OpenID Connect, JWT, dan pengamanan komunikasi antarservice. Tema ini layak diangkat karena pembaca developer butuh contoh nyata bagaimana membangun sistem operasional modern untuk bisnis konektivitas—bukan teori generik yang berhenti di whiteboard.

"Backend yang baik untuk ISP bukan sekadar menghubungkan tabel pelanggan, melainkan menghubungkan operasional, pengalaman pelanggan, dan keputusan bisnis dalam satu alur data yang konsisten."

1. Kenapa Backend untuk ISP Tidak Bisa Disamakan dengan CRUD Biasa

Membangun aplikasi untuk ISP terlihat sederhana di permukaan: ada pelanggan, ada paket internet, ada tagihan, dan ada tiket gangguan. Tetapi saat sistem mulai dipakai tim sales, finance, teknisi lapangan, dan customer support secara bersamaan, kompleksitasnya naik drastis. Di titik ini, backend bisnis isp harus dirancang sebagai fondasi operasional, bukan sekadar panel admin.

Domain bisnisnya lebih hidup daripada kelihatannya

Dalam bisnis ISP, satu pelanggan bisa memiliki beberapa layanan, satu layanan bisa berubah paket, satu invoice bisa memiliki status berbeda, dan satu gangguan bisa melibatkan beberapa tim sekaligus. Itu berarti model datanya perlu menampung histori, perubahan status, dan relasi yang tidak datar.

Beberapa entitas inti yang hampir selalu muncul:

pelanggan
alamat instalasi
paket internet
langganan aktif
invoice dan pembayaran
tiket support
teknisi dan jadwal kunjungan
log aktivitas dan audit trail

Operasional hariannya serba real-time

Yang membuat sistem ini menarik adalah ritme operasionalnya. Data tidak hanya disimpan, tetapi terus bergerak.

Sales membuat prospek dan mengubahnya menjadi pelanggan aktif.
Tim provisioning menjadwalkan instalasi.
Sistem billing membuat invoice bulanan.
Support menerima laporan gangguan.
Teknisi memperbarui status penanganan di lapangan.
Supervisor memantau SLA dan backlog.

Tabel tantangan umum di sistem ISP

Area	Tantangan umum	Dampak bila salah desain
Pelanggan	Data ganda, alamat tidak konsisten	Instalasi kacau, histori sulit ditelusuri
Paket layanan	Perubahan harga dan upgrade paket	Tagihan tidak akurat
Billing	Invoice manual, rekonsiliasi lambat	Cashflow terganggu
Support	Tiket tidak terprioritasi	SLA buruk, churn meningkat
Integrasi	Data tersebar di banyak tool	Reporting tidak sinkron
Keamanan	Hak akses terlalu longgar	Kebocoran data operasional

2. Blueprint Arsitektur yang Masuk Akal untuk Backend ISP

Alih-alih langsung memecah sistem menjadi puluhan service, pendekatan yang lebih sehat adalah mulai dari modular monolith yang rapi, lalu memisahkan service ketika beban dan kebutuhan organisasi memang menuntut. Untuk banyak tim, ini adalah jalur paling realistis agar backend bisnis isp tetap cepat dibangun, tetapi tidak cepat berantakan.

Peta modul yang disarankan

Minimal, backend bisa dibagi ke beberapa bounded context berikut:

Customer Service — mengelola pelanggan, alamat, identitas, dan status akun
Product Service — mengelola paket internet, promo, add-on, dan kebijakan harga
Subscription Service — menghubungkan pelanggan dengan layanan aktif
Billing Service — menghasilkan invoice, pembayaran, dan pengingat jatuh tempo
Support Service — mengelola tiket, kategori gangguan, prioritas, dan SLA
Auth & Access Service — role, permission, session, dan audit login
Notification Service — email, WhatsApp, atau notifikasi in-app

Alur data sederhananya seperti ini

Prospek dibuat oleh sales.
Data diverifikasi dan diubah menjadi pelanggan.
Pelanggan memilih paket internet.
Sistem membuat langganan dan status provisioning.
Billing service menerbitkan invoice sesuai siklus.
Jika ada kendala, support service membuat tiket.
Semua perubahan penting tercatat di audit log.

Referensi internal yang relevan dari CoderLegion

Kalau Anda suka artikel berbasis studi kasus aplikasi bisnis nyata, dua bacaan di CoderLegion ini sangat nyambung dengan tema arsitektur operasional modern:

Yang menarik, dua artikel itu sama-sama menekankan bahwa aplikasi bisnis yang baik lahir dari struktur data dan alur kerja yang jelas—pelajaran yang sangat relevan saat merancang backend bisnis isp.

3. Desain API Inti yang Perlu Ada Sejak Hari Pertama

Sebelum bicara dashboard yang cantik, kita perlu menyepakati kontrak data yang stabil. API yang baik akan membuat frontend lebih cepat berkembang, integrasi lebih mudah, dan perubahan fitur lebih terkendali. Di sinilah kualitas backend bisnis isp benar-benar diuji.

API pelanggan

Endpoint inti untuk domain pelanggan biasanya mencakup:

GET /customers
POST /customers
GET /customers/{id}
PATCH /customers/{id}
GET /customers/{id}/subscriptions
GET /customers/{id}/invoices
GET /customers/{id}/tickets

Contoh payload ringkas:

{
  "full_name": "Rina Pratama",
  "email": "*Emails are not allowed*",
  "phone": "+6281234567890",
  "service_address": {
    "street": "Jl. Melati No. 18",
    "city": "Bekasi",
    "postal_code": "17121"
  },
  "customer_type": "residential"
}

API paket internet

Paket internet sebaiknya diperlakukan sebagai entitas produk yang punya histori harga dan status publikasi.

GET /plans
POST /plans
PATCH /plans/{id}
POST /plans/{id}/publish
POST /plans/{id}/archive

Field yang layak disimpan:

nama paket
kecepatan up/down
FUP atau fair usage policy
harga bulanan
biaya instalasi
status aktif
tag segmen: rumahan, UMKM, kantor, komunitas

API invoice

Billing adalah area yang paling sering memicu drama kalau dari awal tidak dipikirkan dengan serius.

POST /invoices/generate
GET /invoices
GET /invoices/{id}
POST /payments
POST /invoices/{id}/send-reminder

Tabel status invoice yang umum dipakai:

Status	Arti	Aksi sistem
draft	Tagihan baru dibuat	Belum dikirim
issued	Tagihan telah diterbitkan	Siap dibayar
paid	Pembayaran diterima	Update saldo
overdue	Lewat jatuh tempo	Kirim reminder
canceled	Tagihan dibatalkan	Kunci perubahan

API tiket support

Untuk ISP, tiket support bukan sekadar form komplain. Ia adalah pusat koordinasi antara customer service, NOC, dan teknisi lapangan.

POST /tickets
GET /tickets
GET /tickets/{id}
PATCH /tickets/{id}/status
POST /tickets/{id}/assign
POST /tickets/{id}/comment

Struktur prioritas yang umum:

low — gangguan minor, workaround tersedia
medium — layanan terpengaruh sebagian
high — pelanggan tidak bisa menggunakan layanan
critical — outage area atau gangguan massal

4. Skema Data, Rule Bisnis, dan Event yang Sering Terlupakan

Banyak backend terlihat rapi pada demo pertama, lalu mulai retak ketika rule bisnis bertambah. Penyebabnya bukan semata tool, tetapi model data yang tidak disiapkan untuk perubahan. Agar backend bisnis isp tahan tumbuh, desain tabel dan event domain harus memperhitungkan histori.

Rule bisnis yang sebaiknya eksplisit

Satu pelanggan bisa punya lebih dari satu langganan aktif bila produk mendukung.
Perubahan paket tidak boleh menghapus histori paket sebelumnya.
Invoice harus tetap menyimpan snapshot harga pada saat diterbitkan.
Tiket gangguan harus punya jejak assignment dan perubahan status.
Role finance tidak otomatis boleh melihat semua data teknis.

Event yang layak dicatat

Daripada semua log ditumpuk dalam satu tabel generik, lebih baik pisahkan event penting yang memang bernilai operasional.

customer_created
subscription_activated
plan_changed
invoice_issued
payment_received
ticket_opened
ticket_assigned
ticket_resolved

Contoh tabel ringkas relasi inti

Entitas	Relasi utama	Catatan desain
customers	has many subscriptions	Simpan status dan metadata
subscriptions	belongs to customers, plans	Wajib punya tanggal mulai/akhir
plans	has many subscriptions	Hindari hard delete
invoices	belongs to subscriptions	Simpan snapshot amount
payments	belongs to invoices	Rekonsiliasi per transaksi
tickets	belongs to customers	Hubungkan ke SLA dan kategori

5. Security dan Access Control Bukan Fitur Tambahan

Di banyak aplikasi internal, security sering datang belakangan setelah fitur utama selesai. Padahal, untuk sistem operasional ISP, kebiasaan itu berbahaya. Ketika billing, data pelanggan, dan tiket gangguan hidup dalam satu platform, keamanan harus menjadi desain awal, bukan tempelan akhir. Rekomendasi ini juga sejalan dengan literatur microservices yang menyoroti peran OAuth 2.0, OpenID Connect, JWT, dan API Gateway dalam menjaga batas akses sistem.

Checklist keamanan minimum

Area	Praktik yang disarankan
Autentikasi	OAuth 2.0 / OpenID Connect untuk aplikasi modern
Otorisasi	Role-based access control ditambah permission granular
API gateway	Rate limiting, auth verification, request tracing
Audit trail	Catat login, perubahan status, aksi finansial
Secret management	Simpan credential di vault, bukan di repo
Komunikasi service	Gunakan TLS dan validasi antarservice
Data sensitif	Enkripsi field penting dan batasi akses query

Role yang realistis untuk tahap awal

admin — akses konfigurasi dan user management
sales — prospek, pelanggan baru, status instalasi
finance — invoice, pembayaran, aging report
support — tiket, SLA, histori pelanggan
technician — tugas lapangan, update pengerjaan
supervisor — akses dashboard lintas fungsi

Observability juga wajib

Jangan tunggu outage untuk sadar pentingnya observability. Minimal sediakan:

structured logging
request ID / correlation ID
metrics untuk latency dan error rate
dashboard status invoice dan tiket
alert untuk queue backlog dan job gagal

6. Stack Implementasi yang Realistis untuk Tim Kecil sampai Menengah

Kunci dari sistem bisnis yang sehat bukan memilih stack paling keren di X atau GitHub, melainkan yang paling cocok dengan kemampuan tim dan target delivery. Untuk banyak kasus, backend bisnis isp bisa berjalan sangat baik dengan stack yang pragmatis, mudah direkrut, dan gampang dipelihara.

Rekomendasi stack pragmatis

Backend framework: NestJS atau Laravel jika tim sudah kuat di PHP
Database: PostgreSQL
Cache/queue: Redis
ORM: Prisma, TypeORM, atau Eloquent
Auth: Keycloak, Auth.js, atau implementasi OAuth provider terpercaya
API documentation: OpenAPI / Swagger
Background jobs: BullMQ, Laravel Queue, atau Celery bila Python dipakai
Monitoring: Grafana + Prometheus atau stack cloud yang setara

Kapan perlu microservices?

Jangan buru-buru. Pisahkan service bila salah satu kondisi ini muncul:

billing menjadi sangat kompleks dan butuh lifecycle sendiri
support ticketing dipakai lintas brand atau lintas unit bisnis
ada integrasi partner/third-party yang berat di satu domain
deployment cadence tiap modul sudah berbeda jauh

Roadmap 90 hari yang masuk akal

Minggu 1–2: finalisasi domain model, role, dan flow bisnis
Minggu 3–4: bangun auth, customer module, dan product module
Minggu 5–6: subscription + billing draft
Minggu 7–8: payment reconciliation + notification
Minggu 9–10: support ticketing + SLA
Minggu 11–12: audit trail, reporting, hardening, dan observability

7. How to Build MVP Backend Bisnis ISP Tanpa Overengineering

Kalau target Anda adalah bergerak cepat tetapi tetap waras saat scale-up, mulailah dari MVP yang fokus pada alur paling penting. Prinsipnya sederhana: pastikan value chain utama berjalan dulu, lalu baru tambahkan kecanggihan lain.

Langkah-langkah implementasi

Definisikan empat domain inti: pelanggan, paket, invoice, tiket.
Tetapkan role dan permission sejak awal agar UI dan API konsisten.
Buat API contract dengan OpenAPI agar frontend dan backend sinkron.
Bangun billing rule minimal: invoice bulanan, jatuh tempo, status bayar.
Tambahkan ticket workflow: open, assign, in-progress, resolved, closed.
Sediakan dashboard operasional untuk metrik dasar: pelanggan aktif, tagihan overdue, tiket terbuka.
Catat audit trail untuk semua aksi penting.
Uji dengan data realistis dari operasional, bukan data dummy yang terlalu rapi.

Anti-pattern yang sebaiknya dihindari

semua logic ditaruh di controller
status invoice hanya boolean paid/unpaid
tidak ada histori perubahan paket
tiket support tanpa kategori dan prioritas
role akses dibuat belakangan
hard delete untuk data transaksional

FAQ

Apakah backend ISP harus langsung microservices?

Tidak. Untuk banyak perusahaan, modular monolith justru lebih cepat, lebih murah, dan lebih mudah dikelola pada fase awal.

Apa modul pertama yang wajib dibangun?

Mulai dari pelanggan, paket, langganan, dan invoice. Setelah itu, tiket support biasanya menjadi prioritas berikutnya.

Kenapa billing harus dipikirkan sejak awal?

Karena banyak masalah bisnis justru muncul dari tagihan yang tidak sinkron, promo yang tidak tercatat rapi, dan histori perubahan layanan yang hilang.

Apakah frontend dashboard bisa dibangun paralel?

Bisa, asalkan API contract disepakati sejak awal. Ini mempercepat delivery tanpa membuat tim saling menunggu.

Bagaimana kalau bisnis memiliki beberapa brand layanan internet?

Desain multi-brand sejak awal akan sangat membantu. Pisahkan tenant, branding, dan rules tertentu, tetapi tetap jaga core model agar tidak terduplikasi liar.

Checklist Singkat Sebelum Go-Live

[x] Role dan permission sudah diuji
[x] API pelanggan, paket, invoice, dan tiket berjalan stabil
[x] Invoice memiliki histori dan status yang jelas
[x] Audit trail aktif
[x] Monitoring error dan latency tersedia
[x] Backup database dan recovery plan siap
[x] Dokumentasi endpoint tersedia untuk tim internal

Sistem yang Tumbuh Bersama Operasional

Pada akhirnya, backend yang bagus bukan yang paling banyak service-nya, tetapi yang paling jelas memetakan proses bisnis ke alur data yang aman, terukur, dan mudah dikembangkan. Itu sebabnya pembahasan backend bisnis isp menjadi relevan bukan hanya untuk engineer telco, tetapi juga untuk developer yang ingin membangun aplikasi operasional yang benar-benar dipakai setiap hari.

Ada satu kutipan terkenal dari Martin Fowler yang selalu relevan untuk konteks ini: “Any fool can write code that a computer can understand. Good programmers write code that humans can understand.” Artinya, siapa pun bisa menulis kode yang dimengerti komputer, tetapi programmer yang baik menulis kode yang juga dimengerti manusia. Martin Fowler dikenal luas sebagai salah satu figur modern paling berpengaruh dalam dunia software architecture, refactoring, dan desain sistem. Dalam konteks backend ISP, kutipan ini mengingatkan bahwa sistem yang dipakai lintas divisi harus mudah dipahami, mudah dirawat, dan tidak membuat tim operasional bergantung pada satu-dua orang saja.

Jika Anda sedang mengeksplorasi bagaimana infrastruktur konektivitas digital dan sistem operasional yang rapi bisa berjalan beriringan, Anda bisa melihat lebih jauh profil dan ekosistem layanan yang dikembangkan oleh PT Jaringan Lintas Artha sebagai salah satu contoh bisnis yang berfokus pada pengembangan jaringan internet fiber dan layanan konektivitas digital.

How APIs Actually Work (Simple Explanation for Developers)

Mon, 09 Mar 2026 13:34:25 +0000

Most modern applications are powered by APIs.

Mobile apps, websites, and backend systems all communicate through them.

But many developers still find APIs confusing when they first start learning backend development.

I just published a video explaining how APIs actually work using simple real-world examples.

Topics covered:
• API requests and responses
• HTTP methods
• Login system example
• How apps communicate with servers

Thinking auth in producation systems a dive on the backend

Sat, 07 Feb 2026 15:07:52 +0000

When building an application, authentication is unavoidable. You can outsource it to services like Clerk or Better Auth and be productive fast, but that convenience often comes with trade-offs: third-party lock-in, limited control over flows, and opaque behavior when things go wrong.

In production systems, authentication is not just about logging users in. It shapes your security model, influences your database design, and quietly dictates how scalable and debuggable your system will be.

In this article, I’ll walk through how I designed and implemented authentication for a real production project, the decisions I had to make, and the compromises that came with them. Since all systems have different auth requirements, it might vary from your version.

Requirements for my project

For this project, authentication needs to be intentionally simple:
email + password–based registration and login, with Google OAuth as an alternative for both signup and sign-in.

If a user registers using email and password, the email must be verified via a verification link. Once the email is verified, the user is automatically logged in. The frontend receives the required tokens and user context without forcing an additional login step.

Strategy

For this project, I’m using a JWT-based authentication approach with short-lived access tokens and refresh tokens, with refresh token rotation enabled.

private async generateTokens(
    userId: string,
    email: string,
    role: number,
    deviceId: string,
  ) {
    const accessTokenSecret =
      this.configService.get<string>('JWT_ACCESS_SECRET');

    const refreshTokenSecret = this.configService.get<string>(
      'JWT_REFRESH_TOKEN_SECRET',
    );

    const accessTokenExpiryRaw = this.configService.get<string>(
      'JWT_ACCESS_TOKEN_EXPIRE_TIME',
      '1h',
    );
    const refreshTokenExpiryRaw = this.configService.get<string>(
      'JWT_REFRESH_TOKEN_EXPIRE_TIME',
      '7d',
    );

    const accessTokenTtlMs = ms(accessTokenExpiryRaw);
    const refreshTokenTtlMs = ms(refreshTokenExpiryRaw);

    const [accessToken, refreshToken] = await Promise.all([
      this.jwtService.signAsync(
        { sub: userId, email, role },
        {
          secret: accessTokenSecret,
          expiresIn: Math.ceil(accessTokenTtlMs / 1000),
        },
      ),
      this.jwtService.signAsync(
        { sub: userId, email, role },
        {
          secret: refreshTokenSecret,
          expiresIn: Math.ceil(refreshTokenTtlMs / 1000),
        },
      ),
    ]);

    // Extract the signature part of the JWT
    const accessTokenSignature = accessToken.split('.')[2];

    // Keyv expects TTL in ms - cache key includes deviceId for device-specific validation
    await this.cacheManager.set(
      `${CacheKey.ACCESS_KEY}:${userId}:${deviceId}`,
      { tokenSignature: accessTokenSignature, deviceId },
      accessTokenTtlMs,
    );

    return { accessToken, refreshToken };
  }

To minimize authentication lookup latency, access tokens are cached in Redis, mapped to the corresponding user_id and device_id. This allows the system to validate active sessions without hitting the primary database on every authenticated request.

Device Identification
Each client device is assigned a unique deviceId, generated on the backend.

My first approach was to derive this identifier from the IP address and user agent. From a security perspective, this worked well. Tokens were tightly bound to their original context, and replaying them from elsewhere was difficult.

In practice, though, IP addresses change far more often than you’d expect, especially for mobile users. The result was unnecessary session invalidation and a poor user experience.

After weighing those trade-offs, I switched to deriving the deviceId from the user ID and user agent instead.

The goal of the deviceId is simple:

Avoid duplicate session entries in Redis when a user logs in multiple times from the same device
Make session tracking and revocation possible on a per-device basis
Add an extra check during token validation beyond signature and expiry
By anchoring sessions to a deterministic deviceIdRedis maintains a single active access-token entry per device rather than accumulating redundant session data.

In practice, access and refresh tokens are validated not just by their cryptographic signature and expiration, but also by the reason the deviceId they were issued for. This means that even if a token is leaked, it can’t be trivially replayed from a completely different device context unless the attacker also replicates the same user agent.

  private generateDeviceFingerprint(userId: string, userAgent: string): string {
    const raw = `${userId}|${userAgent}`;
    return crypto
      .createHash('sha256')
      .update(raw)
      .digest('hex')
      .substring(0, 32);
  }

User agents aren’t secret, and this is not meant to be a perfect defense. What it does provide is a meaningful reduction in the blast radius of token leakage and protection against a large class of naive replay attacks.

This approach intentionally trades stronger environmental binding for stability and usability. For this system, that trade-off is acceptable. The deviceId isn’t a cryptographic fingerprint; it’s a pragmatic mechanism to keep sessions predictable, revocable, and reasonably safe in production.

Schema design

For authentication, the core requirements can be met with just two tables: one to represent the user, and another to track login sessions via refresh tokens.

Table users {
  id        uuid PK        // uuidv7
  name      string
  email     string UNIQUE NOT NULL
  password  string null // hashed password
  role      int NOT NULL   // 0 = user, 1 = admin, 2 = super_admin
  created_at timestamp
  updated_at timestamp
}
Table user_login_tokens {
  id                     uuid PK
  user_id                uuid FK -> users.id
  device_id              string
  refresh_token_signature string NOT NULL
  user_agent             string
  ip_address             string
  expires_at             timestamp NOT NULL
  revoked_at             timestamp
  created_at             timestamp
}

Rate Limiting & Abuse Protection
Authentication endpoints are a natural target for abuse, so basic rate limiting is enforced at the infrastructure level using Redis.

Rather than applying a single global limit, rate limiting is scoped by action and tracked independently by IP address and email. This allows the system to respond differently to suspicious behavior without punishing legitimate users.

The following flows are protected:

Login attempts.
Registration attempts
Forgot password requests
Each flow has its own threshold, time window, and temporary block duration.

How It Works

For every request, the system checks whether the IP address or email is currently blocked for the given action. If either is blocked, the request is rejected immediately.

On failed attempts, counters are incremented in Redis with a short TTL. Once the configured threshold is exceeded, a temporary block is applied, and the counter is cleared. Successful authentication clears any existing attempt counters.

This keeps rate limiting fast, stateless, and independent of the primary database.

Here’s an example:

this.loginConfig = {
  maxAttempts: this.configService.get<number>('AUTH_LOGIN_MAX_ATTEMPTS', 5),
  windowMs: this.configService.get<number>('AUTH_LOGIN_WINDOW_MS', 60000),
  blockDurationMs: this.configService.get<number>('AUTH_LOGIN_BLOCK_MS',
    60000,
  ),
};

// Registration: 3 attempts per 5 minutes, blocked for 5 minutes
this.registerConfig = {
  maxAttempts: this.configService.get<number>(
    'AUTH_REGISTER_MAX_ATTEMPTS',
    3,
  ),
  windowMs: this.configService.get<number>(
      'AUTH_REGISTER_WINDOW_MS',
      300000,
  ),
  blockDurationMs: this.configService.get<number>(
      'AUTH_REGISTER_BLOCK_MS',
      300000,
    ),
};

// Forgot Password: 3 attempts per 5 minutes, blocked for 5 minutes
this.forgotPasswordConfig = {
   maxAttempts: this.configService.get<number>(
      'AUTH_FORGOT_MAX_ATTEMPTS',
      3,
   ),
   windowMs: this.configService.get<number>('AUTH_FORGOT_WINDOW_MS', 300000),
   blockDurationMs: this.configService.get<number>(
     'AUTH_FORGOT_BLOCK_MS',
     300000,
   ),
}

Token Lifecycle & Rotation
Access tokens in this system are intentionally short-lived. Any long-running session relies on refresh tokens, which makes the refresh flow a critical security boundary.

To handle this safely, refresh tokens are rotated on every use.

Refresh Flow

When a client requests a new access token using a refresh token, the system performs the following steps inside a database transaction:

Revoke the existing refresh token
The refresh token used for the request is immediately marked as revoked. This ensures that a refresh token can only be used once.
Re-evaluate the device context
The request’s device information is re-derived on the backend, ensuring that the new tokens are bound to the correct deviceId.
Issue a new token pair
A fresh access token and refresh token are generated and returned to the client.
Persist the new refresh token
The new refresh token is stored in the database, replacing any existing token for the same user-device combination.
This entire flow is executed atomically. Either all steps succeed, or none do.

Why Rotation Matters

Refresh token rotation prevents an entire class of replay attacks.

If a refresh token is leaked and reused:

The first request succeeds and rotates the token
Any subsequent reuse of the old token fails because it has already been revoked
This ensures that a compromised refresh token has a very limited window of usefulness.

Failure Behavior

If a refresh token is revoked, expired, or does not match the stored signature for the given device, the refresh request is rejected. At that point, the client must re-authenticate.

This makes refresh token misuse noisy, detectable, and self-limiting.

Design Trade-off

This approach slightly increases complexity compared to static refresh tokens, but it dramatically improves security and debuggability.

Token rotation turns refresh tokens from long-lived secrets into single-use credentials, which aligns much better with production threat models.

logout and token revocations
Logout is handled server-side.

When a user logs out:

The refresh token for the current session is revoked in the database
The cached access token for the corresponding userId and deviceId is removed from Redis
After this, the session cannot be refreshed, and any in-flight access tokens are rejected.

Logout is scoped to the current device. Other active sessions remain unaffected unless explicitly revoked.

async logout(userId: string, refreshToken: string, request: Request) {
    return await this.sequelize.transaction(async (t) => {
      const dbOptions = { transaction: t };
      // Revoke refresh token in database
      await this.tokenRepository.revokeRefreshToken(
        userId,
        refreshToken,
        dbOptions,
      );

      // Clear access token from cache
      // Extract device info and clear specific device cache

      const { deviceId } = this.extractDeviceInfo(request, userId);
      deleteFromCache(userId, deviceId)

      return null;
    });
  }

Email Verification Token Handling

On events like registration, or forgot password, or a verification token is created, which is sent through email to the user. where the web passes that token along with the other information, i.e., the new password field, to change the password.

These tokens live shortly in Redis; on successful request processing, these tokens are deleted, and the database is updated properly.

Password reset tokens are intentionally not bound to a device. These flows are designed to work across devices and browsers, and binding them to a device context would introduce unnecessary friction without meaningful security benefits.

// a generic implementation without rate limiting, since it's getting long
 async resetPassword(email: string, token: string, newPassword: string) {
    let payload: any;
    try {
      payload = await this.jwtService.verifyAsync(token, {
        secret: this.configService.get<string>('JWT_VERIFICATION_SECRET'),
      });
    } catch {
      throwBadRequest('auth.INVALID_OR_EXPIRED_TOKEN');
    }
    if (payload.email !== email || payload.type !== 'password_reset') {
      throwBadRequest('auth.INVALID_CREDENTIALS');
    }
    const tokenSignature = token.split('.')[2];
    const cacheKey = `reset_password_token:${tokenSignature}`;
    const userId = await this.cacheManager.get<string>(cacheKey);
    if (!userId) {
      throwBadRequest('auth.INVALID_OR_EXPIRED_TOKEN');
    }
    const hashedPassword = this.hashPassword(newPassword);
    await this.usersService.updateById(userId, { password: hashedPassword });
    await this.cacheManager.del(cacheKey);
    return null;
  }

Closing thoughts
This authentication system is built around clear trade-offs rather than generic patterns. Short-lived access tokens, refresh token rotation, per-device sessions, explicit revocation, and targeted rate limiting together provide predictable behavior and reasonable security without unnecessary complexity.

Follow me on X (formerly known as Twitter): https://x.com/Itstheanurag

My SQL Mistakes Almost Broke Me… But I Learned a Lot

Thu, 16 Oct 2025 21:45:41 +0000

Introduction

Let me be honest... when I first started working with SQL, I made a LOT of mistakes. From messing up queries to accidentally deleting test data, it was exhausting. But each error taught me something valuable about coding, debugging, and patience.

My Biggest SQL Oops Moments

1️⃣ The Missing WHERE Clause
I once ran a DELETE query without a WHERE condition… and yes, I wiped out an entire table. Lesson: Always double-check before hitting execute.

2️⃣ Incorrect Joins
I joined two tables and got a result set that made no sense. After some digging, I realized I was joining on the wrong column. Lesson: Know your schema before writing joins.

3️⃣ Data Type Confusion
I tried comparing numbers stored as strings and got unexpected results. Lesson: Always know your data types.

4️⃣ Installation Woes
At first, I was super excited to install MySQL manually in VS Code. But I hit one stubborn error after another. Even ChatGPT couldn’t save me! Lesson: Tech can be frustrating — patience is key.

What I Learned

SQL mistakes aren’t failures .. they’re learning opportunities.
Test your queries on small datasets first.
Keep backups or work in safe environments.
Document what works and what doesn’t ..it saves future headaches.
Tech journeys are full of errors, tears, and breakthroughs ..and that’s okay!

Advice to Beginners

If you’re new to SQL or databases, know that everyone struggles at first. Don’t give up. Mistakes are part of learning. Patience, persistence, and community support are your best friends.

Let’s Connect

I’d love to hear your SQL horror stories or tips for avoiding mistakes! Share them with me on:
dev.to • GitHub • LinkedIn • X/Twitter

Signature
The Duchess of Hackers
Full-Stack Developer | Digital Marketer

My journey from beginner to backend developer, sharing lessons, projects, and tips.

Tue, 14 Oct 2025 23:23:12 +0000

Title: From Beginner to Backend Developer: My One-Year Journey

Introduction

Hi, I’m The Duchess of Hackers, a Full-Stack Developer and Digital Marketer. Almost a year ago, I started my journey into backend development alongside my studies in Marketing & IT. It’s been full of challenges, discoveries, and growth here’s what I’ve learned.

Why I Started Learning Backend

I wanted to:

Understand how applications work behind the scenes
Build real-world projects
Combine marketing and tech knowledge to create solutions

I started with Python, moved to Flask, and gradually explored databases, APIs, and backend logic.

Key Projects and Lessons Learned

One of my first projects was a simple ATM simulator in Python. Through this, I learned:

Basic programming logic and control flow
How to structure a small project
The importance of testing and debugging

Each small project helped me gain confidence and prepared me for bigger challenges.

Tools and Resources That Helped Me

Languages & Frameworks: Python, Flask
Version Control: Git & GitHub
Development Environment: VS Code, WSL/Ubuntu
Learning Platforms: Online tutorials, coding communities, and practice challenges

Tip for beginners: Break problems into small, manageable steps, practice consistently, and don’t be afraid to ask for help.

Biggest Takeaways

Learning to code is a journey, not a race
Community support and feedback are invaluable
Document your projects and share your learning it helps both you and others

Call to Action

I hope my journey inspires other beginners to start coding, build projects, and keep growing.

If you’re starting your coding journey, I’d love to hear about your first project or challenge share it in the comments or connect with me on dev.to, GitHub, or LinkedIn.

Signature
Full-Stack Developer | Digital Marketer
The Duchess of Hackers
LinkedIn | GitHub | dev.to