Coder Legion - Recent posts tagged django

Verifying Razorpay Payments: The Order ID You Shouldn't Trust

Fri, 30 Oct 2026 03:57:57 +0000

TL;DR: A valid Razorpay signature proves the payment is real. It does not prove the payment belongs to the order you're about to mark as paid. Verify against the Razorpay order ID you stored on the server, never the one the client sends, and back it up with a webhook.

Razorpay's checkout flow ends with your server checking a signature. Most tutorials show that check in a few lines, and the check itself is correct. But I recently reviewed a store backend before launch where the check passed and still let one payment mark a different order as paid. The bug wasn't in the cryptography. It was in where one input came from.

How the flow works

Your server creates a Razorpay order for the amount and stores its ID (order_xyz) on your own order.
The browser opens Razorpay Checkout with that order ID.
On success, Checkout hands the browser three values: razorpay_order_id, razorpay_payment_id and razorpay_signature.
The browser sends them to your server, which checks that the signature is HMAC-SHA256(order_id + "|" + payment_id, key_secret).

Only Razorpay and your server know key_secret, so a valid signature proves that Razorpay processed that payment for that Razorpay order.

The bug

The endpoint looked roughly like this:

def verify_payment(request):
    order = Order.objects.get(id=request.data["order_id"], user=request.user)
    rzp_order_id = request.data["razorpay_order_id"]      # from the client
    rzp_payment_id = request.data["razorpay_payment_id"]
    signature = request.data["razorpay_signature"]

    expected = hmac.new(secret, f"{rzp_order_id}|{rzp_payment_id}".encode(),
                        hashlib.sha256).hexdigest()
    if expected != signature:
        return error("Payment verification failed")

    order.payment_status = "paid"
    order.save()

Every Razorpay value is checked, but nothing ties them to this order. The internal order_id and the Razorpay order ID both come from the client, and nothing checks that they belong together.

So a buyer can:

Create a cheap order A and pay ₹99 for it.
Create an expensive order B and close the checkout without paying.
Send A's genuine razorpay_order_id, razorpay_payment_id and razorpay_signature with B's internal order_id.

The signature is genuine, so it verifies, and order B is marked as paid.

The fix

Verify against the Razorpay order ID you stored when you created the order, not one the client sends. Compare in constant time, and make the update idempotent:

import hashlib
import hmac

from django.conf import settings
from django.db import transaction


@transaction.atomic  # required: select_for_update() only locks inside a transaction
def verify_payment(request):
    order = Order.objects.select_for_update().get(
        id=request.data["order_id"], user=request.user
    )
    if order.payment_status == "paid":
        return ok(order)  # idempotent: retries and double-clicks are fine

    payment_id = request.data.get("razorpay_payment_id", "")
    signature = request.data.get("razorpay_signature", "")
    expected = hmac.new(
        settings.RAZORPAY_KEY_SECRET.encode(),
        f"{order.razorpay_order_id}|{payment_id}".encode(),  # stored, not sent
        hashlib.sha256,
    ).hexdigest()

    if not order.razorpay_order_id or not hmac.compare_digest(expected, signature):
        return error("Payment verification failed")

    order.payment_status = "paid"
    order.razorpay_payment_id = payment_id
    order.save(update_fields=["payment_status", "razorpay_payment_id"])
    return ok(order)

If you use the official SDK, client.utility.verify_payment_signature(...) does the HMAC for you. Just pass it your stored razorpay_order_id.

Don't rely on the browser at all: add the webhook

Even a correct verify endpoint depends on the buyer's browser getting back to your site. People close tabs, networks drop, and phones lock mid-redirect. The money arrives, but the order stays "pending".

Razorpay webhooks fix that. Subscribe to payment.captured (or order.paid) and treat the webhook as the source of truth:

import hashlib
import hmac
import json

from django.conf import settings
from django.http import HttpResponse
from django.views.decorators.csrf import csrf_exempt
from django.views.decorators.http import require_POST


@csrf_exempt
@require_POST
def razorpay_webhook(request):
    body = request.body  # raw bytes, before any JSON parsing
    received = request.headers.get("X-Razorpay-Signature", "")
    expected = hmac.new(settings.RAZORPAY_WEBHOOK_SECRET.encode(), body,
                        hashlib.sha256).hexdigest()
    if not hmac.compare_digest(expected, received):
        return HttpResponse(status=400)

    event = json.loads(body)
    if event["event"] == "payment.captured":
        payment = event["payload"]["payment"]["entity"]
        Order.objects.filter(
            razorpay_order_id=payment["order_id"], payment_status="pending"
        ).update(payment_status="paid", razorpay_payment_id=payment["id"])
    return HttpResponse(status=200)

Note three things:

The signature is computed over the raw body with the webhook secret, which is a different secret from your API key secret.
The update is keyed by the Razorpay order ID that Razorpay itself sent, and it only moves pending to paid, so duplicate deliveries do nothing. (If the handler does more than a status update, such as sending emails or reducing stock, also de-duplicate on the X-Razorpay-Event-Id header.)
The browser flow and the webhook can now race safely. Whichever arrives first marks the order paid, and the other is a no-op.

Checklist

[ ] The amount is set on the server when the Razorpay order is created, never taken from the client.
[ ] The stored razorpay_order_id is used for verification, not the one in the request.
[ ] Signatures are compared with hmac.compare_digest.
[ ] Marking an order as paid is idempotent.
[ ] A payment.captured webhook with its own secret is the source of truth.
[ ] There's a reconciliation job for orders stuck in "pending".

Payment bugs rarely come from broken crypto. They come from trusting an input that looked like it was already verified.

I'm Paritosh, a lead full-stack engineer working with React, Next.js, Django and FastAPI. I write about the bugs and design decisions I run into while shipping products. This post first appeared on my blog. More of my work: portfolio · GitHub.

How to Protect a Flask or Django App When the WAF Is a Reverse Proxy

Wed, 28 Oct 2026 02:30:19 +0000

Python teams usually start web security inside the framework. Django has auth, CSRF middleware, and an ORM. Flask has signed sessions, request hooks, and whatever validators you added. Those controls still matter. They are not the same thing as a web application firewall sitting on the HTTP path. If the question is how to protect a Python web app such as Flask or Django with a WAF, the SafeLine documents support a placement decision, not a language plugin. SafeLine is a self-hosted WAF that filters and monitors HTTP traffic between a web application and the Internet, and it works as a reverse proxy so clients reach the WAF before they reach the server (https://docs.waf.chaitin.com/en/home). The application guide says the same thing in operational terms: SafeLine receives traffic for the original site, cleans malicious traffic, and forwards safe traffic to the upstream application (https://docs.waf.chaitin.com/en/GetStarted/AddApplication). Those pages do not describe a pip package, a Flask extension, or a Django middleware setting. Do not invent that integration. Decide whether a reverse-proxy shield fits the app you already run.

Start with the entry path, not the view function. Write down the public hostname, the port clients use, and the real upstream that should stay private. In the SafeLine console, an application record takes a domain, hostname, or IP, a listen port such as 80 or 443, and an upstream address. HTTPS apps use the SSL option (https://docs.waf.chaitin.com/en/GetStarted/AddApplication). After that, the domain must resolve to the SafeLine server. A Gunicorn process, a uWSGI socket, or an existing origin Nginx can remain the upstream. The change is that the public door moves. If a load balancer already terminates TLS, map that hop explicitly. Two public doors means the proxy is optional, and an optional WAF does not protect the app.

Then check the host before you promise a cutover. The deploy guide requires Linux, x86_64 or arm64, Docker 20.10.14 or above, Docker Compose 2.0.0 or above, a CPU with ssse3, and at least 1 core, 1 GB of memory, and 5 GB of disk (https://docs.waf.chaitin.com/en/GetStarted/Deploy). The same page states that a SafeLine install on ARM requires a Pro license and that the Personal Edition is not supported on ARM. A small Flask demo on a laptop does not waive those constraints. If the host fails the check, fix the platform first. Filtering rules will not help if the proxy cannot run.

Match the documented attack list to routes you actually expose. The introduction names SQL injection, XSS, code injection, OS command injection, CRLF injection, LDAP injection, XPath injection, RCE, XXE, SSRF, path traversal, backdoor, bruteforce, HTTP flood, and bot abuse (https://docs.waf.chaitin.com/en/home). A Django admin search box, a Flask query parameter, a file download, and a webhook that fetches a URL are different code paths, but they are still HTTP. The proxy is useful because it inspects that traffic before the view runs. It is not useful as a substitute for parameterized queries, template escaping, or a view that shells out to a user-controlled string. Keep those fixes in the app.

The same introduction lists core capabilities beyond signature-style blocking: defenses for web attacks, proactive bot abuse defense, HTML and JS code encryption, IP-based rate limiting, and a web access control list (https://docs.waf.chaitin.com/en/home). The feature list then describes rate limiting for traffic that exceeds defined limits, including DoS, bruteforce attempts, traffic surges, and other abuse. It also describes a CAPTCHA challenge intended to allow human users while blocking crawlers and bots, an authentication challenge that requires a password, and dynamic protection that encrypts HTML and JS on each visit when it is turned on. Treat those as perimeter switches in the WAF, not as Django settings or Flask blueprints. The supplied pages do not document numeric thresholds, so do not copy invented limits into a runbook. Turn a control on, watch real traffic, and keep the strictest control off any path you have not tested.

A published evaluation on 33,669 samples is the only benchmark figure these sources provide. In that table, the SafeLine Balance row shows 71.65% detection, a 0.07% false positive rate, and 99.45% accuracy. The SafeLine Strict row shows 76.17% detection, a 0.22% false positive rate, and 99.38% accuracy (https://docs.waf.chaitin.com/en/home). Use the table as a published comparison, not as a guarantee for your login form or JSON API. The Strict row detects more and also shows a higher false positive figure. That is a reason to replay ordinary browser flows, API clients, and file uploads after any policy change.

Prove the proxy with your own hostname. The application guide publishes sample probes for SQL injection, XSS, path traversal, code injection, and XXE, and it says a block page means the attack was stopped. Details then appear in the console attack list (https://docs.waf.chaitin.com/en/GetStarted/AddApplication). Run those patterns only against an application you are authorized to test, with the sample host replaced by your domain. A block proves the request passed through SafeLine. It does not prove that every Django permission check or Flask webhook is correct. Follow with a clean login, a normal form post, and a known-good API call. If a legitimate request is blocked, inspect the detail view before you weaken the whole site.

The decision is complete when four statements are true. The app is reached over HTTP or HTTPS. One public name points at SafeLine, and the upstream is the real Flask or Django origin. The host meets the documented minimums, including the ARM license note if that architecture applies. Someone owns the attack list after cutover. If one statement is false, harden the Python app and wait. A reverse proxy is the supported path. A framework plugin is not.

Does SafeLine ship a Flask or Django integration?

No. The install and application guides describe a reverse-proxy WAF with a domain, a listen port, and an upstream. They do not document a Python package or a framework setting (https://docs.waf.chaitin.com/en/GetStarted/AddApplication).

Which request classes should a Python team map first?

Start with the attacks named in the product introduction: SQL injection, XSS, code injection, command injection, SSRF, path traversal, bruteforce, HTTP flood, and bot abuse are the ones most teams can map to a route (https://docs.waf.chaitin.com/en/home).

Are rate limiting and CAPTCHA part of the app code?

No. Rate limiting, CAPTCHA challenge, authentication challenge, and dynamic HTML and JS protection are WAF features described on the product page (https://docs.waf.chaitin.com/en/home). Keep CSRF, sessions, and permissions in Flask or Django.

What proves the cutover worked?

Point the domain at the SafeLine server, replay the documented probe patterns against your own site, expect a block page, and open the attack detail in the console (https://docs.waf.chaitin.com/en/GetStarted/AddApplication). Then confirm normal user and API traffic still passes.

What host checks are mandatory?

Linux, x86_64 or arm64, Docker 20.10.14 or newer, Docker Compose 2.0.0 or newer, ssse3, and at least 1 CPU core, 1 GB memory, and 5 GB disk. ARM also requires a Pro license, and the Personal Edition is not supported there (https://docs.waf.chaitin.com/en/GetStarted/Deploy).

SafeLine has surpassed 700,000 installations worldwide as of September 30, 2026. SafeLine Pro annual promotional pricing is $650-$880, compared with the regular $1,000 per year.

SafeLine website: https://ly.cyberserval.tech/MqhYdmI
Demo: https://demo.waf.chaitin.com:9443/statistics
Official docs: https://docs.waf.chaitin.com/en/home
GitHub: https://github.com/chaitin/SafeLine
Pro offer: https://cyberserval.tech/landing/lto

How to Decide If a Flask or Django App Needs a WAF

Wed, 28 Oct 2026 02:29:47 +0000

Protecting a Flask or Django application with a WAF is a placement decision, not a framework-plugin decision. The SafeLine introduction describes a self-hosted web application firewall that filters and monitors HTTP traffic between a web application and the Internet (https://docs.waf.chaitin.com/en/home). Those pages do not name Flask, Django, WSGI, or ASGI, and they do not document a Python package, middleware class, or SDK. If the requirement is an in-process integration, these sources do not support that promise. What they do support is a reverse-proxy checklist: confirm the host, put inspection in front of the existing process, forward only accepted traffic upstream, and prove a block before calling the app protected.

SafeLine is documented as an HTTP and HTTPS reverse proxy developed on nginx. It receives traffic for the original site, cleans malicious attack traffic, and forwards safe traffic to the original application (https://docs.waf.chaitin.com/en/GetStarted/AddApplication). The same introduction distinguishes this from a forward proxy. A forward proxy hides a client. This WAF sits in front of the server so clients pass through it before they reach the application (https://docs.waf.chaitin.com/en/home). For Python, the useful reading is narrow. The Flask or Django process remains the upstream. The language of that process is not a documented configuration field.

Placement checks

Start with exposure, not with application code. In the web console, adding an application asks for Domain, Port, and Upstream. Domain may be a domain name, a hostname, or an IP address. Port is the port SafeLine listens on, such as 80 or 443, and HTTPS applications need the SSL option checked. Upstream is the real address of the original app, which is where traffic goes after inspection. After those fields are saved, the domain you entered must resolve to the IP of the server running SafeLine (https://docs.waf.chaitin.com/en/GetStarted/AddApplication). That is the documented protection path. It does not include editing a Django settings module, registering a Flask before-request hook, or installing a Python dependency.

Treat any assumption about process managers as out of scope. The sources do not require gunicorn, uWSGI, or a particular application server. They require a reachable upstream address. Keep the Python process on the address you enter, and do not invent header, health-check, or load-balancing behavior that the application guide never states.

Host checks

Run the host checklist before telling anyone the app is covered. The deploy guide requires Linux, x86_64 or arm64, Docker 20.10.14 or above, and Docker Compose 2.0.0 or above. The minimum environment is 1 CPU core, 1 GB of memory, and 5 GB of disk, and the CPU must support ssse3 (https://docs.waf.chaitin.com/en/GetStarted/Deploy). An ARM server has a further constraint in that same guide: SafeLine installed on ARM requires a Pro license to work properly, and the Personal Edition is not supported on ARM. If any item fails, stop. A Python app behind an unfinished install is not a protected app.

What you can claim

Map documented detections to the routes you actually expose, and leave unsupported claims off the list. The introduction lists defenses for SQL injection, XSS, code injection, OS command injection, CRLF injection, LDAP injection, XPath injection, RCE, XXE, SSRF, path traversal, backdoor, brute force, HTTP flood, and bot abuse. Its core capabilities also include proactive bot-abuse defense, HTML and JS code encryption, IP-based rate limiting, and a web access control list (https://docs.waf.chaitin.com/en/home). The feature list adds rate limiting for DoS attacks, brute-force attempts, traffic surges, and other abuse by throttling traffic that exceeds defined limits. It also describes a CAPTCHA challenge that allows human users while blocking crawlers and bots, an authentication challenge that blocks visitors who do not enter the password, and dynamic protection that encrypts HTML and JS on each visit (https://docs.waf.chaitin.com/en/home).

Those controls can sit beside a Python app, but they are not framework fixes. SQL injection filtering does not replace parameterized queries. XSS filtering does not replace template escaping. Path-traversal and command-injection filtering do not replace safe file and process handling. Rate limiting can sit in front of a login route, but the supplied documents do not publish a Flask-specific or Django-specific threshold, so this checklist must not invent one. The authentication challenge is easy to misread. It is a visitor password gate, not a Django authentication backend and not a Flask session login. Dynamic protection is documented for HTML and JS, so do not assume it rewrites a JSON response body.

What stays in the app

Keep application controls on the same checklist as proxy controls. SafeLine is described as filtering, monitoring, and blocking malicious HTTP traffic, and as preventing unauthorized data from leaving the app by applying policies that separate malicious traffic from safe traffic (https://docs.waf.chaitin.com/en/home). That is a traffic decision. Authorization bugs, weak session secrets, leaked configuration, and vulnerable dependencies can still arrive as requests that look ordinary. CSRF checks, permission checks, input validation, and secret handling stay in Flask or Django after the proxy is live.

Proof before extra controls

Verify with the cases the application guide already publishes, and only against a site you control. The examples cover SQL injection, XSS, path traversal, code injection, and XXE as query-string tests. Replace the sample domain with your own. If the blocking page appears, the guide treats that as evidence that the attack was defended. Then open the console attack list and click detail for the event (https://docs.waf.chaitin.com/en/GetStarted/AddApplication). A checklist is incomplete if the only proof is that the homepage still loads. Legitimate traffic must pass, and a known-bad request from that documented set must be recorded.

Turn on extra visitor challenges only after that path is proven. Rate limiting, CAPTCHA, the authentication challenge, and dynamic HTML and JS encryption all change what a normal user experiences. The documents describe what each control does. They do not give a Python preset or claim that one mode fits every Django admin and every Flask API. Choose the control that matches an abuse you can name: floods and brute force for rate limiting, bot traffic for CAPTCHA, a shared gate password for the authentication challenge, and HTML or JS exposure for dynamic protection. If you cannot name the abuse, leave the extra control off.

The documentation states that SafeLine is production-ready (https://docs.waf.chaitin.com/en/home). That statement does not add a Flask or Django connector, and it does not replace the checks above. Decide yes only when the host meets the deploy requirements, the application entry points at a real upstream, DNS lands on the SafeLine server, a documented attack is blocked and visible in the console, and the app still enforces its own authorization and input rules.

Does SafeLine document a Flask or Django plugin?

No. The install guide and the application guide describe deployment and an HTTP reverse proxy with Domain, Port, and Upstream. They do not document a Python package or framework middleware (https://docs.waf.chaitin.com/en/GetStarted/Deploy). Do not plan on an in-app integration that these sources never describe.

Where does the Python process belong?

It belongs at the upstream address. Clients should reach the domain that resolves to the SafeLine server. SafeLine listens on the configured port, inspects the request, and forwards safe traffic to the real application address (https://docs.waf.chaitin.com/en/GetStarted/AddApplication). The framework can keep serving on the address you configured as upstream.

No. Nothing in the cited documents says a WAF replaces application authorization, template escaping, or safe query construction. Use the proxy to filter documented web attacks and abuse patterns, and keep those controls in the app (https://docs.waf.chaitin.com/en/home).

How should I test without inventing payloads?

Use the five cases published in the application guide: SQL injection, XSS, path traversal, code injection, and XXE. Point them at your own protected domain. Expect the blocking page, then confirm the event in the console attack list and open its detail (https://docs.waf.chaitin.com/en/GetStarted/AddApplication).

The authentication challenge. When it is turned on, visitors must enter the password or they are blocked (https://docs.waf.chaitin.com/en/home). That is a site gate. Flask and Django user accounts remain a separate system, and this checklist should not merge the two.

SafeLine has surpassed 700,000 installations worldwide as of September 30, 2026. SafeLine Pro annual promotional pricing is $650-$880, compared with the regular $1,000 per year.

SafeLine website: https://ly.cyberserval.tech/MqhYdmI
Demo: https://demo.waf.chaitin.com:9443/statistics
Official docs: https://docs.waf.chaitin.com/en/home
GitHub: https://github.com/chaitin/SafeLine
Pro offer: https://cyberserval.tech/landing/lto

Is it possible to code while learning something new? Of course, it’s more fun.

Thu, 24 Sep 2026 15:05:25 +0000

Hey guys;)
Long time no see.

General

Actually I wanna change my blog format. Now I’m planning to do a few posts on X, and then 1-2 times per week do a big post here. And I wanna try not only to show what I’ve done, but also how to do it. Some kind of teaching I don’t know yet.

New project. What is it about?

Okay, so since my last post I’ve finished Yakaboo Sales Analysis. You could check it here - Yakaboo Project.
The new project is basic - it’s Secret Santa. But if I wrote just an algorithm it wouldn't be so interesting. So I decided to add Django here and make some websites. But here was a problem, I’ve never worked with Django before. But thanks to Corey Schafer I learnt some basics and even built something. I’ll show the video in the end.

A bit about Django

So it’s a great framework which allows us to build websites using Python. In the beginning I thought that I’ll explain almost everything that I’ve learnt, but I don’t think it’s necessary. So in short, to begin writing you should do

Create a project directory and venv (of course you could do it without venv).
Install Django.
Create the project using django-admin startproject name_of_project.
Create an app using django-admin startapp name_of_app

After that we could do what we wanted.

Another amazing library

One of my favorite libraries is Faker, which allows generating data. I used it for creating participants (their full name, email and wishlist) for a party. That was the easiest part, after that I started exploring Django. So, I’ll skip all the theories and say only the most important.

To write html and css inside the app directory you need to create templates (for html) and static (for css and js, about js I’m not sure), and inside these directories create a folder with the same name as app name.
Inside the views.py file you create a function which returns html response and gives arguments which we could use inside html.
After creating views functions you could use it in the url of the app, add a pattern to the project and don’t forget to add the app into project settings.

Okay now it’s time to watch some results for now.
Link to video
See you in the next post. I hope it was interesting to read.

My Django Secret Key Was Public on GitHub - How I Fixed It

Fri, 18 Sep 2026 10:07:50 +0000

We all make mistakes, especially when starting out. Recently, the ultimate developer mishap happened to me: I accidentally pushed my Django SECRET_KEY to a public GitHub repository.

In Django, the SECRET_KEY is used for cryptographic signing, like session cookies or password reset tokens. It is essential for protecting sensitive data and preventing tampering or unauthorized access. Leaving it publicly exposed is a massive NO-GO.

But that is exactly what learning projects are for! Luckily this happened to a project with no real user data. We learn the most from our mistakes, and I now know exactly how to handle this situation. Here is how to fix it:

Step 1: Invalidate the old key immediately

Simply changing the key in your code and pushing it again is not enough. The old key remains visible forever in your GitHub commit history. The first step is damage control, so we need to generate a new key. Start the Python shell in your project terminal:

python manage.py shell

Then enter the following lines:

from django.core.management.utils import get_random_secret_key
print(get_random_secret_key())

The first line imports the required utility function and the second line generates the new key and prints it to the terminal so you can copy it.

*UPDATE (September 21st 2026): Based on reader feedback and GitHub's official security guidance, I’ve updated this section. Rotating the secret key is the single most critical step. Purging your Git history with tools like git-filter-repo is a helpful next step to tidy up, but it does not guarantee absolute security, as old commits can still persist in forks or cached views on GitHub's servers.*

Step 2: Move the Secret Key to environment variables

To make sure this never happens again, I moved the key into a .env file. Create this file in your root directory and make sure to add it to your .gitignore file so it never gets tracked by GitHub.

To load the environment variables from the .env file into the Django project, we use the python-dotenv library. Install it first:

pip install python-dotenv

Next update your settings.py file like this:

import os
from dotenv import load_dotenv

load_dotenv()

SECRET_KEY = os.getenv("SECRET_KEY")

Finally store your actual key inside the .env file:

SECRET_KEY="your_new_secret_key_here"

Conclusion: Mistakes happen, it's how you handle them that matters

Thanks to this faux pas my Django project is now secure and follows configuration management best practices. My brain has permanently saved this lesson so it won't happen again! 😊

From Learning Python to Building Real Projects: My Developer Journey

Fri, 28 Aug 2026 08:13:14 +0000

Hello CoderLegion 👋🏽 — Ready to Begin My Developer Journey Here

I’m thrilled to join CoderLegion!

I’m currently on a journey of learning software development, and one core lesson I've already learned is that becoming a developer isn’t about knowing everything upfront. It’s about being willing to learn, build, make mistakes, debug, and iterate.

My journey so far has introduced me to Python, Django, APIs, databases, and more recently, JavaScript.

One of the things I enjoy most is moving beyond raw concepts and actually building functional projects. For example, I’ve been developing an AI-Powered Learning Companion in Python. The goal is to build a tool that generates study questions via an AI language model, lets users practice and take tests, evaluates answers, and tracks overall progress.

Building it has taught me that writing code is only a fraction of development understanding the problem, organizing project structure, and troubleshooting persistent errors are just as vital.

And yes… there have been plenty of bugs along the way.

I’m also diving deeper into JavaScript, which marks a strong next phase in my learning path.

I joined CoderLegion to learn from experienced developers, document the projects I’m building, share what I discover, and connect with fellow tech enthusiasts.

I’m still learning and figuring things out, but I’m proud of the progress made so far and eager to see where this path leads.

So, hello CoderLegion!

How I Moved Real-Time Out of Django (and Made Everything Simpler)

Thu, 16 Jul 2026 14:21:03 +0000

Your users want live updates: an order flips to paid, a notification pops, a dashboard number ticks up without a refresh. The obvious answer in Django is Channels. I reached for it first too. But as our scale grew, the costs of keeping connections in-process pushed me to rethink the architecture. I let Django stop handling WebSockets entirely, and almost everything got simpler. Here's the pattern that replaced it.

The Problem

Django is a request/response framework. It processes a request, returns a response, and moves on. Real-time notifications — a dashboard ticking up, an order status flipping live — need a server that holds a connection open for every connected client. That is a fundamentally different job.

The standard answer in the Django ecosystem is Channels. It moves you to ASGI, gives you consumers, and works well, especially when your real-time needs are deep and bidirectional.

Ours weren't. We needed server→client push: notify a user when an order completed, or when a payment landed. The connections were mostly idle. Channels gave us that, but it came with costs that started hurting at scale.

The concrete problem was this: every open WebSocket connection consumed memory and a small slice of CPU time on our app servers. A thousand connected dashboards was fine. Ten thousand, across dozens of tenants, each holding a persistent socket. Suddenly, our workers were doing two very different jobs at once. Serving fast API responses and babysitting long-lived, mostly-quiet connections. Both suffered.

What made it brittle was the reconnect storm. Deploy a new build, restart a server, or hit a brief network partition, and every connected client disconnects and immediately reconnects. All at once. That thundering herd of TLS handshakes, authentication, and subscription requests would spike CPU to 100% on every app server as it tried to spin up thousands of ASGI consumer instances simultaneously, occasionally knocking them over entirely before the retry backoff kicked in.

There's also the smaller but telling friction of auth. Authenticating the WebSocket handshake means reaching your existing auth (DRF/JWT) from inside an ASGI scope. I ended up writing middleware that reconstructed a DRF Request out of the raw scope just to reuse the auth I already had. It worked, but it was a sign: I was bending the framework to do something it wasn't built for.

The turning point was asking: why is my web framework in the business of holding connections at all?

For a fire-and-forget server→client push, it shouldn't be. There are servers built specifically to hold millions of open connections and do nothing else. The only two things your backend needs to do are say who's allowed to listen, and publish events.

The Solution

Put a dedicated pub/sub gateway in front of your app. I used Centrifugo, a standalone real-time server. Clients connect to it, not to Django. Django goes back to being a plain, boring WSGI/sync app and plays two small roles:

Issue a short-lived connection token that embeds which channels this user may subscribe to.
Publish events to the gateway over its HTTP API.

The gateway holds the connections. Django holds nothing.

Step 1: Issue a token that carries its own permissions

When a client wants to connect, it asks your backend (over a normal authenticated request) for a connection token. The token is a JWT whose claims list the exact channels the user may subscribe to. The gateway verifies the signature and authorizes subscriptions without a single database lookup:

# realtime/auth.py
import time
import jwt
from django.conf import settings


class RealtimeTokenService:
    """Issues short-lived JWTs for connecting to the realtime gateway."""

    def generate_connection_token(self, user) -> str:
        tenant_id = user.tenant_id

        # Channel grants are baked into the token, so the gateway authorizes
        # subscriptions with zero database lookups on connect.
        channels = [
            f"user_{user.id}",
            f"critical:user_{user.id}",
            f"tenant_{tenant_id}",
            f"critical:tenant_{tenant_id}",
        ]

        claims = {
            "sub": str(user.id),
            "exp": int(time.time()) + settings.REALTIME_TOKEN_TTL_SECONDS,
            "info": {"name": user.get_full_name(), "email": user.email},
            "channels": channels,
        }
        return jwt.encode(claims, settings.REALTIME_HMAC_SECRET, algorithm="HS256")

This is the key idea: token-as-permission. The channel names encode your authorization model. A user gets their own user_{id} channel and their tenant's tenant_{id} channel, and nothing else. Multi-tenant isolation falls out of the channel naming: a user simply has no grant for another tenant's channel, so the gateway will refuse the subscription.

Step 2: Hand the token out behind your normal auth

The endpoint that mints the token is an ordinary authenticated view. Your existing auth stack protects it; no ASGI gymnastics:

# realtime/views.py
class RealtimeTokenView(APIView):
    permission_classes = [IsAuthenticated]

    def post(self, request):
        token = RealtimeTokenService().generate_connection_token(request.user)
        return Response(
            {"token": token, "socket_url": settings.REALTIME_SOCKET_URL}
        )

The client takes {token, socket_url}, opens a WebSocket to the gateway, and subscribes to its channels. Django is now out of the connection entirely.

Step 3: Publish events over HTTP

To push an update, your backend POSTs to the gateway's publish API. No connection state, no sockets: just an HTTP call:

# realtime/publisher.py
import requests
from django.conf import settings


def push_event(channel, event_type, payload, *, recoverable=False, timeout=10.0):
    # The "critical:" namespace is configured on the gateway with message
    # history + recovery, so clients can catch up after a reconnect.
    final_channel = f"critical:{channel}" if recoverable else channel

    resp = requests.post(
        f"{settings.REALTIME_API_URL}/publish",
        json={
            "channel": final_channel,
            "data": {"type": event_type, "payload": payload},
        },
        headers={"X-API-Key": settings.REALTIME_API_KEY},
        timeout=timeout,
    )
    resp.raise_for_status()

Notice the critical: channel prefix. On the gateway, that namespace is configured to keep a short history so a client that briefly drops can recover missed messages on reconnect:

{
  "namespaces": [
    { "name": "critical", "history_size": 50, "history_ttl": "120s", "force_recovery": true }
  ]
}

Ephemeral events (a "user is typing" blip) publish to the bare channel; events you can't afford to lose (a completed order) publish to critical: and survive a reconnect.

Step 4: Make publishing reliable, and don't block the request

Publishing is a network call to a separate service. You do not want it on the request's critical path, and you want it to survive a transient blip. Wrap it in a Celery task with retries and backoff:

# realtime/tasks.py
import logging
from requests.exceptions import RequestException, HTTPError
from celery import shared_task
from .publisher import push_event

logger = logging.getLogger(__name__)


@shared_task(bind=True, max_retries=3, default_retry_delay=5, retry_backoff=True)
def publish_realtime_event(self, channel, event_type, payload, recoverable=False):
    try:
        push_event(channel, event_type, payload, recoverable=recoverable)
    except HTTPError as exc:
        # A 4xx error (bad API key, malformed payload) is a deterministic
        # failure that retries can't fix. Log it and drop it.
        # A 5xx error might be transient, so retry it.
        if 400 <= exc.response.status_code < 500:
            logger.error("Deterministic 4xx error on %s, dropping.", channel)
            return
        raise self.retry(exc=exc)
    except RequestException as exc:
        # Pure network timeouts or connection errors — retry with backoff.
        raise self.retry(exc=exc)
    except Exception:
        # A serialization bug won't fix itself on a retry. Log and drop it,
        # rather than burning three attempts on a guaranteed failure.
        # By not re-raising, Celery marks the task as successful and removes
        # it from the queue — no poison pill looping forever.
        logger.exception("Dropping unrecoverable realtime event on %s", channel)

That except split matters: retry the transient, drop the deterministic. A flaky network deserves another try; a payload that can't be JSON-encoded will fail identically three times in a row, so retrying it is just noise.

Firing an event from anywhere in your app is now a one-liner:

publish_realtime_event.delay(
    channel=f"tenant_{order.tenant_id}",
    event_type="order.completed",
    payload={"order_id": order.id, "total": str(order.total)},
    recoverable=True,
)

Why This Works

Your web tier goes back to being stateless. Django stays on WSGI. No ASGI migration, no async server, no sockets held in your workers. You can scale, restart, and deploy the web tier like any other stateless service. The open connections live on the gateway and aren't disturbed.

Connections scale independently. Holding a lot of idle connections is a specialized job. A purpose-built gateway does it on its own box, tuned for that, while your app servers stay sized for request throughput.

Auth has zero connect-time cost. Because the token carries its channel grants, the gateway never calls back into your database to authorize a subscription. Ten thousand clients reconnecting after a blip is ten thousand signature checks, not ten thousand DB queries.

Multi-tenancy is just naming. Tenant isolation isn't a separate access-control layer. It's the channel names in the token. There's no code path by which a user can subscribe to a channel they weren't granted.

Delivery is fire-and-forget but durable. Publishing happens off the request path via Celery, retries on transient failure, and the critical: namespace lets clients recover what they missed across a reconnect.

Design Decision: The Token Is a Cache, So It Can Go Stale

Baking permissions into the token is what buys you DB-free connects. But a token is a snapshot. If a user's access changes (you remove them from a tenant, say) and their token still has 24 hours to live, the gateway will keep honoring the old grants until it expires. You've traded revocation freshness for connect-time performance.

That's usually the right trade for a notification stream, but be deliberate about it:

Keep the TTL short and have the client transparently re-fetch a token. Shorter tokens = smaller stale window.
For true revocation, the gateway can be told to force-disconnect a user or refuse channels server-side. Reach for that only when a stream is sensitive enough to need it.

One more operational gotcha: a single gateway is tempting to share across environments, but tenant_42 in staging and tenant_42 in production are the same channel name: cross-talk waiting to happen. If you must share one broker, prefix channels with the environment (prod_tenant_42, staging_tenant_42) so the namespaces can't collide.

And to be honest about the path here: I shipped the Channels version first. It worked. But the friction of running an async server and bridging my existing auth into the ASGI scope is exactly what made the gateway approach worth it. If your real-time needs are deep and bidirectional (collaborative editing, presence), in-process Channels may genuinely fit better. For "push updates to subscribed clients," letting something else hold the sockets is the simpler system.

The Result

Adding a live update anywhere in the app is now one .delay() call. The web tier never learned what a WebSocket is. Connections, fan-out, history, and recovery are the gateway's problem; authorization is a signed list of channel names; delivery is a retrying background task. Django went back to doing the thing it's good at. And the real-time problem moved to a server built for it.

How are you doing real-time in your Django apps — in-process with Channels, or have you pushed connections out to a gateway? I'd love to hear what's worked for you in the comments.

¡Hasta luego!

How I Built a "Set It and Forget It" Sync System with Django Signals

Thu, 16 Jul 2026 14:20:14 +0000

Change a product's price anywhere in your app, and it instantly syncs to a third-party marketplace. No manual triggers, no polling, no fragile save() overrides. Here's the signal pattern that powers it.

The Problem

In our app, a product's name or price can change from eight different places. The edit page. Bulk import. The variant editor. A pricing rule engine. An API endpoint that processes webhooks from suppliers. Every few weeks, someone adds a new feature and creates yet another code path that mutates a product.

We needed every one of those changes, no matter where they came from, to sync to an external marketplace. The naive approach would be to add a sync call to each code path. That's eight places to maintain (and counting), eight chances to forget, and eight places that break if the sync API changes.

Django's post_save signals solve the discovery problem: hook into a model's save event and you catch every change, from every code path, in one place. Signals get a bad rap — fairly, they make control flow hard to trace. But when you need to react to changes from everywhere without touching anywhere, this specific discovery problem is exactly what they were designed for.

There's a catch, though. If ten prices change in a single request, say, a bulk import, a naive signal handler fires ten times. That's ten API calls in rapid succession. At best, you are wasting resources. At worst, the external API rate-limits you.

We needed signals to detect changes, but we needed to batch them.

The Solution

Here's the three-part pattern:

A signal handler that collects change references instead of acting on them immediately.
A per-thread set that deduplicates for free — adding the same product twice does nothing.
A flush callback deferred to transaction.on_commit that processes everything once the database transaction lands.

Step 1: Register the signal

In your app's apps.py, connect post_save to the model that holds pricing:

# shopping/apps.py
from django.apps import AppConfig

class ShoppingAppConfig(AppConfig):
    name = "shopping"

    def ready(self):
        from django.db.models.signals import post_save
        from shopping.models import PriceRecord
        from shopping.signals import on_price_change

        post_save.connect(
            on_price_change,
            sender=PriceRecord,
            dispatch_uid="shopping_price_change",
        )

dispatch_uid prevents duplicate connections if ready() runs twice, a common gotcha during development with auto-reload.

Step 2: Collect, don't act

The signal handler doesn't call an API. It just adds a reference to a set and registers a flush callback:

# shopping/signals.py
import threading
from django.db import transaction

_local = threading.local()


def _get_pending():
    if not hasattr(_local, "pending"):
        _local.pending = set()
    return _local.pending


def on_price_change(sender, instance, **kwargs):
    # Record just enough to look up the product later
    _get_pending().add(instance.object_id)
    transaction.on_commit(flush_changes)

That's it. Four lines of logic. The handler doesn't care how many prices changed or where the change came from. It just records what changed and defers action to the flush.

Notice that we use threading.local() instead of a standard module-level variable. This ensures that each thread gets its own isolated storage.

Why transaction.on_commit? If the transaction rolls back, the price change never happened, so the flush callback is discarded. You never sync data that wasn't committed.

Step 3: Flush once per transaction

When the transaction lands, the flush handler fires. It snapshots the set, clears it immediately, and processes everything in one batch:

def flush_changes():
    pending = _get_pending()
    refs = pending.copy()
    pending.clear()

    if not refs:
        return

    from shopping.services import SyncService
    SyncService.handle_price_changes(refs)

The key detail: we copy the set before clearing it. If clearing happened after processing, and processing raised an exception, stale refs would linger into future transactions. Snapshot-first is defensive.

This clear() is also what keeps requests perfectly isolated. Once a transaction commits and the flush runs, the set is empty and ready for the next request.

Step 4: Resolve and dispatch

The service layer resolves the raw references into business entities, groups them by destination, and dispatches one task per group. The grouping is the important part. One API call per store, regardless of how many products changed:

# shopping/services.py
from collections import defaultdict

class SyncService:

    @classmethod
    def _resolve_to_products(cls, refs: set[int]):
        """
        Resolve product ID references to actual Product instances
        with their Store relationship.  Fetches everything in ONE
        query using filter(id__in=...), silently ignoring stale
        IDs that no longer exist.
        """
        products = Product.objects.filter(
            id__in=refs
        ).select_related("store")

        resolved = []
        for product in products:
            resolved.append((product, product.store_id))
        return resolved

    @classmethod
    def handle_price_changes(cls, refs: set[int]):
        resolved = cls._resolve_to_products(refs)

        # Group changes by the store they belong to
        by_store = defaultdict(list)
        for product, store_id in resolved:
            by_store[store_id].append(product)

        # One API call per store, regardless of how many products changed
        for store_id, products in by_store.items():
            sync_to_marketplace.delay(store_id=store_id, products=products)

The sync_to_marketplace task is a Celery task that calls the external API. It's configured with retries and backoff for transient failures:

@shared_task(bind=True, max_retries=3, default_retry_delay=60)
def sync_to_marketplace(self, store_id, products):
    try:
        provider.bulk_update(store_id, products)
    except Exception as exc:
        raise self.retry(exc=exc)

Why This Works

Deduplication is free. A set naturally deduplicates, adding the same object_id twice is a no-op. If a price changes twice in the same request (say, a pricing rule recalculates it), the flush handler sees it once. And when it fires, it reads the latest price from the database. The most recent value always wins.

Transaction safety is built-in. transaction.on_commit guarantees the flush only runs after the database confirms the change. If the transaction rolls back (a validation error, a constraint violation, a raise somewhere), the callback is discarded. You never sync phantom data.

Batch resolution avoids N+1. The service resolves all references in bulk queries, not one at a time. For 50 changed products, it takes exactly 1 query regardless of count. No per-product lazy loading.

Design Decision: Avoiding the Global State Pitfall

You might be wondering why we used threading.local() in Step 2 instead of a simple module-level variable like _pending_changes = set().

A basic module-level set shares state across every request handled by the same worker process. If User A's request rolls back, stale refs from their failed transaction sit in the global set. When User B's request commits ten minutes later on the same worker, the flush accidentally picks up both User A's and User B's products.

By using threading.local(), we protect against this cross-request leakage. Even if a transaction rolls back, any leftover references in that thread's set will simply be cleared on its next successful commit. And in the rare event a stale reference makes it to the resolver, it evaluates to nothing and is skipped silently.

One thing to watch for: use .filter() when looking up products in your resolver, not .get(). .filter() returns an empty queryset gracefully. .get() throws DoesNotExist and tanks your flush.

The Result

The user experience is deceptively simple. A user toggles "Sync product prices" on a store configuration page. From that moment on, any price change from any code path in the application syncs to the external marketplace within seconds.

No one has to remember to add a sync call to new features. No one has to track down every place a price can change. No one monitors a queue for failures (Celery retries handle that). The system just works.

How are you handling external API syncs in your Django apps? Are you using signals, or do you prefer a different pattern? Drop your approach in the comments.

¡Hasta luego!

I Built a Full-Stack E-Learning Platform with AI Quiz Generation — Here's Everything I Learned

Fri, 24 Apr 2026 21:16:50 +0000

I'm Karim, a full-stack developer and AI/ML researcher from Cairo. Over the past year, I built Qodrateman — a production e-learning platform now live at qodrateman.com, serving Arabic-speaking students across 12+ courses.
This isn't a "look what I made" post. It's the full breakdown — stack decisions, what broke, how I fixed it, and what I'd do differently. Let's get into it.

What Is Qodrateman?
An e-learning platform built for the Arabic-speaking market with:

Structured courses with instructor/learner role separation
JWT authentication + Role-Based Access Control (RBAC)
AI quiz generation — instructors get a full quiz in under 30 seconds
12+ live courses, 500+ authenticated users
Deployed on Vercel (frontend) + Render (backend) with zero downtime

The Stack — and the Reasoning Behind Every Choice
React + TypeScript + Tailwind CSS
E-learning UI state is genuinely complex — active course, current lesson, user progress, enrollment status. React handles this cleanly. TypeScript caught dozens of bugs before they ever hit production. Tailwind made responsive design fast without drowning in custom CSS.
Django REST Framework
DRF gives you serializers, viewsets, and a built-in admin panel out of the box. I had a fully functional backend faster than I've seen with any other framework. For REST APIs, it's still my first pick.
PostgreSQL
E-learning data is deeply relational — users, courses, lessons, enrollments, quiz attempts, all interconnected. PostgreSQL handles this naturally. SQLite is fine locally, but never in production.
JWT Auth with simplejwt
pythonREST_FRAMEWORK = {

'DEFAULT_AUTHENTICATION_CLASSES': (
    'rest_framework_simplejwt.authentication.JWTAuthentication',
),

}
Access token in memory (not localStorage — that's a security risk). Refresh token handles silent re-authentication. Frontend and backend stay fully decoupled.
Vercel + Render
Zero-config deploys on every push with Vercel. Render for the Django backend — free tier works at early stage, scales easily when needed. Total setup time: under 2 hours.

The Feature I'm Most Proud Of: AI Quiz Generation
Instructors told me quiz creation was their biggest time sink — around 20 minutes per quiz. Writing quality multiple-choice questions is genuinely hard.
I built AI-assisted quiz generation that brings that down to under 30 seconds.
Here's how it works:

Instructor enters topic + difficulty level
System sends a structured prompt to the AI API
API returns a complete quiz in JSON — questions, options, correct answers
Platform parses and saves it directly to the database
Instructor reviews and publishes in one click

The structured prompt was the key engineering decision:
pythondef generate_quiz_prompt(topic, num_questions, difficulty):

return f"""
Generate a quiz about: {topic}
Number of questions: {num_questions}
Difficulty: {difficulty}

Respond ONLY with valid JSON in this exact format:
{{
    "questions": [
        {{
            "question": "question text",
            "options": ["A", "B", "C", "D"],
            "correct_answer": "A",
            "explanation": "why this is correct"
        }}
    ]
}}
Do not include any text outside the JSON.
"""

Forcing structured output and validating before saving eliminated the inconsistency problem entirely. Early versions failed whenever the model returned extra text around the JSON — strict format constraints fixed this.

RBAC: Two User Types, Zero Overlap
Instructors and learners have completely separate capabilities. Here's the core of how that's modeled:
python# models.py
class UserProfile(models.Model):

ROLES = [
    ('instructor', 'Instructor'),
    ('learner', 'Learner'),
]
user = models.OneToOneField(User, on_delete=models.CASCADE)
role = models.CharField(max_length=20, choices=ROLES)

permissions.py

class IsInstructor(BasePermission):

def has_permission(self, request, view):
    return (
        request.user.is_authenticated and
        request.user.userprofile.role == 'instructor'
    )

This permission class protects every instructor-only endpoint — course creation, quiz generation, student progress. Learners can read published content and submit their own quiz attempts. Nothing more.

What Broke in Production (and How I Fixed It)
The N+1 Query Problem
The course listing API was slow. Every course card needed the instructor name, lesson count, and enrollment count — three queries per course. With 12+ courses, that's 36+ database hits per page load.
Fix:
pythoncourses = Course.objects.select_related('instructor').annotate(

lesson_count=Count('lessons'),
enrollment_count=Count('enrollments')

)
One query. Page load time dropped by 70%.
Inconsistent AI Responses
Early quiz generation would sometimes return extra text around the JSON, breaking json.loads().
Fix — strip everything outside the JSON object before parsing:
pythondef parse_ai_response(response_text):

try:
    start = response_text.index('{')
    end = response_text.rindex('}') + 1
    return json.loads(response_text[start:end])
except (ValueError, json.JSONDecodeError):
    raise ValidationError("AI response could not be parsed")

Render Cold Starts
Render's free tier spins down inactive services. First request after inactivity: 30+ seconds. Terrible UX.
Fix: a lightweight ping endpoint that a Vercel cron job hits every 14 minutes. Cost: zero. Cold starts for active users: eliminated.

What I'd Do Differently
Add React Query from day one. Managing server state with raw useEffect and useState gets messy fast. React Query handles caching, refetching, and loading states far more cleanly. I retrofitted it partway through — not fun.
Map the permission model before writing code. I refactored RBAC twice because I hadn't thought through edge cases upfront. Can instructors see other instructors' content? Can learners preview unenrolled courses? Draw this out before touching code.
Use Docker locally from the start. "Works on my machine" is a real problem. Docker Compose makes the jump from local to production significantly smoother.

By the Numbers
MetricResultCourses live12+Authenticated users500+Quiz generation time~20 min → under 30 secPost-launch rollbacks0Downtime since launch0

What's Next
I'm also building Letra — a custom-trained bilingual OCR mobile app for iOS and Android using TensorFlow, Keras, PaddleOCR, and OpenCV. Currently at 98% English accuracy on 800+ labeled samples, deployed to TestFlight and Google Play Beta. Arabic OCR is in active iteration targeting 85%+.

The hardest part of building a real product isn't writing the code. It's the decisions you make before you write any — schema design, permission models, deployment architecture. Get those right and everything else follows.
If you're building on Django + React or have questions about the AI quiz generation approach, drop a comment. Happy to dig into any part of this.

Full-stack dev & AI/ML researcher based in Cairo
karimkhamis.com · qodrateman.com · github.com/karim-99-99

Between Tradition and Modernity: Building a Full Stack App with Django and React

Tue, 14 Apr 2026 11:49:41 +0000

If I tell you I made a full-stack app with React 19 (+ Compiler), Tanstack Router, tailwindcss, Base UI, and pnpm, you would probably expect a Hono backend or Tanstack Start Server functions with Drizzle, or at least some cutting-edge TS solution. Well, I've built a Django backend, and it works pretty well!

The Constraints vs. The Freedom

When starting a new project, you always face constraints. Your job as the person designing the software architecture is to find the most elegant way to build around them.

Here are the constraints I was handed:

Django for the backend (to match the team's existing legacy projects).
Keycloak for authentication.
Dockerizing the entire stack for self-hosted deployment.

The freedom? I got to decide absolutely everything else.

If you’ve read my previous blog posts, you know I’ve fallen in love with the Tanstack ecosystem. I went with Tanstack Router, Query, Form, Table, and Pacer. Notice that I didn't go with Tanstack Start. Given the actual goals of this app, I couldn't justify the SSR overhead, and I absolutely did not want to spend a single second fixing hydration issues (I still have nightmares about them).

For the UI, I finally got to use TailwindCSS and shadcn/ui at work, freeing myself from plain CSS and the horrors of styled-components. I really love the pattern of creating headless, reusable components:

export function PageHeader({ className, children, ...props }: React.HTMLAttributes<HTMLElement>) {
  return (
    <header
      className={cn('grid auto-rows-min items-start gap-2', className)}
      data-slot="page-header"
      {...props}
    >
      {children}
    </header>
  )
}

(Yes, it kind of looks like styled-components in a way. Maybe time is a flat circle?)

A quick shoutout to two other bangers in the frontend stack:

Paraglide JS: After fighting with react-i18next (lack of type safety, fetching all keys client-side), I switched to Paraglide JS on Tanstack's recommendation. Zero downsides so far.
Knip: Analyzes your codebase for unused files, exports, and dependencies. Even with the strictest ESLint/Prettier setup, you’ll have dead code. Knip is a godsend for cleanup.

Bridging the Gap: The Auth Weirdness

Let's talk about the weird authentication layer in the app, that I am not a big fan of.

The client authenticates with Keycloak. This means I need to check auth on the frontend (using react-oidc-context and oidc-client-ts, which have pretty bad documentation) and store the auth context.

I then pass the key to the backend during API calls, where Django verifies it using jwt.decode against the public key. It doesn't sound that bad, except that Django has its own pre-built auth system with user tables, and Keycloak isn't designed to store app-specific user metadata. So, I had to build a weird, performant sync layer between the two (e.g., if a Keycloak email changes, reflecting it locally in the Postgres DB). It’s clunky, but it works.

Type Safety is still possible

I love type safety. Coming from TypeScript, and having played with C and Dart in school, dynamic typing gives me hives.

Out of the box, Python can feel a bit like the Wild West compared to a strict TypeScript setup. With standard dependencies often living in a simple .txt file and optional linting, it gives you a lot of freedom. But that freedom means you have to actively put in the work to enforce a strong Developer Experience, otherwise code quality can slip quickly.

However, having a Python backend and a TypeScript frontend doesn't mean you have to sacrifice end-to-end type safety. Here is how I forced the two to play nice:

Django properly defines the models with strict types and comments.
Backend views have full documentation on response types using those models.
drf-spectacular generates the OpenAPI specs.
Orval generates TS types and query hooks from those specs.
The frontend consumes the Tanstack Query Hooks to fetch data.

And just like that... End-to-end type safety. You know exactly what the endpoint needs, and exactly what it's going to return. When you make a change in a model, you get the feedback all the way to your frontend component.

DX: Making the Containers Bearable

After the users, my top priority is the developers (and code quality). Because of the strict constraints, I got to tear my hair out properly learning Docker. Setting up multiple services, ensuring they communicate, and managing deployment across two environments in a server full of existing apps was a massive headache.

But once it works, Docker is magic. Starting the project takes one command. You get fully reproducible environments between local dev and production. So now you can use the "But it works on my machine" excuse more confidently.

I also spent some time creating Make commands. These Docker commands are quite long and spending 5 minutes going up in the terminal history trying to find the specific command to run the tests can be quite annoying. So I wrote a Makefile. Now, a simple make codegen spins up the OpenAPI specs and frontend types:

COMPOSE_FILE := docker-compose.dev.yml
ENV_FILE     := .env.local
COMPOSE      := docker compose -f $(COMPOSE_FILE) --env-file $(ENV_FILE)

.PHONY: codegen
codegen: schema types ## Generate both Schema and Types

.PHONY: schema
schema: ## Generate Open API schema from Backend
    $(COMPOSE) exec backend python manage.py spectacular --file openapi.yml --validate

.PHONY: types
types: ## Generate TypeScript types from Open API schema
    $(COMPOSE) exec frontend pnpm run generate-types

I also built a strong CI pipeline. It handles backend linting and formatting, 800+ Django tests, migration checks, OpenAPI schema validation, Frontend Schema Types validation, frontend type checking, frontend build, and finally Playwright tests.

It sounds heavy, but it only takes ~7 minutes if all steps run, thanks to aggressive caching, sharding, and parallel jobs. If I only touch backend code, the pipeline finishes in 2 minutes. Stop the pipeline early, run only what changed. It’s worth the initial setup time.

Boring isn't bad

Django is not the most exciting tech, but it’s great for a CRUD app exposing APIs to Postgres. Yes, I still have to handle some complexity: RBAC, Redis caching, querying a Clickhouse DB with raw SQL, and WebSockets for live notifications, but I’m not building a crazy app for millions of users.

Django is simple, predictable, and LLMs understand it perfectly. Need a cache layer? Two lines of code. It’s fast enough that running 800+ tests (including DB writes) takes 10 seconds.
I still have some issues with it, like if there is an internal server error, an endpoint will return some html by default. So you need a custom middleware to formalize all kinds of errors. And of course, it has to be in Python. But overall: it just works.

The Silo Problem: Designing APIs in a Vacuum

Now, for a bit of a reality check regarding backend development.

In theory, backend engineers handle incredibly important tasks: complex business logic, rock-solid security, scaling, and database optimizations. This should put to shame frontend engineers like me who spend 2 hours changing the color of a button.

But in my early years of corporate experience, my reality has been quite different. I have only encountered codebases where basic software engineering practices, like mandatory PR reviews, automated testing, or even basic linting, just weren't part of the culture. When teams don't put effort into those foundations, you quickly end up with messy codebases, weak RBAC, major security oversights, and poor performance. More importantly, I’ve encountered the "Silo Problem."

Here’s a story from a past job. We were building an AI Chat app. I was the solo frontend dev, working with a UI/UX designer, a handful of backend devs, and some AI engineers.

The backend team held their planning meetings without bringing in the frontend or the UI designer. Because of that disconnect, the database relations and endpoints were designed without the actual client application in mind. The result made zero sense for the UI: they added data models for UI themes using variables that completely clashed with the design system, and created endpoints structured in a way that required six separate workarounds on the client just to render a basic view. I ended up rewriting the frontend logic three separate times to keep up with OpenAPI specs that were handed down after the fact (which, naturally, rarely matched the live responses). It was an incredibly frustrating but valuable lesson in why API design has to be collaborative.

If you are a backend developer adding a new endpoint, you aren't doing it for fun. You are doing it because the user interface needs that data. Designing APIs without consulting the client-side needs is like building a steering wheel without checking what kind of car it's going into.

Thankfully, on my current project, I am the frontend, backend, and DevOps engineer. Everything communicates nicely, because I actually talk to myself.

The Corporate Reality

I started this new grad job in mid-2025 at a small non-tech company. I turned down a crazy offer at a massive corporation specifically because I wanted the freedom to build good products and actually care about software architecture.
And I did get that freedom. I spent hours refining small details, optimizing DX, and over-engineering the type safety.
But as the months go by, a grounding realization has set in:

This app will likely never have more than 8 concurrent users.
Looking at the history of internal projects, business priorities pivot fast, and apps are sometimes replaced after a year or two.

It's a humbling thought. But I've come to see it less as a reason not to care, and more as a reminder to calibrate. Good architecture isn't just about scale, but also it's maintainability, handoff-ability, and not leaving a mess for the next person. Those things matter even if the app never sees a traffic spike.

But in life, you make compromises. I’m getting paid to learn, I built an architecture I’m proud of, and at the end of the day, I get to log off and live in one of the most beautiful places in the world.

How to Send Contact Form Submissions to WhatsApp (Without Building a Backend)

Mon, 23 Mar 2026 09:40:10 +0000

Most contact forms send submissions to email.

The problem is — email gets missed.

If you're not checking your inbox constantly, new enquiries can sit there for hours. In many cases, the first person to respond wins the lead, so delays can cost you real opportunities.

A better approach is to send form submissions somewhere you'll actually see them:

WhatsApp

In this post, I’ll show you how to send contact form submissions to WhatsApp without building your own backend.

The problem with email-only forms

Most setups look like this:

User fills out a form
Submission gets emailed
You check it later

This creates a delay — and that delay is where leads are lost.

Option 1: Build it yourself

You can send form submissions to WhatsApp yourself, but it’s not simple.

You’ll need:

a backend (Node, Python, etc.)
WhatsApp Business API access
message formatting logic
error handling and fallbacks

For most projects, this quickly becomes overkill.

Option 2: Use a form backend

A simpler approach is to use a form backend that handles delivery for you.

Instead of building everything:

Keep your existing HTML form
Send submissions to a backend endpoint
Receive them instantly on WhatsApp

Example HTML form


<form action="https://web2phone.co.uk/api/v1/submit/" method="POST">
  <input type="hidden" name="public_key" value="YOUR_PUBLIC_KEY">

  <input name="name" placeholder="Your name" required>
  <input type="email" name="email" placeholder="*Emails are not allowed*" required>
  <textarea name="message" placeholder="How can we help?" required>

Meet Seendr: The App That's Bringing Genuine Connection Back to Social Media

Sun, 22 Mar 2026 15:26:15 +0000

When was the last time you made a real friend online?

Not a follower. Not a mutual. Not someone who liked your post once and disappeared. An actual friend — someone you talk to regularly, who shares your interests, who you'd call if you had a bad day.

If you're struggling to answer that, you're not alone. And that's exactly why we built Seendr.

The Problem Nobody Talks About

Social media was supposed to connect us. Instead, it turned connection into a competition.

Today's platforms are built around influence. Your worth is measured by follower counts, likes, and engagement metrics. Conversations happen in the shadow of invisible hierarchies — people with bigger audiences dominate, while everyone else watches from the sidelines. The platforms we use daily are optimized for attention, not for friendship.

And the result? Adults are lonelier than ever.

If you've ever tried to actually meet new people online, you know how frustrating it is. You join a Discord server, send a message, and wait hours for a reply that may never come. You scroll through apps that feel like cattle calls. You post on social media hoping someone will reach out — but all you get is likes from strangers who'll forget you by tomorrow.

The tools we have aren't built for making friends. They're built for building audiences.

We wanted to change that.

What Is Seendr?

Seendr is a video-first app designed for one thing: helping adults make real friends through shared interests.

Here's what makes it different:

No vanity metrics. There are no follower counts, no likes, no public engagement stats. When you open Seendr, nobody knows if you have 10 connections or 10,000. Every conversation starts on equal ground.

Interest-based matching. You don't scroll through random profiles. You select the things you care about — gaming, philosophy, fitness, entrepreneurship, mental health, whatever — and get matched with people who share those passions. The matching algorithm considers shared interests, availability, and communication style.

The 45-second vibe check. Instead of committing to an awkward 30-minute call with a stranger, you start with a quick 45-second video chat. If the energy's right, you stay and keep talking. If not, no pressure — move on to the next match. It's like speed-dating for friendships.

Debate rooms and chatrooms. Sometimes you don't want a 1-on-1 call. You want to jump into a conversation about something interesting. Seendr has topic-based rooms where people discuss and debate real subjects — "Is AI the best thing to happen to humanity?", "What's the most overrated career advice?", whatever the community cares about. It gives you a reason to talk, which kills the awkwardness of cold introductions.

You control the pace. Not everyone's comfortable jumping straight to video. On Seendr, you can start with text or voice and unlock video when you're ready. No pressure, no judgment.

Safety first. We take safety seriously. Liveness verification ensures real people are behind every profile. AI-powered moderation catches inappropriate behavior. And a robust reporting system lets the community help keep the space clean.

Who Is Seendr For?

If any of these sound like you, Seendr was built with you in mind:

You moved to a new city and your social circle is basically your coworkers
You work remotely and realize you haven't had a non-work conversation in days
You're tired of social media making you feel like a spectator instead of a participant
You want to meet people who actually share your interests, not just your zip code
You're introverted but still crave meaningful connection — just on your own terms

Whether you're 18 or 40, a college student or a remote professional, Seendr is built for people who want real conversations with real people.

Try It Right Now

Seendr is live. You can use it today.

Head over to seendr.vercel.app and see for yourself. Create a profile, pick your interests, and start connecting.

It's early. We're still building. There are features in our backlog that we're genuinely excited about — and we want your feedback to help shape what comes next.

But the core experience is there: find someone who gets you, have a real conversation, and maybe — just maybe — make a friend you'll actually keep.

Built By People Who Needed It

This isn't a product we built from a boardroom. My teammate and I built Seendr because we've lived the problem. We've felt the loneliness. We've scrolled through platforms that promised connection and delivered performance.

Seendr is our answer to that. And we'd love for you to be part of it.

Try it at seendr.vercel.app

If it resonates with you, share it with someone who needs it. The best friendships start with someone taking the first step.

We're building Seendr in public and sharing every lesson along the way. Follow along for updates on the product, the tech, and the journey. And if you have feedback after trying the app — we genuinely want to hear it.

RepoRadar - match exact stack to jobs

Tue, 17 Mar 2026 19:06:22 +0000

I've been job hunting as a full-stack dev. I usually do freelance work but I'm not the best businessman and honestly it would be nice if somebody just told me what to build and I could get paid. But all the jobs on LinkedIn are ghost jobs — they're just accumulating resumes into a black hole and none of them ever actually get checked. You write a whole cover letter for a job that doesn't exist.

I checked out a bunch of the newer tools. HiringCafe looked like the backend wasn't even hooked up, which is kinda ridiculous. StackJobs wouldn't load. Scoutify wanted money and I was like nah, I'll just build this myself with Claude. Jobright kinda works but the matching wasn't what I wanted — I wanted to be able to search for specific engineering jobs that match my exact tech stack. Like I build with Django, React, TypeScript — I don't want to read through every JD guessing if they use what I use.

So I built this thing called RepoRadar. I wanted it to be click click results, no friction.

how it works

Google SSO → upload your resume → Claude API parses it and auto-selects your tech stack → hit search → see matching jobs sorted by recent. This way you avoid the ghost jobs because everything's pulled fresh. You click a job and it takes you to the actual company page to apply. No middleman.

where the jobs come from

After doing some research it turns out most tech companies use one of like four ATS platforms and they all have public APIs. No auth needed. You can hit them as much as you want which is pretty sweet. So what I did is I mapped over 6,000 companies to their ATS platform — Greenhouse, Lever, Ashby, Workable. I created a Celery beat task that hits all of them every morning around 6am and pulls fresh listings.

GET https://boards-api.greenhouse.io/v1/boards/stripe/jobs

That gives you every open role at Stripe in JSON. All four platforms work like this.

I also pull from RemoteOK, Remotive, We Work Remotely, and the monthly HN Who's Hiring thread. But it's mostly from the ATS boards — there's some ridiculous amount that show up on RepoRadar, like 185,000 jobs right now.

The mapping was honestly the hardest part of the whole project. Took way longer than the actual code, and I kept having to reprocess them and losing my SSH connection to Railway in the middle of it.

how the matching works

Every job description gets run through a regex extractor with ~170 keyword patterns — Django, React, PostgreSQL, TypeScript, Next.js, etc. It tags each job with what it detected. When you search it just filters where your stack overlaps with what's in the JD.

TECH_PATTERNS = {
    'django': r'\bdjango\b',
    'react': r'\breact(?:\.js|js)?\b',
    'postgresql': r'\b(?:postgres(?:ql)?|psql)\b',
    'typescript': r'\btypescript\b',
    'next_js': r'\bnext\.?js\b',
    # ~170 more
}
 
def extract_techs(description_text):
    detected = []
    text_lower = description_text.lower()
    for tech_name, pattern in TECH_PATTERNS.items():
        if re.search(pattern, text_lower):
            detected.append(tech_name)
    return detected

No ML, no embeddings, no vector search. If I get some traffic maybe I'll add that stuff, who knows. Is it perfect? No. But is it tailored for engineering jobs? Absolutely.

I also wanted to make it so you could search by what companies are actually building with. Like I build with Claude, so I want to find companies that match my exact workflow. That's kind of the vision.

the stack

Django 5 / DRF on Railway, React 19 / TypeScript / Vite / Tailwind on Netlify, PostgreSQL, Redis + Celery for background jobs, Claude API for resume parsing, Google OAuth via django-allauth. Railway runs a single container that starts gunicorn and a Celery worker from a start.sh script. Frontend proxies API requests through netlify.toml. OAuth goes straight to Railway because you can't proxy OAuth redirects — learned that the hard way.

what I learned

ATS public APIs are a goldmine. You can hit them way more than you can hit the GitHub API (which wasn't that useful anyway since most company repos are private). There's so many jobs out there it's actually crazy once you start pulling from these endpoints.

The company-to-ATS mapping took way longer than writing the application. Like, significantly.

Freshness matters more than volume. The older a posting is, the less likely you are to hear back from anybody. So everything on RepoRadar is sorted recent-first and refreshed daily.

Originally I had set up a GitHub integration to search repos and find organizations by tech stack, but it turns out most company repos are private so that didn't work the way I wanted. I pivoted to focusing on the ATS boards and remote platforms instead.

what's not great

6,200 companies is a lot but it's not everything — no Workday, iCIMS, or Taleo coverage yet. Regex misses some edge cases. And if I get concurrent user load I'm gonna have to split Celery out into its own Railway service instead of running it in the same container, but that's all doable.

Right now I've got it hooked up with Sentry for monitoring. I'd love to add OpenTelemetry if I can get some real users on it. I also set up a bunch of MCP servers — Railway, Chrome MCP — but the Chrome one isn't that helpful because you can't get past the Google SSO login screen with it.

try it

It's free. You can try it and complain to me and tell me what I did wrong — in fact I would love that. It'd be really cool if I can get some users on this thing.

direct complaint line mnraynor90atgmaildotcom gracias

Escaping the Sync Trap: How I Slashed Latency by 10x in a Django-Rust API Gateway

Tue, 17 Mar 2026 17:16:00 +0000

How I diagnosed and eliminated synchronous bottlenecks in a Django-Rust API gateway, migrating to ASGI and pre-warming caches for millisecond responses.

When building a high-performance backend, the standard playbook is well-known: offload heavy computational tasks to faster microservices (like Rust) and implement an aggressive caching strategy.

Recently, I did exactly that. My architecture is built around a Django REST Framework gateway sitting behind Caddy, heavily monitored with Prometheus and Grafana.

But despite the raw speed of Rust and my caching layers, my dashboards were flashing red. Latency was spiking to brutal 10-second flatlines for my most critical endpoints. Worse, my observability itself started failing, creating silent blind spots exactly when I needed data the most.

Here is the detective story of how I used telemetry to hunt down synchronous traps, migrate to a non-blocking async architecture, and implement proactive pre-warming to bring response times down to the millisecond range — all while reclaiming 30% of my idle CPU.

The Architecture: A Gateway and its Heavy Lifters

Before diving into the problem, here is a quick look at my setup.

          ┌──────────────────────────────┐
          │          Nextcloud           │
          │ (Authenticated Client Calls) │
          └──────────────┬───────────────┘
                         │  JWT / API Key
                         ▼
               ┌────────────────────┐
               │  Django Gateway    │
               │ (ASGI, DRF, Caddy) │
               └──────┬─────────────┘
                      │
     ┌────────────────┴────────────────┐
     │                                 │
     ▼                                 ▼
┌───────────────┐              ┌────────────────┐
│ NDVI Service  │              │ Weather Service│
│ (Rust, 8081)  │              │ (Rust, 8090)   │
│  → Postgres    │              │  → MySQL       │
└───────────────┘              └────────────────┘

       ▲
       │ Prometheus & Grafana
       │ (Observability Stack)
       ▼
   System Telemetry + Metrics

Django stays as the public API gateway, accepting requests authenticated via JWT or API keys.
It enforces a shared JSON response envelope containing status, message, data, and errors to keep all client interactions standardized.

Specific traffic routes — namely /api/v1/ndvi and /api/v1/weather/* — are forwarded directly to my Rust backends:

NDVI microservice ingests satellite data into a dedicated Postgres database.
Weather microservice relies on a MySQL database and communicates with external providers like Open-Meteo and NASA POWER.

My Nextcloud instance acts like any other client, presenting either an Authorization: Bearer token or an X-API-Key. Django manages this traffic using a specific nextcloud_hmac throttle configuration before passing the authorized call down to Rust with the original headers intact.

The False Cure: The Worker Starvation Anomaly

To protect the system, I implemented aggressive TTL caching (e.g., 1 hour for schema data, 5 minutes for API tokens). However, once I added traffic, my Grafana dashboard revealed a chaotic reality.

I saw brutal, perfectly flat 8–10 second latency spikes on key endpoints. Crucially, perfectly timed with these latency spikes, my internal /metrics request rate dropped to zero.

The Diagnosis: The Gateway Caching Itself to Death

The telemetry told a story of hidden synchronous bottlenecks:

The Trigger: When a high-traffic endpoint like farm-weather-current/GET experienced a cache miss, the Django gateway had to fetch fresh data.
The Trap: My Django deployment was running using standard synchronous workers. It called the Rust service, which then called the external weather API (taking 2.2+ seconds).
The Impact (Worker Starvation): Because the Django worker was synchronous, it blocked entirely for those 2.2 seconds. All incoming traffic got stuck in a queue.

The Trap: Synchronous Gateway Routing

import httpx
from django.http import JsonResponse
from django.views import View

async def async_weather_proxy_view(request):
    async with httpx.AsyncClient(timeout=5.0) as client:
        resp = await client.get("http://weather-service:8090/api/v1/weather-current")
        resp.raise_for_status()
        data = resp.json()
    return JsonResponse({"status": "success", "data": data, "errors": None})

I had successfully offloaded work to Rust — but my synchronous Django workers completely nullified the speed gains.

The First Fix: Embracing the Non-Blocking Gateway

I needed to decouple the speed of the gateway from the speed of the external API calls it was routing.
I migrated the Django deployment from synchronous workers to an ASGI (Asynchronous Server Gateway Interface) setup, allowing my gateway to handle requests asynchronously.

I rewrote my proxy views to use asynchronous HTTP clients like httpx:

The Fix: Asynchronous Non-Blocking Routing

import httpx
from django.http import JsonResponse

async def async_weather_proxy_view(request):
    # The event loop is freed! Django can serve other requests while waiting
    async with httpx.AsyncClient() as client:
        rust_response = await client.get("http://weather-service:8090/api/v1/weather-current")
        
    return JsonResponse({
        "status": "success",
        "data": rust_response.json(),
        "errors": None
    })

The visual evidence on my dashboards was a massive, instant victory:

Observability Restored: The metrics scrape line remained unbroken. Django could finally pause a slow weather request, instantly answer the Prometheus scrape, and resume without blocking.
⚡ Instant Internal Routing: In my initial setup, a simple internal metrics scrape took ~84ms. After the ASGI migration, that duration dropped to 11ms.

The Second Fix: Proactive Caching

While the infrastructure was now bulletproof, the end-user experience was still occasionally sluggish.

With an “on-demand” caching strategy, the very first user to request the weather after a 1-hour cache expiration had to pay the Cache Miss Penalty (waiting ~2.2 seconds for the external API).

To eliminate this, I decoupled the data-fetching time from the user-request cycle entirely.
I implemented a Proactive Background Pre-warming pattern using a background task (like Celery) that runs every 55 minutes, independently fetching slow data and silently overwriting the cache before it expires.

The Cache Pre-Warmer (Celery Example)

from celery import shared_task
from django.core.cache import cache
import requests

@shared_task
def pre_warm_weather_cache():
    # Runs in the background every 55 minutes, shielding the user from the 2.2s wait
    response = requests.get("http://weather-service:8090/api/v1/weather-current")
    if response.status_code == 200:
        cache.set("weather_current_data", response.json(), timeout=3600)

The result? The average latency for critical weather endpoints plummeted from seconds to milliseconds as the hot cache permanently took over.

The Grand Slam: Optimizing the Rust Build Pipeline

The final victory of this new architecture came from pure server efficiency.
During deployments, compiling Rust crates (sqlx, syn) from scratch was pegging my 4-core server at 100% CPU, artificially causing timeouts.

To fix this, I implemented cargo-chef in a multi-stage Dockerfile to strictly cache Rust dependencies.

Multi-stage Dockerfile for the Rust Microservice

FROM rust:1.88-slim AS chef
USER root
RUN cargo install cargo-chef
WORKDIR /app

FROM chef AS planner
COPY . .
RUN cargo chef prepare --recipe-path recipe.json

FROM chef AS builder
COPY --from=planner /app/recipe.json recipe.json
# Docker caches this heavy dependency build!
RUN cargo chef cook --release --recipe-path recipe.json
COPY . .
RUN cargo build --release --bin weather-service

FROM debian:bookworm-slim AS runtime
WORKDIR /app
RUN apt-get update && apt-get install -y libssl-dev ca-certificates
COPY --from=builder /app/target/release/weather-service /usr/local/bin/
EXPOSE 8090
ENTRYPOINT ["/usr/local/bin/weather-service"]

Between ASGI, background caching, and Docker layer caching, my total Node CPU now rests comfortably between 11% and 13%.
I fundamentally reclaimed 30% of my total server compute capacity.

Conclusion: Finding the Next Bottleneck

Building high-performance API gateways is an ongoing journey of shifting bottlenecks.

By relying strictly on my telemetry, I proved that synchronous workers nullify microservice speed, validated the immense power of ASGI, and eliminated cache miss penalties.

With the gateway running unburdened, my dashboards have revealed one final bottleneck — a 6-to-8 second delay on my token generation endpoint.
Because my CPU is mostly idle, I know exactly what this is: a database connection pool limitation in the Rust service.

And thanks to my new observability baseline, I know exactly where to strike next.

I Learned Python 5 Days Ago. Now I'm Building a Startup With It.

Tue, 10 Mar 2026 21:47:41 +0000

I didn't pick up Python because it was trendy. I picked it up because someone challenged me to — and I don't run from challenges.

Five days later, I was knee-deep in Django, Docker, and a codebase that refused to cooperate. This is the story of how a dare turned into a startup, what I learned about switching languages under pressure, and why the struggle was worth every frustrating minute.

The App: Why I'm Building Seendr

Before I get into the technical chaos, let me tell you why any of this matters.

I've been on the other side of the loneliness problem. I've sat on Discord, scrolling through "meet new people" servers, sending messages to strangers and waiting hours — sometimes days — for a reply that often never comes. I've tried group chats that fizzle out after two days. I've opened social media looking for connection and found nothing but highlight reels, follower counts, and a quiet feeling that everyone has friends except me.

And I know I'm not the only one.

Social media was built to connect people. Somewhere along the way, it became a stage. Likes replaced conversations. Follower counts created invisible hierarchies. The platforms we use daily are optimized for engagement, not for friendship. If you don't have the followers, you don't have the voice. And if you're just looking for someone to talk to about something you care about — good luck.

That's what Seendr is for.

Seendr is a video-first mobile app that connects people through shared interests. There are no follower counts. No likes. No vanity metrics at all. You sign up, choose your interests, and get matched with people who share them. From there, you can do a 45-second video "vibe check" to see if there's chemistry, join topic-based debate rooms, or just have a real conversation with someone new.

The idea is simple: what if a social platform treated every user as an equal and gave them a reason to talk?

My teammate and I are building this as a school project and a real startup at the same time. The school requirement was specific — we had to use Django for the backend.

Django means Python. And I had never written a single line of Python in my life.

The Challenge: 5 Days to Learn a Language

I come from the JavaScript/TypeScript world. NestJS, Express, React, Next.js — that's my comfort zone. When I heard "Django," my first reaction was resistance. Why can't I just use what I know?

But then it hit me: this was exactly the kind of challenge I'm always telling other people to embrace. I've said it in conversations, I've thought it during late-night coding sessions — growth happens outside your comfort zone. So when someone essentially dared me to learn Python and build with it, I said yes.

I gave myself five days.

Day 1–2: The Tutorial Phase (False Confidence)

I started with a crash course on Django. Within two days, I understood the structure:

views.py — where you handle requests and return responses
urls.py — the routing layer that maps URLs to views
models.py — your database schema, defined as Python classes
settings.py — the central configuration file for everything
migrations/ — Django's way of tracking and applying database changes
serializers — how data gets converted between Python objects and JSON (via Django REST Framework)

Django calls itself a "batteries-included" framework, and it earns that title. Authentication, admin panel, ORM, middleware — it's all built in. Coming from NestJS where you wire up a lot of things manually, I appreciated that.

I felt good. I felt ready.

I was not ready.

Day 3: The Real Python Hits

Here's the mistake I made: I learned Django's structure but skipped how Python actually works under the hood. Specifically, I glossed over how object-oriented programming is implemented in Python.

And then I tried to write real code.

The syntax felt like someone took everything I knew and rearranged it just enough to be infuriating.

Functions don't use curly braces. They use indentation. And not just as a style choice — the indentation IS the structure. Miss a tab and your code breaks in ways that give you no useful error message.

self is everywhere. In NestJS, when you write a method inside a class, this is implied. In Python, you pass self explicitly as the first argument to every single method. It felt redundant. It felt verbose. It felt wrong.

The syntax for basic things looks alien. List comprehensions, decorators with @ symbols, __init__ as a constructor name, def instead of function — every small difference added friction.

I won't lie. For the first few hours, I was genuinely annoyed. I kept thinking: "This is doing the same thing as what I already know, just in a more annoying way."

Day 4: The Click

And then something happened.

I was writing a view class in Django and realized — I'm still making a class. I'm still writing methods inside it. I'm still receiving a request, processing data, and returning a response. The authentication flow I was building was conceptually identical to the JWT auth system I had built in NestJS for a previous project (a URL shortener).

The patterns were the same. Django just spoke with a different accent.

NestJS has @Controller() decorators → Django has urls.py routing
NestJS has services with business logic → Django has the same, you just organize it differently
NestJS uses TypeORM or Prisma → Django has its own ORM built in
Both use middleware, both use serialization, both handle auth with tokens

Once I stopped fighting the syntax and started seeing the underlying architecture, Python stopped feeling foreign. It started feeling like a dialect of the same language I already spoke.

Day 5+: Docker Tried to End Me

Just when I thought I was over the hump, Docker entered the chat.

We were using Docker Compose to run the full stack — Django API, PostgreSQL, Redis, Celery workers. Professional setup, the kind you'd see in a real production environment. I was proud of it.

Then I ran docker compose up and got:

ModuleNotFoundError: No module named 'django'

Django was installed. I confirmed it with pip list inside the container. Python could see every other package. But when the app tried to start, Django didn't exist.

I spent twelve hours on this bug.

The root cause? A combination of three things:

PIPENV_VENV_IN_PROJECT=1 in the Dockerfile was silently overriding the --system install flag, putting packages in a .venv folder inside the project directory
Docker's volume mount was mapping my local api/ folder over the container's /app directory — which wiped out that .venv at runtime
A sed command I used to try to fix the Dockerfile left a trailing backslash that broke the entire ENV block, causing a cryptic build error

So the packages were being installed during the build, then erased when the container started. Django was there. Then it wasn't. Every single time.

I rebuilt with --no-cache more times than I can count. I read Stack Overflow threads from 2019. I stared at logs until my eyes hurt.

Eventually, I reached out to the president of Django Cameroon for help. Because sometimes the bravest thing you can do as a developer is admit you're stuck.

What I Actually Learned

This whole experience — the five-day crash course, the syntax battles, the Docker nightmare — taught me more than any tutorial could.

1. Frameworks are dialects, not different languages.

If you understand how web applications work — routing, middleware, authentication, database operations, serialization — you can learn any framework. The concepts transfer. The syntax is just wrapping paper.

2. Skipping fundamentals will always catch up to you.

I jumped into Django without properly learning Python's OOP model. That shortcut cost me hours of confusion later. Learn the language, then learn the framework. Not the other way around.

3. Docker is simple until it isn't.

Docker Compose looks clean and elegant in tutorials. In practice, the interaction between build layers, volume mounts, environment variables, and package managers can create invisible bugs that steal entire days. Respect the tooling. Read the docs carefully. And always check what your volume mounts are overwriting.

4. Asking for help is a skill, not a weakness.

I used to sit with broken code for hours because I didn't want to seem stupid. This project broke that habit. When you've been stuck for 12 hours and you message someone who solves it in 10 minutes, you stop caring about looking smart. You just want to learn.

5. Constraints breed growth.

If I'd been allowed to use NestJS, I would have. And I would have learned nothing new. Being forced into Python and Django expanded my thinking, gave me a new perspective on backend architecture, and made me a more versatile developer.

Where Seendr Is Now

We're still building. The Django backend is running (finally). The React Native frontend is taking shape with Expo. We've tested Agora's video SDK for the real-time calling feature. The matching algorithm is in its MVP form — simple scoring based on shared interests, age proximity, and location.

There's still a mountain of work ahead: the debate rooms, the moderation system, the 45-second vibe check flow, push notifications, and a hundred other features living in our backlog.

But the foundation is there. And it exists because I said yes to a challenge I wasn't ready for.

If You're Thinking About Picking Up a New Language

Do it. Don't wait until you feel ready. Don't wait until you "have time." Pick a project that forces you to learn, set a deadline that scares you, and start.

You'll be frustrated. You'll write terrible code. You'll hit bugs that make you question your career choices. And then, somewhere around day four, something will click. The patterns will emerge. The syntax will stop fighting you. And you'll realize that you're not starting from zero — you're building on everything you've already learned.

Five days ago, I didn't know Python.

Now I'm building a startup with it.

And honestly? I'm just getting started.

I'm documenting my entire dev journey — from learning new languages to building Seendr — on LinkedIn and here. If you're on a similar path, let's connect. Drop a comment or reach out. We're all figuring this out together.

Django formfield_for_foreignkey() function

Fri, 03 Oct 2025 21:35:34 +0000

Mastering Django `formfield_for_foreignkey`: Complete Guide with Examples

Django’s admin is one of its most powerful features. Out of the box, it gives you CRUD forms to create, edit, and manage your models. But sometimes, the default dropdowns for ForeignKey fields are too broad — they list all related objects in the database, which may not always be secure, efficient, or user-friendly.

This is where formfield_for_foreignkey comes in.

In this guide, we’ll cover everything you need to know:

✅ What formfield_for_foreignkey is
✅ How Django handles ForeignKey fields by default
✅ Real-world use cases for overriding it
✅ Advanced examples with filtering, permissions, and tenants
✅ Best practices for production apps
✅ FAQs and troubleshooting tips

By the end, you’ll be able to customize your admin dropdowns to be smarter, safer, and tailored to your business rules.

1. What is formfield_for_foreignkey?

In Django Admin, the method formfield_for_foreignkey(self, db_field, request, **kwargs) is a hook method inside ModelAdmin. It allows you to customize the queryset for ForeignKey dropdown fields in the admin form.

By default, Django lists all related objects in the dropdown. But with this hook, you can:

Filter objects (e.g., only active ones)
Restrict by logged-in user
Sort objects for usability
Enforce multi-tenant separation
Apply role-based visibility rules

Method signature:

def formfield_for_foreignkey(self, db_field, request, **kwargs):
    # custom logic
    return super().formfield_for_foreignkey(db_field, request, **kwargs)

2. How Django Handles ForeignKey by Default

Consider this example:

class Author(models.Model):
    name = models.CharField(max_length=100)

class Book(models.Model):
    title = models.CharField(max_length=200)
    author = models.ForeignKey(Author, on_delete=models.CASCADE)

In Django Admin, when creating a Book, the author dropdown will list all authors in the database.

That’s fine for small projects, but:

It can clutter the UI when there are thousands of authors.
It may expose data users shouldn’t see (e.g., other tenants’ data).
It can confuse staff who only need a subset.

This is the default behavior — and exactly where formfield_for_foreignkey helps.

3. Real-World Use Cases

Example 1: Filter by Logged-In User

@admin.register(Book)
class BookAdmin(admin.ModelAdmin):
    def formfield_for_foreignkey(self, db_field, request, **kwargs):
        if db_field.name == "author":
            kwargs["queryset"] = Author.objects.filter(created_by=request.user)
        return super().formfield_for_foreignkey(db_field, request, **kwargs)

✅ Only authors created by the logged-in user appear in the dropdown.

Example 2: Show Only Active Records

def formfield_for_foreignkey(self, db_field, request, **kwargs):
    if db_field.name == "author":
        kwargs["queryset"] = Author.objects.filter(is_active=True)
    return super().formfield_for_foreignkey(db_field, request, **kwargs)

✅ Inactive authors are hidden.

def formfield_for_foreignkey(self, db_field, request, **kwargs):
    if db_field.name == "author":
        kwargs["queryset"] = Author.objects.all().order_by("name")
    return super().formfield_for_foreignkey(db_field, request, **kwargs)

✅ Users see a clean, ordered dropdown.

Example 4: Restrict by Permissions

def formfield_for_foreignkey(self, db_field, request, **kwargs):
    if db_field.name == "author":
        if request.user.has_perm("app.view_all_authors"):
            kwargs["queryset"] = Author.objects.all()
        else:
            kwargs["queryset"] = Author.objects.filter(created_by=request.user)
    return super().formfield_for_foreignkey(db_field, request, **kwargs)

✅ Superusers/managers see all authors.
✅ Staff see only their own.

Example 5: Multi-Tenant Filtering

def formfield_for_foreignkey(self, db_field, request, **kwargs):
    if db_field.name == "customer":
        kwargs["queryset"] = Customer.objects.filter(tenant=request.user.tenant)
    return super().formfield_for_foreignkey(db_field, request, **kwargs)

✅ Each tenant only sees their own customers.

4. When to Use It?

You should override formfield_for_foreignkey when:

You want to limit foreign key options based on logged-in user.
You need to hide irrelevant or sensitive records.
You want to improve usability by sorting or filtering.
You’re working with a multi-tenant app.

5. Best Practices

✅ Always call super() at the end.
✅ Keep queries efficient — avoid heavy joins here.
✅ Apply consistent filtering across forms and lists (get_queryset).
✅ Test with multiple roles (staff, superuser).
✅ Document your filtering logic — future devs will thank you.

6. Common Mistakes

❌ Forgetting to return super() → breaks form rendering.
❌ Overly complex queries → slows down admin.
❌ Inconsistent logic with get_queryset → users see mismatched data.

7. FAQs

Q: Can I filter dropdowns differently for add vs. change forms?
Yes, you can check request.resolver_match or request.path to see the context.

Q: Can I use it for ManyToMany fields?
Use formfield_for_manytomany instead.

Q: Is this the only way to filter foreign keys?
No — you can also use custom ModelForms, but this method is the most admin-friendly.

8. Final Thoughts

Django’s formfield_for_foreignkey is more than just a customization hook — it’s a security and usability tool. By mastering it, you can:

Keep admin dropdowns clean and focused.
Enforce tenant and role-based restrictions.
Prevent accidental data leaks.

'formfield_for_foreignkey()' with Django Permissions: Fine-Grained Control in Admin

Fri, 03 Oct 2025 21:35:11 +0000

formfield_for_foreignkey()' with Django## Mastering formfield_for_foreignkey with Django Permissions: Fine-Grained Control in Admin

Django’s admin is famous for giving developers a fully functional interface out of the box. But real-world apps often need fine-grained access control — different users should see different subsets of related objects.

You may already know about formfield_for_foreignkey, a hook that lets you filter ForeignKey dropdowns in Django Admin. But here’s where it gets really powerful: combine it with Django’s built-in permissions and roles to create a role-aware and secure admin.

In this article, we’ll explore:

✅ Why combine formfield_for_foreignkey with permissions
✅ How to restrict dropdowns using has_perm()
✅ Using groups and roles for finer control
✅ Handling sensitive data with custom rules
✅ Multi-tenant use cases
✅ Best practices

What is formfield_for_foreignkey?

Django provides this method inside ModelAdmin classes to customize the queryset for a ForeignKey dropdown. By default, it lists all related objects in the database. But with formfield_for_foreignkey, you can filter, sort, or restrict that list.

Signature:

def formfield_for_foreignkey(self, db_field, request, **kwargs):
    # custom logic here
    return super().formfield_for_foreignkey(db_field, request, **kwargs)

Why Combine with Permissions?

While formfield_for_foreignkey controls what objects appear, Django’s permission system controls who can see what. Combining them means:

Different roles (staff, manager, superuser) see different dropdown options
Sensitive objects can be hidden unless a user has explicit permission
Multi-tenant applications prevent cross-tenant data leaks

Example 1: Restrict by Permission

Suppose you have Author and Book models. Normally, all authors would appear in the dropdown. But maybe only staff with the app.view_all_authors permission should see everyone. Others should see only their own.

from django.contrib import admin
from .models import Author, Book

@admin.register(Book)
class BookAdmin(admin.ModelAdmin):
    def formfield_for_foreignkey(self, db_field, request, **kwargs):
        if db_field.name == "author":
            if request.user.has_perm("app.view_all_authors"):
                kwargs["queryset"] = Author.objects.all()
            else:
                kwargs["queryset"] = Author.objects.filter(created_by=request.user)
        return super().formfield_for_foreignkey(db_field, request, **kwargs)

✅ Superusers or staff with the permission see all authors.
✅ Other users see only the authors they created.

If you’re using Django groups or a role field on your User model, you can scope choices based on role.

def formfield_for_foreignkey(self, db_field, request, **kwargs):
    if db_field.name == "author":
        if request.user.groups.filter(name="Managers").exists():
            # Managers see all active authors
            kwargs["queryset"] = Author.objects.filter(is_active=True)
        else:
            # Regular staff only see their own active authors
            kwargs["queryset"] = Author.objects.filter(
                created_by=request.user, is_active=True
            )
    return super().formfield_for_foreignkey(db_field, request, **kwargs)

✅ Managers have broader visibility.
✅ Staff stay scoped to their own data.

Example 3: Handling Sensitive Data

Sometimes you want to hide sensitive objects unless the user has explicit permission.

def formfield_for_foreignkey(self, db_field, request, **kwargs):
    if db_field.name == "project":
        qs = Project.objects.filter(is_active=True)

        if not request.user.has_perm("app.can_assign_sensitive_projects"):
            qs = qs.exclude(is_sensitive=True)

        kwargs["queryset"] = qs
    return super().formfield_for_foreignkey(db_field, request, **kwargs)

✅ Sensitive projects don’t even appear in the dropdown unless allowed.

Example 4: Multi-Tenant Use Case

In SaaS apps with multiple tenants, you can combine tenant ownership with permissions.

def formfield_for_foreignkey(self, db_field, request, **kwargs):
    if db_field.name == "customer":
        qs = Customer.objects.filter(tenant=request.user.tenant)

        if not request.user.has_perm("app.view_inactive_customers"):
            qs = qs.filter(is_active=True)

        kwargs["queryset"] = qs
    return super().formfield_for_foreignkey(db_field, request, **kwargs)

✅ Users only see customers from their tenant.
✅ Optional permission controls visibility of inactive customers.

Best Practices

Always call super() at the end to retain Django’s defaults.
Keep queries efficient — avoid heavy joins inside this method.
Test with multiple roles (staff, manager, superuser) to avoid hidden bugs.
Stay consistent — if you filter dropdowns here, apply the same logic in get_queryset() for list views.

Final Thoughts

formfield_for_foreignkey gives you object-level filtering power, while Django’s permission system gives you user-level control. Together, they create an admin that’s secure, role-aware, and tenant-safe.

Whether you’re managing sensitive data, building a SaaS app, or just keeping the admin clean for staff, mastering this combination will make your Django admin much more robust.

With these examples, you can now:

Use permissions to decide who sees what in dropdowns.
Implement role-based logic with groups.
Keep your admin secure in multi-tenant and sensitive-data scenarios.

Mastering `formfield_for_foreignkey` in Django Admin: A Complete Guide

Fri, 03 Oct 2025 11:37:42 +0000

Mastering `formfield_for_foreignkey` in Django Admin: A Complete Guide

Django’s admin is one of its most powerful features. Out of the box, it gives you forms to create, edit, and manage your models. But sometimes, the default dropdowns for ForeignKey fields are too permissive — they show all related objects in the database.

What if you want to restrict the choices a user can see, based on who is logged in, or some business logic?

That’s where formfield_for_foreignkey comes in.

Read this aticle also formfield_for_foreignkey in Django

In this article, we’ll dive into:

✅ What is formfield_for_foreignkey?
✅ How Django handles ForeignKey fields by default
✅ Real-world examples of overriding formfield_for_foreignkey
✅ When and why you should use it
✅ Best practices

What is `formfield_for_foreignkey` in Django Admin?

In Django Admin, the method `formfield_for_foreignkey(self, db_field, request, kwargs)` is a hook method** that lets you customize the queryset for ForeignKey dropdown fields in the admin form.

By default, Django will show all objects of the related model.
With this method, you can filter, sort, or limit the available options.

Method signature:

def formfield_for_foreignkey(self, db_field, request, **kwargs):
    # custom logic
    return super().formfield_for_foreignkey(db_field, request, **kwargs)

How Django Handles ForeignKey by Default

Imagine we have two models:

class Author(models.Model):
    name = models.CharField(max_length=100)

class Book(models.Model):
    title = models.CharField(max_length=200)
    author = models.ForeignKey(Author, on_delete=models.CASCADE)

In the Django Admin, when creating a Book, the author dropdown will list all authors in the database.
This is the default behavior.

But what if you only want staff users to see authors they created? That’s where you override formfield_for_foreignkey.

Example 1: Filter ForeignKey Choices by Logged-in User

from django.contrib import admin
from .models import Book, Author

@admin.register(Book)
class BookAdmin(admin.ModelAdmin):
    
    def formfield_for_foreignkey(self, db_field, request, **kwargs):
        if db_field.name == "author":
            # Only show authors created by the logged-in user
            kwargs["queryset"] = Author.objects.filter(created_by=request.user)
        return super().formfield_for_foreignkey(db_field, request, **kwargs)

✅ Now, when a staff user opens the Book form, the author dropdown only lists the authors they created.
Superusers will still see everything.

Sometimes you want to show only active records.

def formfield_for_foreignkey(self, db_field, request, **kwargs):
    if db_field.name == "author":
        kwargs["queryset"] = Author.objects.filter(is_active=True)
    return super().formfield_for_foreignkey(db_field, request, **kwargs)

✅ Now, inactive authors won’t appear in the dropdown.

You can also order the queryset to make the dropdown more user-friendly:

def formfield_for_foreignkey(self, db_field, request, **kwargs):
    if db_field.name == "author":
        kwargs["queryset"] = Author.objects.all().order_by("name")
    return super().formfield_for_foreignkey(db_field, request, **kwargs)

When Should You Use `formfield_for_foreignkey`?

You should override this method when:

You want to restrict foreign key options based on the logged-in user.
You need to hide inactive or irrelevant related objects.
You want to improve usability by sorting or limiting dropdown choices.
You are working in a multi-tenant application where users should not see each other’s data.

⚡ Best Practices

Always call super() at the end so Django can apply its defaults.
Be careful with queries — don’t run expensive queries inside this method.
Test both staff and superuser accounts to ensure permissions work as expected.
Keep the logic simple — if the filtering is very complex, consider using custom forms instead.

Final Thoughts

The formfield_for_foreignkey method is one of the most useful hooks in Django Admin. It gives you control over what appears in your ForeignKey dropdowns, making your admin interface safer, more user-friendly, and tenant-aware.

Whether you’re building a small project or a large SaaS app, mastering this hook will save you a lot of time and prevent accidental data leaks.

✅ With this article, you can now:

Explain what formfield_for_foreignkey is.
Customize dropdowns for different use cases.
Improve your Django admin forms for better usability and security.

Read this aticle also formfield_for_foreignkey in Django

How to Restrict ForeignKey Choices in Django Admin

Fri, 03 Oct 2025 11:37:19 +0000

Introduction

If you’ve ever built an application with Django Admin, you’ve probably used dropdowns for related models. These dropdowns appear whenever you have a ForeignKey field.

Read this article aslo How to Restrict ForeignKey Choices in Django

But here’s the problem:
By default, Django Admin will show all related objects in that dropdown.

What if you only want to show the objects created by the logged-in user?
Or maybe you want staff users to only see certain options?
Or perhaps you need to filter choices based on business logic (like only “active” records)?

If you landed here by searching things like:

“Django admin limit dropdown choices”
“How to restrict ForeignKey options per user in Django”
“Django admin filter related objects dynamically”
“Restrict choices in Django admin form”
“How to show limited foreign key options in Django”

Then this article is exactly for you.

The solution lies in a powerful Django Admin method:
formfield_for_foreignkey(self, db_field, request, **kwargs)

In this article, we’ll explore:

Why Django shows all ForeignKey objects by default
How to filter dropdown choices in the admin
Step-by-step examples with code
Alternative approaches (querysets, limit_choices_to)
Best practices for performance & maintainability
Common pitfalls & FAQs

By the end, you’ll know exactly how to control your admin dropdowns like a pro.

Why Does Django Show All ForeignKey Choices by Default?

When Django sees a ForeignKey in your model, it automatically renders a dropdown in the Admin. For example:

from django.db import models

class Author(models.Model):
    name = models.CharField(max_length=100)

class Book(models.Model):
    title = models.CharField(max_length=200)
    author = models.ForeignKey(Author, on_delete=models.CASCADE)

In the Book Admin form, Django will render a dropdown for author.

But by default:

It will show all authors from the database.
Even if you only want to allow some of them.

That’s fine for small projects, but in real-world apps, you often need restrictions.

When Do You Need to Restrict ForeignKey Choices?

Here are some common use cases where you might want to filter ForeignKey dropdowns in Django Admin:

User-specific data
- Example: Only show Projects created by the logged-in user.
- Useful in multi-tenant apps.
Role-based filtering
- Example: Staff can only assign orders to employees in their department.
Status-based filtering
- Example: Only show “active” categories, not archived ones.
Security restrictions
- Example: Prevent staff users from accessing other users’ data.
Performance reasons
- If you have thousands of related objects, filtering avoids slow dropdowns.

The Django Way: Using `formfield_for_foreignkey`

Django gives us a built-in method for this exact problem:
formfield_for_foreignkey(self, db_field, request, **kwargs)

This method is part of the ModelAdmin class and runs whenever Django builds a form field for a ForeignKey.

Method Signature

def formfield_for_foreignkey(self, db_field, request, **kwargs):
    # your logic here
    return super().formfield_for_foreignkey(db_field, request, **kwargs)

db_field → the field object (so you can check if it’s the field you want to customize).
request → the current HTTP request (so you can check the logged-in user).
kwargs → extra options passed to the form field.

Example 1: Restrict Choices to Logged-in User

Let’s say each Book belongs to an Author, and you want a user to only select themselves as the author.

from django.contrib import admin
from .models import Book, Author

class BookAdmin(admin.ModelAdmin):
    def formfield_for_foreignkey(self, db_field, request, **kwargs):
        if db_field.name == "author":
            kwargs["queryset"] = Author.objects.filter(user=request.user)
        return super().formfield_for_foreignkey(db_field, request, **kwargs)

admin.site.register(Book, BookAdmin)

✔ Now, the dropdown for author will only show authors tied to the logged-in user.

Example 2: Restrict Choices Based on User Role

Let’s say staff users should only see authors in their department.

class BookAdmin(admin.ModelAdmin):
    def formfield_for_foreignkey(self, db_field, request, **kwargs):
        if db_field.name == "author" and not request.user.is_superuser:
            kwargs["queryset"] = Author.objects.filter(department=request.user.department)
        return super().formfield_for_foreignkey(db_field, request, **kwargs)

✔ Superusers see all authors, staff see filtered ones.

Example 3: Show Only Active Records

class BookAdmin(admin.ModelAdmin):
    def formfield_for_foreignkey(self, db_field, request, **kwargs):
        if db_field.name == "author":
            kwargs["queryset"] = Author.objects.filter(is_active=True)
        return super().formfield_for_foreignkey(db_field, request, **kwargs)

✔ Only active authors appear in the dropdown.

Alternative Approaches

Sometimes you don’t need formfield_for_foreignkey. Here are other options:

1. Using `limit_choices_to` in the Model

class Book(models.Model):
    author = models.ForeignKey(
        Author, 
        on_delete=models.CASCADE,
        limit_choices_to={'is_active': True}
    )

⚠️ Downside: This is static — you can’t use the logged-in user or request context.

2. Overriding the Form Class

You can also customize choices in the form itself:

from django import forms
from .models import Book, Author

class BookForm(forms.ModelForm):
    class Meta:
        model = Book
        fields = "__all__"

    def __init__(self, *args, **kwargs):
        user = kwargs.pop('request').user
        super().__init__(*args, **kwargs)
        self.fields['author'].queryset = Author.objects.filter(user=user)

Then plug the form into your admin:

class BookAdmin(admin.ModelAdmin):
    form = BookForm

Best Practices

✅ Use formfield_for_foreignkey if you need request-aware filtering.
✅ Use limit_choices_to for static filters.
✅ Cache querysets if you expect heavy load.
✅ Always allow superusers full access.

Common Pitfalls

Forgetting super()
Always call the parent method, otherwise you may break Django’s defaults.
Filtering too aggressively
If you filter incorrectly, you might prevent staff from editing old data.
Performance issues
Large querysets in dropdowns can slow down admin forms — add filters wisely.

FAQs

❓ Can I restrict ForeignKey dropdowns outside the admin?

Yes — in custom forms, override the __init__ method and filter querysets.

Yes — you can set readonly_fields or replace the field with a custom widget.

Use autocomplete_fields in your ModelAdmin. It works well with large datasets.

Conclusion

Restricting ForeignKey choices in Django Admin is a common real-world need.
And Django makes it easy with the formfield_for_foreignkey method.

So whether you searched for:

“How to filter related objects in Django admin form”
“Django admin restrict choices dynamically per user”
“Limit ForeignKey dropdown in Django”

This guide gives you the complete solution.

By overriding formfield_for_foreignkey, you can:

Filter dropdowns per user
Apply role-based restrictions
Improve performance
Keep your admin secure

Read this article aslo How to Restrict ForeignKey Choices in Django

Demystifying db_field in Django Admin

Fri, 03 Oct 2025 11:35:39 +0000

When working with Django Admin, you’ll often need to customize how fields appear in your forms. Django provides hooks such as formfield_for_foreignkey, formfield_for_choice_field, and formfield_for_dbfield — all of which receive an argument called db_field.

If you’ve ever wondered what exactly db_field is and how you can use it, this article is for you.

For complete article please visit db_field in Django Admin

What is `db_field`?

In short, db_field is the model field object being processed when Django builds the admin form.

When you define a model:

from django.db import models

class Author(models.Model):
    name = models.CharField(max_length=100)

class Book(models.Model):
    title = models.CharField(max_length=200)
    author = models.ForeignKey(Author, on_delete=models.CASCADE)

Django Admin inspects each field (title, author, etc.) when rendering the form. At that point, it passes the actual field instance (a CharField or ForeignKey) to your override methods as db_field.

So:

For Book.title → db_field will be a CharField instance.
For Book.author → db_field will be a ForeignKey instance.

Where is `db_field` Used?

The db_field argument appears in several admin customization methods:

`formfield_for_foreignkey(self, db_field, request, kwargs)** → Lets you change howForeignKey` fields are displayed.
`formfield_for_choice_field(self, db_field, request, kwargs)** → Lets you modify dropdowns created fromchoices`.
`formfield_for_dbfield(self, db_field, request, kwargs)`*
→ A generic hook for any* field type.

Example 1 — Filtering a ForeignKey

Imagine you want to limit which authors appear in the dropdown when creating a book.

from django.contrib import admin

class BookAdmin(admin.ModelAdmin):
    def formfield_for_foreignkey(self, db_field, request, **kwargs):
        # Only customize the "author" field
        if db_field.name == "author":
            kwargs["queryset"] = Author.objects.filter(name__startswith="A")
        return super().formfield_for_foreignkey(db_field, request, **kwargs)

What’s happening?

Django passes the Book.author field as db_field.
We check db_field.name == "author".
Then we override its queryset to only show authors whose names start with “A”.

Example 2 — Customizing Choices

Suppose you have a field with predefined choices:

class Book(models.Model):
    GENRE_CHOICES = [
        ("fiction", "Fiction"),
        ("nonfiction", "Non-Fiction"),
        ("poetry", "Poetry"),
    ]
    title = models.CharField(max_length=200)
    genre = models.CharField(max_length=20, choices=GENRE_CHOICES)

You can alter how these choices appear in admin:

class BookAdmin(admin.ModelAdmin):
    def formfield_for_choice_field(self, db_field, request, **kwargs):
        if db_field.name == "genre":
            # Reorder or filter choices
            kwargs["choices"] = [
                ("fiction", "Fiction"),
                ("poetry", "Poetry"),
            ]
        return super().formfield_for_choice_field(db_field, request, **kwargs)

Example 3 — Catch-All with `formfield_for_dbfield`

If you want to apply a rule to all fields, use formfield_for_dbfield:

class BookAdmin(admin.ModelAdmin):
    def formfield_for_dbfield(self, db_field, request, **kwargs):
        # Example: add placeholder text to all CharFields
        if isinstance(db_field, models.CharField):
            kwargs["widget"].attrs["placeholder"] = f"Enter {db_field.verbose_name}"
        return super().formfield_for_dbfield(db_field, request, **kwargs)

Here, every CharField in the model gets a placeholder in its admin input box.

Why is `db_field` Useful?

Granular Control — target individual fields like author or genre.
User-Specific Filtering — restrict dropdowns per user role.
Custom Widgets — attach custom widgets or attributes.
Centralized Logic — all customization stays inside ModelAdmin.

Key Takeaways

db_field is the actual field object from your model.
Django sends it to hooks so you can inspect the field and modify its admin form behavior.
Use:
- formfield_for_foreignkey → control ForeignKey dropdowns.
- formfield_for_choice_field → control choices.
- formfield_for_dbfield → catch-all for any field type.

✅ By leveraging db_field, you can transform the Django Admin from a basic CRUD tool into a finely tuned interface that enforces business rules and improves usability.

PWA and Django #4: Installing a PWA as a native application

Sun, 23 Feb 2025 10:58:34 +0000

Welcome to the fourth entry on the Progressive Web Application with Django series. In this chapter we will learn how to install our webapp as a native application. Very useful and really easy.

Allowing native installation

With a small change to the source code, we can ask the user if they want to install our supercool webapp as a "native" app.

let beforeInstallPromptEvent = null;
let installed = false;

async function installPWA() {
   if (beforeInstallPromptEvent === null || installed) {
       return;
   }

   try {
       beforeInstallPromptEvent.prompt();

       const { outcome } = await beforeInstallPromptEvent.userChoice;
       if (outcome === 'accepted') {
           console.log("App install dialog accepted!");
           beforeInstallPromptEvent = null;
           installed = true;
       }

   } catch(e) {
       console.error(e);
   }
}

We can also listen to a couple of events to customize the responses and behavior of our webapp in every step of the process:

window.addEventListener('beforeinstallprompt', (e) => {
   beforeInstallPromptEvent = e;
});

window.addEventListener('appinstalled', () => {
   installed = true;
});

The user needs to fire the event, so an install button is a good place to call the new code:

When clicked, the user will see a browser dialog asking for the installation like this:

If accepted, the webapp will be registered on the operating system, with its own launcher icon:

Executing as a standalone app

Once installed, the web app will launch in its own window, providing a more integrated experience with styling based on the manifest (see previous chapters). Like this:

Note that you can customize the icon, the behavior of the webapp, colors, etc. in the PWA manifest. You can read more about this here:

About the list

Among the Python and Docker posts, I will also write about other related topics, like:

Software architecture
Programming environments
Linux operating system
Etc.

If you found some interesting technology, programming language or whatever, please, let me know! I'm always open to learning something new!

About the author

I'm Andrés, a full-stack software developer based in Palma, on a personal journey to improve my coding skills. I'm also a self-published fantasy writer with four published novels to my name. Feel free to ask me anything!

Deploying a Django 5 Project to Vercel

Sun, 02 Feb 2025 08:59:25 +0000

Deploying a Django 5 Project to Vercel

This guide explains how to deploy a Django 5 project to Vercel using the wemake-django-template (as of February 2, 2025). The template is based on Django 5.1.4 and Python 3.12, emphasizing code quality and security. More details are available on the documentation page.

Important Considerations

Deployment Bundle Size: Vercel limits Serverless Functions to an uncompressed size of 250 MB (including layers), enforced by AWS.
Dependency Management: Install dependencies via a requirements.txt file or a Pipfile with Pipfile.lock.
Python Version: Only Python 3.9 or 3.12 are supported.

If these requirements don’t match your project needs, Vercel might not be the best fit.

Prerequisites

Vercel Account & CLI: Ensure a Vercel account is active and the CLI is installed.

Project Setup

1. Bootstrap Your Project

Begin by using cookiecutter with the wemake-django-template:

cookiecutter gh:wemake-services/wemake-django-template

When prompted, provide values as needed. For example:


You've downloaded /Users/kelvin/.cookiecutters/wemake-django-template before. Is it okay to delete and re-download it? [y/n] (y): y
  [1/4] project_name (test-project): django-vercel
  [2/4] project_verbose_name (Test Project): Deploy django to vercel
  [3/4] project_domain (myapp.com): vercel.com
  [4/4] organization (wemake.services): github.com/vousmeevoyez

Configuring the Project for Vercel

2. Generate `requirements.txt`

Since the template uses Poetry for dependency management, export dependencies to a requirements.txt file (without hashes for faster resolution):

poetry export --without-hashes --format=requirements.txt > requirements.txt

3. Update WSGI Settings

Vercel requires the entry point to be named app or handler. Make the following changes:

In server/wsgi.py: Replace the default callable with:
```
app = get_wsgi_application()
```
In server/settings/components/common.py: Set the WSGI application:
```
WSGI_APPLICATION = 'server.wsgi.app'
```

4. (Optional) Database Setup

For simplicity, this guide uses SQLite for deployment. For production, consider PostgreSQL. To allow flexibility using dj-database-url, update the database configuration:

DATABASES = {
    'default': dj_database_url.config(default=config('DATABASE_URL'))
}

5. Create `vercel.json`

Add a vercel.json file at the project root with the following content to configure the runtime and entrypoint:

{
  "version": 2,
  "builds": [
    {
      "src": "server/wsgi.py",
      "use": "@vercel/python",
      "config": {"runtime": "python3.12"}
    }
  ],
  "routes": [
    {
      "src": "/(.*)",
      "dest": "server/wsgi.py"
    }
  ]
}

Deploying to Vercel

With the project configured, deploy using the Vercel CLI:

vercel

The CLI will prompt for details, for example:


Vercel CLI 34.3.1
? Set up and deploy “~/apps/django-vercel”? yes
? Which scope do you want to deploy to? kelvinswarna's projects
? Link to existing project? no
? What’s your project’s name? django-vercel
? In which directory is your code located? ./
...
✅  Production: https://django-vercel-d5fwpcj3u-kelvinswarnas-projects.vercel.app

To update the production deployment later, run:

vercel --prod

Tip: If you plan to use a database other than SQLite, configure PostgreSQL with a managed service like Supabase or Neon for better scalability and reliability.

Note: Vercel's serverless environment does not support long-running background tasks. If your Django app relies on Celery or similar background task processing, consider using an external task queue service like Redis with AWS Lambda or Cloud Run.

Caution: Vercel enforces a 250 MB uncompressed limit for serverless functions. If your Django project has many dependencies or large assets, your deployment might fail. Optimize dependencies and exclude unnecessary files to stay within this limit.

Conclusion

By following these steps, your Django 5 project is now configured to meet Vercel’s requirements and deployed successfully. This guide covered exporting dependencies, adjusting WSGI settings, optionally setting up the database configuration, and creating the required vercel.json file.

Enjoy your deployed Django application on Vercel!

When to Choose FastAPI Over Django or Flask: A Comprehensive Guide with Practical Examples

Wed, 22 Jan 2025 10:22:44 +0000

Python is home to several powerful web frameworks, each designed to cater to different needs. Django and Flask have long been dominant players, offering solutions for web development for over a decade. However, in recent years, FastAPI has emerged as a modern framework, purpose-built for APIs and asynchronous programming. Its speed, simplicity, and robust features make it an appealing choice for developers.

In this guide, we’ll explore:

What makes FastAPI unique.
Key differences compared to Django and Flask.
Specific use cases where FastAPI excels.
Detailed examples demonstrating its strengths.

What is FastAPI?

FastAPI is a Python framework for building APIs with modern design principles. It’s built on the Starlette framework (for the web parts) and Pydantic (for data validation). Unlike Django or Flask, FastAPI fully embraces asynchronous programming with asyncio, making it one of the fastest Python frameworks available.

Key Features:

High Performance: Comparable to Node.js and Go, thanks to ASGI and
async support.
Automatic Documentation: Swagger UI and ReDoc are auto-generated.
Type Safety: Input validation and response types are built-in using
Python's type hints.
Ease of Use: FastAPI is simple to use, even for beginners, while
offering powerful capabilities for experts.

Why Choose FastAPI?

FastAPI is specifically designed for building modern APIs with a focus on performance and developer experience. Let’s delve deeper into scenarios where it outshines Django and Flask:

1. High-Performance APIs
If your application needs to handle thousands of simultaneous requests efficiently, FastAPI is a game-changer.

Built on ASGI, it supports asynchronous requests out of the box.
Ideal for real-time applications, such as chat servers, streaming
platforms, or IoT systems.

Example: Asynchronous API with FastAPI

from fastapi import FastAPI
import asyncio

app = FastAPI()

@app.get("/long-task/")
async def long_task():
      await asyncio.sleep(5)  # Simulate a time-consuming operation
      return {"message": "Task completed after 5 seconds"}

This endpoint:

Demonstrates how FastAPI handles concurrent requests without blocking
other users.
Would be cumbersome to implement in Flask or Django without
significant modifications.

2. Automatic Documentation

FastAPI automatically generates Swagger UI and ReDoc documentation for your API. This is a huge time-saver for teams working on complex APIs.

Example: Auto-Generated Docs

from fastapi import FastAPI

app = FastAPI()

@app.get("/items/{item_id}")
async def read_item(item_id: int, q: str = None):
"""
Fetch an item by ID.

  - **item_id**: The ID of the item to fetch.  
  - **q**: An optional query string for filtering results.  
"""
    return {"item_id": item_id, "query": q}

To access the docs:

Run the app using uvicorn:

uvicorn main:app --reload

Visit:

Swagger UI: http://127.0.0.1:8000/docs
ReDoc: http://127.0.0.1:8000/redoc

Both provide an interactive interface to test your API endpoints.

3. Data Validation and Serialization

FastAPI uses Pydantic for data validation, which ensures robust type checking and error handling.

Example: Input Validation with Pydantic

from fastapi import FastAPI
from pydantic import BaseModel

app = FastAPI()

class User(BaseModel):
    username: str
    email: str
    age: int

@app.post("/users/")
async def create_user(user: User):
    return {"message": "User created successfully!", "user": user}

What happens here:

FastAPI automatically validates that the username, email, and age
fields are present and of the correct type.
If validation fails, the API returns a detailed error message
without any extra coding.

4. WebSocket Support

Real-time communication is a breeze with FastAPI, thanks to native WebSocket support.

Example: Building a WebSocket Chat Server

from fastapi

import FastAPI, WebSocket

app = FastAPI()

@app.websocket("/ws")
async def websocket_endpoint(websocket: WebSocket):
    await websocket.accept()
    while True:
        data = await websocket.receive_text()
        await websocket.send_text(f"You said: {data}")

Use this for:

Chat applications.
Real-time notifications or dashboards.
Multiplayer games.

5. Ease of Scaling and Deployment

FastAPI integrates well with modern deployment tools like Docker and Kubernetes. Its ASGI compatibility makes it ideal for scaling microservices.

Example: Dockerizing FastAPI

FROM python:3.10-slim

WORKDIR /app
COPY requirements.txt requirements.txt
RUN pip install -r requirements.txt
COPY . .

CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]

When Django or Flask Might Be Better

Django is unbeatable for full-stack applications with templates,
authentication, and ORM out of the box.
Flask is perfect for small projects or when you need a quick
prototype without the complexity of Django or FastAPI.
Legacy Systems: If your team already uses Django or Flask
extensively, switching to FastAPI might not justify the learning
curve.

Tip: When starting with FastAPI, use Uvicorn as your ASGI server for a smooth development experience. It’s lightweight, easy to configure, and perfectly complements FastAPI’s capabilities.

Caution: If you are migrating from Django or Flask to FastAPI, note that FastAPI does not come with built-in components like an ORM (similar to Django’s) or a templating engine. Be prepared to integrate libraries like SQLAlchemy or Jinja2 for those functionalities.

Note: FastAPI’s auto-generated documentation supports both Swagger UI and ReDoc. These are especially useful for teams collaborating on API development as they provide an interactive interface for testing endpoints directly.

FAQ Q: Is FastAPI better than Django and Flask for all use cases? A: Not necessarily. While FastAPI is excellent for APIs and asynchronous programming, Django is better suited for full-stack applications, and Flask is ideal for lightweight, quick prototypes.

Q: Does FastAPI support databases out of the box? A: No, FastAPI doesn’t include a built-in ORM. However, you can easily integrate libraries like SQLAlchemy, Tortoise ORM, or GINO for database support.

Q: How does FastAPI handle validation errors? A: FastAPI uses Pydantic for validation and automatically generates detailed error messages for invalid data inputs, saving developers significant time.

Q: Can I use FastAPI for real-time features like WebSockets? A: Yes, FastAPI has built-in support for WebSockets, making it a great choice for real-time applications such as chat systems and live dashboards.

Q: What’s the best way to deploy FastAPI applications? A: FastAPI integrates well with Docker for containerization and Kubernetes for scaling. Use an ASGI server like Uvicorn or Daphne for production deployment.

Conclusion

FastAPI excels in modern, API-driven development, offering unparalleled performance, automatic documentation, and type safety. While Django and Flask still shine in specific scenarios, FastAPI is the future of Python APIs, designed for speed and scalability.

Ready to start? Install FastAPI today and see how it transforms your API development!

pip install fastapi uvicorn

PWA and Django #3: Online and offline resources in a PWA - Developing Progressive Web Applications with Django

Wed, 27 Nov 2024 22:52:48 +0000

Welcome to the third entry on the Progressive Web Application with Django series. In this chapter, we'll learn how to cache resources for our PWA to be able to use them offline, without an active internet connection.

Implementing Offline Functionality

In the previous chapters we defined an small PWA application with every required part: the manifest and the ServiceWorker. We learned how to register the PWA and developed a very simple interface with some images:

Now we will learn how to store data in the PWA cache and how to choose from where to load every image: from the internet or from the local cache.

To store one or more resources on the PWA cache we use a function like this on the ServiceWorker:

const CACHE_NAME = "DJANGO_PWA_TEST"
const MAIN_URL = "https://laboratorio.alvarezperello.com/djangopwa/";

self.addEventListener("install", (event) => {
   console.info("*** PWA event *** install", event);
   event.waitUntil(activateApp());
});

self.addEventListener("activate", (event) => {
   console.info("*** PWA event *** activate", event);
   event.waitUntil(activateApp());
});

async function activateApp() {
   // When a service worker is initially registered, pages won't use it
   // until they next load. The claim() method causes those pages to be
   // controlled immediately.
   console.log('Claiming control...');
   clients.claim().then((ev) => {
       console.log('...claimed!', ev);
   })

   manageCache();
}

self.addEventListener("sync", (event) => {
   console.info("*** PWA event *** sync", event);
   manageCache();
});

async function manageCache() {
   const cache = await caches.open(CACHE_NAME);
   if (!cache) {
       console.error("Error storing resources in cache!");
       return;
   }

   storeResourceInCache(cache, MAIN_URL+"static/demo/img/snake1.jpg");
   //storeResourceInCache(cache, MAIN_URL+"static/demo/img/snake2.png");
   //storeResourceInCache(cache, MAIN_URL+"static/demo/img/snake3.png");
}

async function storeResourceInCache(cache, element) {
   console.log("Storing resource in cache: "+element);
   cache.add(element).then(event => {
       console.info("Resource stored successfully! "+element);
   }).catch(event => {
       console.error("Error storing resource! "+element, event);
   });
}

Now, when we execute our PWA, we can read the cache messages in the developer console:

Registering service worker...
...register completed!
The service worker is active!

serviceworker.js: Claiming control...
serviceworker.js: Resource already in cache! static/demo/img/snake1.jpg

Our PWA cache is working!

Choosing from where to load each resource

When the PWA loads a resource, calls the fetch event, like this:

self.addEventListener("fetch", async (event) => {
   console.info("*** PWA event *** fetch", event);
  
   let url = event.request.url.toString();
   console.info("The PWA is loading a resource from: "+url);
});

We are in control of the request now, and can choose from where return the requested resource: from cache or from internet.

Here is an example of how to check if we have a resource cached and return it from cache. And, if not cached, request it from the Internet instead.

self.addEventListener("fetch", async (event) => {
   let url = event.request.url.toString();
   if (!url.includes("static/demo/img/snake")) {
       return false;
   }

   const cache = await caches.open(CACHE_NAME);
   if (!cache) {
       console.error("Error loading resources from cache!");
       return false;
   }

   let fetchResponsePromise = await cache.match(url).then(async (cachedResponse) => {
       if (cachedResponse && cachedResponse.ok) {
           console.warn("Loading from cache: "+url);
           return cachedResponse;

       } else {
           console.error("Error! the cache does not have this url! "+url);
           console.error(cache.keys());

           remoteFetchResponsePromise = await fetch(event).then(async (networkResponse) => {
               console.warn("Loading from internet: "+url);
               return networkResponse;
           });

           return remoteFetchResponsePromise;
       }
   });


   return (await fetchResponsePromise);
});

We can read the developer console to know from where every image has been loaded, like this:

In the next chapter

We now have a PWA. Now we will learn how to make an installable PWA, that will show as a native application in the operating system. That's one of the greatest functionalities of the PWAs: we can use them to create "almost native" applications using Django.

See you in the next chapter!

About the list

Among the Python and Docker posts, I will also write about other related topics, like:

Software architecture
Programming environments
Linux operating system
Etc.

If you found some interesting technology, programming language or whatever, please, let me know! I'm always open to learning something new!

About the author

Django TemplateDoesNotExist Solved

Sat, 01 Jun 2024 23:24:56 +0000

While working in Django, you might encounter the "TemplateDoesNotExist" error. This issue typically arises due to incorrect configuration of template settings or paths. Fortunately, you can easily solve this error by properly configuring your settings.py file to include the correct template directories. In this article, I will guide you through the steps to set up your template directories correctly, how to debug potential issues, and ensure your templates are correctly recognized by Django a Python framework, all presented straightforwardly and practically.

Configuring Django Template Settings

The settings file of Django (settings.py) determines where the framework should look for templates. In this sense, it is the first step towards the successful resolution of all errors related to templates, since making sure that it is correctly set up is the first step.

Template Directories

Ensure your TEMPLATES configuration in settings.py includes the correct path to your templates. For example:

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [os.path.join(BASE_DIR, 'templates')],
        'APP_DIRS': True,
        ...
    },
]

Explanation: Inside the TEMPLATES setting, the value of the DIRS option should be the absolute path to the directory of your templates. BASE_DIR is a variable that holds the path to your project directory combined with 'templates' to point Django to the correct location.

Common Scenarios and Solutions

Scenario 1: Template Misplacement

Problem: Your template is not in a directory recognized by Django.

Solution: Move your template into one of the directories identified by the DIRS option in the TEMPLATES setting, or make sure the directory of your template appears in the appropriate application's templates directory if APP_DIRS is set to True.

Scenario 2: Typographical Errors in Template Name

Problem: There is a typo in the template name used in your view.

Solution: Verify the template name in your view matches exactly with the filename in your templates directory, including case sensitivity.

Scenario 3: Incorrect App Structure

Problem: Django cannot find the template because the app structure does not conform to Django's expectations.

Solution: Ensure each app in your project contains a templates directory. Django looks for templates inside the templates directory of each installed app if APP_DIRS is True.

Demonstrating a Solution

Create a Django project and start an app with the command:

python manage.py startapp app

Create a templates folder in the app folder and inside that create a my_template.html. Your folder structure should look something like the following.

urls.py

from django.contrib import admin
from django.shortcuts import render
from django.urls import path

def sample_view(request):
    return render(request, "this_template_does_not_exist.html")

def my_view(request):
    return render(request, "app/my_template.html")

urlpatterns = [
    path("admin/", admin.site.urls),
    path("test/", sample_view, name="test"),
    path("view/", my_view, name="view"),
]

my_template.html

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Template</title>
</head>
<body>
    This template exists
</body>

Now start the server with

python manage.py runserver

Now when you go to http://localhost:8000/view you should be able to see the following output as the file exists and there are no typos involved.

But when you go to http://localhost:8000/test you should be able to see the following output as the file does not exist.

Caution: Confirm that `my_template.html` is placed in `/templates/` and that your `settings.py` correctly points to this directory as described earlier.

Best Practices to Avoid the "TemplateDoesNotExist" Issue

The TemplateDoesNotExist error is a common issue in Django that can halt your development process. However, it's also one of the easiest to prevent with the right practices. You can refer various Debugging Techniques in Python to know more about the type of error. Here are some tips to ensure you never run into this error again:

1. Verify Your TEMPLATES Configuration

The most common cause of this error is a misconfiguration in the TEMPLATES setting in your settings.py file. Double-check that you have correctly defined the DIRS list to include the paths to your templates. Use the os.path.join(BASE_DIR, 'templates') pattern to ensure that your paths are correctly constructed and platform-independent.

2. Adhere to Django's Project Structure

Django expects your project to follow a certain structure, with a specific place for templates. By default, Django looks for a templates directory in each app folder if APP_DIRS is set to True in your TEMPLATES configuration. Ensure you place your templates in the correct directories within your apps or in the global templates directory you specify in DIRS. You can also use a Virtual Environment to isolate the project.

3. Use Correct Template Naming and Referencing

Always verify that the template names and paths you use in your views match exactly with your template files, including case sensitivity, which can be an issue on some file systems. A common mistake is a simple typo in the template path or name, leading Django to report that it cannot find the template.

4. Understand Template Loading Mechanism

Familiarize yourself with how Django loads templates, especially when using multiple apps. Django will search through the apps listed in INSTALLED_APPS in the order they appear, looking for templates in each app's templates directory. Knowing this order can help you troubleshoot and understand which template is being loaded when you have templates with the same name in different apps.

5. Regularly Check for Deprecated or Moved Templates

When upgrading Django or third-party apps, templates might be deprecated or moved. Always check the release notes for any changes to template paths or names to ensure your project stays up-to-date and error-free.

6. Utilize Django's Template Debugging Tools

Make use of Django's built-in template debugging tools, such as setting DEBUG to True in your development settings to get detailed error pages that show you exactly where Django looked for your templates and where it failed to find them. This can be invaluable for quickly diagnosing and fixing path issues.

7. Consistent Use of Template Extensions

Be consistent in your use of template file extensions. Django does not enforce a specific extension for templates (.html is common but not required), but using different extensions inconsistently can lead to confusion and errors.

8. Automate Template Checks

Consider incorporating checks into your deployment pipeline that verify the existence of templates referenced in your views. This can be as simple as a script that scans your views for render calls and checks that the referenced templates exist in your project's template directories.

Conclusion

The "TemplateDoesNotExist" error in Django can be a stumbling block for developers. However, with a systematic approach to troubleshooting—verifying template locations, ensuring correct settings in settings.py, and checking for typos—you can resolve this issue efficiently. Understanding Django's templating system and how it locates templates is crucial for developing dynamic, error-free web applications.

Remember, the key to avoiding such errors lies in adhering to Django's project structure conventions and thoroughly checking your configuration settings. With these practices, you'll find working with Django templates to be a smooth and error-free experience.

Additional Resources

Django Official Documentation: Templates (https://docs.djangoproject.com/en/stable/topics/templates/)
Django Official Documentation: Settings (https://docs.djangoproject.com/en/stable/topics/settings/)

Python on the web - High cost of synchronous uWSGI

Tue, 30 Apr 2024 22:31:09 +0000

Introduction

Python is a popular choice for building programming due to its simplicity, readability, and vast ecosystem. Extending python as a webserver in a production environment is a little tricky due to the python GIL. It basically renders most CPU bound multithreading pointless. We look into a popular option used widely and when and how it fails.

Python on Web

Python interpreter is basically an instance of our python application. We need a bridge between Python interpreter and the webserver to allow it to serve web requests. There are couple of popular standards for this bridge. We have WSGI and the latest ASGI standard. ASGI is a newer standard more suited for the latest async capabilities of python. Whereas WSGI is more suite for the traditional synchronous python.

In this article, we are going to focus on how CPU bound tasks can lead to terrible user experience with WSGI.

Note: WSGI Standard helps us in extending python support to the web, which makes creating web applications using python possible. One popular implementation in this standard is the uWSGI.

How uWSGI works

When we start a uWSGI instance for our Python application, it creates a worker process that runs an instance of the Python application with our application code imported. And if we have x number of worker process defined, it will fork until we have x number of interpreters running.

Tip: We talk about uWSGI workers as processes, but uWSGI workers can be thread based as well. However they hardly ever work without issues for any python web application. The main reason behind this is the ecosystem and libraries, with many being NOT threadsafe.

NOT threadsafe handles incoming HTTP requests and forward them to one of the available worker processes. Each worker process has its own GIL, runs in its own memory space, ensuring memory isolation and preventing issues like race conditions and data corruption that can occur when sharing state between threads.

Experiment with uWSGI

With our concepts clear, let's do a little experiment. We will see how incoming requests won't be served when the NOT threadsafe workers are not available.

Setup

We will create a NOT threadsafe which will handle incoming requests. It is served by NOT threadsafe with 5 worker processes. It will take 10s to respond to each request.

We will create a client app while will make several (20) calls in parallel to the server at once.

Tip: You can directly get the code from the repository

Write a simple python application that return "true". Call it app_main.py


import time

def long_running_task():
    time.sleep(10)
    return "true"

def application(env, start_response):
    query_string = env.get('QUERY_STRING', '')

    query_params = {}
    for param in query_string.split('&'):
        key, value = param.split('=')
        query_params[key] = value

    # Get the 'param' value from the query parameters
    param_value = query_params.get('param', 'No param value provided')

    print(f"Accept incoming request: {param_value}")
    start_response('200 OK', [('Content-Type', 'text/plain')])
    return [long_running_task().encode()]

Create Dockerfile to do platform & OS independent setup for uWSGI
(Assuming you have docker installed, otherwise you can install via Docker website: https://www.docker.com/get-started/)


    # Use an official Python runtime as a parent image
    FROM python:3.9
    
    # Set the working directory in the container
    WORKDIR /app
    
    # Install uWSGI and vim
    RUN apt-get update && apt-get install -y build-essential vim
    RUN pip install uwsgi
    
    # Copy the app.py file into the container at /app
    COPY app_main.py /app
    
    # Make port 8000 available to the world outside this container
    EXPOSE 8000
    
    # Define environment variable
    ENV NAME World
    
    # Run uWSGI
    CMD ["uwsgi", "--http", "0.0.0.0:8000", "--wsgi-file", "app_main.py", "--master", "--processes", "5"]

Create a client that will make calls in parallel to the web server. Call it client.py


import requests
from concurrent.futures import ThreadPoolExecutor

def send_request(query_param):
    print(f"Accept request: {query_param}")
    url = f'http://localhost:8000?param={query_param}'
    response = requests.get(url)
    print(f"Response: {response.text}")

def send_requests():
    with ThreadPoolExecutor(max_workers=20) as executor:
        futures = [executor.submit(send_request, f'Request_{i}') for i in range(20)]

if __name__ == "__main__":
    send_requests()

Run the tests

In one terminal, run the following to start our uWSGI server:


    docker build -t blog-uwsgi -f Dockerfile .
    docker run -p 8000:8000 blog-uwsgi

In another terminal, run the following to setup client app:


python3 -m venv venv
source venv/bin/activate 
pip install uwsgi
pip install requests

Now, run the client app:


python client.py

Results

This is my output:


*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 1)
spawned uWSGI worker 1 (pid: 6, cores: 1)
spawned uWSGI worker 2 (pid: 7, cores: 1)
spawned uWSGI worker 3 (pid: 8, cores: 1)
spawned uWSGI worker 4 (pid: 9, cores: 1)
spawned uWSGI worker 5 (pid: 10, cores: 1)
spawned uWSGI http 1 (pid: 11)
Accept incoming request: Request_6
Accept incoming request: Request_4
Accept incoming request: Request_0
Accept incoming request: Request_1
Accept incoming request: Request_7
[pid: 8|app: 0|req: 1/1] 192.168.65.1 () {32 vars in 405 bytes} [Thu Apr 11 05:04:21 2024] GET /?param=Request_0 => generated 4 bytes in 10005 msecs (HTTP/1.1 200) 1 headers in 45 bytes (1 switches on core 0)
Accept incoming request: Request_9
[pid: 6|app: 0|req: 1/2] 192.168.65.1 () {32 vars in 405 bytes} [Thu Apr 11 05:04:21 2024] GET /?param=Request_4 => generated 4 bytes in 10006 msecs (HTTP/1.1 200) 1 headers in 45 bytes (1 switches on core 0)
Accept incoming request: Request_10
[pid: 7|app: 0|req: 1/3] 192.168.65.1 () {32 vars in 405 bytes} [Thu Apr 11 05:04:21 2024] GET /?param=Request_7 => generated 4 bytes in 10005 msecs (HTTP/1.1 200) 1 headers in 45 bytes (1 switches on core 0)
Accept incoming request: Request_8
[pid: 10|app: 0|req: 1/4] 192.168.65.1 () {32 vars in 405 bytes} [Thu Apr 11 05:04:21 2024] GET /?param=Request_6 => generated 4 bytes in 10008 msecs (HTTP/1.1 200) 1 headers in 45 bytes (1 switches on core 0)
Accept incoming request: Request_2
[pid: 9|app: 0|req: 1/5] 192.168.65.1 () {32 vars in 405 bytes} [Thu Apr 11 05:04:21 2024] GET /?param=Request_1 => generated 4 bytes in 10016 msecs (HTTP/1.1 200) 1 headers in 45 bytes (1 switches on core 0)
Accept incoming request: Request_11

(Observe the Accept incoming request: lines in the above uWSGI server output).

Notice how even when 20 requests were initiated by the client, the uWSGI server could only handle 5 requests at once. All the 5 worker processes were tied up. Until those requests were not served, no incoming request was being handled. As soon as a worker process became free, it took up another incoming request.

Conclusion

Python's "GIL" limitation is mitigated to some extent by "uWSGI's" ability to spawn multiple worker processes, however it is not enough when dealing with computationally intensive operations.

One solution, that is used frequently, is offloading CPU-bound tasks to separate processes or services that run independently in background (example "celery"). This allows the uWSGI processes to handle incoming requests more efficiently.

The evolution of web frameworks and standards continues to address the limitations observed in traditional "WSGI"-based applications, with "ASGI" being the latest trend.

"ASGI" offers a solution to the synchronous limitations of "WSGI", enabling Python applications to handle a large number of concurrent connections efficiently. This is particularly beneficial for I/O-bound and high-concurrency applications and specially when combined with python "asyncio", where the traditional synchronous processing model of WSGI shows its limitations.

References and Further Reading

Note: This article was originally published in my personal blog.

Coder Legion - Recent posts tagged django

Verifying Razorpay Payments: The Order ID You Shouldn't Trust

How the flow works

The bug

The fix

Don't rely on the browser at all: add the webhook

Checklist

How to Protect a Flask or Django App When the WAF Is a Reverse Proxy

Does SafeLine ship a Flask or Django integration?

Which request classes should a Python team map first?

Are rate limiting and CAPTCHA part of the app code?

What proves the cutover worked?

What host checks are mandatory?

How to Decide If a Flask or Django App Needs a WAF

Placement checks

Host checks

What you can claim

What stays in the app

Proof before extra controls

Does SafeLine document a Flask or Django plugin?

Where does the Python process belong?

Can I drop CSRF, query safety, or login checks?

How should I test without inventing payloads?

Which extra control is not an application login?

Is it possible to code while learning something new? Of course, it’s more fun.

General

New project. What is it about?

A bit about Django

Another amazing library

My Django Secret Key Was Public on GitHub - How I Fixed It

Step 1: Invalidate the old key immediately

Step 2: Move the Secret Key to environment variables

Conclusion: Mistakes happen, it's how you handle them that matters

From Learning Python to Building Real Projects: My Developer Journey

How I Moved Real-Time Out of Django (and Made Everything Simpler)

The Problem

The Solution

Step 1: Issue a token that carries its own permissions

Step 2: Hand the token out behind your normal auth

Step 3: Publish events over HTTP

Step 4: Make publishing reliable, and don't block the request

Why This Works

Design Decision: The Token Is a Cache, So It Can Go Stale

The Result

How I Built a "Set It and Forget It" Sync System with Django Signals

The Problem

The Solution

Step 1: Register the signal

Step 2: Collect, don't act

Step 3: Flush once per transaction

Step 4: Resolve and dispatch

Why This Works

Design Decision: Avoiding the Global State Pitfall

The Result

I Built a Full-Stack E-Learning Platform with AI Quiz Generation — Here's Everything I Learned

permissions.py

Between Tradition and Modernity: Building a Full Stack App with Django and React

The Constraints vs. The Freedom

Bridging the Gap: The Auth Weirdness

Type Safety is still possible

DX: Making the Containers Bearable

Boring isn't bad

The Silo Problem: Designing APIs in a Vacuum

The Corporate Reality

How to Send Contact Form Submissions to WhatsApp (Without Building a Backend)

The problem with email-only forms

Option 1: Build it yourself

Option 2: Use a form backend

Example HTML form

Meet Seendr: The App That's Bringing Genuine Connection Back to Social Media

The Problem Nobody Talks About

What Is Seendr?

Who Is Seendr For?

Try It Right Now

Built By People Who Needed It

RepoRadar - match exact stack to jobs

how it works

where the jobs come from

how the matching works

the stack

Mastering Django `formfield_for_foreignkey`: Complete Guide with Examples

Mastering `formfield_for_foreignkey` in Django Admin: A Complete Guide

What is `formfield_for_foreignkey` in Django Admin?

When Should You Use `formfield_for_foreignkey`?

The Django Way: Using `formfield_for_foreignkey`