Coder Legion - Recent posts tagged performance

What Is Idempotency?

Fri, 30 Oct 2026 09:28:35 +0000

🚀 What Happens When Your API Receives the Same Request Twice?

Imagine a customer clicks “Pay Now” twice because the page freezes.

Or a payment gateway retries a request because it never received your response.

💥 Without proper protection, your application might charge the customer twice!

This is where Idempotency becomes essential.

🔍 What Is Idempotency?

Idempotency means that performing the same operation multiple times produces the same result as performing it once.

For example, if a client retries a payment request, the system should return the original result instead of creating another payment.

🛠️ How Do We Implement It in Ruby on Rails?

A common approach is using an Idempotency-Key.

The client sends a unique key with the request:

POST /api/v1/payments
Idempotency-Key: order-123-payment

Rails can use that key to identify duplicate requests.

Step 1: Add a database constraint

add_index :payments, :idempotency_key, unique: true

Step 2: Store the key with the payment

Payment.create!(
  idempotency_key: idempotency_key,
  amount: amount,
  status: "pending"
)

Important: This is only a simplified example. A production implementation must handle duplicate-key exceptions, concurrent requests, and retries safely.

You should also define what happens when the same key is reused with different request parameters.

For payment APIs, store and return the original result for a matching key rather than processing the payment again.

💎 Why Is Idempotency Important?

✅ Prevents Duplicate Payments — Avoid charging customers twice when requests are retried.

✅ Makes Retries Safer — Network failures become easier to handle.

✅ Improves API Reliability — Clients can retry requests without accidentally repeating business operations.

✅ Protects Background Jobs — Prevent duplicate emails, orders, or other side effects when jobs run more than once.

✅ Supports Distributed Systems — Helps manage unreliable networks and repeated messages.

⚠️ One Important Detail

Idempotency is not just checking whether a request has been received before.

You must handle concurrent requests, database transactions, unique constraints, and external service calls carefully.

For example, a database transaction alone cannot guarantee that an external payment provider won't charge a customer twice.

💡 The goal is simple: make retries safe, even when failures happen.

Because in real-world applications, requests will fail, networks will time out, and background jobs will retry.

Your architecture should be ready for it!

💬 How do you handle duplicate requests in your Rails applications?

RubyOnRails #API #Idempotency #BackendDevelopment #SoftwareArchitecture #DistributedSystems #WebDevelopment #SoftwareEngineering

Redux Refactoring Playbook: From Action-Types Soup to RTK Query — Without Rewriting the App

Thu, 29 Oct 2026 08:51:02 +0000

Every React codebase of a certain age has one: the legacy Redux store. Hundreds of thunk actions, manually managed loading flags, per-feature cache invalidation, and a polling timer per data type pulling the full payload whether anything changed or not.

Ours lived under a real-time monitoring dashboard tracking 1,000+ GSM installations at Orange Romania — and by the time we migrated to RTK Query, the story ended with dashboard responsiveness up ~40%, telemetry traffic down 20%, and "frozen dashboard" complaints down 30%.

The previous article told that performance story. This one is the refactoring mechanics: in what order things moved, which decisions unlocked the next one, and the one rule that made a no-rewrite migration possible.

Rule 1 — Strangle, don't rewrite

The store wasn't broken in one place; it was broken everywhere: a switch-case reducer file nobody fully read, five middleware, isLoading / isRefreshing / hasError flag families per resource, and component-level useEffect chains firing fetches in a specific order "because it worked."

A big-bang rewrite was a non-starter: the dashboard was used every day by operations teams. So the rule was:

One page migrates per PR. The rest of the app keeps running on the legacy store untouched.

Redux makes this possible because combineReducers is natural strangler scaffolding:

const rootReducer = combineReducers({
  // new pages live here
  ordersApi: ordersApi.reducer,          // RTK Query
  invoices: invoicesSlice.reducer,       // RTK slice
  // legacy pages keep their ducks until their turn
  legacyTelemetry: legacyTelemetryReducer, // switch-case reducer, retired last
});

Each migration PR carried its own before/after render counts and INP p75 from a fixed scripted interaction. If the numbers moved the wrong way, the PR didn't merge — refactoring without a measurement loop is just rewriting with extra steps.

The code was the easy half. The pitch mattered as much: I sold this as risk control, not cleanup. Every PR shippable alone meant no feature-freeze window that anyone had to buy; the cost I did have to name up front was the parallel-store tax: two stores side by side meant duplicated fixtures and slower reviews while a slice waited for its turn. Leadership signed off because every migration PR ended with a measurement, not with the word "done". One more org-side lesson: the change-flag contract needed a backend buy-in, and I opened that conversation politely, late. Open it on day one; that delay cost us a month.

Rule 2 — Selectors are the API. Freeze them first.

The trick that removed 90% of the migration risk: components never touched state shapes directly.

The legacy store had selector functions (selectOrders, selectNodeById, selectIsNodeActive). Before migrating anything, we froze them as the contract and then made each selector a re-export shim that could source data from either world:

// selectors/orders.js
import { legacyOrders } from './legacySelectors';
import { useGetOrdersQuery } from '../api/ordersApi';

export function useOrders(selector) {
  const { data, isFetching } = useGetOrdersQuery();
  // legacy pages keep working against the same contract
  return useMemo(() => selector(data ?? legacyOrders()), [data, selector]);
}

(Simplified — the real shims carried fallbacks during the window when a page's data half-lived in the old store. The point is the shape: the component-facing API never moved, only its plumbing did.)

Result: a "migration PR" changed the data layer under a page, not forty components over it. That made page-by-page migration realistic rather than a nice idea.

Rule 3 — Kill loading flags with RTK Query, not with better thunks

There was a middle step available: keep the thunks and clean them up. I'd have kept every race. Request A finishing after request B silently overwrote fresher data, and that was our most common production bug, because each flag family carried its own private version of it.

So server state moved into RTK Query, and components declare what they need, not how to fetch it:

// before: every call site, every page
dispatch(fetchOrders());
// + component-local: isLoading, ordersError, lastRefreshed...

// after: the component declares the requirement
const { data: orders, isLoading } = useGetOrdersQuery(undefined, {
  pollingInterval: 0, // see Rule 4 — we don't poll blindly
});

Seventeen loading-flag families disappeared in this step. After it, a component tracking its own fetch lifecycle existed in exactly one place: a page that legitimately needed manual control.

Rule 4 — Polling moves from timers to tags

Old polling: one setInterval per data type, full payload every tick. On a dashboard watching 1,000+ nodes, that's thousands of identical requests an hour and constant re-render churn over data that hadn't changed.

New polling is no polling — it's a change-flag endpoint: a ten-byte version string the dashbo
ard polls on a timer, and the real data refetches when that string actually changes.

export const api = createApi({
  reducerPath: 'api',
  baseQuery: fetchBaseQuery({ baseUrl: '/api' }),
  tagTypes: ['Nodes'],
  endpoints: (build) => ({
    nodesChanged: build.query<{ version: string }, void>({
      query: () => 'nodes/changed',       // returns a version string only
    }),
    getNodes: build.query<NodeState[], void>({
      query: () => 'nodes',
      providesTags: ['Nodes'],
    }),
    // refetch tagged data only when the flag version changes:
    // a listener middleware polls `nodesChanged` (cheap, ~10 lines) and,
    // on version change, dispatches api.util.invalidateTags(['Nodes'])
  }),
});

const flagListener = createListenerMiddleware();

flagListener.startListening({
  predicate: (action, currentState, previousState) => {
    // the version lives in RTK Query's own cache, so the store
    // snapshot gives us both "current" and "what it used to be"
    const cur = nodesChanged.select()(currentState)?.data;
    const prev = nodesChanged.select()(previousState)?.data;
    return cur != null && cur !== prev;
  },
  effect: (action, api) => {
    api.dispatch(api.util.invalidateTags(['Nodes']));
  },
});

// the flag endpoint itself: cheap on purpose
//   query: () => 'nodes/changed',   // returns ~10 bytes: a version string
//   pollingInterval: 5000,

What happens on an update, end to end: the version goes into the cache and
predicate sees it differ from the previous snapshot, invalidateTags marks
every tagged entry stale without removing it (the screen keeps the old
data), mounted subscribers immediately refetch — deduplicated to one request
per endpoint+args no matter how many components listen — and one re-render
lands the new data. A component sees isLoading: false, isFetching: true
during that window, which is why nothing flashes. If the flag endpoint dies,
data simply goes stale until the next successful tick; nothing breaks. If
nodes changed five times between two ticks, one refetch covers it: the
version is a snapshot of "what is true now", not an event queue.

What stayed in Redux core

A mistake we avoided (mostly): RTK Query is for server state only. UI-owned state — selected node, filter panel state, column layout, wizard steps — stayed in plain createSlice. Migrating that into queries would have replaced a working pattern with a worse one.

The end state: slices for decisions, RTK Query for data, zero custom cache code, zero hand-rolled loading flags.

The honest ledger

From the published case, measured as perceived dashboard responsiveness (how fast fresh data lands after a change, without UI-thread churn):

Dashboard responsiveness: +~40%
"Frozen dashboard" complaints: −30%
REST telemetry traffic: −20%

What I can't give you is a LOC table with a precise count — we deleted more code than we wrote every migration PR, and by the end the store directory had visibly halved; I kept no exact tally, and a made-up precision here would be dishonest.

What I'd do differently

Start with RTK Query for any new page. Half the migration tail was legacy thunks for features that could have been endpoints from day one.
Freeze selectors before touching anything. The component-facing API is the migration's unit of safety.
Measure per PR, not per epic. The scripted interaction + INP check cost minutes per PR and caught two regressions a "it looks fine" test never would.
Write the change-flag backend contract early. The polling redesign was the single biggest win and it needed a backend buy-in — politeness about it cost us a month.

Your refactoring checklist

Inventory actions/thunks/loading flags — count them, this is your "before" baseline.
Freeze selectors as the component-facing contract.
Add RTK Query via combineReducers alongside the legacy store.
Migrate one page per PR; measure INP p75 + render count on a fixed interaction.
Server state → endpoints with tags; UI state stays in slices.
Replace timers with a change-flag contract before touching fetch logic.
Delete: action-type strings, flag families, custom cache code. The deletion count is the score.

The performance numbers come from the same production system as my earlier piece, A 40% Faster Fault-Detection Pipeline Without WebSockets — that one tells the polling-protocol story; this is the data-layer refactoring around it.

You Upgraded to Next.js 15 and Your Site Got Slower. The Caching Defaults Flipped

Tue, 27 Oct 2026 16:13:23 +0000

You upgrade from Next.js 14 to 15. The app compiles, the tests pass, nothing visibly breaks. A week later your database bill is higher, your pages feel slower, and your Time to First Byte graph has a step in it that lines up exactly with the deploy.

Nothing is broken. Something that used to happen for free stopped happening.

What Changed

In Next.js 14, data fetching was cached by default. A fetch in a Server Component ran once, the result was stored, and later requests reused it. You had to opt out of caching.

In Next.js 15, the default flipped. Per the upgrade notes:

fetch requests are no longer cached by default.
GET route handlers are no longer cached by default.
The client-side router cache no longer holds page segments between navigations by default.

So code you wrote once and never touched now behaves differently:

// app/products/page.tsx
export default async function ProductsPage() {
  const res = await fetch('https://api.example.com/products');
  const products = await res.json();
  return <ProductList products={products} />;
}

On 14, this fetched the data once and served the page from cache. On 15, every visitor triggers the API call, and because the page now depends on data fetched at request time, the route is rendered dynamically.

Why It Is Easy to Miss

Nothing errors. The code is valid. It just does more work.
Dev mode hides it. In next dev, nearly everything is rendered per request anyway.
The slowdown is gradual. With low traffic, an extra API call per request is invisible. It shows up when the traffic does.
Third-party limits bite later. If the API you call has rate limits or per-request billing, you find out from the invoice.

How to See It in 30 Seconds

Run a production build and read the route table at the end:

next build

Route (app)
┌ ○ /                 (Static)
├ ƒ /products         (Dynamic)
└ ƒ /blog/[slug]      (Dynamic)

○ means the page is pre-rendered once at build time. ƒ means it runs on the server for every request. A page that used to be static and is now ƒ is exactly what this upgrade can cause. Compare the table before and after the upgrade.

Fix 1: Cache One Request on Purpose

const res = await fetch('https://api.example.com/products', {
  cache: 'force-cache',
});

This restores the old behavior for that single request. Use it for data that does not change per user and rarely changes at all.

Fix 2: Cache With an Expiry

Most data is not permanent, just slow-changing. Revalidate it on a timer:

const res = await fetch('https://api.example.com/products', {
  next: { revalidate: 3600 }, // seconds
});

Visitors get the cached copy, and after an hour the next request refreshes it in the background. This is the right default for product lists, blog posts and prices.

Fix 3: Set It Once for the Whole Page

If several fetches on a page should share the same freshness, declare it at the route level:

// app/products/page.tsx
export const revalidate = 3600;

Everything that page renders is treated as cacheable for an hour, without repeating options on each call.

Fix 4: Route Handlers Need Their Own Opt-In

A GET route handler that returns the same data for everyone used to be cached automatically. Now you must say so:

// app/api/products/route.ts
export const dynamic = 'force-static';

export async function GET() {
  const products = await getProducts();
  return Response.json(products);
}

Without that line, the handler runs on every request.

The Part People Forget: Your Database Was Never Covered

Here is the subtle thing. The fetch cache only ever applied to fetch. If you read from MongoDB or Postgres directly with an ORM, this change did not touch those calls. Depending on whether the page used request data such as cookies, it was either rendered once at build time with the data frozen in, or rendered on every request. Neither is controlled by the fetch cache, so check your build table for those pages too.

To cache a database call at runtime, wrap it:

import { unstable_cache } from 'next/cache';

const getProducts = unstable_cache(
  async () => Product.find().lean(),
  ['products'],
  { revalidate: 3600, tags: ['products'] }
);

The tags option lets you clear that cache on demand with revalidateTag('products') after an edit. I wrote about why revalidateTag appears to do nothing when the data is not tagged in the first place. Newer Next.js versions keep moving toward explicit opt-in caching, so check the docs for the version you are on before copying any of this.

Should You Cache Everything Again?

No. The new default exists because stale data surprised people. Show a user their own cart, balance or notifications from a shared cache and you have a bug, or a data leak.

Data	Choice
Public, rarely changes (marketing pages, docs)	Cache it, long expiry
Public, changes often (product list, prices)	Cache with a short `revalidate`
Per-user (dashboard, cart, account)	Do not cache
Admin views	Do not cache

The Rule

When a framework changes a default, your old code changes behavior without a single line changing. After any major upgrade, run next build before and after and compare which routes are static (○) and which are dynamic (ƒ). Then opt back in on purpose, for the data that deserves it.

Slow server responses hurt the metrics Google measures. I break those down in plain language in my guide to Core Web Vitals, and if you want a Next.js starting point, you can browse my templates on pixelanas.com.

Get the templates: https://pixelanas.gumroad.com

Did a framework upgrade ever quietly change your app's behavior? Tell me what it was in the comments.

Anas, full-stack Next.js developer building SaaS products and premium templates. X: @ASheikh69751

Seventeen Seconds to Parse One Name: A Security Scan Across Eleven PHP Projects

Tue, 27 Oct 2026 13:32:42 +0000

This is a 16 KB name:

$row = 'Smith, ' . str_repeat('V ', 8188);
(new \Iliaal\NameParser\Parser())->parse($row);

On nameparser 1.5.0, the last version before the fix, that call takes 17.4 seconds of CPU on my i9-13950HX. On 1.7.0 it takes 0.064 seconds. Each V is a candidate roman-numeral suffix, and the old suffix scan walked back over every earlier token for each one, so the cost grew with the square of the input. Nothing about it is exotic. It is a plausible shape for a junk row in a CSV import, and it pins a PHP worker for as long as the row keeps coming.

I didn't find it. A security scan did, and the same scan ran across all ten of my PHP extensions as well. In the first three weeks of October each project got a full pass: one coordinating session, a set of subagents reading code and writing proof-of-concept exploits, and a separate verifier that re-ran every finding before it counted, on its own ASan and release builds for the C code. Everything below ships in releases from the last few weeks.

Two other things landed in the same window. PHP's October security releases changed what memory_limit does, which forced code changes in three extensions and a same-day round of re-releases in five more. And DuckDB 2.0 went GA; pdo_duckdb bundles it as of today.

What the scans found, and what is fixed

Every project in the table has a release that carries its fixes. The counts are findings confirmed by the verifier, fixed, and released; severities are the verifier's, not the scanner's first guess.

Project	Fixed in	Confirmed and fixed
phpser	0.7.1	19: 1 high, 5 medium, 13 low
mdparser	0.8.0	12: 1 high, 5 medium, 5 low, 1 informational
fastchart	1.8.0	15 reproduced, highest medium, including vendored plutovg and plutosvg; two UBSan sites in vendored code only partly addressed
php_clickhouse	0.13.0	1 high, 1 medium, plus low-severity binding fixes and 7 server-triggered crashes in clickhouse-cpp
pdo_duckdb	0.8.0	6: 1 high, 1 medium, 1 low-medium, 2 low, 1 informational
fastjson	0.10.0	4: 1 medium, 3 low, plus 1 documentation correction
php_excel	2.9.0	5: 3 low, 2 informational
statgrab	2.3.0	4: 2 medium, 2 low, plus a build bug
fast_uuid	0.8.1	5: 4 low, 1 by-design behaviour changed anyway
phonetic	0.4.4	3: 1 medium, 2 low
nameparser	1.6.0	5: 1 medium, 1 low, 3 informational

A few findings in upstream libraries are reported privately to their maintainers and aren't described here until fixes ship.

The coordinating model was Mythos 5.1, but most of the work ran on other models: on fastjson, 695 of the 702 assistant messages came from Opus 5.5 subagents doing the reading, fuzzing, and PoC writing. The phpser scan doubled as an A/B/C test of my own tooling against a bare prompt, which I wrote up on X at the time. So the table measures the process more than any one model.

fast_uuid and phonetic came back with zero memory-safety bugs, and php_excel's wrapper with nothing above low severity.

Bugs that needed no attacker

The worst php_clickhouse finding in practice wasn't the high-severity one. It was B-4, which corrupted data on ordinary queries. Decimal and DateTime64 decoding cached each column's scale in a memo keyed on the raw address of the column's type object. clickhouse-cpp allocates a fresh type object per column per block and frees the block after each callback, so the next block's first column could land at the same address and inherit the previous column's scale. Select a scale-2 column next to a scale-6 one with max_block_size=100, and a stored 12.34 came back as 0.001234, with no error. With default settings the verifier still saw 1 wrong cell in 70,000 rows.

The test suite never caught it because ASan builds hide it: the sanitizer's quarantine stops freed memory from being reused, so the address never repeats. It took a release build and a real server. 0.13.0 keys the memo correctly.

0.13.0 also fixes a behaviour you should check for before upgrading. A whole-number insert into a Decimal column, 7 into Decimal(9,2), now stores 7.00 as the server does; 0.12 stored 0.07. Code that compensated by passing pre-scaled integers (700 meaning 7.00) will now store 700.00 without complaint.

fast_uuid's main finding has the same character. uuid1() and uuid6() with a fixed node and clock sequence kept no last-tick state, so uuid1(1, 7) called 500,000 times produced 61,723 duplicates. The default arguments were fine. 0.8.1 fixes it.

Bugs a server, a file, or a document could trigger

php_clickhouse's high-severity finding was a heap overflow write. With compression enabled, a malicious or compromised server could send a compressed frame whose declared size was smaller than the 9-byte frame header, and the client wrote past its buffer with bytes the server chose. clickhouse-cpp had already fixed it on master in PR #566, but no clickhouse-cpp release contains the fix yet, and php_clickhouse's prebuilt binaries ship the library. 0.13.0 carries the patch. The same release rejects seven more server-triggered crash shapes, from FixedString(0) columns to Array offsets that run backwards, and stops a packet of a few dozen bytes from committing gigabytes of memory outside memory_limit. I sent the fixes upstream as 15 pull requests; three have merged so far and twelve are open.

fastchart's two medium findings were both resource bombs. A 44 KB SVG passed to svgTo*() reached 2,121,264 KiB of resident memory through data URIs and <use> references that slipped past the input gate, all of it outside memory_limit; a 30,000-deep <use> chain crashed the process. A 1 MB CgBI PNG with decoy dimensions decoded into a 16000 by 16000 buffer. 1.8.0 runs the gate on the parsed tree, bounds <use> depth and expansion, and caps decoded images at 4,096 pixels a side. Some heavy but legitimate SVG that used to render slowly now throws a ValueError instead, so test your largest inputs.

mdparser's high-severity finding was quadratic parsing: blank lines inside deeply nested lists took 1.7 seconds at 200 KB and 55 seconds at 1 MB. Seven of mdparser's thirteen findings had their root cause in md4c rather than in my code, and the fixes went upstream; md4c merged five of my pull requests for them on October 13. One remaining low-severity signed overflow is still waiting in md4c#374. 0.8.0 also changes generated heading IDs to start with user-content-, which closes a DOM-clobbering finding and breaks existing links to heading fragments when headingAnchors is on.

The other high and medium findings

pdo_duckdb's high finding was a use-after-free: an appender left open inside a reference cycle could outlive its PDO handle when gc_collect_cycles() or request shutdown freed the handle first. Two of the medium-range findings were the same shape. A memory_limit bailout at the wrong moment during column setup left uninitialised type pointers that were destroyed later; the PoC got the sanitizer to report a destructor running on 0x4141414141414141. 0.8.0 fixes all six.

phpser's high finding was that a nested SPL unserialize() ignored allowed_classes. Its legacy handler path ran outside the decoder's context, so a class allowlist set on the outer call never reached the inner one. 0.7.1 fixes it along with eighteen others, most of them bounds on work an untrusted payload could demand.

nameparser's quadratic case, from the top of this post, is fixed in 1.6.0. Its release notes say plainly that earlier notes describing hostile-input parsing as linear "did not hold for these shapes". phonetic's medium finding was a 14-byte input, bmpm("de akeaxhaeasz"), that caused an uncatchable fatal error. 0.4.4 turns budget overruns into a catchable ValueError and caps BMPM output at 256 times the input size, after a 1,560-byte input had produced 3.9 MB.

memory_limit stopped stopping things

PHP 8.2.35, 8.3.36, 8.4.27, and 8.5.12 shipped on October 21 with the fix for CVE-2026-109618, rated critical. Part of that fix changes what happens when an allocation crosses memory_limit. The engine used to bail out on the spot with a longjmp. Now it records the error, lets the allocation succeed, and raises the fatal at the next safe point in the VM.

For PHP code that's invisible. For an extension it isn't. A C loop that decodes a payload or builds rows never reaches a VM safe point, so it keeps allocating past memory_limit until it hits a hard limit, by default twice memory_limit below 1 GiB. At the hard limit the engine calls _exit(): no shutdown functions, no destructors, no session write, and php-fpm loses the worker instead of failing one request.

So every loop whose length comes from input has to notice the pending error and stop on its own. This is how phpser does it:

/* PHP 8.2.35, 8.3.36, 8.4.27 and 8.5.12 queue a memory_limit error instead of
 * bailing out; the allocation still succeeds until a hard limit (2x the soft
 * limit below 1G) that _exit()s the process. Loops that keep allocating must
 * stop on their own once the error is pending. */
#ifdef ZEND_HAS_OOM
# define PHPSER_HAS_OOM() ZEND_HAS_OOM()
#else
# define PHPSER_HAS_OOM() 0
#endif

A loop check cannot help when a single allocation is sized from a count the payload declares. Those go through the new fallible allocators instead (zend_hash_real_finit(..., true), safe_femalloc, fecalloc), which queue the error and return NULL rather than running into the hard limit. phpser 0.10.0 uses both. mdparser 0.8.1 and pdo_duckdb 0.9.0 do the same for their input-sized loops and presizes.

The re-release day

There is a catch. The October headers route smart_str growth through a new engine function, smart_str_ferealloc(), from inline header code. An extension compiled against 8.4.27 headers imports that symbol even if it never mentions it, and 8.4.26 doesn't export it. My release workflows built prebuilt binaries against the newest patch of each PHP minor, which had always been the safe choice.

On October 26 that meant mdparser 0.8.1, phpser 0.10.0, fastjson 0.10.1, php_clickhouse 0.13.1, and phonetic 0.4.5 all shipped Linux and macOS binaries that died with undefined symbol: smart_str_ferealloc on every PHP 8.4 before 8.4.27 and every 8.5 before 8.5.12. Source builds were fine, because they compile against the PHP they load into. All five were re-released the same day as mdparser 0.8.2, phpser 0.10.1, fastjson 0.10.2, php_clickhouse 0.13.2, and phonetic 0.4.6, and fastchart 1.8.1 picked up the same change before it could break. Their prebuilts now compile against the oldest patch of each minor (8.4.1, since 8.4.0 was never released, and 8.5.0), and each binary has to load on both the oldest and the newest patch before anything uploads.

That fix has a cost, and the phpser changelog states it: a binary built against 8.4.1 headers doesn't have the memory_limit stop. To get it, you build from source on a patched PHP.

Refusing vulnerable PHP releases

The longer-term answer is the opposite trade. pdo_duckdb 0.9.0 makes it today: its prebuilt binaries are built against the patched headers and require PHP 8.2.35, 8.3.36, 8.4.27, or 8.5.12 or newer. Load one on 8.4.26 and the process stops at the first call:

php: symbol lookup error: /ext/pdo_duckdb.so: undefined symbol: smart_str_ferealloc

That message is honest but unfriendly, and it arrives at runtime rather than at startup. The next releases of pdo_duckdb, phpser, and mdparser fix that with a startup check. On PHP 8.3 and newer, the module compares the running patch with the one it was compiled against and refuses to start on an older one, naming both versions. This is mdparser's:

if (php_version_id() < PHP_VERSION_ID) {
    zend_error(E_CORE_WARNING,
        "mdparser was built for PHP %s and cannot run on PHP %s; "
        "install a build for this PHP version or build from source",
        PHP_VERSION, php_version());
    return FAILURE;
}

With that check in place, pdo_duckdb keeps the binaries it already builds against the patched headers, and phpser and mdparser can move theirs back to the patched headers and regain the memory_limit stop. A server still on 8.4.26 gets a clear startup warning instead of a crash on the first request, and with a critical CVE fixed in 8.4.27 it should move anyway.

pdo_duckdb 0.9.0: DuckDB 2.0

DuckDB 2.0.0 went GA on October 21, and pdo_duckdb 0.9.0, out today, bundles it in every prebuilt binary. 0.8.0 had already built against the 2.0 previews. Source builds still work against DuckDB 1.5.3 and newer; configure detects 2.0 from duckdb_v2.h, because the 2.0 headers still report C API version 1.5.6, and it refuses to build when 2.0 headers meet an older libduckdb.

The upgrade is not transparent, and the README has a full section on it. The one most likely to bite is the file format. DuckDB 2.0 writes new files at storage version 69, and DuckDB 1.5 reads only 64 to 68. If a 1.5 process has to read the file too, ask for the older format when you open it:

$db = new PDO('duckdb:/data/shared.duckdb;storage_compatibility_version=v1.5.6');

On the 0.9.0 binary, a file opened that way reports v1.5.0+ in duckdb_databases(), and a default one reports v2.0.0+. Existing 1.5 files keep their format after 2.0 writes to them. The rest is DuckDB's own behaviour: a failed statement now invalidates the whole transaction, integer division by zero throws, single-arrow lambdas are rejected, and the new PEG parser is the default. That last one has a sharp edge for parameter binding: $1_0 means parameter 10 under PEG and parameter 1 under the legacy parser. Each change has a setting that restores 1.5 behaviour.

0.9.0 also adds PDO::DUCKDB_ATTR_NATIVE_STRUCTURES, an opt-in that fetches LIST, ARRAY, STRUCT, TUPLE, MAP, UNION, JSON, and VARIANT values as lossless PHP structures instead of canonical strings, with exact numerics, binary keys, and SQL NULL kept distinct from VARIANT null. VARIANT works top-level and nested, and the appender accepts PHP arrays and objects for it.

One fix in 0.9.0 matters even if you never touch 2.0: two PDO connections to the same database file could lose writes and see stale reads. Equivalent paths now share one DuckDB instance while transactions stay connection-local.

Performance, with the hardware attached

Most of the speed work came before the scans. Apart from the nameparser row, which I re-ran for this post, these are the numbers from each project's changelog with the conditions it states.

Project	Change	Measured on
nameparser 1.6.0	16 KB hostile row: 17.4 s to 0.064 s	i9-13950HX, PHP 8.4.27 (my rerun of 1.5.0 vs 1.7.0)
phpser 0.6.0	`range(0, 9999)` encodes to 7 bytes; encode 57% faster, decode 66% faster	aarch64
phpser 0.7.0	Signed decode 53 to 85% faster; 1,000-row rowset decodes in 429 KB instead of 645 KB	aarch64
fastjson 0.9.0	`fastjson_validate()` costs 0.47 to 0.90x a full parse, down from 1.1 to 1.9x	Neoverse-N1
fastjson 0.9.0 to 0.10.0	Key-heavy decode peak memory about 15% lower; file decode peak 9 to 33% lower	Neoverse-N1
fast_uuid 0.8.0	`uuid_v4()` 76 to 42 ns; `uuid_v4_bin_batch(100)` 61 to 27 ns per UUID; `uuid_v7_bin_batch()` 25.9 to 16.6 ns	Neoverse-N1
mdparser 0.6.2	`toAst()` 5 to 9% faster; retained AST memory 47% lower on softbreak-dense input; code-heavy `toXml()` output 49% smaller in memory	aarch64, PHP 8.4.25
php_clickhouse 0.12.0	Temporal, string, and IP inserts about 19% faster; IPv4 selects about 12%	200,000-row probes

The phpser integer-run encoding is the one I'd point at. A run with a constant step is stored as a start, a step, and a count, so range(0, 9999) encodes to 7 bytes and range(0, 999999) to 8. Native serialize() needs 137,790 bytes for the first one.

fast_uuid's speedup comes from refilling its random buffer with a userspace AES-256-CTR generator with fast key erasure, instead of a getrandom() call per 8 KiB, on CPUs with AES instructions. It has one caveat, and the changelog states it: clones of one VM snapshot can repeat up to 64 KiB of output per thread.

Breaking changes worth reading before you upgrade

Most of these releases are drop-in. These are not:

php_clickhouse 0.13.0: whole-number Decimal inserts scale like the server's (above).
mdparser 0.8.0: heading IDs carry a user-content- prefix.
phpser 0.10.0: Closures and other unserializable objects throw instead of quietly encoding NULL. Check what you put in $_SESSION before upgrading. New wire tags since 0.7.0 mean readers must upgrade before writers.
php_excel 2.9.0 and 2.9.1: dates use PHP's timezone rather than the OS one, and text is UTF-8 regardless of the process locale. Apps that relied on the locale to pass non-UTF-8 bytes through will see different output. 2.7.0 also rejects a font from a different ExcelBook in addText().
fastchart 1.8.0: heavy SVG that used to render slowly may now throw ValueError.
phonetic 0.4.4: oversized BMPM and Daitch-Mokotoff work throws a catchable ValueError rather than ending the request.
statgrab 2.2.2: Statgrab::ERROR_UNSUPPORTED is 44, matching libstatgrab; it was 41.

On the feature side, phpser 0.8.0 and 0.9.0 added signed sessions (phpser.session_key, HMAC-SHA256), an APCu serializer (apc.serializer=phpser), keyed payloads, and an exported C API. nameparser 1.4.0 understands joint salutations, so Mr. and Mrs. Brad Smith keeps Brad as the first name, with new isJoint(), getSalutations(), and getPartner(), and 1.4.1 raised first-and-last accuracy on 30,000 NPPES provider rows from 95.94% to 97.18%. statgrab 2.3.1 fixes a heap overflow in the bundled libstatgrab's Linux command-line handling that the scan didn't cover.

What a scan is worth

None of these projects were neglected before October. Each had a test suite and documentation that said where its limits were. The scans still found an allowlist that didn't reach a nested call, a Decimal that came back ten thousand times too small, and a 16 KB name that took 17 seconds. Two of the most consequential findings sat in code I had already called linear or already tested under ASan, and for the Decimal bug the ASan testing is exactly what hid it.

The fixes are released and listed above. Upgrade pdo_duckdb if you are moving to DuckDB 2.0, php_clickhouse if you talk to a server you don't control, and PHP itself to 8.4.27 or 8.5.12 whatever else you do.

Mastering React Performance: Optimizations That Actually Matter 🚀

Sun, 25 Oct 2026 05:57:59 +0000

Introduction

React is a powerful JavaScript library, but as applications grow, performance bottlenecks can arise. Unnecessary re-renders, inefficient state management, and large bundle sizes can lead to slow applications.

This guide covers practical performance optimizations that actually matter. Whether you’re working on a small project or a large-scale application, these best practices will help you write faster, scalable, and efficient React applications.

By the end of this blog, you’ll learn:
✅ Why React apps slow down
✅ Best practices for optimizing re-renders
✅ How to efficiently manage state
✅ How to optimize large lists
✅ lazy loading for faster load times

Let’s dive in! 🚀

1️⃣ Understanding React’s Rendering Behavior

Before optimizing, it’s crucial to understand how React renders components.

📌 React re-renders a component when:

State changes in that component.
Props change from the parent component.
A parent re-renders, causing child components to re-render.

🔍 How to detect unnecessary re-renders?
Use React DevTools Profiler to analyze component renders.

✅ Solution: Memoization & Avoiding Unnecessary Renders

2️⃣ Optimize Component Re-Renders with `React.memo`

By default, React re-renders a component when its parent renders, even if props haven’t changed.

🚀 Solution: Use React.memo() to prevent unnecessary renders

🔴 Without React.memo (Unnecessary re-renders)

import React, { useState } from "react";

const MovieDetails = ({ title, director }) => {
  console.log("🎬 MovieDetails component re-rendered");
  return (
    <div>
      <h2>{title}</h2>
      <p>Directed by: {director}</p>
    </div>
  );
};

const App = () => {
  const [count, setCount] = useState(0);

  return (
    <div>
      <h1>React.memo Example</h1>
      <button onClick={() => setCount(count + 1)}>Click Me ({count})</button>
      <MovieDetails title="Inception" director="Christopher Nolan" />
    </div>
  );
};

export default App;

👉 Here, MovieDetails re-renders every time the parent renders, even if props hasn't changed.

🟢 Optimizing with React.memo
To prevent unnecessary re-renders, wrap MovieDetails with React.memo():

const MovieDetails = React.memo(({ title, director }) => {
  console.log("🎬 MovieDetails component re-rendered");
  return (
    <div>
      <h2>{title}</h2>
      <p>Directed by: {director}</p>
    </div>
  );
});

👉 Now, MovieDetails will only re-render if its props (title or director) change.

Clicking the button will no longer trigger a re-render of MovieDetails unless new props are passed.

📌 When to use React.memo?
✅ The component receives static or rarely changing props (e.g., user profile, movie info).
✅ The component is expensive to render (e.g., complex UI elements like charts).
✅ The parent re-renders frequently, but child props remain the same.

3️⃣ Minimize Re-Renders with Derived State

Unnecessary state updates can cause frequent re-renders, slowing down your React app. Instead of storing derived state, compute values when needed.

🔴 Problem: Storing Derived State in useState (Causes Extra Re-Renders)

const [items, setItems] = useState([...]); 
const [filteredItems, setFilteredItems] = useState([]);

useEffect(() => {
  setFilteredItems(items.filter(item => item.active));
}, [items]); // Extra re-render every time `items` change

📌 Issue:
Every update to items triggers an unnecessary state update for filteredItems.

This causes an extra re-render that could have been avoided.

🟢 Solution: Use Derived State Instead of Storing It

const filteredItems = items.filter(item => item.active);

✅ Now, filteredItems is computed only when needed, avoiding extra state updates and re-renders.

4️⃣ Optimize Function Props with `useCallback`

Passing new function instances as props causes child components to unnecessarily re-render.

🔴 Problem: Without useCallback (Function re-creates on every render)

const ParentComponent = () => {
  const handleClick = () => {
    console.log("Button clicked");
  };

  return <ButtonComponent handleClick={handleClick} />;
};

👉 Even if handleClick doesn’t change, it gets re-created every time ParentComponent renders.

🟢 Solution: Use useCallback to memoize the function

const handleClick = useCallback(() => {
  console.log("Button clicked");
}, []);

✅ Now, handleClick retains the same reference, preventing unnecessary re-renders.

5️⃣ Optimize Expensive Computations with `useMemo`

If a component performs heavy calculations on every render, it can slow down the UI.

🔴 Problem: Without useMemo (Expensive re-computation)

const heavyComputation = (num) => {
  console.log("Expensive calculation running...");
  return num * 2;
};

const Component = ({ number }) => {
  const result = heavyComputation(number); // Runs on every render
  return <p>Result: {result}</p>;
};

🟢 Solution: Use useMemo to cache expensive computations

const result = useMemo(() => heavyComputation(number), [number]);

✅ heavyComputation() only re-runs when number changes.

6️⃣ Optimize Lists & Large Data Rendering

🔍 Problem: Rendering large lists slows down UI
Solution: Use Virtualization with react-window

🔴 Without Virtualization (Slow performance)

const List = ({ items }) => {
  return (
    <ul>
      {items.map((item) => (
        <li key={item.id}>{item.name}</li>
      ))}
    </ul>
  );
};

🟢 With Virtualization (react-window)

import { FixedSizeList as List } from "react-window";

const VirtualizedList = ({ items }) => (
  <List height={500} itemCount={items.length} itemSize={35} width={300}>
    {({ index, style }) => (
      <div style={style}>{items[index].name}</div>
    )}
  </List>
);

✅ Only visible items are rendered, improving performance significantly.

7️⃣ Reduce Bundle Size with Code Splitting & Lazy Loading

Large JavaScript bundles slow down initial page load.

✅ Solution: Use React.lazy() for dynamic imports

const HeavyComponent = React.lazy(() => import("./HeavyComponent"));

const App = () => (
  <Suspense fallback={<div>Loading...</div>}>
    <HeavyComponent />
  </Suspense>
);

👉 Components load only when needed, reducing bundle size.

Conclusion: Build Fast & Scalable React Apps

Optimizing React apps is crucial for fast UI interactions and smooth performance. By implementing:
✅ Memoization (React.memo, useCallback, useMemo)
✅ Efficient state management
✅ Virtualization for large lists
✅ Lazy loading for faster page loads

You’ll significantly improve performance! 🚀

💡 What performance tricks do you use in React? Drop a comment below! 👇

Finding the slow part of a WooCommerce page with Query Monitor

Sat, 24 Oct 2026 22:32:40 +0000

Query Monitor is a free WordPress plugin by John Blackbourn. It adds a panel to the admin toolbar that shows what one page load did: database queries, hooks, HTTP calls, PHP errors and the template that rendered the page. On a WooCommerce store it answers the usual question, "which plugin makes the product page slow", with numbers instead of guesses.

Install it on staging first

Query Monitor shows its panel to administrators only, so customers on a live store never see it. It still does its work on every request you make while logged in, which is why staging is the better home for it. Install it from the plugin directory and activate it. The toolbar then shows four figures for the current page: generation time, peak memory, database query time and the number of queries.

Read the queries by component

Open the panel and go to Queries by Component. Every query is attributed to the plugin, theme or core file that made it, with a count and a total time. Sorting by time usually points at one or two plugins.

Duplicate Queries is the next view to open. The same query repeated dozens of times in one request often comes from a loop that reads product meta one product at a time.

Slow Queries lists every query that took longer than 0.05 seconds. That threshold is the QM_DB_EXPENSIVE constant, and you can lower it in wp-config.php when the whole page is slow but no single query stands out.

Check outgoing HTTP calls

The HTTP API Calls panel lists the requests a page sent to other servers while it was being built. Licence checks, currency rates and shipping quotes are the usual suspects. A plugin that calls an external API on every product page adds that server's response time to yours. For each call the panel shows the URL, the response code and how long it took.

Time your own code

Query Monitor also has a small profiling API. Wrap a block of code in two actions and it shows up in the Timings panel:

do_action( 'qm/start', 'shipping_estimates' );
// the code you want to measure
do_action( 'qm/stop', 'shipping_estimates' );

For a value you want to inspect, do_action( 'qm/debug', $variable ); sends it to the Logs panel instead of the page output.

Look at a page as a guest

Some slowness only appears for logged-out visitors, for example when a cache plugin treats them differently. The Query Monitor settings can set an authentication cookie, and with it the panel keeps working in the same browser after you log out.

When you are done

Deactivate the plugin once the slow part is found. It is light, but there is no reason to keep it running on a store that nobody is debugging.

Shipping a 4K Wallpaper App Without Running Out of Memory: ImageIO Downsampling and Core Image

Sat, 24 Oct 2026 12:40:06 +0000

Originally published at https://bestlatestapps.com/blog/shipping-a-4k-wallpaper-app-without-running-out-of-memory/

Decoded-image maths, thumbnails that don't decode the full file, a blur editor that keeps edges clean, saving with the minimum permission, and why your app can't set the wallpaper itself

A wallpaper app looks like the simplest kind of app there is: a grid of pictures, tap one, save it. Then you load forty 4K images into a scrolling grid on an older iPhone, and the system terminates your app for using too much memory.

My app Wallpapers HD: Exotic 4K lists "fresh 4K HD wallpapers", "parallax-ready backgrounds that look sharp on every iPhone display", a Create mode to "edit photos and add blur effects", categories from Nature and Anime to Space and Minimalism, search, and saving so you can set an image as your Lock Screen or Home Screen. It's been on the App Store since December 2022.

Every one of those features runs into the same physical fact: a decoded image costs width × height × 4 bytes, no matter how small the file is. This article covers how I'd build a wallpaper app around that fact. That means downsampled thumbnails with ImageIO, full-resolution images only when needed, a Core Image blur editor, saving with add-only Photos access, and an honest explanation of the "set as wallpaper" step. API names and availability are checked against Apple's documentation, and this is the approach I'd use rather than the app's exact source.

1. The memory maths everyone forgets

A 4K image (3840 × 2160) might be a 2 MB JPEG on disk. Decoded for display, it's:

3840 × 2160 pixels × 4 bytes (RGBA) ≈ 33 MB

A grid showing 12 of those at once is around 400 MB of bitmap memory before you've scrolled anywhere. Yet a grid cell might be 120 × 260 points, about 360 × 780 pixels on a 3× display, which needs roughly 1.1 MB decoded.

So the rule is: never decode a bigger image than you're going to display. UIImage(data:) decodes at full size. ImageIO can decode straight to the size you need.

2. Downsampling with ImageIO

CGImageSourceCreateThumbnailAtIndex (iOS 4+) creates a smaller image directly from the compressed data, without ever holding the full-size bitmap in memory:

import ImageIO
import UIKit

enum Downsampler {
    /// Decodes `data` at most `maxPixelSize` on its longest side.
    static func image(from data: Data, maxPixelSize: CGFloat, scale: CGFloat) -> UIImage? {
        let sourceOptions = [kCGImageSourceShouldCache: false] as CFDictionary
        guard let source = CGImageSourceCreateWithData(data as CFData, sourceOptions) else { return nil }

        let options = [
            kCGImageSourceCreateThumbnailFromImageAlways: true,
            kCGImageSourceShouldCacheImmediately: true,      // decode now, off the main thread
            kCGImageSourceCreateThumbnailWithTransform: true, // respect EXIF orientation
            kCGImageSourceThumbnailMaxPixelSize: maxPixelSize
        ] as CFDictionary

        guard let cg = CGImageSourceCreateThumbnailAtIndex(source, 0, options) else { return nil }
        return UIImage(cgImage: cg, scale: scale, orientation: .up)
    }

    /// Same, from a file URL: avoids loading the whole file into memory first.
    static func image(at url: URL, maxPixelSize: CGFloat, scale: CGFloat) -> UIImage? {
        let sourceOptions = [kCGImageSourceShouldCache: false] as CFDictionary
        guard let source = CGImageSourceCreateWithURL(url as CFURL, sourceOptions) else { return nil }
        let options = [
            kCGImageSourceCreateThumbnailFromImageAlways: true,
            kCGImageSourceShouldCacheImmediately: true,
            kCGImageSourceCreateThumbnailWithTransform: true,
            kCGImageSourceThumbnailMaxPixelSize: maxPixelSize
        ] as CFDictionary
        return CGImageSourceCreateThumbnailAtIndex(source, 0, options)
            .map { UIImage(cgImage: $0, scale: scale, orientation: .up) }
    }
}

Two options do the important work:

kCGImageSourceShouldCache: false on the source stops ImageIO from caching a full-size decode as a side effect.
kCGImageSourceShouldCacheImmediately: true on the thumbnail forces decoding now, which you'll do on a background task, instead of lazily on the main thread during the first draw, where it causes scroll hitches.

On iOS 15+, UIImage.preparingThumbnail(of:) is a convenient alternative when you already have a UIImage. But a UIImage made from data hasn't necessarily been decoded yet, and ImageIO gives you the most control.

3. A grid that stays smooth

Better still: don't download 4K images for the grid at all. Have your image host serve a small rendition for browsing and the full file only for the detail screen. Then downsample whatever arrives to the cell's pixel size anyway, because servers change.

import SwiftUI

actor ThumbnailCache {
    static let shared = ThumbnailCache()
    private let cache = NSCache<NSURL, UIImage>()
    init() { cache.totalCostLimit = 80 * 1024 * 1024 }    // ~80 MB of decoded thumbnails

    func image(for url: URL, pixelSize: CGFloat, scale: CGFloat) async -> UIImage? {
        if let hit = cache.object(forKey: url as NSURL) { return hit }
        guard let (data, _) = try? await URLSession.shared.data(from: url),
              let image = Downsampler.image(from: data, maxPixelSize: pixelSize, scale: scale)
        else { return nil }
        let cost = Int(image.size.width * image.scale * image.size.height * image.scale * 4)
        cache.setObject(image, forKey: url as NSURL, cost: cost)
        return image
    }
}

struct WallpaperCell: View {
    let thumbnailURL: URL
    @Environment(\.displayScale) private var scale
    @State private var image: UIImage?

    var body: some View {
        GeometryReader { geo in
            ZStack {
                Color(.secondarySystemBackground)
                if let image { Image(uiImage: image).resizable().scaledToFill() }
            }
            .frame(width: geo.size.width, height: geo.size.height)
            .clipped()
            .task(id: thumbnailURL) {
                let px = max(geo.size.width, geo.size.height) * scale
                image = await ThumbnailCache.shared.image(for: thumbnailURL, pixelSize: px, scale: scale)
            }
        }
        .aspectRatio(9.0 / 19.5, contentMode: .fit)
        .clipShape(RoundedRectangle(cornerRadius: 12))
    }
}

The details that matter:

NSCache with a cost limit sized in decoded bytes, not image count, so the system can also evict under memory pressure.
.task(id:) cancels the load when a cell scrolls off-screen before finishing.
@Environment(\.displayScale), not UIScreen.main.scale. Apple's docs now mark UIScreen.main as deprecated in iOS 26. Read the scale and screen size from the view's environment or window scene instead.

Why not just AsyncImage (iOS 15+)? It's fine for small images, but it decodes whatever it downloads at full size and has no memory-cost cache. For a wallpaper grid, owning the pipeline pays off.

4. The detail screen: full resolution, exactly once

When the user opens a wallpaper, they deserve every pixel, but only one full-size image should be alive at a time. Download to a file (it streams to disk instead of RAM) and decode to the device's native screen size rather than the source's 4K:

import UIKit

enum FullImageLoader {
    /// Downloads to a temporary file, then decodes at the screen's native pixel size.
    static func load(_ url: URL, nativeScreenSize: CGSize, scale: CGFloat) async throws -> (UIImage, URL) {
        let (tempURL, _) = try await URLSession.shared.download(from: url)
        // Move out of the system's temporary location before it's cleaned up.
        let kept = FileManager.default.temporaryDirectory.appendingPathComponent(url.lastPathComponent)
        try? FileManager.default.removeItem(at: kept)
        try FileManager.default.moveItem(at: tempURL, to: kept)

        let longest = max(nativeScreenSize.width, nativeScreenSize.height)
        guard let image = Downsampler.image(at: kept, maxPixelSize: longest, scale: scale) else {
            throw CocoaError(.fileReadCorruptFile)
        }
        return (image, kept)    // keep the file: saving uses the original bytes
    }
}

UIScreen.nativeBounds (iOS 8+) gives the screen's size in physical pixels. Get the screen from the current window scene (view.window?.windowScene?.screen) rather than UIScreen.main.

What about "parallax-ready"? iOS can apply motion and depth effects to wallpapers, and the system crops and zooms to fit the device. The practical rule: supply images at least as large as the native screen, in portrait, with the subject away from the edges, so the system has room to crop and move. That's why a 4K source is useful even though you decode the preview smaller. You save the original, not the preview.

5. Create mode: blur that keeps clean edges

"Create your own wallpapers — edit photos and add blur effects." Core Image's Gaussian blur is the right tool, with one classic gotcha: blurring pulls in transparent pixels from outside the image, which gives you a dark, soft border. Clamp the image to its extent first, then crop back:

import CoreImage
import CoreImage.CIFilterBuiltins
import UIKit

final class BlurEditor {
    private let context = CIContext(options: [.useSoftwareRenderer: false])
    private let source: CIImage

    init?(image: UIImage) {
        guard let ci = CIImage(image: image) else { return nil }
        source = ci
    }

    /// radius in pixels of the *output*; scale it if you preview at a smaller size.
    func render(radius: Double, dim: Double = 0) -> UIImage? {
        let blur = CIFilter.gaussianBlur()
        blur.inputImage = source.clampedToExtent()     // no dark edges
        blur.radius = Float(radius)
        var output = blur.outputImage?.cropped(to: source.extent) ?? source

        if dim > 0 {                                   // darken for legible clock and widgets
            let controls = CIFilter.colorControls()
            controls.inputImage = output
            controls.brightness = Float(-dim)
            output = controls.outputImage ?? output
        }

        guard let cg = context.createCGImage(output, from: source.extent) else { return nil }
        return UIImage(cgImage: cg)
    }
}

Performance tips for a live slider:

Preview on the downsampled image and render the full-resolution output only when the user taps Save. Blur cost grows with the pixel count and the radius.
Scale the radius between preview and export. A 20-pixel blur on a 1,290-pixel-wide preview is about a 60-pixel blur on a 3,840-pixel export.
Reuse one CIContext. Creating a context per slider tick is the slowest possible way to use Core Image.

The optional dim parameter is a small design touch: a slightly darker, blurred background keeps the Lock Screen clock and widgets readable.

6. Saving with the minimum permission

Saving an image to Photos doesn't require read access to the user's library. iOS has an add-only access level for exactly this:

import Photos
import UIKit

enum PhotoSaver {
    enum SaveError: Error { case denied }

    /// Saves the original file bytes (not a re-encoded UIImage) to Photos.
    static func saveOriginal(fileURL: URL) async throws {
        let status = await PHPhotoLibrary.requestAuthorization(for: .addOnly)   // iOS 14+
        guard status == .authorized || status == .limited else { throw SaveError.denied }

        try await PHPhotoLibrary.shared().performChanges {
            let request = PHAssetCreationRequest.forAsset()
            request.addResource(with: .photo, fileURL: fileURL, options: nil)
        }
    }

    /// For edited images from Create mode.
    static func save(_ image: UIImage) async throws {
        let status = await PHPhotoLibrary.requestAuthorization(for: .addOnly)
        guard status == .authorized || status == .limited else { throw SaveError.denied }
        try await PHPhotoLibrary.shared().performChanges {
            PHAssetChangeRequest.creationRequestForAsset(from: image)
        }
    }
}

Declare NSPhotoLibraryAddUsageDescription (iOS 11+), not the full-library key. For example: "Save wallpapers you choose to your photo library."
requestAuthorization(for: .addOnly) (iOS 14+) asks only for what you need. Users are far more likely to allow it, and you never see their photos.
Save the original file with PHAssetCreationRequest.addResource(with:fileURL:options:) (iOS 9+). Re-encoding a UIImage can cost quality and strip metadata.

7. The honest part: you can't set the wallpaper for the user

There's no public iOS API that lets an app set the Lock Screen or Home Screen wallpaper. A wallpaper app's job ends at "saved to Photos", and the user sets it from the Photos app or from Settings → Wallpaper.

So make that last step effortless:

After saving, show a short, illustrated guide: Photos → open the image → Share → Use as Wallpaper.
Offer a button that opens Photos. Don't promise a one-tap "Set" button you can't deliver; App Review and users both dislike that.
Mention that the system lets you pinch and position the image, and that depth and motion options depend on the device and iOS version.

The listing's own wording handles this well: "Save the image… Set it as your Lock Screen or Home Screen wallpaper". It's a user step, described as one.

8. Real-world example: Wallpapers HD: Exotic 4K

These are the techniques I'd use, and recommend, for Wallpapers HD: Exotic 4K.

At the time of writing, its App Store listing shows:

Free with an optional Premium tier, in Photo & Video and Graphics & Design
First released in December 2022
6 ratings in the US store, all 5 stars
Supports iOS 16.0 and later

How the listing maps to this article:

What the listing says	Where it's covered
"Fresh 4K HD wallpapers added regularly" / "Browse by category… Quick search"	A small rendition for the grid, downsampled and cost-cached (sections 2–3)
"Parallax-ready backgrounds that look sharp on every iPhone display"	Native-size preview, original saved, room for system cropping (section 4)
"Create your own wallpapers — edit photos and add blur effects"	Clamped Gaussian blur with optional dimming (section 5)
"Save the image… Set it as your Lock Screen or Home Screen wallpaper"	Add-only Photos saving, then a guided user step (sections 6–7)
"Free: Browse and download HD wallpapers with ads / Premium: … advanced blur effects… ad-free"	Gate the content and editor features by entitlement; the pipeline is the same

9. Before you ship an image-heavy app: a checklist

Nothing is decoded larger than it's displayed. Grid cells use downsampled thumbnails sized in pixels.
The cache is costed in decoded bytes with a sensible limit, and it responds to memory pressure.
Scrolling is profiled on your oldest supported device with Instruments' Allocations and Animation Hitches tools.
Full-resolution images are downloaded to disk, previewed at screen size, and saved as original bytes.
Blur previews run on the small image and export at full size with a scaled radius.
Photos access is add-only, with NSPhotoLibraryAddUsageDescription and graceful handling of denial.
You don't promise "set wallpaper" automation. Guide the user through Photos instead.
UIScreen.main is gone from new code. Use the environment's displayScale and the window scene's screen.
Image licensing is clear for every image you serve.

Wrapping up

A wallpaper app is really a memory-management app with nice pictures. Downsample with ImageIO, cache by decoded cost, keep exactly one full-size image alive, preview edits small and export big, save with the narrowest permission, and be straight with users about the last step only they can take.

Try it, or explore my other apps:

Wallpapers HD: Exotic 4K (iOS): https://apps.apple.com/us/app/wallpapers-hd-exotic-4k/id6443850234
All of my free iOS apps: https://apps.apple.com/us/developer/alok-singh/id1508795536
My portfolio: https://bestlatestapps.com/alok/

What's the largest memory spike an image grid has ever given you? Tell me in the comments.

— Alok Kumar Singh

Originally published at https://bestlatestapps.com/blog/shipping-a-4k-wallpaper-app-without-running-out-of-memory/

PS4 Pro Belize DDR3 patch v2

Sat, 24 Oct 2026 08:36:00 +0000

The last time the DDR3 memory on PS4 pro has a critical problem it was capped under 11mb/s speed how much I tied I failed every time

then I notice something while debugging through the southbridge well it turns out DDR3 has it's own southbridge DMA engine but I don't really notice at first.

and after some more debugging it turns out my DDR3 chip (1 GB on the southbridge) was being read by the CPU personally, 8 bytes at a time, through a tiny PCI window ~10.6 MB/s. and that was the main bottleneck of all.

now it feels like I almost fix the issue but nope the southbridge DMA engine has no Linux driver for PS4 to use it.

and after more debugging it turns out Sony's PS4 gameOS aka orbis uses that chip (their sbram driver), but the code is closed.

So, I used AI to reverse engineer the kernel binary and everything started making sense now

the chip lives at 0xC0000000 + offset on the internal bus and everything moves through the southbridge DMA engine with exact protocol byte-count, source, dest, then the 0x1d40 "go" register, and write 0x11 to the interrupt-mask register.

after that it's time to do the main patch The implementation

On the ps4linux it sees the southbridge DMA engine as a pci device: 104d:90cd. The driver asks linux for it, switches it on, and maps its register memory. Now the engine can't touch program memory directly. everything goes through one special buffer the driver allocates at startup a small chunk of RAM that is always below 4 GB and always coherent (DMA-visible) think of it as the loading dock between the CPU and the DDR3.

After the patch bake into the kernel and the result

	old patch	new patch
reads	10.6 MB/s	504 MB/s
writes	~138 MB/s	425 MB/s
ceiling	unknown	~1.4 GB/s (65,408 B jobs @ 1,383 MB/s)

The engine itself moves data at up to ~1.4 GB/s. what used to be a 10.6 MB/s unusable brick is now a fast little swap chip on PS4linux

Image SEO in 2026: Alt Text, Modern Formats and Fast Loading

Fri, 23 Oct 2026 04:00:46 +0000

Image SEO in 2026 boils down to three jobs: describe every image accurately, deliver it fast, and make it discoverable and attributable. This guide covers each with copy-ready HTML, sticking to what Google, web.dev and the browser makers actually document.

It starts with a rule that catches many modern sites: Google indexes images in an img element (including inside picture) but not images set as CSS backgrounds. Then description: alt text that says what the image shows and why it's there, in plain words, with empty alt for decoration and any text inside a graphic repeated in the alt. Filenames, captions, nearby text and page titles all help Google understand an image, and the title attribute isn't a substitute for alt.

For speed, I recommend serving AVIF or WebP through a picture element with a JPEG or PNG fallback (and making sure the server sends the right content type), setting width and height on every image to prevent layout shift, using srcset with sizes so phones don't download desktop files, lazy-loading below the fold, and never lazy-loading the main image, which should get fetchpriority="high" instead.

I also cover image sitemaps, the image property in structured data, licence metadata for Google's Licensable badge, and what's actually documented about Google Lens and multimodal AI Mode: use your own photos and explain them in text nearby. A checklist and honest limits close it out.

Key takeaways

Use real img elements; CSS background images aren't indexed
Write alt text that describes what the image shows
Serve AVIF or WebP with a fallback
Always set width and height to prevent layout shift
Lazy-load below the fold, but prioritise the hero image

Related on BuiltToWinWeb: what makes a website load fast

Read the full guide on BuiltToWinWeb: The image SEO guide for 2026

A summary by Jacob Campbell of an article first published on built2winweb.com on October 7, 2026.

Why Does a Website Become Slow, and How Can We Fix It?

Thu, 22 Oct 2026 03:20:22 +0000

As a web developer, one thing I find interesting is that building a website is not just about making it look good. A website also needs to be fast, reliable, and easy to use.

Sometimes, a website looks perfect during development, but after adding more features, images, and data, it starts getting slower. Pages take longer to load, buttons respond late, and users may leave before the website even finishes loading.

So, what causes this problem, and how can we solve it?

The Problem: A Website That Loads Too Slowly

Imagine you are building an online store. At first, everything works fine because there are only a few products, images, and pages.

After some time, the store grows. Hundreds of products are added, images become larger, and the database has to handle more requests.

Suddenly, the homepage takes several seconds to load.

Users start complaining, and the website becomes difficult to use, especially on mobile devices or slower internet connections.

The first reaction might be to rewrite the entire website or upgrade the server. However, that is not always necessary.

The better approach is to identify what is actually making the website slow before changing anything.

Step 1: Find the Real Cause

Before fixing performance issues, I would first investigate where the delay comes from.

There are several possible reasons:

Large images that take too long to download.
Too much JavaScript running in the browser.
Database queries that retrieve unnecessary information.
Too many HTTP requests.
Missing caching for frequently requested data.
A server that takes too long to respond.

Browser developer tools and performance testing tools can help identify which part needs attention.

For example, if the server responds quickly but the page still takes a long time to appear, the problem might be related to images, CSS, or JavaScript rather than the backend.

This is why guessing is not a good debugging strategy.

Step 2: Optimize Images and Static Files

Images often take up a large part of a webpage's total download size.

Uploading a large image directly from a camera can make a website unnecessarily heavy.

A few practical solutions include:

Compressing images before uploading them.
Using modern image formats such as WebP or AVIF when appropriate.
Serving images at dimensions suitable for their display size.
Loading below-the-fold images only when needed.
Minifying CSS and JavaScript files where appropriate.

These changes can reduce download time without requiring major changes to the application.

Step 3: Improve Database Queries

Database performance is another common issue, especially when a website starts handling more users and records.

Consider a product page that retrieves every column from a database table even though it only needs the product name, price, and image.

Instead of retrieving unnecessary data, the application can request only the fields it needs.

For example, a query might look like this:

SELECT name, price, image_url
FROM products
WHERE category_id = 5;

Indexes can also help when queries frequently filter or sort records using particular columns.

However, indexes should be chosen carefully because they also require storage and can slow down certain write operations.

For larger datasets, pagination can prevent the application from loading thousands of records at once.

Small database improvements can make a noticeable difference when applied to frequently used pages.

Step 4: Use Caching Where It Makes Sense

Some information does not need to be calculated or retrieved from the database every time someone visits a page.

For example, a website might display the same list of categories to thousands of users throughout the day.

Instead of repeatedly running the same query, the application could temporarily store the result in a cache.

Depending on the application, caching might happen in the browser, at the server, through a reverse proxy, or using a service such as Redis.

The important thing is to decide which data can safely be cached and how it should be refreshed.

Caching everything is not a good solution. Information such as account balances or current order status may require more careful handling to avoid displaying outdated data.

Step 5: Reduce Unnecessary JavaScript

Modern websites often depend on JavaScript for interactive features, but adding too many libraries can increase loading time.

A simple page does not always need a large framework or multiple libraries.

Developers can improve performance by removing unused dependencies, splitting code into smaller bundles, and loading nonessential features only when required.

It is also worth checking whether an animation, widget, or third-party script is slowing down the initial page load.

The goal is not to avoid JavaScript. The goal is to use it where it provides real value.

Step 6: Test the Changes

After making improvements, the next step is to test the website again.

I would compare the results before and after the changes, paying attention to:

Page loading time.
Server response time.
Database query duration.
JavaScript execution time.
Performance on mobile devices.
Overall user experience.

Tools such as Chrome DevTools and Lighthouse can help identify performance problems and measure improvements.

Testing matters because a change that improves one page might not improve every part of the application.

What If the Website Is Still Slow?

Sometimes, optimizing images, queries, and scripts is not enough.

The application may be receiving more traffic than the current infrastructure can comfortably handle. In that situation, it may be necessary to review server resources, application architecture, background jobs, or database capacity.

A content delivery network can help deliver static files to users from locations closer to them. Background processing can also move time-consuming tasks, such as sending emails or generating reports, away from the main request.

But these solutions should follow evidence, not assumptions.

Upgrading infrastructure before understanding the problem can increase costs without fixing the real cause.

My Main Takeaway

Web development does not end when a page looks correct or a feature starts working. A good website should continue to perform well as its content, features, and number of users grow.

When a website becomes slow, I would start by measuring its performance, finding the biggest bottleneck, and fixing one issue at a time.

Sometimes, compressing a few images is enough. In other cases, improving a database query or introducing caching can make a much bigger difference.

The most useful lesson is to understand the problem before choosing the solution.

Every performance issue is an opportunity to learn something new about how websites work, from the browser to the server and the database.

I think this is one of the things that makes web development interesting: there is always another way to make an application more reliable, efficient, and easier to use.

What do you usually check first when a website starts getting slow: the frontend, the backend, or the database?

I'd be interested to hear how other developers approach this problem.

INP, the Core Web Vital That Replaced FID: How to Fix It

Sun, 18 Oct 2026 07:09:33 +0000

Interaction to Next Paint tracks how fast a page shows a visible reaction to a click, tap or key press. It took over from First Input Delay as a Core Web Vital on March 12, 2024. Good is 200 milliseconds or less and poor is over 500, measured at the 75th percentile of real visits. Many sites that passed FID easily fail INP because it covers every interaction in a visit and includes running your code and painting the result.

In the guide I explain what counts as an interaction (clicks, taps and key presses, not scrolling or hovering) and the three phases of each one: input delay, processing duration and presentation delay. Knowing which phase is slow tells you which fix applies.

Measurement starts with field data from CrUX, PageSpeed Insights and Search Console, then moves to DevTools with CPU throttling to reproduce problems. The web-vitals library's attribution build tells you which element was slow for real visitors and how long each phase took.

The fixes come in three groups: break up long tasks and yield to the main thread (with scheduler.yield() where supported); cut rendering work by avoiding layout thrashing, keeping the DOM small and using content-visibility; and tame input handlers with debouncing while deferring or removing third-party scripts. I'm clear about limits too: INP isn't a ranking shortcut, field data moves slowly, and a fast spinner doesn't fix slow work.

Key takeaways

Good INP is 200 ms or less at the 75th percentile
Each interaction has input, processing and presentation phases
Use field data first; Lighthouse doesn't measure INP
Break up long tasks and yield to the main thread
Debounce heavy handlers and audit third-party scripts

Related on BuiltToWinWeb: Core Web Vitals explained

Read the full guide on BuiltToWinWeb: The full INP explainer and fixes

A summary by Jacob Campbell of an article first published on built2winweb.com on October 6, 2026.

🔥 30 Days of Frontend — Day 5: React DevTools Profiler

Sat, 17 Oct 2026 16:39:44 +0000

When a React application starts feeling slow, one of the first things developers often do is reach for useMemo, useCallback, or React.memo.

We've all seen this pattern:

A component renders more often than expected.
The UI feels sluggish when interacting with a page.
A large list becomes noticeably slower as more items are added.
Updating one piece of state seems to trigger work across an entire component tree.

The natural reaction is to start memoizing components and functions.

But here's the problem: you might be optimizing code that isn't responsible for the slowdown in the first place.

React performance optimization should start with understanding what is happening, not guessing what might be happening.

That's where the React DevTools Profiler becomes useful.

1. What Is the React DevTools Profiler?

React DevTools is a browser extension that helps developers inspect React component trees, props, state, and other component-related information.

It includes two particularly useful panels:

Components: Inspect the component hierarchy, props, state, and component details.
Profiler: Record rendering activity and investigate how React updated the component tree.

The Profiler helps you understand the rendering work associated with a particular interaction.

For example, imagine you're working on an e-commerce application with a product grid containing hundreds of products.

When a user changes a filter, several things might happen:

The parent component updates.
The product list renders again.
Individual product cards render.
Sorting or filtering calculations run.
Some components render even though their visible data hasn't changed.

Some of this work might be necessary. Some might be avoidable.

Instead of assuming which component is responsible, you can record the interaction and inspect the results.

The goal isn't to eliminate every render. It's to understand whether the work is necessary and whether it affects the user experience.

2. Understanding How React Rendering Relates to Profiling

Before opening the Profiler, it's worth understanding what it actually measures.

A React update generally involves two important phases.

The render phase

During the render phase, React calls components as needed to determine what the next UI should look like.

This phase includes calculating the next React element tree and determining the changes required.

A component rendering doesn't automatically mean that the browser DOM will change.

For example:

function Counter({ count }) {
  console.log("Counter rendered");

  return <p>Count: {count}</p>;
}

Whenever React calls Counter, the component function executes and returns a React element describing the UI.

React may then determine that the resulting output doesn't require a DOM mutation.

A render is not the same thing as a DOM update.

This distinction is important because developers sometimes interpret every component render as expensive browser work.

The commit phase

After React finishes the necessary render work, it can commit the changes to the host environment, such as updating the DOM in a browser.

The Profiler helps you investigate rendering activity, but its measurements shouldn't be interpreted as a complete measurement of every part of browser performance.

For example, a page can have relatively inexpensive React rendering but still feel slow because of:

Expensive JavaScript outside React.
Large amounts of DOM layout or painting.
Long-running event handlers.
Network delays.
Large images or other resources.
Main-thread work that blocks user interactions.

This is why the React Profiler is one tool in a broader performance investigation, not a replacement for the browser's Performance panel.

3. What Can You Learn From the Profiler?

When you record a profiling session, the Profiler provides several useful ways to understand React's work.

A. Which components rendered?

The Profiler helps you identify the components involved in an update.

Suppose your component tree looks like this:

App
├── Header
├── SearchBar
├── FilterSidebar
└── ProductGrid
    ├── ProductCard
    ├── ProductCard
    ├── ProductCard
    └── ...

When the user types into the search field, you might discover that SearchBar, ProductGrid, and many ProductCard instances rendered.

That doesn't automatically mean all those renders are unnecessary.

You need to understand why they rendered and whether the work was expensive.

The component tree gives you a starting point for that investigation.

B. How often did components render?

Repeated rendering can be perfectly normal.

For example, typing into a controlled input typically updates state as the user types:

function SearchBar() {
  const [query, setQuery] = React.useState("");

  return (
    <input
      value={query}
      onChange={(event) => {
        setQuery(event.target.value);
      }}
      placeholder="Search products..."
    />
  );
}

Each keystroke can trigger an update.

That's expected behavior.

The more interesting question is what else happens during those updates.

If typing into the search field also triggers expensive calculations or unnecessary rendering across a large product grid, the Profiler can help you investigate that work.

Render frequency is a clue, not a performance verdict.

C. What caused a component to render?

This is one of the most useful parts of profiling.

A component might render because:

Its state changed.
Its parent rendered.
Its props changed.
Its context value changed.
An external store or another subscription triggered an update.

The exact cause depends on the component and the update mechanism.

For example:

function ProductPage() {
  const [query, setQuery] = React.useState("");
  const [selectedCategory, setSelectedCategory] =
    React.useState("all");

  return (
    <>
      <SearchInput
        value={query}
        onChange={setQuery}
      />

      <CategoryFilter
        value={selectedCategory}
        onChange={setSelectedCategory}
      />

      <ProductGrid query={query} />
    </>
  );
}

When selectedCategory changes, ProductPage renders again.

Because ProductGrid is a child of ProductPage, React may also render ProductGrid, even though its query prop hasn't changed.

That can be a candidate for optimization, but it isn't automatically a problem. First, determine whether the work is expensive enough to matter.

Also, remember that React DevTools' ability to display detailed render reasons depends on the React version, development tooling, and available settings. Don't assume every render will always have a definitive explanation displayed.

4. How to Use React DevTools Profiler: A Step-by-Step Workflow

Let's turn this into a practical workflow.

Step 1: Reproduce the actual performance problem

Start with the user interaction that feels slow.

Examples include:

Typing into a search field.
Changing a product filter.
Switching tabs.
Expanding a large navigation menu.
Updating a dashboard with multiple charts.
Scrolling through a long list.

Avoid profiling random interactions without a clear question.

A useful starting question might be:

Why does changing a product filter cause so much rendering work?

This gives you a specific behavior to investigate.

Step 2: Open React DevTools

Install React Developer Tools for your browser if you haven't already.

Open your application, launch the browser's developer tools, and locate the Profiler tab.

Make sure you're profiling the application and interaction you want to investigate.

For realistic performance measurements, prefer a production build or a profiling-enabled production build where appropriate. Development mode can introduce extra overhead, and Strict Mode may intentionally invoke rendering-related logic more than once to help reveal bugs.

Development profiling is still valuable for understanding component behavior, but don't assume its timings represent production performance.

Step 3: Start recording

In the Profiler tab, start a recording.

Then perform the exact interaction that causes the slowdown.

For example:

Start recording.
Type a search query.
Change a category filter.
Wait for the UI to settle.
Stop recording.

Keep the recording focused.

Step 4: Inspect the recorded commits

The Profiler organizes recorded updates into commits.

A commit represents a point at which React applied an update to the host environment

Select a commit related to your interaction, then inspect the available visualization and component details.

Look for patterns such as:

A large subtree rendering during a small interaction.
An expensive component repeatedly appearing in relevant commits.
Components rendering even though their inputs appear unchanged.
Expensive calculations associated with frequently occurring updates.

Don't immediately optimize the component with the largest-looking bar. First establish what the measurement represents and whether that component is related to the user-visible problem.

Step 5: Make one targeted change

Change one thing at a time whenever possible.

If you simultaneously add React.memo, introduce several useMemo calls, restructure the component tree, and change how state is managed, you won't know which change made the difference.

A focused experiment makes the results easier to interpret.

Step 6: Profile again

Repeat the same interaction under comparable conditions.

Compare:

Actual rendering duration.
Which components rendered.
How much of the subtree participated in the update.
Whether the interaction feels more responsive.
Whether the optimization introduced additional complexity or bugs.

If the measured React work decreased but the interaction still feels slow, investigate other parts of the performance pipeline.

An optimization isn't complete until you've verified its effect.

The React DevTools Profiler helps turn vague performance complaints into concrete questions about rendering behavior.

Instead of asking:

"Why is React so slow?"

Start by asking:

"Where is my application actually spending time, and what evidence do I have?"

That shift—from guessing to measuring—is what makes performance optimization more systematic.

The Memory Leak in addEventListener You’re Ignoring

Thu, 15 Oct 2026 06:48:00 +0000

Event listeners on dynamic elements or SPA routing are silent performance killers. If you attach an event listener to window or a persistent parent node inside a component and don’t clean it up when the component unmounts, that DOM reference stays in memory forever.

JavaScript
// ❌ Dangerous in Single Page Applications
function setupListener() {
window.addEventListener('resize', handleResize);
}

// ✅ Use AbortController for clean, single-call teardown
const controller = new AbortController();
const { signal } = controller;

window.addEventListener('resize', handleResize, { signal });

// Call this anywhere during unmount/cleanup:
controller.abort(); // Automatically removes ALL listeners bound to this signal!
Why it changes your career: Knowing how browser memory management works at runtime elevates you from a basic script author to a senior engineer who writes production-ready apps that don't crash under load.

Why Does a Website Get Slower Over Time? A Web Developer’s Approach to Finding the Real Problem

Wed, 14 Oct 2026 12:11:34 +0000

As a web developer, one thing I’ve learned is that a website can work perfectly during development but behave very differently after real users start using it.

A page loads quickly when there are only a few records in the database. Buttons respond immediately, and everything feels smooth. But after a few months, the same page starts taking longer to load. Users complain about delays, search results appear slowly, and sometimes the browser freezes for a few seconds.

The first reaction might be to upgrade the server or rewrite part of the application. But the real problem could be much simpler.

A slow website is not always a server problem. Sometimes, the issue is hidden in the way we write queries, load data, or handle things in the browser.

Let’s look at a common web development problem and how to approach it step by step.

The Problem: Everything Works Fine Until the Data Grows

Imagine building an online store with a product listing page.

At the beginning, the database contains only 100 products. The page loads quickly, and users can search, filter, and browse products without any trouble.

A few months later, the store has 50,000 products. Suddenly, the product page takes several seconds to load.

The application still works, but the user experience is getting worse.

There are several possible reasons:

The application loads too many records at once.
Database queries are taking longer than expected.
The same data is requested multiple times.
Images and JavaScript files are unnecessarily large.
The frontend performs too much work before displaying the page.
The server makes repeated requests to other services.

The important thing is not to assume which problem is responsible. The first step should be finding out where the time is actually being spent.

Step 1: Find the Bottleneck Before Changing the Code

When a page becomes slow, I would start by checking its behavior instead of immediately changing the implementation.

The browser's developer tools can help identify slow network requests, large resources, and JavaScript execution issues. Server logs and application monitoring can reveal slow endpoints, while database query logs can show expensive queries.

For example, suppose the product page takes four seconds to load.

The investigation might reveal that the initial HTML response takes only 200 milliseconds, but the API request takes 2.5 seconds. The remaining time is spent loading images and rendering the page.

That changes the direction of the investigation. There is little reason to optimize the initial HTML if the API is responsible for most of the delay.

A useful habit is to measure the important parts separately:

Frontend: How long does the browser take to render the page?
Backend: How long does the API take to return a response?
Database: Which queries consume the most time?
Network: Are requests or assets unnecessarily large?

Without measurements, optimization quickly becomes guesswork.

Step 2: Stop Loading Everything at Once

One common mistake is returning every matching record from the database and allowing the frontend to deal with the result.

Consider this simplified JavaScript example:

async function getProducts() {
  const response = await fetch("/api/products");
  const products = await response.json();

  return products;
}

This code is not inherently wrong. The problem appears when /api/products returns tens of thousands of records, including fields the page does not need.

The browser must download the response, parse the JSON, and process the data. The server and database also have to handle a much larger request than necessary.

A better approach is to introduce pagination.

For example:

async function getProducts(page = 1) {
  const response = await fetch(
    `/api/products?page=${page}&pageSize=20`
  );

  if (!response.ok) {
    throw new Error("Failed to load products");
  }

  return response.json();
}

Now the frontend requests a smaller set of products at a time.

The backend must also implement pagination correctly; adding query parameters alone does not reduce the amount of data returned.

For large datasets, the database query should limit the number of records it retrieves. The API should return only the fields the page needs.

The goal is not to load all available data. It is to load the data needed for the current task.

Step 3: Look for Inefficient Database Queries

Sometimes, pagination helps but the page is still slow.

The next place to investigate is the database.

Imagine displaying a list of 100 products alongside their category names. A poorly designed implementation might execute one query to retrieve the products and then another query for every product to retrieve its category.

That could mean 101 database queries for a single page.

This pattern is often called the N+1 query problem.

A better solution is to retrieve related information efficiently, using an appropriate join, eager loading, or another strategy supported by the database framework.

For example, in Ruby on Rails:

@products = Product
  .includes(:category)
  .order(created_at: :desc)
  .limit(20)

This tells Rails to load the associated categories efficiently instead of triggering an individual association query every time the view accesses a category.

The correct approach depends on the associations and how the data is used, so the generated SQL should still be inspected.

Database indexes are another important consideration. If users frequently search products by SKU or filter by a particular field, an appropriate index may improve query performance significantly.

However, adding indexes everywhere is not a good solution. Indexes consume storage and can slow down writes. They should be selected based on actual query patterns and execution plans.

Step 4: Make the Frontend Do Less Work

A website can have a fast API and still feel slow.

For example, a frontend might render hundreds of unnecessary elements, download oversized images, or execute expensive calculations whenever a component updates.

Some practical improvements include:

Loading images only when they are needed.
Serving appropriately sized, compressed images.
Removing unused JavaScript and unnecessary dependencies.
Avoiding repeated calculations during rendering.
Reducing unnecessary API requests.
Displaying useful loading states while data is being fetched.

It is also important to remember that users may have slower devices or unreliable internet connections. A page that feels fast on a powerful development computer might perform poorly on a budget phone.

Performance should be tested under realistic conditions rather than judged only by how quickly a developer's own machine loads the page.

Step 5: Use Caching Where It Actually Helps

Suppose the store's homepage displays the same featured products to thousands of visitors.

If the application repeatedly runs the same expensive query to retrieve information that rarely changes, caching might reduce the amount of repeated work.

Depending on the application, useful options include HTTP caching, application-level caching, and database query optimization.

For example, Rails provides a caching interface that can be used to store frequently requested results:

featured_products = Rails.cache.fetch(
  "featured_products",
  expires_in: 10.minutes
) do
  Product.where(featured: true)
         .order(created_at: :desc)
         .limit(10)
         .to_a
end

The next request can reuse the cached result until it expires or is invalidated.

But caching introduces another problem: keeping the data up to date.

If a featured product changes and the cache still contains an old result, users may see outdated information. Sensitive or user-specific data also needs careful cache handling to avoid exposing one user's information to another.

Caching should be used when repeated work is expensive and the freshness requirements are understood. It is not a replacement for fixing an inefficient query.

Step 6: Measure the Improvement

After making changes, the job is not finished.

The application should be tested again under comparable conditions. Otherwise, it is difficult to know whether the changes actually helped.

Suppose the product API originally took 2.5 seconds to respond. After implementing pagination and fixing inefficient database queries, it takes 400 milliseconds under the same test conditions.

That would be a meaningful improvement.

I would also check whether the changes introduced any new problems:

Are all required products still displayed?
Do filters and search work correctly?
Does pagination behave properly when records are added or removed?
Are database queries still efficient with realistic data volumes?
Does the application remain responsive when multiple users access it?

Performance improvements should not come at the cost of correctness.

What About Websites Built With Ruby on Rails?

Ruby on Rails provides useful tools for addressing these problems, but it does not automatically prevent every performance issue.

For a Rails application, I would pay particular attention to Active Record queries, association loading, database indexes, caching, background jobs, and response sizes.

For example, sending emails, generating reports, or processing large files may not need to happen during the user's web request. Moving suitable work to background jobs can make the application more responsive while allowing the task to finish separately.

For frontend-heavy applications that use JavaScript or Angular, I would also inspect bundle sizes, rendering behavior, and API communication.

The exact solution depends on the application. There is no single optimization that makes every website faster.

The Bigger Lesson: Do Not Optimize Blindly

One of the easiest mistakes in software development is changing code because it looks inefficient without checking whether it causes a real problem.

A complicated rewrite might take days and produce little improvement. A small database index or a simple pagination change might solve the actual bottleneck.

That is why a disciplined approach matters.

Measure first, identify the bottleneck, make one meaningful change, and measure again.

This approach makes debugging easier because each change has a clear purpose. It also reduces the risk of introducing unnecessary complexity into an application that already works.

Final Thoughts

A website becoming slower as it grows is a normal challenge in web development, but it should not be ignored. Slow pages frustrate users, waste server resources, and can make an otherwise useful application feel unreliable.

The solution usually begins with understanding what has changed. More data, inefficient queries, oversized responses, unnecessary frontend work, and repeated operations can all contribute to the problem.

As developers, we do not need to make every part of an application complicated in the name of performance. We need to understand the problem well enough to choose the simplest solution that works.

Every slow page is an opportunity to learn more about how the browser, application server, and database work together.

And sometimes, the best performance improvement is not writing more code. It is finding the code that does not need to run in the first place.

I’d be interested to hear how other developers approach this in their projects.

Python 3.15 Is Not a Speed Release for Data Teams. It Is an Operations Release.

Mon, 12 Oct 2026 07:13:52 +0000

Originally published on DEV.to: https://dev.to/firfircelik/python-315-is-not-a-speed-release-for-data-teams-it-is-an-operations-release-5fbo

Python 3.15.0 shipped on October 9, 2026 (release post). The headline number most people will repeat is in the JIT section of What's New: a "7-8% geometric mean performance improvement" over the standard interpreter on x86-64 Linux, and 11-12% on AArch64 macOS.

For a data team, the tempting reading is "upgrade the job images, pipelines get faster." I installed 3.15.0 on a VM, ran a small pandas/pyarrow workload with the JIT off and on, and then spent most of my time on three features that do not touch throughput at all. Those three are the ones that change how you run Python next week.

The thesis

For data teams, Python 3.15 is an operations release, not a speed release. The features that matter are about startup (PEP 810 lazy imports), observability (the Tachyon sampling profiler and frame pointers on by default) and supply chain (PEP 829 .start files replacing import lines in .pth files).

Each of them also has a trap that the release notes do not spell out. I hit all of them on the VM, and they are documented below with the commands to reproduce them.

The antithesis, at full strength

The best counter-case is not silly. It goes like this:

The JIT is real now. What's New lists a new tracing frontend, basic register allocation, more constant propagation and reference count elimination. The pyperformance range is wide, "from roughly 15% slowdown to over 100% speedup", but the geometric mean is positive on both platforms.
Data engineers write more Python-level loops than they admit. Row-wise apply, per-record parsing, UDFs, validation code. That is exactly the code a JIT can see.
Free-threading got a stable ABI. PEP 803 adds abi3t: extensions that target the Stable ABI can now be compiled so they are compatible with free-threaded builds. That is the long road to real multi-core Python for CPU-bound pipeline code.

So the antithesis is: the startup and tooling features are nice, but the throughput work is what will show up on the cloud bill.

Here is what that looked like on the VM.

Setup

Everything below ran on one shared VM: 8 vCPU Intel Xeon, 15 GiB RAM, Debian 13, kernel 6.12. Python 3.15.0 is the uv-managed build (Clang 22, tail-calling interpreter, JIT compiled in but off by default, turned on with PYTHON_JIT=1). Libraries: numpy 2.5.4, pandas 3.0.6, pyarrow 26.0.0, all installed from cp315 wheels three days after the release. The VM is shared with other workloads, so I report medians of repeated fresh processes and repeat whole rounds.

One thing I did not expect: the VM had uv 0.12.24, and uv python install 3.15 installed 3.15.0rc3, not the final release. uv 0.13.0 installed 3.15.0. If your images resolve "3.15" through an older tool, check python -VV before you trust anything. That matters again in the profiler section.

JIT off vs on

Three workloads, timed in-process, 7 fresh processes per setting, two rounds:

Workload	JIT off (median)	JIT on (median)	Change
Pure-Python parsing loop, 1M rows	337.6 / 332.4 ms	309.7 / 304.6 ms	about 8% faster
pandas row-wise `apply`, 100k rows	1176.9 / 1183.6 ms	1126.8 / 1144.3 ms	about 3 to 4% faster
pandas vectorized `groupby().sum()`, 2M rows	63.5 / 76.3 ms	63.3 / 71.4 ms	within noise (single runs ranged 60 to 103 ms)

jit_bench.py (the three workloads)

"""Time three data-team-shaped workloads in-process. Run with PYTHON_JIT=0 or 1."""
import sys
import time

import numpy as np
import pandas as pd


def pure_python_parse(n=1_000_000):
    total = 0
    for i in range(n):
        line = f"{i},{i % 97},{i * 3}"
        a, b, c = line.split(",")
        if int(b) > 50:
            total += int(c)
    return total


def vectorized_groupby(n=2_000_000):
    rng = np.random.default_rng(42)
    df = pd.DataFrame({"k": rng.integers(0, 50_000, n), "v": rng.random(n)})
    return df.groupby("k")["v"].sum().iloc[0]


def rowwise_apply(n=100_000):
    df = pd.DataFrame({"email": [f" User{i}+*Emails are not allowed* " for i in range(n)]})
    return df.apply(lambda r: r["email"].strip().lower().replace("+spam", ""), axis=1).iloc[0]


WORKLOADS = {"pure_python_parse": pure_python_parse,
             "vectorized_groupby": vectorized_groupby,
             "rowwise_apply": rowwise_apply}

fn = WORKLOADS[sys.argv[1]]
fn(10_000)  # warm-up
t0 = time.perf_counter()
fn()
print(f"{(time.perf_counter() - t0) * 1000:.1f}")

Run it as PYTHON_JIT=0 python jit_bench.py pure_python_parse and PYTHON_JIT=1 ..., several times each.

What the numbers support (interpretation): the JIT does what the release notes say for Python-level code. The pure-Python loop landed right in the 7-8% band. But a pipeline whose hot path is already in numpy, pandas or Arrow does not get faster, because the JIT never sees that code. And the row-wise apply is the honest middle: 3 to 4% is real, but the fix for a 1.2 second apply is to vectorize it, not to wait for a JIT.

The free-threading ABI is a packaging milestone. What's New says moving an extension to abi3t "usually requires some non-trivial changes to the source code", and it pays off only when the extensions you depend on ship those wheels and you run a free-threaded build (cp315t). That is worth tracking, not something you collect by bumping a base image.

So the antithesis is partly right: there is a free win for Python-level code. It is small, it is opt-in, and it is experimental. The three features below change your runbooks this week.

1. Lazy imports fix startup, not throughput

The wrong assumption: "lazy imports make Python faster."

The mechanics: lazy import pandas as pd binds pd to a lightweight proxy. The module is loaded the first time the name is used, a step PEP 810 calls reification. If loading fails, the error is raised at first use, and the traceback shows both places. Lazy imports are only allowed at module scope: inside a function, a class body or a try/except they are a SyntaxError. I confirmed the try case on 3.15.0, which rules out the classic try: import orjson as json fallback.

I wrote a toy pipeline CLI, pipectl, with the imports most data tools have at the top: numpy, pandas, pyarrow, requests. It has a version command, a check command that validates a JSON config, and a run command that aggregates 2M rows and writes Parquet. The lazy variant changes only the import lines:

lazy import json
import argparse
import sys

lazy import numpy as np
lazy import pandas as pd
lazy import pyarrow as pa
lazy import pyarrow.parquet as pq
lazy import requests

Each command was run 30 times in a fresh process after warm-up, in three clean rounds:

Command	Eager (median per round)	Lazy (median per round)
`python -c pass` (floor)	9.3 ms
`pipectl version`	351.4 / 381.2 / 344.4 ms	20.0 / 19.6 / 19.9 ms
`pipectl check cfg.json`	352.1 / 342.7 / 352.1 ms	21.2 / 21.1 / 20.8 ms
`pipectl run` (2M rows)	453.2 / 441.0 / 450.8 ms	400.5 / 392.8 / 400.5 ms

-X importtime shows where the eager 350 ms goes: pandas about 180 ms cumulative, numpy about 40 ms, requests about 35 ms (medians of three runs, and importtime adds its own overhead).

The version and check rows are the point of the feature: about 94% less wall time, because those commands never touch pandas. That is more than the "50-70% in practice" that PEP 810 cites for command-line tools, and it should be: my CLI is the extreme case where the cheap commands use none of the heavy imports.

The run row is the one to read twice. Lazy saved about 50 ms, so did it make the work faster? No. I ran a third variant, eager but with the import requests line deleted:

`pipectl run`, 20 runs each	Median
eager	454.9 ms
eager without `import requests`	410.4 ms
lazy	413.3 ms

The whole gain is the import that the run path never uses. Lazy imports remove the imports you don't use. They don't make the ones you do use any cheaper. If your job imports pandas and uses pandas, it pays the full 180 ms either way.

Where this pays off for data teams (interpretation): CLIs, --help in CI, pre-commit hooks, tests that import a package to run one function, notebook kernels, and anything that gets parsed often. The Airflow docs already warn that "import statements also count as top-level code" in Dag files and that expensive imports there hurt the scheduler (Airflow best practices). The lazy keyword looks built for that pattern. I did not test it inside Airflow, and you need an Airflow version that supports 3.15 first.

Trap 1: the global switch broke pandas

3.15 also has an interpreter-wide mode: -X lazy_imports=all or PYTHON_LAZY_IMPORTS=all makes every module-level import in every module potentially lazy, third-party code included. It is tempting to flip it in a job image and collect the startup win without touching code. On 3.15.0 with pandas 3.0.6 and numpy 2.5.4, that failed 5 out of 5 times:

$ PYTHON_LAZY_IMPORTS=all python -c "import pandas as pd; pd.DataFrame({'a': [1]})"
  File ".../pandas/_typing.py", line 401, in <module>
    TakeIndexer: TypeAlias = Sequence[int] | Sequence[np.integer] | npt.NDArray[np.integer]
                                                                    ~~~~~~~~~~~^^^^^^^^^^^^
TypeError: 'lazy_import' object is not subscriptable

The minimal trigger is npt.NDArray[np.integer] with numpy.typing imported under the global mode. numpy alone and requests alone worked. A tiny pure-Python package with the same re-export chain worked too, so this looks specific to how numpy.typing (which defines a module-level __getattr__) interacts with global laziness. I did not chase it further. The lesson does not depend on the root cause: the global mode changes the import semantics of code you did not write, and the lazy keyword in your own entry point does not.

Trap 2: side-effect imports silently stop happening

Plenty of data code imports a module only for what it does at import time: registering a pandas accessor, a SQLAlchemy dialect, a plugin. Linters flag these as unused, and they are the first thing someone will mark lazy:

lazy import pandas as pd
lazy import dq_accessors  # noqa: F401  (registers df.dq at import time)

df = pd.DataFrame({"a": [1, None, 3], "b": [None, None, 1]})
print(df.dq.null_ratio())

AttributeError: 'DataFrame' object has no attribute 'dq'. Did you mean '.eq' instead of '.dq'?

The name dq_accessors is never used, so it is never reified, and the registration never runs. The eager version prints {'a': 0.333, 'b': 0.667}. PEP 810 says this directly: side effects are "deferred until first use", and it recommends explicit initialization functions instead of import-time registration.

Trap 3: the PEP's kill switch does not exist in 3.15.0

PEP 810 describes a third mode, "none", to force every import eager. The shipped 3.15.0 accepts only normal and all, which matches the sys docs. Copying the PEP is not harmless:

$ PYTHON_LAZY_IMPORTS=none python -c pass
Fatal Python error: config_init_lazy_imports: PYTHON_LAZY_IMPORTS: invalid value; expected 'all' or 'normal'

Exit code 1, before any of your code runs. sys.set_lazy_imports("none") raises ValueError. The PEP page itself now says it is "a historical document". If you want an emergency "everything eager" switch, a filter that returns False does it (verified: json loads at the lazy import line):

import sys

sys.set_lazy_imports_filter(lambda importer, name, fromlist: False)

Runbook: lazy imports

Measure first: python -X importtime -m yourtool --help 2> imports.txt, then sort by the cumulative column.
Mark only the heavy imports in your own entry point lazy. Leave third-party internals alone.
Keep side-effect imports eager, or replace them with an explicit register() call.
Do not set PYTHON_LAZY_IMPORTS=all in a job image unless the full test suite passes under it.
If you must support 3.14 too, use __lazy_modules__ = ["pandas", ...] with normal import statements. 3.15 treats those imports as lazy, and older versions just see a module variable and import eagerly (checked on 3.15.0 and 3.13.5 with json).

2. Tachyon: attach to the stuck job by PID, and read the error rate

The wrong assumption: "a sampling profiler is free and always right."

The mechanics: profiling.sampling (PEP 799, named Tachyon) reads the target process's memory from outside. It can run a script, attach to a PID for a time window, or dump a single traceback-style snapshot of a live process. Output formats include pstats tables, collapsed stacks, flame graphs, Firefox Profiler and a live TUI. The docs list the constraints that matter in production:

Same Python minor version for profiler and target. If either side is a pre-release, both must be the exact same version.
Permissions. On Linux, root, CAP_SYS_PTRACE, or a Yama ptrace_scope of 0. The docs note the default ptrace_scope of 1 restricts ptrace to parent processes.
Non-blocking mode (the default) never stops the target, but "can occasionally produce incomplete or inconsistent stack traces". --blocking pauses the target for each sample.

I wrote the classic slow nightly job: a pandas row-wise apply with a per-row Python function, looping forever, and pretended I did not know why it was slow.

slow_job.py

"""A 'stuck' nightly job: a row-wise pandas apply that nobody profiled."""
import time

import numpy as np
import pandas as pd


def normalize_email(row):
    return row["email"].strip().lower().replace("+spam", "")


def enrich(df):
    df["email_norm"] = df.apply(normalize_email, axis=1)
    return df


def main():
    rng = np.random.default_rng(0)
    n = 200_000
    df = pd.DataFrame({
        "id": np.arange(n),
        "email": [f" User{i}+*Emails are not allowed* " for i in rng.integers(0, 10**6, n)],
    })
    while True:
        enrich(df.copy())
        time.sleep(0.01)


if __name__ == "__main__":
    main()

The first thing you want during an incident is "what is it doing right now":

$ python -m profiling.sampling dump --blocking 440208
Stack dump for PID 440208, thread 440208 (main thread, has GIL; most recent call last):
  File "slow_job.py", line 31, in <module>
    main()
  File "slow_job.py", line 26, in main
    enrich(df.copy())
  File "slow_job.py", line 14, in enrich
    df["email_norm"] = df.apply(normalize_email, axis=1)
  ...
  File "slow_job.py", line 10, in normalize_email
    return row["email"].strip().lower().replace("+spam", "")
  File ".venv/lib/python3.15/site-packages/pandas/core/series.py", line 959, in Series.__getitem__
    return self._get_value(key)

No restart, no code change, and the culprit is on screen. On this kernel there is no Yama module (/proc/sys/kernel/yama/ptrace_scope does not exist), so attaching as the same unprivileged user just worked. On a host with the default ptrace_scope of 1, plan for root or CAP_SYS_PTRACE before the incident, not during it.

Then the part I did not expect. Without --blocking, dump failed 8 times out of 20 with RuntimeError: Failed to parse initial frame in chain. With --blocking it succeeded 20 out of 20. For attach, the profiler prints an Error rate line, the percentage of samples it could not read. I measured it on a fixed workload (median of six apply rounds per run, 1 kHz default rate, three repetitions):

Mode	Target round time	Error rate
No profiler	1170 / 1150 / 1186 ms
`attach` (non-blocking, default)	1179 / 1180 / 1150 ms	67.8% / 62.0% / 63.7%
`attach --blocking`	1606 / 1760 / 1657 ms	1.5% / 1.6% / 1.3%

Both halves of the docs are true at once. Non-blocking mode had no overhead I could measure on the target. And on this VM, 62 to 68% of its samples were unreadable in this test (46 to 50% in two earlier 5-second runs against slow_job.py, 38% against a pure-Python recursive function). Blocking mode was clean, and it made the job 37 to 53% slower while attached.

Interpretation: treat a non-blocking profile like a survey with a response rate. If 60% of samples failed, the remaining 40% may still point at the right function (here the row-wise apply dominated either way), but check the error rate before you trust percentages. Use dump --blocking for a one-shot snapshot, where a single pause costs nothing, and keep --blocking profiles short on anything user-facing.

And the version rule bit me right away. A job started on the 3.15.0rc3 that the older uv had installed could not be inspected from 3.15.0:

Can't attach to a pre-release Python interpreter from a process running a different Python version

From rc3 to rc3 it worked. Pin the profiler to the job image's exact interpreter, which in practice means running it from inside that image.

The quieter observability change is PEP 831: CPython is now built with frame pointers by default. What's New says this makes native stack unwinding "faster and more reliable" for system profilers, debuggers, crash analysis tools and eBPF-based tools. The uv build I used has -fno-omit-frame-pointer in its CFLAGS. The catch, in the release notes' words: "A single native component built without frame pointers can break stack unwinding for the whole Python process." Your own Rust or C extensions need the same flags if you want mixed native and Python profiles.

Runbook: profiling a stuck job

python -m profiling.sampling dump --blocking PID first. One snapshot, one short pause.
Then python -m profiling.sampling attach -d 30 --flamegraph -o job.html PID, and read Error rate before the flame graph.
Run the profiler with the job's exact interpreter. A pre-release on either side means exact versions only.
Decide now who is allowed CAP_SYS_PTRACE on worker hosts, and where.

3. `.pth` import lines become `.start` files: auditable, not safe

The wrong assumption: "a package can only run code when I import it."

The mechanics: at startup the site module reads every .pth file in site-packages. Lines are paths to add to sys.path, except lines that start with import, which are executed with exec(). Every Python process in that environment runs them: python -c pass, pip, pytest, the notebook kernel, the scheduler. PEP 829 calls this "a broad attack surface".

This is not theoretical. The advisory for the compromised litellm 1.82.7 and 1.82.8 releases (PYSEC-2026-2, published March 24, 2026) describes credential-harvesting malware, and its references point to a litellm_init.pth file inside the 1.82.8 package.

And it is not only malware. setuptools 84.0.0, installed fresh on the VM, still ships distutils-precedence.pth, whose single line starts with import os; ....

What PEP 829 and the site docs change in 3.15:

A new name.start file lists entry points as pkg.mod:callable. They run after all .pth path lines are applied.
If name.start exists next to name.pth, the import lines in that .pth are ignored.
Without a matching .start, import lines still run in 3.15, and the deprecation message appears only with -v. The docs say they will be "silently ignored in Python 3.18 and 3.19", with a warning in 3.20.
Errors on one line no longer abort the rest of the file. A failing entry point prints a traceback and startup continues.

On the VM, with a demo zz_demo.pth containing a path line and an import line:

$ python -c "import sys; print('/tmp/extra_path_dir' in sys.path)"
[zz_demo.pth] import line executed at startup
True

python -v shows the new deprecation notice for both the demo file and setuptools' file:

import lines in .../site-packages/distutils-precedence.pth are deprecated, use entry points in a .../site-packages/distutils-precedence.start file instead.

After adding zz_demo.start with one line, zz_demo_boot:init, the import line is suppressed and the entry point runs:

import lines in .../zz_demo.pth are suppressed due to matching .../zz_demo.start file.
Executing entry point: zz_demo_boot:init from .../zz_demo.start
[zz_demo.start] entry point zz_demo_boot:init called

Here is the part to be precise about. A .start file still runs code at every interpreter start. A malicious wheel can ship one just as easily as a .pth. The PEP says so: "The overall pre-start code execution attack surface is not eliminated by this PEP." What 3.15 gives you is a narrower, auditable shape: startup code is a named callable in an importable module, it goes through the import system so audit hooks (PEP 578) can see it, and "you can tell by listing the files in the site-dir, exactly where arbitrary code execution occurs."

That listing is the useful part, and you can do it today on any Python version:

startup_audit.py (runs on Python 3.9+)

"""List everything that can run code at interpreter start in this environment.

Run it with the interpreter you want to audit:
    .venv/bin/python startup_audit.py
"""
import site
import sys
from pathlib import Path

dirs = list(dict.fromkeys(site.getsitepackages() + [site.getusersitepackages()]))
found = 0
for d in map(Path, dirs):
    if not d.is_dir():
        continue
    for f in sorted(d.glob("*.pth")):
        lines = f.read_text(encoding="utf-8-sig", errors="replace").splitlines()
        code = [l for l in lines if l.startswith(("import ", "import\t"))]
        if code:
            found += 1
            shadow = f.with_suffix(".start").exists()
            note = "skipped on 3.15+ (matching .start), runs on older Pythons" if shadow else "RUNS on every start"
            print(f"{f}  [{note}]")
            for l in code:
                print(f"    {l[:110]}")
    for f in sorted(d.glob("*.start")):
        found += 1
        print(f"{f}  [.start entry points]")
        for l in f.read_text(encoding="utf-8-sig").splitlines():
            if l.strip() and not l.lstrip().startswith("#"):
                print(f"    {l.strip()}")
    for name in ("sitecustomize.py", "usercustomize.py"):
        if (d / name).exists():
            found += 1
            print(f"{d / name}  [runs on every start]")
print(f"{sys.version.split()[0]}: {found} startup code hook(s) found")

Output in the demo environment (with setuptools installed):

.../site-packages/distutils-precedence.pth  [RUNS on every start]
    import os; var = 'SETUPTOOLS_USE_DISTUTILS'; enabled = os.environ.get(var, 'local') == 'local'; enabled and __
.../site-packages/zz_demo.pth  [skipped on 3.15+ (matching .start), runs on older Pythons]
    import sys; sys.stderr.write("[zz_demo.pth] import line executed at startup\n")
.../site-packages/zz_demo.start  [.start entry points]
    zz_demo_boot:init
3.15.0: 3 startup code hook(s) found

Most .pth files are harmless. An editable install of a src-layout package with setuptools wrote a path-only .pth on the VM, with no code at all. The ones that execute code are few enough to put on an allowlist.

python -S disables both mechanisms (I checked), but it also skips site-packages entirely, so it is a debugging tool, not a production setting.

Runbook: startup code

Run the audit in your image build and fail the build on any startup hook that is not on an allowlist.
If you maintain a package with a .pth import line, move the code into a zero-argument function and add a matching .start file. Keep the .pth line as import pkg.mod; pkg.mod.callable() while you still support 3.14.
Treat a new .start file in a dependency diff like a new postinstall script in npm: a reason to read the release.

What to do this week

If you run...	Do this	Skip this
Python CLIs, CI tooling, notebook kernels	Measure with `-X importtime`, add `lazy` to your entry point's heavy imports	`PYTHON_LAZY_IMPORTS=all` in images
Long batch jobs on VMs or Kubernetes	Install the exact job interpreter in the image, rehearse `dump --blocking` and `attach` on a staging job, sort out `CAP_SYS_PTRACE`	Trusting a profile without reading `Error rate`
Anything that installs from PyPI	Add the startup audit to the image build, review new `.pth` and `.start` files in dependency updates	Assuming `.start` files make startup code safe
Python-heavy loops or row-wise UDFs	Try `PYTHON_JIT=1` in a benchmark, expect single digits	Counting on the JIT to fix a row-wise `apply`

The JIT will show up in your benchmarks as a few percent. Lazy imports, Tachyon and .start files will show up in your incident timeline: the CLI that starts in 20 ms instead of 350, the stuck job you can inspect without a restart, and the list of files that run code before your main() does. Upgrade for those, and measure the rest.

Build a Pure CSS Radar Chart — Lightweight, Max Performance

Sun, 11 Oct 2026 20:07:46 +0000

Most radar charts pull in SVG path math or a JavaScript charting library. You can skip both.

With st-core.fscss, a spider (radar) chart is:

an FSCSS array of scores (0–100)
polar coordinates stored as CSS variables (--st-sxN, --st-syN)
shapes drawn with clip-path: polygon()
rings and spokes from gradients

No canvas. No SVG. No chart runtime on the critical path.

Source: github.com/fscss-ttr/st-core.fscss

Here’s how it works

1. Data is an array

@arr skills[80, 65, 90, 55, 70, 85]

Length can be 5, 6, 8 — whatever you need. Geometry (number of polygon vertices) is fixed when FSCSS expands the mixins.

2. One mixin writes coordinates

.spider {
  @st-spider-points(skills, 42)  /* array, max radius % of the box */
  position: relative;
  width: 280px;
  height: 280px;
  border-radius: 50%;
}

For each index i:

angle = (i - 1) × 360° / N (0° at the top)
radius = (value / 100) × rmax
CSS gets:

--st-sxN: calc(50% + (r * 1%) * sin(θ));
--st-syN: calc(50% - (r * 1%) * cos(θ));

Trig stays in the browser’s calc(). Updating values later is rewriting those variables.

3. Renderers only read variables

@st-spider-fill(.spider-fill, skills)
@st-spider-line(.spider-line, skills)
@st-spider-dots(.spider-dot-, skills, 9px)
@st-spider-grid(.spider-grid)
@st-spider-spokes(.spider-spokes, skills)

Layer	Role
fill	Closed polygon through every `--st-sx/sy` pair
line	Dual-pass polygon (outer path + path scaled toward center) so the stroke has real thickness
dots	Absolutely positioned markers at each vertex
grid	Concentric rings (`radial-gradient`)
spokes	Thin radial axes (`conic-gradient`)

Rule: call @st-spider-points on the host. Children inherit. Same model as linear @st-chart-points / fill / line in st-core.

4. Markup is plain HTML

<div class="spider">
  <div class="spider-grid"></div>
  <div class="spider-spokes"></div>
  <div class="spider-fill"></div>
  <div class="spider-line"></div>
  <div class="spider-dot-1"></div>
  <div class="spider-dot-2"></div>
  <div class="spider-dot-3"></div>
  <div class="spider-dot-4"></div>
  <div class="spider-dot-5"></div>
  <div class="spider-dot-6"></div>
</div>

Dots need real nodes — the mixin emits .spider-dot-1 … .spider-dot-N rules; it does not invent elements.

Here’s how values can be displayed on the dot nodes

Markers are normal DOM nodes, so you can show the score next to (or inside) each dot without a chart API.

Option A — `data-value` + CSS `::after`

<div class="spider-dot-1" data-value="80"></div>
<div class="spider-dot-2" data-value="65"></div>
<!-- … -->

.spider-dot-1,
.spider-dot-2,
.spider-dot-3,
.spider-dot-4,
.spider-dot-5,
.spider-dot-6 {
  /* st-core already sets left/top from --st-sx / --st-sy */
}

[data-value]::after {
  content: attr(data-value);
  position: absolute;
  left: 50%;
  top: -1.4em;
  transform: translateX(-50%);
  font-size: 0.65rem;
  font-variant-numeric: tabular-nums;
  color: var(--st-text);
  white-space: nowrap;
  pointer-events: none;
}

Push labels slightly outward per axis if they collide with the polygon (different top / left offsets per index).

Option B — nested label element

<div class="spider-dot-1">
  <span class="dot-label">80</span>
</div>

.dot-label {
  position: absolute;
  left: 50%;
  bottom: calc(100% + 4px);
  transform: translateX(-50%);
  font-size: 0.65rem;
  color: var(--st-muted);
}

Option C — update labels from JS with the series

function setSeries(host, values, rmax = 42) {
  const n = values.length;
  let css = "";
  values.forEach((v, i) => {
    const ang = (i * 360) / n;
    const r = (v / 100) * rmax;
    css += `--st-sx${i + 1}: calc(50% + (${r} * 1%) * sin(${ang} * 1deg));`;
    css += `--st-sy${i + 1}: calc(50% - (${r} * 1%) * cos(${ang} * 1deg));`;
    const dot = host.querySelector(`.spider-dot-${i + 1}`);
    if (dot) {
      dot.dataset.value = v;
      // or: dot.querySelector(".dot-label").textContent = v;
    }
  });
  host.style.cssText = css;
}

Geometry and labels stay in sync because both come from the same array pass.

Axis names (Speed, Quality, …) stay outside the dots — absolute labels around the circle — so vertices stay readable at small sizes.

Here’s why it’s lighter

Traditional radar	st-core radar
Chart.js / D3 / Recharts bundle	Mixins → CSS
SVG path string updates	Custom properties + `clip-path`
Layout + paint on every redraw	Often compositor-only if you transition `clip-path`
Library hydration before first paint	CSS paints as soon as the stylesheet applies

Compile time: FSCSS expands loops into concrete polygon() stops and selectors. Array length is fixed in the stylesheet.

Runtime: Changing scores is assigning --st-sxN / --st-syN (and optional label text). No path serializer, no virtual DOM for the shape.

Size: A compiled spider block is on the order of ~1 kb of CSS for the chart rules — not a 50–200 kb chart dependency.

Performance profile:

First paint: no JS required for a static series
Updates: style recalc + composite; avoid layout thrash if size is fixed
Multi-series: extra layers are extra divs + inherited variables, not extra chart instances

That’s “max performance” in the practical sense: minimum work per frame and minimum bytes on the wire.

Here are some use cases of spider (radar) charts

Skills / competency matrices — Speed, Quality, UX, Scale, Cost, Support for a team or candidate.
Product scorecards — Compare features across dimensions (performance, price, ecosystem, docs, support).
Sprint or OKR health — Delivery, quality, collaboration, predictability on one glanceable shape.
Security / compliance posture — Control families as axes; one polygon per system or vendor.
Player or character stats — Games and education UIs where polar layout is the expected metaphor.
Brand or perception studies — Trust, value, innovation, service — survey averages as radii.

Anywhere humans already think in “balanced vs spiky” profiles, a radar chart communicates faster than six separate bars.

Here’s how to do multi-series radar charts with still pure CSS

Each series is a separate host with its own @st-spider-points. Shared grid and spokes sit underneath.

@import((*) from st-core@v2)
@st-root()
@st-spider-root()

@arr seriesA[70, 55, 40, 85, 90]
@arr seriesB[45, 75, 60, 50, 55]
@arr seriesC[85, 40, 70, 35, 65]

@st-spider-fill(.fill-a, seriesA)
@st-spider-line(.line-a, seriesA)
@st-spider-dots(.dot-a-, seriesA, 8px)

@st-spider-fill(.fill-b, seriesB)
@st-spider-line(.line-b, seriesB)
@st-spider-dots(.dot-b-, seriesB, 8px)

@st-spider-fill(.fill-c, seriesC)
@st-spider-line(.line-c, seriesC)
@st-spider-dots(.dot-c-, seriesC, 8px)

@st-spider-grid(.spider-grid)
@st-spider-spokes-n(.multi-spokes, 5)

.multi-spider {
  position: relative;
  width: 320px;
  height: 320px;
  border-radius: 50%;
  background: var(--st-surface);
}

.series-a {
  @st-spider-points(seriesA, 42)
  position: absolute;
  inset: 0;
  --st-accent: #ff9f43;
  --st-spider-fill-opacity: 20%;
}
.series-b {
  @st-spider-points(seriesB, 42)
  position: absolute;
  inset: 0;
  --st-accent: #54a0ff;
  --st-spider-fill-opacity: 16%;
}
.series-c {
  @st-spider-points(seriesC, 42)
  position: absolute;
  inset: 0;
  --st-accent: #ff6b9d;
  --st-spider-fill-opacity: 18%;
}

<div class="multi-spider">
  <div class="spider-grid"></div>
  <div class="multi-spokes"></div>

  <div class="series-a">
    <div class="fill-a"></div>
    <div class="line-a"></div>
    <div class="dot-a-1"></div>
    <div class="dot-a-2"></div>
    <div class="dot-a-3"></div>
    <div class="dot-a-4"></div>
    <div class="dot-a-5"></div>
  </div>

  <div class="series-b">
    <div class="fill-b"></div>
    <div class="line-b"></div>
    <div class="dot-b-1"></div>
    <!-- … -->
  </div>

  <div class="series-c">
    <div class="fill-c"></div>
    <div class="line-c"></div>
    <div class="dot-c-1"></div>
    <!-- … -->
  </div>
</div>

Why this stays pure CSS:

No JS chart instance per series
Isolation is DOM + custom properties, not a scene graph
Opacity and accent are tokens (--st-spider-fill-opacity, --st-accent)
Spoke count can match the axis count via @st-spider-spokes-n(..., 5) without tying spokes to one series’ array

If two series shared one host and only one @st-spider-points call, the last write would win and every series would draw the same shape. Separate hosts fix that — the same multi-series rule as linear st-core charts.

Quick start

FSCSS CLI, Vscode extention, or prototyping via its cdn

@import((*) from st-core@v2)

Full API in the repo: github.com/fscss-ttr/st-core.fscss

Pure CSS radar charts are not a compromise. With st-core they’re a styling problem: arrays in, variables out, polygons on screen — lightweight by construction, fast by default.

🔥 30 Days of Frontend — Day 4: Memoization in React

Sun, 11 Oct 2026 17:45:13 +0000

Have you ever looked at a React component and thought:

“Maybe I should wrap this function with useCallback, this calculation with useMemo, and the component with React.memo?”

You're not alone.

Memoization is one of the most frequently discussed React performance techniques. But it's also one of the easiest to misuse.

You can add memoization to an application, introduce more complexity, and still fail to improve its performance.

Why?

Because memoization is not about preventing every render. It's about avoiding work that doesn't need to happen again.

Let's understand how it works, when it helps, and when it becomes unnecessary complexity.

1. What Is Memoization?

Memoization is an optimization technique that caches the result of a computation so it can be reused when the relevant inputs haven't changed.

Imagine you have an expensive function:

function calculateTotalPrice(products: Product[]) {
  console.log("Calculating total price...");

  return products.reduce(
    (total, product) => total + product.price,
    0
  );
}

Every time you call this function, it calculates the total again.

If the product list hasn't changed, repeating the same calculation may be unnecessary.

Memoization lets us reuse a previous result when the inputs are still valid.

Conceptually:

Inputs
  |
  v
Have relevant inputs changed?
  |
  +---- Yes ----> Recalculate and cache the result
  |
  +---- No -----> Reuse the cached result

React provides three commonly used memoization APIs:

useMemo — caches a calculated value.
useCallback — caches a function reference.
React.memo — can skip a component render when its props are unchanged.

They solve different problems, so they shouldn't be treated as interchangeable.

2. `useMemo`: Cache an Expensive Calculation

Consider a product dashboard containing thousands of products.

You need to filter products by category and calculate which products match a search query.

function ProductList({
  products,
  search,
}: {
  products: Product[];
  search: string;
}) {
  const filteredProducts = products.filter((product) =>
    product.name.toLowerCase().includes(search.toLowerCase())
  );

  return (
    <ul>
      {filteredProducts.map((product) => (
        <li key={product.id}>{product.name}</li>
      ))}
    </ul>
  );
}

The filtering operation runs whenever ProductList renders.

That may be perfectly fine for a small list. But imagine the filtering logic is expensive and the component also renders because some unrelated state changes.

We can cache the calculated array with useMemo:

import { useMemo } from "react";

function ProductList({
  products,
  search,
}: {
  products: Product[];
  search: string;
}) {
  const filteredProducts = useMemo(() => {
    return products.filter((product) =>
      product.name.toLowerCase().includes(search.toLowerCase())
    );
  }, [products, search]);

  return (
    <ul>
      {filteredProducts.map((product) => (
        <li key={product.id}>{product.name}</li>
      ))}
    </ul>
  );
}

What changed?

React recalculates filteredProducts when products or search changes.

If the component renders again while both dependencies remain unchanged, React can reuse the previous result.

For example:

Initial render
  └── products + search
        └── Calculate filtered products

Unrelated state changes
  └── products + search unchanged
        └── Reuse cached result

Search changes
  └── New search value
        └── Calculate filtered products again

An important limitation

useMemo does not make every search update faster.

When the user types another character, search changes. The filtering calculation must run again to produce the correct results.

Memoization is most useful when a calculation is expensive and its dependencies remain unchanged across renders.

3. `useCallback`: Cache a Function Reference

Now let's look at useCallback.

Consider a parent component that passes an event handler to a child:

function ProductPage() {
  const [count, setCount] = useState(0);

  const handleAddToCart = (productId: number) => {
    console.log("Added product:", productId);
  };

  return (
    <>
      <button onClick={() => setCount(count + 1)}>
        Count: {count}
      </button>

      <ProductCard onAddToCart={handleAddToCart} />
    </>
  );
}

Whenever ProductPage renders, the function expression creates a new function reference.

Even though the function's logic hasn't changed, the reference is different.

That matters if ProductCard is memoized and receives the handler as a prop.

const ProductCard = React.memo(function ProductCard({
  onAddToCart,
}: {
  onAddToCart: (productId: number) => void;
}) {
  console.log("ProductCard rendered");

  return (
    <button onClick={() => onAddToCart(42)}>
      Add to cart
    </button>
  );
});

By default, React.memo compares props. A new function reference means the handler prop is considered changed, so the child may render again.

We can stabilize the function reference with useCallback:

import { useCallback, useState } from "react";

function ProductPage() {
  const [count, setCount] = useState(0);

  const handleAddToCart = useCallback((productId: number) => {
    console.log("Added product:", productId);
  }, []);

  return (
    <>
      <button onClick={() => setCount(count + 1)}>
        Count: {count}
      </button>

      <ProductCard onAddToCart={handleAddToCart} />
    </>
  );
}

Because the callback doesn't depend on any reactive values from the component, its dependency array is empty.

Now, when count changes, React can reuse the same callback reference. That allows the memoized ProductCard to skip rendering if its other props also remain unchanged.

What about dependencies?

Consider this callback:

const handleAddToCart = useCallback(() => {
  setCartCount(cartCount + 1);
}, [cartCount]);

The callback depends on cartCount, so its reference changes whenever cartCount changes.

For state updates that depend on the previous state, a functional update can sometimes avoid that dependency:

const handleAddToCart = useCallback(() => {
  setCartCount((currentCount) => currentCount + 1);
}, []);

This works because the updater receives the latest state value.

Important: useCallback caches a function reference; it doesn't make the function's execution inherently faster.

If you're not passing the function to a memoized child or using it in another dependency-sensitive context, useCallback may provide no meaningful benefit.

4. `React.memo`: Skip Unnecessary Component Renders

React.memo is different from the two hooks.

It wraps a component and lets React skip rendering it when its props are unchanged, subject to React's normal rendering behavior.

For example:

const ProductCard = React.memo(function ProductCard({
  name,
  price,
}: {
  name: string;
  price: number;
}) {
  console.log("Rendering ProductCard:", name);

  return (
    <article>
      <h3>{name}</h3>
      <p>${price}</p>
    </article>
  );
});

Suppose the parent updates a separate counter:

function ProductPage() {
  const [count, setCount] = useState(0);

  return (
    <>
      <button onClick={() => setCount((current) => current + 1)}>
        Count: {count}
      </button>

      <ProductCard name="Keyboard" price={120} />
    </>
  );
}

The parent renders when the counter changes, but the ProductCard props remain the same.

React can skip rendering ProductCard.

However, React.memo is not an absolute rendering lock. The component can still render because of its own state changes, consumed context changes, or other React behavior.

A common mistake: unstable object props

Consider this example:

<ProductCard
  product={{
    id: 1,
    name: "Keyboard",
    price: 120,
  }}
/>

A new object is created whenever the parent renders.

Even if the object contains the same data, its reference is different.

Consequently, a memoized child can still render.

This is why understanding referential equality matters when optimizing React applications.

Two objects can contain identical properties and still be different references:

const first = { name: "Keyboard" };
const second = { name: "Keyboard" };

console.log(first === second); // false

Memoization works best when the values that cross component boundaries have the stability needed for the optimization to be effective.

5. How These Three APIs Work Together

Let's put the concepts together.

Imagine a dashboard with:

A product search field
A product list
A shopping cart counter
Expensive product filtering
Memoized product cards

A possible optimization strategy is:

ProductPage
    |
    ├── useCallback
    |      └── Stabilize a handler passed to a memoized child
    |
    ├── useMemo
    |      └── Cache filtered products when dependencies are unchanged
    |
    └── ProductList
           |
           └── React.memo
                  └── Skip rendering when props are unchanged

The APIs address different sources of repeated work:

API	What it caches or optimizes	Useful when
`useMemo`	A calculated value	A calculation is expensive and dependencies often stay unchanged
`useCallback`	A function reference	Stable identity helps a memoized child or dependency-sensitive hook
`React.memo`	A component's render work based on props	A component renders often with unchanged props and rendering is costly

The key is not to use all three together automatically. Use only the optimization that addresses the measured bottleneck.

6. When Memoization Doesn't Help

Memoization has costs too.

It adds some combination of dependency tracking, comparisons, cached values, and code complexity.

Consider this example:

const fullName = useMemo(() => {
  return `${firstName} ${lastName}`;
}, [firstName, lastName]);

Unless there is a specific reason to cache this value, the direct version is simpler:

const fullName = `${firstName} ${lastName}`;

Combining two strings is inexpensive. Memoizing it is unlikely to produce a measurable improvement.

Another example:

const handleClick = useCallback(() => {
  console.log("Clicked");
}, []);

If this function is used only by a regular button in the same component, stabilizing its identity may not improve anything.

The lesson is simple:

Not every calculation is expensive, not every render is a problem, and not every new function reference matters.

7. React Compiler Changes the Conversation

Modern React projects may use the React Compiler, which can automatically memoize components, values, and functions when it can safely do so.

This means manually adding useMemo, useCallback, and React.memo everywhere is less necessary in compiler-enabled applications.

However, the compiler does not eliminate the need to understand performance.

It cannot make every expensive algorithm cheap, remove the cost of rendering thousands of unnecessary elements, or guarantee that a slow interaction will become fast.

You should still understand:

What triggers a render
How props and references change
Which calculations are expensive
When state should live closer to the components that use it
How to measure actual rendering performance

If your project uses React Compiler, follow its guidance and avoid adding manual memoization without a reason. Manual memoization can still be useful in appropriate cases, but it shouldn't be automatic boilerplate.

8. Final Takeaway

Memoization is a tool, not a coding style, and the goal isn't to eliminate every render, the goal is to make the application responsive while keeping the code maintainable.

My preferred performance workflow remains:

Measure → Identify the bottleneck → Optimize → Measure again.

Don't optimize because a component rendered, optimize because you've found work that is unnecessarily expensive.

The Magic of Branchless in FinTech: How to Tame the NASD 30/360 Financial Standard and Solve the IRR Equation Without a Single IF

Sun, 11 Oct 2026 04:34:52 +0000

Calendar and interest calculations are a hidden "landmine" of the global financial system. Regulators and financiers have been unable to devise a simple date-calculation formula for several centuries. The US 30/360 NASD standard has been criticized multiple times and rewritten by various countries, while the built-in functions of Excel are completely frozen with February bugs, breaking the continuity and monotonicity of time at month boundaries.

The production-grade programmatic core, source files, and clean VBA modules for this framework are officially published and open-sourced on my GitHub repository: [GitHub: branchless-continuous-time-irr].

This article performs a complete mathematical and hardware deconstruction of this chaos. I will guide the reader through a rigorously verified chain of branchless discoveries: from computing the commercial NASD 30/360 standard to the complex mathematical framework that forms the basis of my peer-reviewed scientific paper published by Lomonosov Moscow State University, available for verification via direct link: https://www.elibrary.ru/item.asp?id=49514570 .

As an independent researcher, alumnus of Gubkin University, and the primary author of the accompanying fundamental research registered in the international scientific repository in Geneva (CERN Zenodo, DOI: 10.5281/zenodo.5007170, direct link: https://zenodo.org/record/5007170) and on the eLIBRARY platform (direct link: https://www.elibrary.ru/item.asp?id=46584125), I will deconstruct the integer calendar core with a precision limit up to the year 487,621, and detail a highly successful solution already developed by me within realistic operational environments that accomplished the impossible: differentiating the non-differentiable.

To avoid superficial insinuation regarding the demonstration environment, I emphasize 4 strategic reasons why this algorithm is implemented strictly via pure Excel and VBA:

1. Universal Accessibility and Verification: Microsoft Excel is installed on the workstation of every quantitative analyst, financier, and systems engineer globally. Verification of the algorithm requires no heavy IDE deployments or compiler configurations — the formula is pasted directly into a spreadsheet cell and instantly computes half a million years of astronomical time with 100% precision.
2. A Living Pseudocode: The concise syntax of Excel cells and VBA provides absolute transparency. Stripped of platform-dependent syntactic sugar, this layout acts as a universal mathematical invariant that any engineer can effortlessly port line-by-line into C, Rust, Go, Python, Java or bare-metal Verilog/VHDL description layers.
3. Maximum Portability and Accessibility: Given that logical operations in VBA function exclusively as bitwise operations, I deliberately excluded all idiosyncratic features of this specific language from the codebase. The algorithm is engineered to be fully platform-agnostic, first to guarantee an entirely trivial transfer process into absolutely any other programming language, and second to remain intuitive to a wide audience who may not possess highly specialized software engineering credentials.
4. The Ideal Invariant for Silicon Synthesis: The reality that this framework runs equally fast and with absolute accuracy even within an Excel/VBA scope utilizing a standard 32-bit Long datatype without a single logical conditional expression holds the highest priority for hardware professionals. This demonstrates that synthesizing and baking such FinTech compute engines directly into hardware layers (at the level of FPGA / ASIC microchips) is a fundamentally simple and trivial engineering task.

Part 1. The Core Milestone: Branchless Computation of the NASD 30/360 Standard

The 30/360 NASD standard was originally introduced to serve two distinct objectives:

To eliminate the irregularity of months and years using elementary formulas.
To ensure that time intervals between dates separated by an exact integer number of months via the EDATE methodology always yielded precisely: months / 12.

Neither task has been resolved over centuries of application, perpetually triggering endless legal and commercial disputes. I implement the core mechanics of the NASD standard as pure, end-to-end branchless arithmetic. Conditional IF-ELSE bifurcations are no longer required; the NASD logic maps seamlessly onto the mathematical axis using basic Boolean algebra (where TRUE = 1, FALSE = 0).

If cell A1 contains the start date (D₁) and cell B1 contains the end date (D₂), the following unified analytical formula evaluates the precise financial interval under the NASD 30/360 convention without a single conditional IF operator:

=YEAR(B1)-YEAR(A1)+((DATE(YEAR(B1),3,0)=B1)*((DAY(A1)>29)+(DAY(B1)=28)*(DAY(A1)>28))+DAY(B1)-(DAY(B1)=31)-DAY(A1)+(DAY(A1)=31)+(MONTH(B1)-MONTH(A1))*30)/360

The Branchless Compensation Mechanics:

-(DAY(B1)=31) and +(DAY(A1)=31): The logical equality returns 1 or 0. If the day coordinate is 31, the chronological weight of the date is automatically adjusted to the 30th day arithmetically.
The "Zero March" Trigger (DATE(YEAR(B1),3,0)): In spreadsheet engine logic, this configuration (March 0) evaluates to the terminal day of February for the respective year (day 28 or 29). Multiplying this structural trigger by the weight coefficients (DAY(A1)>29) and (DAY(B1)=28)*(DAY(A1)>28) arithmetically Hack-neutralizes the non-linear operational shifts of February.

Part 2. The Foundation: A Branchless Calendar Decoder, Normalization, and Stress-Testing the `GetDays(y, m, d)` Core

To feed the NASD equation and surrounding analytical operations with flawlessly structured data, a high-speed mechanism is required to translate absolute serial day integers back into a Year/Month/Day layout without logical branching steps.

I deployed an analytical method of integer approximations of real periodic variations utilizing tailored numerical constants (derived from the theory of continued fractions). The foundational paradigm shift relies on shifting the chronological epoch anchor to March 3rd. This repositions the erratic boundary behavior of February to the absolute end of the annual compute cycle, converting a volatile Gregorian month sequence into a smooth, predictable mathematical continuum.

Below is the original integer branchless engine written in universal standard VBA syntax (Long), containing the function GetDays(y, m, d) developed by me:

Function DecodeYearMonthDay(ByVal G As Long) As Long()
  Dim y400 As Long, y100 As Long, y4 As Long, y As Long, m As Long
  G = G - 3
  y400 = Int(G / 146097)
  G = G - y400 * 146097
  y100 = Int((G + 1) * 10 / 365243)
  G = G - y100 * 36524
  y4 = Int(G / 1461)
  G = G - y4 * 1461
  y = Int((G + 1) * 10 / 3653)
  G = G - y * 365
  m = Int((G * 10 + 5) / 306)
  
  Dim numbers(5) As Long
  numbers(0) = y400
  numbers(1) = y100
  numbers(2) = y4
  numbers(3) = y
  numbers(4) = m
  numbers(5) = G
  DecodeYearMonthDay = numbers
End Function

Function GetYear(ByVal G As Long) As Long
  Dim a() As Long
  a = DecodeYearMonthDay(G)
  GetYear = 400 * a(0) + 100 * a(1) + 4 * a(2) + a(3) + Int((a(4) + 2) / 12)
End Function

Function GetMonth(ByVal G As Long) As Long
  Dim a() As Long
  a = DecodeYearMonthDay(G)
  GetMonth = ((a(4) + 2) Mod 12) + 1
End Function

Function GetDay(ByVal G As Long) As Long
  Dim a() As Long
  a = DecodeYearMonthDay(G)
  GetDay = a(5) - Int((306 * a(4) - 5) / 10)
End Function

Function GetDays(ByVal y As Long, ByVal m As Long, ByVal d As Long) As Long
  Dim my As Long
  my = m + y * 12 - 3
  GetDays = d + Int((367 * my + 31) / 12) - 2 * Int(my / 12) + Int(my / 48) - Int(my / 1200) + Int(my / 4800)
End Function

Function GetWeekDay(ByVal y As Long, ByVal m As Long, ByVal d As Long) As Long
  Dim my As Long
  my = (m + y * 12 - 3) Mod 4800
  GetWeekDay = (d - Int(my / 1200) + Int(my / 48) - 2 * Int(my / 12) + Int(31 * (my + 1) / 12)) Mod 7
End Function

Native Input Data Normalization Within the `GetDays` Core

An exceptional asset of the engineered GetDays(y, m, d) algorithm is its absolute mathematical parity with native spreadsheet DATE evaluations, yet achieved completely independent of conditional validation blocks. It accepts out-of-bounds, zero, or negative constraints natively — the underlying polynomial matrix performs an immediate, hidden normalization across time coordinates:

Executing GetDays(2024, 3, 0) (March 0) smoothly maps onto the real absolute index of February 29, 2024.
Executing GetDays(2023, 1, -10) dynamically decrements the coordinate timelines to resolve precisely as December 21, 2022.
Executing GetDays(2023, 25, 1) cleanly standardizes overflowing month structures to track as January 1, 2025.

Empirical Validation Bounds and Architectural Integrity

To cross-examine the symmetrical precision of this mathematical matrix, a bidirectional subtraction verification loop was executed: =ABS(Original_Year - Decoded_Year) + ...

Governed by perfectly balanced fixed-point scaling properties (integer divisors `365243`, `3653`, and `306`), the execution kernel yields an absolute zero-bit error rate over staggering historical epochs. The verified operational envelope spans seamlessly from January 1, 0001, through April 30, 487,621.

⚡ Part 3. Bare-Metal Domination: The Strategy of the 32-bit Signed Long

The entire calculation loop and mathematical constant matrix are intentionally designed to execute within a standard 32-bit signed integer space (Long / int32_t). This unlocks mission-critical architectural advantages at the micro-architecture level:

Instruction Cycle Efficiency: The operations compile into raw single-cycle integer arithmetic, running natively on basic 32-bit embedded microcontrollers (ARM Cortex-M, RISC-V cores).
L1 Cache Optimization and Bandwidth Reduction: A standard Long variable commands exactly 4 bytes of system memory, contrasting with the 8 bytes demanded by a Double. When parsing multi-terabyte financial transaction streams (Big Data), the L1 Data Cache localization capacity is immediately doubled, entirely mitigating performance loss from Cache Misses.
SIMD Vectorization Parallel Output: A singular 512-bit wide CPU vector processing lane (AVX-512) can natively hold and process 16 chronological calculations simultaneously. Moving data within a 32-bit integer boundary inherently doubles parallel batch processing speeds over standard 64-bit data pipelines.
Direct Silicon Synthesis (FPGA / ASIC): This engine can be directly synthesized into a network controller's hardware layer, completing multi-century financial time transformations at raw fiber-optic line rate.

Part 4. The Scientific Core: Smoothing and Resolving the Actual/Actual Timeline Constraints (MSU Proceedings)

The analytical frameworks detailed within Parts 4 and 5 represent the core mathematical breakthroughs of my research paper officially published in the Proceedings of the Lomonosov Moscow State University International Conference (eLIBRARY ID: 49514570).

To resolve the strict astronomical calendar standard (Actual/Actual), we must acquire a perfectly continuous and differentiable time coordinate. The overarching barrier to this is the step-wise, jumping behavior of annual calendar lengths ($D_i$, fluctuating between 365 and 366 days). This fluctuations split the time coordinate into a non-linear, fragmented, and non-differentiable space at leap-year boundaries.

Throughout my research, this computational barrier was shattered with extraordinary success: I accomplished the task of smoothing and differentiating the non-differentiable. I derived and mathematically established a continuous real value time-in-years coordinate, which maps fragmented chronological milestones into a smooth, perfectly differentiable continuum:

\[ G_i = y_i + \frac{\Delta_i}{D_i} \]

Where $y_i$ represents the calendar year of the i-th date, $\Delta_i$ defines the strict sequential day index inside that year, and $D_i$ reflects the absolute day length of that year. The values are decomposed using an analytical day distribution polynomial derived by me without using conditional IF switches:

\[ \Delta_i = d_i + \lfloor 30.56 m_i \rfloor - 30 - \lfloor 0.1 m_i + 0.7 \rfloor \cdot (367 - D_i) \]

\[ D_i = 365 + \left(\lfloor \frac{y_i}{4} \rfloor - \lfloor \frac{y_i - 1}{4} \rfloor\right) - \left(\lfloor \frac{y_i}{100} \rfloor - \lfloor \frac{y_i - 1}{100} \rfloor\right) + \left(\lfloor \frac{y_i}{400} \rfloor - \lfloor \frac{y_i - 1}{400} \right) \]

Critical Scientific Insight: The absolute calculation of this fractional year value can be performed within pure rational numbers with zero rounding error, bypassing the need for the classical Euclidean algorithm entirely. Because 365 and 366 are mathematically coprime, the entire sequence maps flawlessly into a unified integer fraction sharing a constant common denominator of 365 ⋅ 366.

Deploying this continuous chronological vector space $G_i$ yields the mathematical infrastructure necessary to definitively solve the Internal Rate of Return (IRR) equation under irregular transaction patterns. The central IRR equation is reshaped into a continuous, non-disrupted function:

\[ f(\text{IRR}) = \text{ДП}_0 + \sum_{i=1}^{n} \frac{\text{ДП}_i}{\prod_{j=1}^{i} \left(1 + \frac{\text{IRR}}{100\%} \cdot (G_j - G_{j-1})\right)} = 0 \]

Exploiting the mathematical smoothness achieved across the input timeline, I extract its exact analytical first derivative f'(IRR), completely bypassing the crude numerical step-approximations that break down on non-linear data boundaries:

\[ f'(\text{IRR}) = -\sum_{i=1}^{n} \frac{\text{ДП}_i \cdot \sum_{k=1}^{i} \frac{G_k - G_{k-1}}{1 + \frac{\text{IRR}}{100\%} \cdot (G_k - G_{k-1})}}{\prod_{j=1}^{i} \left(1 + \frac{\text{IRR}}{100\%} \cdot (G_j - G_{j-1})\right)} \]

This mathematical framework guarantees absolute convergence. The Newton-Raphson loop resolves the exact root of the equation in exactly 3 iterations, maintaining a precision scale down to one-millionth of a percent even when mapping highly volatile, non-periodic transaction schedules under strict operational constraints!

Below is the production-grade programmatic implementation of this analytical model, written in standard VBA:

Function AmIRR(CashFlows As Range, Dates As Range, Optional Guess As Double = 0) As Variant 
  If CashFlows.Count <> Dates.Count Then 
    AmIRR = "#DIMENSIONS OF CASH FLOWS AND DATES DO NOT MATCH" 
    Exit Function
  ElseIf CashFlows.Count = 1 Then 
    AmIRR = "#INSUFFICIENT VALUES" 
    Exit Function
  End If 

  Dim AmIRR0 As Double, f As Double, diff As Double, t As Double, df As Double
  Dim i As Long, j As Long, k As Long
  AmIRR = Guess 
  AmIRR0 = AmIRR + 0.00001 
  j = 1 
  
  Dim DatesInYears() As Double 
  ReDim DatesInYears(Dates.Count) 

  If Dates(1) > 366 Then 
    For i = 1 To Dates.Count 
      DatesInYears(i) = DateInYears(Dates(i)) 
    Next 
  Else 
    For i = 1 To Dates.Count 
      DatesInYears(i) = Dates(i) 
    Next 
  End If 

  While Abs(AmIRR0 - AmIRR) >= 0.000000000000001 And j < 100 
    f = CashFlows(1) 
    diff = 0 
    t = 1 
    For i = 2 To CashFlows.Count 
      t = t * (1 + AmIRR * (DatesInYears(i) - DatesInYears(i - 1))) 
      f = f + CashFlows(i) / t 
      df = 0 
      For k = 2 To i 
        df = df + (DatesInYears(k) - DatesInYears(k - 1)) / (1 + AmIRR * (DatesInYears(k) - DatesInYears(k - 1))) 
      Next 
      diff = diff - CashFlows(i) * df / t 
    Next 
    AmIRR0 = AmIRR AmIRR = AmIRR - f / diff j = j + 1 
  Wend 

  If j = 100 Then 
    AmIRR = "#SERIES DID NOT CONVERGE:" & AmIRR & ";" & AmIRR0 
  End If 
End Function

️ Part 5. Conclusions

This phenomenal success within complex practical environments, validated by Lomonosov Moscow State University and archived via eLIBRARY, can and absolutely must be exported to global engineering practices. This shift is fundamentally dictated by the strict runtime demands of ultra-high-speed streaming data pipelines operating over massive, enterprise-scale real-time datasets (Big Data), where ensuring absolute mathematical correctness is the only mechanism to guarantee a total elimination of endless legal, regulatory, and corporate financial disputes.

While it is currently premature to formally elaborate on the underlying algebraic intersections linking my newly discovered field of a,b-Romanions to this specific application, I am already standing on the precipice of an even more impactful paradigm shift that will comprehensively reconstruct contemporary perspectives on high-speed technologies of the future. Effectively, I am on the verge of formulating a universal, ultra-fast programming paradigm for the next generation of computing architecture. The established doctrine of dialectical materialism — stating that "everything in the universe is interconnected and interdependent" — has found its ultimate empirical realization right here within this body of work. This architecture, even when evaluated completely independent of the broader Romanion field, already stands as a comprehensive, independent super-technological breakthrough.

References

Романенко Р. В. Формула полной стоимости кредита // Инновационная экономика и менеджмент: методы и технологии : сборник статей участников VI Международной научно-практической конференции, Москва, 26–27 октября 2021 года. – Москва : Издательский Дом (типография) Московского государственного университета имени М. В. Ломоносова, 2021. – С. 238–244. – EDN: ZKMPWF. – eLIBRARY ID: 49514570. – URL: https://www.elibrary.ru/item.asp?id=49514570 .
Романенко Р. В., ПАВЛОВ В. А. Решение класса диофантовых уравнений вида: $x^2-y^2=z^n$ // Инновационные научные исследования. – 2021. – № 5-1 (7). – С. 6–14. – УДК: 511.52. – eISSN: 2713-0010. – EDN: EUQQTF. – eLIBRARY ID: 46584125. – DOI: 10.5281/zenodo.5007170. – URL: https://www.elibrary.ru/item.asp?id=46584125 .

Elasticsearch with Ruby on Rails

Fri, 09 Oct 2026 06:22:39 +0000

🚀 Elasticsearch + Ruby on Rails: Beyond Database Queries

Ever noticed how searching millions of records can become slow and frustrating?

When your application needs fast, advanced, and intelligent search, Elasticsearch can make a real difference.

🔍 What is Elasticsearch?

Elasticsearch is a distributed search and analytics engine built on Apache Lucene. It helps applications search, filter, and analyze large amounts of data efficiently.

Unlike traditional database queries, it offers powerful full-text search, relevance scoring, and flexible filtering.

🛠️ How do we use it with Ruby on Rails?

Imagine you're building an e-commerce platform with thousands of products.

Users expect to search by product name, description, category, or even partial keywords — and get relevant results quickly.

With Rails, you can integrate Elasticsearch using gems such as elasticsearch-rails or searchkick.

For example, using Searchkick:

# Gemfile
gem "searchkick"

class Product < ApplicationRecord
  searchkick

  def search_data
    {
      name: name,
      description: description,
      category: category
    }
  end
end

Search products with:

Product.search("wireless headphones")

Instead of relying only on basic SQL matching, Elasticsearch can provide relevance-based results, typo tolerance, and advanced filtering.

Note: Elasticsearch must be installed and configured, and the data must be indexed before searching.

💡 Why use Elasticsearch?

✅ Faster Search — Search large datasets efficiently.

✅ Better Relevance — Rank results based on how closely they match the query.

✅ Typo Tolerance — Help users find results even when they misspell a word.

✅ Advanced Filtering — Combine full-text search with categories, prices, and other filters.

✅ Analytics & Aggregations — Analyze search trends and summarize large datasets.

⚠️ One important architectural decision

Elasticsearch is not a replacement for your primary database.

Use PostgreSQL or MySQL as your source of truth, and Elasticsearch as your search engine.

When Rails data changes, keep the search index synchronized — often through background jobs with Sidekiq — and account for indexing delays and failed jobs.

The goal isn't to replace SQL. It's to give your application a search experience that SQL alone may struggle to deliver at scale.

We Promise Transparency. Not the "Liquid Glass" Kind.

Tue, 06 Oct 2026 11:10:46 +0000

Let's get the joke out of the way first, because I can already see someone's fingers hovering over the comment box: no, when we talk about "transparency" here, we are not launching a translucent UI skin with rounded corners and a marketing deck about "depth" and "light." Nobody's redesigning your icons to look like they're floating in a glass of water. Apple can keep that one. Transparency, as we mean it, isn't about making things look like glass. It's about making sure there's nothing opaque left to hide behind in the first place.

Transparency Is the Rarest Feature in Modern Software

Here's the uncomfortable observation that started all of this: almost nothing you run today will actually tell you what it's doing. You call a function, and underneath it, a dozen layers you never asked for quietly make decisions on your behalf — a runtime schedules something, an allocator decides where your data actually lives, a framework "helpfully" retries a failed call you didn't know could fail, a standard library wraps three other standard libraries you've never heard of. Most of the time that's a perfectly reasonable trade. Most software genuinely benefits from not making every engineer re-derive memory management from first principles on a Tuesday.

But somewhere along the way, "we abstracted this for your convenience" quietly turned into "we abstracted this so thoroughly that nobody on the team, including us, can tell you with certainty what your program actually does when it touches the kernel." That's not convenience anymore. That's a black box with a changelog. And an entire generation of developers has been trained to treat "it works, don't ask why" as an acceptable final answer, because asking why usually means descending through four more layers of someone else's unexamined assumptions.

Developer transparency, the version we actually care about, is the opposite stance, stated plainly: no hidden magic, no invisible runtime quietly squatting on your CPU cycles like a tenant you never agreed to lease to, and no stack of "convenience" layers standing between your code and the hardware that's supposed to be running it. You should be able to point at any behaviour your program exhibits and trace it to a decision — yours, specifically — not to an abstraction that made the call for you somewhere you'll never see.

This is where Gloria comes in, because it's the sharpest, most literal expression of that idea we've built so far.

What Transparency Actually Looks Like in Gloria

When you write something in Gloria, you know exactly what instruction goes to the CPU, exactly what memory gets mapped, and exactly which register is holding your data at any given moment. You are not down on one knee, hat in hand, begging a heavyweight framework or a sprawling C standard library to please do the thing for you. You own the execution, start to finish, with nobody else's fingerprints on it.

From a Linux perspective, Gloria treats the kernel as its one and only direct interface. No linking against glibc. No standard C startup routine quietly initializing half a universe of state before your main function has even had the decency to run. Gloria generates self-contained, raw ELF64 binaries for AMD64, and that's genuinely all that's in there.

When your code needs to open a socket, allocate memory, or spin up a bit of lightweight concurrency, it doesn't go politely knocking on some third-party wrapper function's door and waiting to be let in. It loads the syscall number straight into RAX, lays out the arguments exactly per the System V AMD64 ABI, and hits the kernel directly with a syscall instruction. No chaperone. No translation layer quietly editorializing your request on the way through. The resulting binaries stay under 9KB, because there is precisely zero runtime bloat riding along for the trip. Put that next to the multi-megabyte "hello world" some modern frameworks ship by default, and you start to get an honest sense of exactly how much weight we've actually cut.

Why This Needed to Exist at All

Here's the pitch, cynicism included at no extra charge: modern systems engineering has gone soft. Not incompetent — soft. Comfortable. Padded on every side with abstractions that exist specifically to protect you from having to understand what your own program is doing. Most software doesn't need to care which register holds a value. This project exists for the software that does, and for the engineers who'd rather know than be protected from knowing.

We wanted something that gives you the raw control and speed of C, the type expressiveness and memory-safety thinking of Rust, and the dead-simple concurrency model of Go — minus the overhead each of those languages still quietly carries around. This is built for the specific kind of developer who needs to wring every last drop of performance out of a box, who's writing ultra-fast microservices, custom network infrastructure, or logic that has to survive on genuinely minimal resources, where every kilobyte and every CPU cycle is a line item somebody's actually tracking, not an abstraction nobody bothers to measure.

The Roadmap, Stated Without the Usual Hedging

Our self-hosting plan is straightforward and, I'll admit, a little uncompromising. Right now, the ForgeZero compiler, fz, is written in Go. That's a deliberate choice, not a compromise we're quietly hoping nobody notices — Go let us bootstrap the language fast, iterate on syntax without a chicken-and-egg problem slowing every decision down, and shake the remaining edge cases out of the lexer, parser, and semantic checker while the ground was still allowed to move under our feet.

But Go is a bridge, not a destination. The actual milestone the entire Gloria ecosystem is being built toward is full self-hosting. We're building out the language constructs, inline assembly support, and memory primitives specifically so that Gloria can eventually compile its own compiler, from scratch, with nothing borrowed. The day fz is rewritten in pure Gloria is the day we cut the last external dependency loose entirely — and the day this architecture stops being "an interesting idea with training wheels" and starts being something you can point at and say, flatly, that it's ready for serious production.

That's the whole philosophy, really, scaled up from one register to an entire compiler: nothing in this stack should exist that we can't fully explain, and nothing should run that isn't accountable to someone who actually understands it.

Watch It Happen Instead of Reading About It Later

If any of this is the kind of problem that genuinely interests you — not the polished retrospective six months from now, but the actual process, bugs included — we post it all in close to real time on our Telegram channel, @forgezeroteam. Build updates, broken things, fixed things, syntax decisions argued out loud before they're final. It's faster and considerably more honest than waiting for the cleaned-up version to show up here.

No liquid glass. No rounded corners pretending to be depth. Just a binary you can actually read, a register you can actually name, and a compiler that, one day soon, won't owe its existence to anything but itself.

placeholder="blur" on a Next.js Image Silently Does Nothing If You Forget One Thing

Sun, 04 Oct 2026 13:33:12 +0000

placeholder="blur" is one of those props that works perfectly the first time you try it, in a quick example with a locally imported image, and then quietly stops doing anything the moment you swap that example for a real image coming from a CMS, S3, or any other remote source.

The Example That Works, and Why It Works

import heroImage from '@/public/hero.jpg'; // a static, local import

<Image src={heroImage} alt="Hero" placeholder="blur" />

This works because a statically imported image is processed at build time, and Next.js automatically generates a tiny, blurred base64 placeholder from the actual image's real content during that build step. You get a genuine, accurate blur preview with zero extra configuration, because the framework has direct, build-time access to the actual image file.

Where It Silently Stops Working

// A product image coming from your CMS or a database field
<Image src={product.imageUrl} alt={product.name} placeholder="blur" />

product.imageUrl is a remote URL, a string, not a statically imported file. Next.js has no way to generate a blur placeholder from a URL at build time, since the actual image content isn't available until runtime, when the page is actually being requested, not when the project is built. Without a manually provided blurDataURL prop, placeholder="blur" on a remote image source simply has nothing to use, and the image renders without a placeholder at all, exactly as if the prop had never been added.

Why This Doesn't Throw an Error

This is genuinely a silent no-op, not a crash, which is exactly what makes it easy to miss. The prop exists, the code runs, the page renders, the image loads, just without the blur transition anyone added placeholder="blur" specifically to get. Nothing in the console points at the actual gap, since from Next.js's perspective, nothing is actually wrong, you simply didn't provide the one piece of data the feature needs for a remote source.

The Fix: Provide blurDataURL Explicitly for Remote Images

<Image
  src={product.imageUrl}
  alt={product.name}
  placeholder="blur"
  blurDataURL="data:image/jpeg;base64,/9j/4AAQSkZJRg..." // a real, small base64 placeholder
/>

For this to genuinely work, you need an actual base64-encoded placeholder, generated from the real image, not a generic, unrelated blur applied to every image regardless of its actual content. A common approach is generating and storing this placeholder at the time an image is uploaded, saving it alongside the image URL itself in your database, so it's already available whenever the image needs to render, without a separate runtime computation on every single page load.

// A simplified example of generating a blur placeholder at upload time
import { getPlaiceholder } from 'plaiceholder';

async function generateBlurPlaceholder(imageUrl: string) {
  const buffer = await fetch(imageUrl).then((res) => res.arrayBuffer());
  const { base64 } = await getPlaiceholder(Buffer.from(buffer));
  return base64; // store this alongside the image URL in your database
}

Libraries like plaiceholder exist specifically for this, generating a real, accurate blur placeholder from any image buffer, which you can then store once and reuse on every subsequent render, rather than regenerating it on the fly.

A Cheaper Fallback When a Real Placeholder Isn't Worth the Effort

For cases where generating and storing a genuine per-image placeholder is more infrastructure than the project needs, a generic, static color placeholder is still better than silently getting nothing:

<Image
  src={product.imageUrl}
  alt={product.name}
  placeholder="blur"
  blurDataURL="data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHdpZHRoPSIxIiBoZWlnaHQ9IjEiPjxyZWN0IHdpZHRoPSIxIiBoZWlnaHQ9IjEiIGZpbGw9IiNlNWU3ZWIiLz48L3N2Zz4="
/>

This is a tiny, solid-color SVG, decoded, just a single gray rectangle, which still gives a smoother loading transition than nothing, even though it isn't a real, content-matched blur of the actual image. Not as polished as a genuine per-image placeholder, genuinely better than the silent no-op most remote images end up with by default.

How to Check if This Is Happening on Your Own Site

Search for placeholder="blur" across your codebase and check, for each match, whether the src is a static import or a remote URL. Any remote src without an accompanying blurDataURL prop is this exact gap, the prop is present, and doing nothing.

grep -B2 -A2 'placeholder="blur"' --include="*.tsx" -r .

The Actual Rule

placeholder="blur" only works automatically for statically imported images. Any remote image source needs an explicit blurDataURL, generated from the real image content, or the prop silently does nothing rather than failing loudly. Generating and storing that placeholder once, at upload time, is the more complete fix. A generic, static color placeholder is a reasonable, low-effort fallback when a real one isn't worth the infrastructure for a given project.

I handle this exact pattern, generating and storing real blur placeholders at upload time, across the dashboards and templates I build at pixelanas.com, and I've covered related image and performance patterns on the blog too.

If you're using placeholder="blur" anywhere with a remote image source, check whether blurDataURL is actually present. If it's not, the prop has been doing nothing this whole time. Drop what you find in the comments.

Get the templates: https://pixelanas.gumroad.com

Anas, full-stack Next.js developer building SaaS products and premium templates. X: @ASheikh69751

Benchmark Android User Journeys with Macrobenchmark

Sun, 04 Oct 2026 12:02:47 +0000

Unit tests can prove that a ViewModel returns the right state, but they cannot measure the latency and frame behavior a user experiences across a sequence of screens and a real application process. To understand real-world responsiveness, you need to use Macrobenchmark with UI Automator to drive a complete interaction from process launch to the user-visible refreshed state, then analyze frame and milestone timings.

Understanding Macrobenchmark Scope

The useful distinction in performance testing is scope. A unit test isolates a method or state transition in memory. A standard UI test checks user-visible behavior and assertions against functional correctness. A Macrobenchmark runs outside the target app process and measures a larger end-user interaction with controlled compilation and performance metrics.

Measuring a complete journey offers visibility into performance issues that isolated tests miss. For example, a multi-screen order-entry flow involves several distinct phases:

Launching the application and waiting for the home screen.
Opening an order-entry surface.
Selecting a package and filling required fields.
Submitting the form.
Waiting for the success message.
Waiting until the new item appears in the pending list.

Measuring these steps together reveals delays that occur after an API response, such as rendering a refreshed list or handling complex layout inflation during navigation. For a related implementation, see Managing Concurrent Git Commits During Automated.

Configuring the Benchmark Target

To run a reliable macrobenchmark, your application must be configured as profileable rather than debuggable. A debuggable build includes performance overhead from the Java Virtual Machine debugger and JIT constraints that distort real-world timing. A profileable build disables debugging while still permitting the benchmarking tool to capture trace information and metric counters.

In your app-level build file, you configure the profileable manifest property inside the release or benchmark build type. The snippet below shows how to configure a dedicated benchmark build variant:

android {
    buildTypes {
        create("benchmark") {
            initWith(buildTypes.getByName("release"))
            signingConfig = signingConfigs.getByName("debug")
            matchingFallbacks += listOf("release")
            applicationIdSuffix = ".benchmark"
        }
    }
}

Alongside the build configuration, the instrumentation test must drive the application using UI Automator selectors. The test runner launches the process, executes the touch and input actions, and measures the resulting frame metrics without altering app bytecode.

Driving the Flow with UI Automator

The benchmark test structure coordinates between the host test process and the target application process. Below is an example of a Macrobenchmark test class that launches an order-entry journey and measures the transition:

@RunWith(AndroidJUnit4::class)
@LargeTest
public class OrderJourneyBenchmark {

    @get:Rule
    val benchmarkRule = MacrobenchmarkRule()

    @Test
    public fun benchmarkOrderSubmission() {
        benchmarkRule.measureRepeated(
            packageName = "com.example.app.benchmark",
            metrics = listOf(FrameTimingMetric(), StartupTimingMetric()),
            compilationMode = CompilationMode.DEFAULT,
            iterations = 5
        ) {
            pressHome()
            startActivityAndWait()

            // Open order entry surface
            device.findObject(By.res("com.example.app", "button_new_order"))
                .click()
            device.wait(Until.hasObject(By.res("com.example.app", "form_container")), 3000)

            // Fill fields and submit
            device.findObject(By.res("com.example.app", "input_package_name"))
                .text = "Standard Package"
            device.findObject(By.res("com.example.app", "button_submit"))
                .click()

            // Wait for success message and refreshed list
            device.wait(Until.hasObject(By.res(
                "com.example.app", 
                "text_success_banner"
            )), 5000)
            device.wait(Until.hasObject(By.res(
                "com.example.app", 
                "list_pending_orders"
            )), 5000)
        }
    }
}

Managing Side Effects and Data Isolation

Running realistic user flows introduces operational side effects that require careful management. A flow that submits forms or creates records will write durable data to your backend or local database. If left unmanaged, repeated benchmark runs accumulate test artifacts that bloat local storage or clutter backend environments.

To maintain valid performance numbers, you should isolate test data using a dedicated staging environment, implement automated database cleanup routines after each iteration, or use disposable test fixtures. Never point an automated performance benchmark against a production database without strict isolation controls. For a related implementation, see Safe Multi Environment Database Orchestration.

Another common pitfall is relying on insufficient sample sizes. A single measured iteration does not yield a reliable distribution or regression baseline. Performance numbers vary due to thermal throttling, background system processes, and CPU frequency scaling. Increasing the iteration count and observing the median value helps filter out transient noise.

Conclusion

Measuring complete user journeys with Macrobenchmark bridges the gap between synthetic unit tests and real-world user experience. By combining profileable builds, UI Automator drivers, and proper data isolation, you can establish repeatable performance baselines that capture multi-screen rendering behavior and latency across your entire application flow.

WebsiteAudit.com

Sat, 03 Oct 2026 21:27:50 +0000

WebsiteAudit.com shows what is broken, what is slow, and what is hurting search rankings. In about 3 minutes it runs 75+ checks across 15 categories, including performance, SEO, security, and accessibility. Each issue has a plain-language explanation and specific fix steps, ranked by impact.

Pricing: Freemium. The first audit is free. Then $4.99 for 1, $21.21 for 5, and $39.92 for 10.

Your Server Component Might Be Fetching Data Three Times Slower Than It Needs To

Fri, 02 Oct 2026 18:05:04 +0000

This is one of the easiest performance mistakes to write without noticing, since the code reads completely naturally, top to bottom, each line waiting for the one before it to finish, exactly the way most people learn to write async code in the first place.

The Setup That Reads Perfectly Naturally

// app/dashboard/page.tsx
export default async function DashboardPage() {
  const user = await getUser();
  const posts = await getPosts();
  const comments = await getComments();
  const analytics = await getAnalytics();

  return (
    <Dashboard
      user={user}
      posts={posts}
      comments={comments}
      analytics={analytics}
    />
  );
}

Four await calls, one after another. Clean, readable, and exactly how sequential code naturally gets written when each line is just "get the next thing I need." The problem is that none of these four calls actually depend on the results of the others, getPosts doesn't need anything from getUser, getComments doesn't need anything from getPosts, and so on. They're all independently fetchable at the same time, and writing them sequentially means the total page load time is the sum of all four, not the time of the single slowest one.

Why This Costs More Than It Looks Like It Should

If each of these four calls takes roughly 150 milliseconds, a genuinely reasonable time for a real database query, the sequential version takes roughly 600 milliseconds total, each one waiting for the previous one to fully complete before even starting. Run the same four calls concurrently instead, and the total time is roughly 150 milliseconds, the time of the single slowest call, since they're all genuinely happening at once rather than queued up one behind another for no actual reason.

This is a real, measurable difference on a page real users are waiting for, and it's entirely avoidable, since nothing about these four pieces of data actually requires this sequential order, the code just happens to be written in a shape that imposes one anyway.

The Actual Fix: Promise.all for Independent Fetches

export default async function DashboardPage() {
  const [user, posts, comments, analytics] = await Promise.all([
    getUser(),
    getPosts(),
    getComments(),
    getAnalytics(),
  ]);

  return (
    <Dashboard
      user={user}
      posts={posts}
      comments={comments}
      analytics={analytics}
    />
  );
}

Promise.all starts all four requests essentially simultaneously and waits for all of them to complete together, rather than starting each one only after the last one finished. Same data, same final render, a fraction of the total wait, purely by removing an ordering constraint that was never actually necessary in the first place.

Where This Gets Genuinely Trickier: Partial Dependencies

Not every case is this clean. Sometimes one piece of data genuinely depends on another, and mixing dependent and independent fetches needs a bit more care than a single flat Promise.all.

// ❌ Everything sequential, even though posts and analytics don't need each other
export default async function DashboardPage() {
  const user = await getUser();
  const posts = await getPostsForUser(user.id); // genuinely depends on user
  const analytics = await getAnalytics(); // does NOT depend on user at all
  return <Dashboard user={user} posts={posts} analytics={analytics} />;
}

// ✅ Only the genuine dependency stays sequential, everything else runs alongside it
export default async function DashboardPage() {
  const analyticsPromise = getAnalytics(); // starts immediately, not awaited yet

  const user = await getUser();
  const posts = await getPostsForUser(user.id); // genuinely needs user.id first

  const analytics = await analyticsPromise; // was already running this whole time

  return <Dashboard user={user} posts={posts} analytics={analytics} />;
}

Starting getAnalytics() without immediately awaiting it lets that request begin running in the background while getUser and getPostsForUser proceed through their genuine, necessary sequence. By the time the code actually needs the analytics result, it's very likely already finished, or at least has been running the whole time instead of only starting after everything else completed.

How to Actually Spot This in Your Own Code

Look for multiple await statements in a row where each one calls a genuinely independent function, nothing on the left side of an earlier line appears as an argument to a later one. That absence of a real data dependency is the signal, these calls have no actual reason to be sequential, and wrapping them in Promise.all is very likely a free, meaningful performance improvement with zero behavior change.

grep -B2 -A2 "await get" app/**/page.tsx

Scanning for clusters of consecutive await calls to different functions, then checking whether any of them genuinely reference the previous line's result, is a quick, practical way to find real waterfall candidates across an existing codebase.

Why This Matters More on Pages With Several Independent Data Sources

A dashboard, a page with several distinct widgets or sections each backed by their own query, is exactly where this pattern accumulates the most cost, since it's common to have four, five, or more genuinely independent pieces of data feeding one page, each one innocently written as its own await line, each one adding its full latency on top of all the others instead of overlapping.

The Actual Rule

Before awaiting multiple async calls sequentially, check whether any of them genuinely need a previous call's result. If none do, Promise.all turns their combined wait time from a sum into a maximum, for free, with no behavior change. For a genuine mix of dependent and independent calls, start the independent ones early without immediately awaiting them, so they run in the background while the real, necessary sequence proceeds.

I check every Server Component I write for exactly this pattern now, across client work and the dashboards and templates I build at pixelanas.com, since it's one of the highest-value, lowest-effort performance fixes available, and I've written about related data-fetching patterns on the blog too.

Go check your own dashboard or any data-heavy page for consecutive await calls with no genuine dependency between them. If you find some, wrapping them in Promise.all is very likely a free win sitting right there. Drop what you find in the comments.

Get the templates: https://pixelanas.gumroad.com

Anas, full-stack Next.js developer building SaaS products and premium templates. X: @ASheikh69751

Stop Overworking Your Server: A Developer’s Guide to HTTP Caching

Tue, 29 Sep 2026 10:10:51 +0000

HTTP caching headers are instructions sent between a web server and a browser that dictate whether a file (like an image, script, or stylesheet) should be saved locally by the user or requested again from the server. By managing these instructions, developers can prevent unnecessary data transfers and ensure the browser only fetches content when absolutely required.

The Library Analogy

Imagine you are a researcher visiting a massive national library. The first time you need a book, you wait in line for the librarian to retrieve it from the archives. This takes time. Now, imagine the librarian gives you a sticky note on the cover saying, "Keep this on your personal desk for three days." Instead of walking back to the archives every time you need that information, you simply look at your desk. The library is less crowded, you work faster, and the librarian has more time to help others. HTTP caching headers are those sticky notes; they tell the browser how long to "keep the book on the desk" before it must go back to the library server to check for a newer version.

Why It Matters

In the tech industry, "performance" is often synonymous with "user retention." Every time a browser asks for a file, it consumes server bandwidth and adds latency—the delay between a user's request and the site's response. Engineers use caching headers to minimize "round-trips" to the server. Without them, a user’s browser would re-download your entire JavaScript bundle and all your CSS files on every single page load, even if the files haven't changed. This causes slow page loads and unnecessarily high cloud infrastructure costs.

Implementation in Express.js

As a Node.js developer, you can control this behavior easily within your Express.js routes. Here is how you might set a "Cache-Control" header to tell the browser to keep a static file for one hour:

const express = require('express');
const app = express();

app.get('/api/data', (req, res) => {
  // max-age is defined in seconds (3600 seconds = 1 hour)
  res.set('Cache-Control', 'public, max-age=3600');
  res.json({ message: 'This data will be cached by the browser for an hour' });
});

app.listen(3000);

The Takeaway

Caching is not just about speed; it is about intelligence. By properly configuring headers, you shift the workload from your server to the user's browser, resulting in a more resilient application that saves resources while providing a near-instant experience for returning visitors.

Originally published on my blog. You can read the alternative breakdown here.

API Performance Profiler: Live route latency directly inside VS Code ⚡

Sun, 27 Sep 2026 10:25:03 +0000

Hey everyone! 👋 I'm Nahid, the creator of API Performance Profiler.
As backend developers, we often fly blind when building APIs locally. You write a route, but you don't really know how fast it runs until you leave your editor, set up Postman, or configure a heavy APM tool.
I wanted performance testing to feel native and frictionless, so I built an open-source, zero-dependency local profiling extension for Node.js (Express & NestJS).
Here is what it brings directly to your editor:
🟢 Inline Latency Metrics: See real-time execution speeds (e.g., 🟢 84ms) rendered right beside your route definitions in the code.
🚀 1-Click Load Testing: A '▶ Load test' CodeLens sits directly above your route. Click it to replay the last recorded request 1000 times under heavy load.
📊 Routes Dashboard: A dedicated VS Code sidebar showing all observed routes in your app, sorted from slowest to fastest.
🔐 100% Local & Secure: Everything is stored in local memory. No cloud accounts, no external dashboards.
It takes less than a minute to set up. I’d love for you to try it out!
🔗 VS Code Marketplace: https://marketplace.visualstudio.com/items?itemName=nahid625.api-profiler-vscode
🐙 GitHub Repo: https://github.com/Nahid625/api-performance-profiler

JSON vs MessagePack in Laravel: A Practical API Benchmark

Thu, 24 Sep 2026 13:39:50 +0000

JSON is the right default for most Laravel APIs. It is readable, supported everywhere, easy to debug, and familiar to every client.

MessagePack is worth considering when payload size, bandwidth, or a controlled client ecosystem matters. It is a compact binary format that keeps a schema-free data model, but it is not automatically faster or smaller for every workload.

This article compares both formats in a Laravel API and shows how to adopt MessagePack without breaking existing JSON clients.

The examples use Laravel Msgpack, an open-source package that adds opt-in MessagePack content negotiation, request decoding, safety limits, and a repeatable benchmark command.

The short answer

Use JSON when:

Your clients include browsers, third-party integrations, or unknown consumers.
Human-readable responses and simple debugging are important.
Your payloads are already small or compressed effectively.
You do not control the client decoder and deployment environment.

Use MessagePack when:

You control the clients and can ship a binary decoder.
Network transfer size is important for mobile, edge, or internal services.
Your payloads are large enough for serialization and decoding trade-offs to matter.
You want to keep JSON compatibility while allowing capable clients to opt in.

For many Laravel applications, the best answer is not choosing one format globally. Keep JSON as the default and negotiate MessagePack per request.

What are we comparing?

JSON is a text representation. It is self-describing, easy to inspect, and supported by the web platform.

MessagePack is a binary representation. It encodes the same basic data model using compact type markers and lengths instead of textual punctuation and names repeated in a text format.

The comparison should use the same data, the same PHP runtime, and the same operation count. Otherwise, the result is a benchmark of different workloads rather than a comparison of formats.

Add MessagePack to Laravel

Install the package:

composer require smmehdisharifi/laravel-msgpack

Apply the middleware to a route or route group:

use Illuminate\Support\Facades\Route;

Route::middleware('msgpack')->get('/api/profile', function () {
    return [
        'id' => 1042,
        'name' => 'Laravel MessagePack',
        'roles' => ['maintainer', 'developer'],
        'features' => [
            'negotiation' => true,
            'fallback' => 'json',
        ],
    ];
});

The route does not need separate JSON and MessagePack controllers. Existing clients continue to receive JSON:

curl -i http://localhost/api/profile

A capable client opts into MessagePack with the Accept header. Save the binary response instead of printing it as terminal text:

curl -sS -D - \
  -H 'Accept: application/msgpack' \
  -o profile.msgpack \
  http://localhost/api/profile

The negotiated response includes:

Content-Type: application/msgpack
Vary: Accept

Vary: Accept is important because a cache must not serve a MessagePack response to a client that asked for JSON, or the other way around.

How content negotiation works

The middleware keeps the migration opt-in:

A client that does not request MessagePack receives JSON.
A client that sends Accept: application/msgpack receives a binary response when the route supports the middleware.
If both formats are accepted, the higher quality factor wins.
Wildcards are evaluated by media-type specificity.
A request that rejects every supported representation receives 406 Not Acceptable.

The same negotiation applies to relevant error responses, including unmatched routes and route exceptions. Incoming requests can use application/msgpack or the legacy application/x-msgpack media type.

Run a repeatable benchmark

Laravel Msgpack includes an Artisan command so the comparison is easy to reproduce:

php artisan msgpack:benchmark --iterations=1000 --json

The command uses a deterministic API-shaped fixture with nested values and Unicode text. It measures:

Raw JSON and MessagePack byte size.
Encode time per iteration.
Decode time per iteration.
Optional gzip size and compression timings when ext-zlib is available.

The command warms each operation once before timing it. It reports per-iteration timings so runs with different iteration counts remain comparable.

Results from this package

The following result was produced by the package benchmark on PHP 8.4 with rybakit/msgpack 0.7.2 and 1,000 iterations:

Measurement	JSON	MessagePack	Difference
Raw payload	591 B	435 B	26.40% smaller
Encode time	0.002 ms	0.009 ms	JSON faster
Decode time	0.003 ms	0.008 ms	JSON faster
Gzip payload	377 B	371 B	1.59% smaller
Gzip compress time	0.009 ms	0.010 ms	Nearly equal
Gzip decompress time	0.004 ms	0.003 ms	Nearly equal

There are three useful conclusions here.

First, MessagePack is noticeably smaller before compression for this fixture. That can matter when transfer size is the bottleneck.

Second, the smallest representation is not automatically the fastest representation. For this small payload, JSON serialization is faster in the measured run.

Third, compression changes the size comparison. Once gzip is applied, the gap is much smaller because gzip already removes a lot of JSON's repetitive syntax.

These are indicative measurements, not universal promises. A real decision should use payloads, client runtimes, compression settings, and network conditions that match your application.

Decode MessagePack requests

MessagePack request bodies are decoded automatically when the middleware is applied:

use Illuminate\Http\Request;

Route::middleware('msgpack')->post('/api/profile', function (Request $request) {
    return [
        'received' => $request->msgpack(),
        'name' => $request->input('name'),
    ];
});

Map payloads are also merged into Laravel's normal request input. The original decoded value remains available through request()->msgpack(), including scalar and list payloads.

The package rejects malformed payloads before they reach the route handler. Configurable request size, nesting-depth, and value-count guards provide additional protection against oversized or unexpectedly deep input.

When a route must always return MessagePack

Negotiation is useful when the same endpoint serves different clients. For a route that should always return MessagePack, use the response macro:

return response()->msgpack([
    'message' => 'Created',
], 201, [
    'X-Request-Id' => $requestId,
]);

Direct serialization is also available through the facade:

use Msgpack;

$packed = Msgpack::encode($data);
$unpacked = Msgpack::decode($packed);

A practical decision guide

Do not change every endpoint to MessagePack because a synthetic benchmark produced a smaller payload. Instead, measure the part of the system that matters.

Choose JSON first when interoperability, browser support, observability, or simple debugging is the priority.

Test MessagePack when bandwidth, payload volume, or mobile transfer cost is important and the clients are controlled.

Keep both formats when you serve a mixed ecosystem. JSON remains the safe default, while clients that understand MessagePack can opt into it without a second controller implementation.

Limitations to keep in mind

MessagePack is not encryption. Sensitive data still needs transport security and application-level authorization.

Binary responses are less convenient to inspect in logs and command-line terminals. Save them to a file or decode them with a client library.

Compression may reduce the practical size advantage. Measure compressed and uncompressed responses for the transport configuration you actually deploy.

The benchmark fixture is intentionally stable, not universal. A list-heavy payload, a text-heavy payload, a large nested object, or a response with repeated keys can produce different results.

Try it in your project

Install Laravel Msgpack, apply the msgpack middleware to one endpoint, and compare both representations with your real data.

The repository contains the full implementation, compatibility matrix, configuration reference, safety limits, and benchmark command:

https://github.com/smmehdisharifi/laravel-msgpack

If the package helps your API, a GitHub star helps other Laravel developers discover it.

How to Use st-core.fscss with Svelte

Mon, 21 Sep 2026 12:45:40 +0000

st-core.fscss draws charts with pure CSS, clip-path polygons plus custom properties, no JS chart library, no SVG, no canvas. Svelte doesn't need to draw anything for it. It just needs to own the data and reactively set --st-p1, --st-p2, and so on whenever your series changes. Wanted to write up the actual integration because getting the responsibilities split correctly took me a minute the first time through.

Why compiled mode, not the runtime CDN

Svelte already has a build pipeline. Shipping the FSCSS runtime to a Svelte app just to expand a few chart mixins into static CSS is unnecessary work for the browser to do on every load. The runtime CDN script is fine for a CodePen or a quick static HTML demo, but for an actual Svelte or SvelteKit app you compile once at build time and import plain CSS.

Three layers, each with one job:

Layer	Job
FSCSS CLI	Expand `@st-chart-fill` / `@st-chart-line` into static CSS
Svelte	Reactive `data[]` → `--st-pN`
Browser CSS	Paint, plus optional `clip-path` transitions

Svelte state  →  style="--st-p1: 30%; --st-p2: 45%; …"
                      ↓
              compiled st-core.css
                      ↓
                 chart on screen

1. Write the FSCSS entry file

src/styles/st-core.fscss:

@import((*) from st-core@v2)

@st-root()

/* Length at compile time = number of polygon stops */
@arr chartData[0, 0, 0, 0, 0, 0, 0]

@st-chart-fill(.chart-fill, chartData)
@st-chart-line(.chart-line, chartData)

.chart {
  @st-chart-points(chartData)
  position: relative;
  width: 100%;
  height: 220px;
  background: var(--st-surface);
  border-radius: var(--st-radius-lg);
  overflow: hidden;
}

.chart-fill,
.chart-line {
  transition: clip-path 0.5s cubic-bezier(0.4, 0, 0.2, 1);
}

The zeros are placeholders, they only matter for their count. Svelte overwrites the real Y values through CSS variables at runtime. What you can't change without recompiling is the array's length — match it to your series (7 days of data → 7 entries).

2. Compile it as part of your build

npm install -D fscss@latest
npx fscss src/styles/st-core.fscss src/styles/st-core.css

Wire it into package.json so it runs automatically before every build:

{
  "scripts": {
    "build:st-core": "fscss src/styles/st-core.fscss src/styles/st-core.css",
    "prebuild": "npm run build:st-core"
  }
}

Then import the compiled CSS once, wherever makes sense for your app (root layout, app.css, or the chart's parent component):

import '../styles/st-core.css';

3. Map your data to `--st-pN`

Chart geometry in st-core is top-down, the same way CSS itself is. A high value needs to sit near 0%, not 100%. So every point gets inverted:

--st-pᵢ = (100 - humanValue) %

A score of 90 ends up 10% from the top, which is exactly where you'd expect it visually.

Here's a reusable Chart.svelte that does that mapping reactively, Svelte 4 style:

<script>
  /** @type {number[]} 0–100, higher = taller */
  export let data = [];
  export let height = '200px';
  export let color = '#9d7eff';

  $: cssVars = [
    ...data.map((value, index) => `--st-p${index + 1}: ${100 - value}%`),
    `--st-accent: ${color}`,
    `height: ${height}`
  ].join('; ');
</script>

<div class="chart" style={cssVars}>
  <div class="chart-fill"></div>
  <div class="chart-line"></div>
</div>

And the Svelte 5 version, using $props / $derived instead of the old export-let / reactive-statement pattern:

<script>
  let { data = [], height = '200px', color = '#9d7eff' } = $props();

  let cssVars = $derived(
    [
      ...data.map((value, index) => `--st-p${index + 1}: ${100 - value}%`),
      `--st-accent: ${color}`,
      `height: ${height}`
    ].join('; ')
  );
</script>

<div class="chart" style={cssVars}>
  <div class="chart-fill"></div>
  <div class="chart-line"></div>
</div>

Both do the exact same job, just with whichever reactivity API your Svelte version uses.

4. Use it like any other component

<script>
  import './styles/st-core.css';
  import Chart from './lib/Chart.svelte';

  let weekly = [45, 70, 30, 90, 60, 80, 50];

  function randomize() {
    weekly = Array.from({ length: 7 }, () => Math.floor(Math.random() * 80) + 10);
  }
</script>

<Chart data={weekly} color="#4fffb0" height="250px" >

<!-- Svelte 4 -->
<button type="button" on:click={randomize}>Randomize</button>

<!-- Svelte 5 -->
<!-- <button type="button" onclick={randomize}>Randomize</button> -->

Reassigning weekly recomputes cssVars, the browser applies the new custom properties, and the compiled CSS's clip-path transition animates the shape change. No chart library re-rendering, no SVG path recalculation, no manual DOM diffing on your end.

Fixed length vs changing length

Because geometry is baked in at compile time, the array length is the one thing you can't change on the fly:

Situation	What to do
Always exactly N points	Compile `@arr chartData` with N zeros, always pass exactly N values from Svelte
Length varies, up to a max of M	Compile M zeros, then `data.slice(0, M)` and pad shorter series with `0`

If you compile a template expecting 5 points and hand it 8 values from Svelte, the extra --st-p6, --st-p7, --st-p8 have nowhere to go, there's no polygon stop for them until you recompile with a longer array.

What not to do in production

<!-- Fine for a demo, avoid as your only path in a real Svelte app -->
<script src="https://cdn.jsdelivr.net/npm/fscss@1.2.3/runtime.min.js"></script>

Runtime mode is for CodePen and quick static HTML, not for a build pipeline that already exists. See the plain HTML integration sample if that's what you actually want.

The mental model, condensed

@st-chart-points, fill, and line define the structure and only ever read --st-p*.
Svelte's only job is to replace those variables when the data changes.
transition: clip-path gives you motion for free, no animation library needed.

Nothing here is Svelte-specific past step 3, the same pattern maps directly to Vue, React, or vanilla JS, only the reactivity layer changes. Contributions for other frameworks welcome under integration/.

Repo: github.com/fscss-ttr/st-core.fscss
Svelte sample: integration/svelte

The Silent Performance Killer: Demystifying the N+1 Query Problem in Node.js

Mon, 21 Sep 2026 09:18:48 +0000

Have you ever noticed a feature that works beautifully on your local computer, only to crawl at a snail's pace once it gets deployed to production? One of the most common, yet invisible, reasons for this sudden slowdown is a database fetching behavior known as the N+1 query problem.

In short, the N+1 query problem occurs when an application makes one initial database query to fetch a list of records (the "1"), and then makes a separate database query for every single record returned in that list (the "N") to fetch related data. Instead of fetching all the required information in a single, efficient request, the application ends up chatting back and forth with the database dozens, hundreds, or thousands of times.

The Restaurant Server Analogy

To understand why this is so inefficient, imagine sitting down at a busy restaurant with a table of ten friends. It is time to order drinks.

An efficient server (who behaves like an optimized database query) would take everyone's drink orders on a notepad, walk to the kitchen once, load all ten glasses onto a single tray, and bring them back to the table in one trip. This requires just one round-trip to the kitchen.

Now imagine an N+1 server. This server walks to the table, asks the first person what they want, and walks all the way back to the kitchen to fetch that single drink. After serving it, they walk back to the table, ask the second person for their order, and make another complete round-trip to the kitchen. They repeat this individual process for all ten people. By the time everyone has a drink, the server has made eleven total trips (1 trip to inspect the table, plus N trips for the N guests). Your dining experience would be incredibly slow, and the poor server would be completely exhausted.

Why This Matters in Daily Engineering

In real-world software engineering, database round-trips are remarkably expensive. Every single query you run introduces network latency, database parsing overhead, and thread consumption in your database's connection pool.

If you are building a social media dashboard that displays 50 posts, and you load the comments for each post individually, you are executing 51 database queries. If 1,000 users visit that page simultaneously, your application will suddenly bombard your database with 51,000 queries. This easily exhausts your database connection pool, drives database CPU usage to 100%, and results in catastrophic timeouts for your users. Engineers must actively prevent this to keep application response times low and infrastructure costs manageable.

Seeing It in Action: Node.js and MySQL

Let's look at a typical Express.js endpoint where this mistake happens, followed by how we fix it.

Here is the problematic, naive approach:

// BAD: This triggers N+1 queries
app.get('/api/posts', async (req, res) => {
  try {
    // 1. Fetch all posts (The "1" query)
    const [posts] = await db.query('SELECT id, title, content FROM posts LIMIT 50');
    
    // 2. Loop through each post to fetch its comments (The "N" queries)
    for (let post of posts) {
      const [comments] = await db.query(
        'SELECT id, text FROM comments WHERE post_id = ?',
        [post.id]
      );
      post.comments = comments;
    }
    
    res.json(posts);
  } catch (error) {
    res.status(500).json({ error: error.message });
  }
});

Here is how we resolve this using a SQL JOIN to pull all the information in a single, efficient query:

// GOOD: Resolved using a single JOIN query
app.get('/api/posts', async (req, res) => {
  try {
    const query = `
      SELECT 
        p.id AS post_id, 
        p.title, 
        p.content, 
        c.id AS comment_id, 
        c.text AS comment_text
      FROM posts p
      LEFT JOIN comments c ON p.id = c.post_id
      LIMIT 50
    `;
    
    const [rows] = await db.query(query);
    
    // Group the flat SQL results into structured JSON
    const postsMap = {};
    for (const row of rows) {
      if (!postsMap[row.post_id]) {
        postsMap[row.post_id] = {
          id: row.post_id,
          title: row.title,
          content: row.content,
          comments: []
        };
      }
      if (row.comment_id) {
        postsMap[row.post_id].comments.push({
          id: row.comment_id,
          text: row.comment_text
        });
      }
    }
    
    res.json(Object.values(postsMap));
  } catch (error) {
    res.status(500).json({ error: error.message });
  }
});

The Takeaway

The N+1 query problem highlights that how you fetch your data is just as important as the data itself. By switching from sequential, iterative loops to combined database operations like SQL JOINs, you minimize the highly expensive overhead of network round-trips. Designing your data fetching strategies proactively keeps your APIs incredibly fast and ensures your databases stay healthy, stable, and cost-efficient under high production loads.

Originally published on my blog. You can read the alternative breakdown here.

⚛️ Controlled vs Uncontrolled Components in React – A Deep Dive

Sat, 19 Sep 2026 19:08:29 +0000

When building forms in React, you’ll inevitably come across controlled and uncontrolled components.
At first glance, they look similar — both accept user input and manage form data.
But how they manage that data under the hood can significantly impact performance, maintainability and UX.

In this blog, let’s break down the difference, see them in action, explore real-world issues (like cursor position and undo/redo bugs), and understand when to use each.

🧠 What Are Controlled Components?

In controlled components, React state is the single source of truth.
That means every keystroke or change updates the component’s state and that state determines what’s rendered.

Example:

import { useState } from "react";

function ControlledInput() {
  const [name, setName] = useState("");

  return (
    <input
      value={name}
      onChange={(e) => setName(e.target.value)}
      placeholder="Type your name"
    />
  );
}

Here, the <input> value is fully controlled by React via the name state.
Each keystroke triggers setName(), re-rendering the component with the updated value.

✅ Pros

Easy to validate or transform input on change.
Centralized state makes debugging and unit testing easier.
Integrates seamlessly with complex UIs or form libraries (Formik, React Hook Form).

⚠️ Cons

Frequent re-renders can impact performance in large forms.
Need to carefully manage performance and input lag.
Cursor position bugs can occur with advanced state management.

🧩 What Are Uncontrolled Components?

In uncontrolled components, the DOM maintains its own state — React just references it.
You don’t handle every keystroke; instead, you access the value when needed.

Example:

import { useRef } from "react";

function UncontrolledInput() {
  const inputRef = useRef();

  const handleSubmit = () => {
    alert(`Input value: ${inputRef.current.value}`);
  };

  return (
    <>
      <input ref={inputRef} placeholder="Type your name" />
      <button onClick={handleSubmit}>Submit</button>
    </>
  );
}

✅ Pros

Better performance for simple forms.
Minimal React re-renders.
Perfect when you just need the final value.

⚠️ Cons

Harder to validate or modify values dynamically.
Not ideal when input data must always reflect application state.

⚔️ Controlled vs Uncontrolled: Key Differences

Feature	Controlled	Uncontrolled
Data Source	React State	DOM (via refs)
Performance	May re-render often	Faster for simple inputs
Validation	Easy to handle dynamically	Requires manual check
Use Case	Complex, validated forms	Simple or performance-critical inputs

⚡ Valtio & Controlled Inputs: Why Cursor Jumps Happen

When you use a state library like Valtio in a React controlled input, you might encounter this issue: you edit text in the middle of the input, but the cursor jumps to the end. This frustrates users and breaks the typing experience.

🔍 What’s going on?

By default, Valtio batches updates before triggering re-renders. The documentation states:

“By default, state mutations are batched before triggering re-render. Sometimes, we want to disable the batching.”
Because of this batching behavior, when you type and update the proxy state, multiple changes may be queued and then re-rendered in one go. During that delay, the browser loses context of the input cursor position, so after render the input’s value is applied and the cursor jumps to the end.

✅ The fix: { sync: true }

Valtio provides the option to disable batching for specific use-cases like inputs:

const snap = useSnapshot(state, { sync: true });

With sync: true, updates go through immediately rather than being deferred/batched. This keeps the input cursor where the user expects it.

⚠️ Trade-off:

Using sync: true disables batching optimizations, which can reduce render efficiency in large/complex components. Therefore:

Use sync: true only in input fields where cursor behaviour matters.
Avoid it for general state where performance is more important than instantaneous update.

🔁 Undo/Redo (Cmd+Z / Cmd+Shift+Z) in Controlled Components

Another subtle but important issue:
Undo (Cmd+Z) and Redo (Cmd+Shift+Z) might not work as expected in controlled inputs.

Why?
Because React replaces the entire value on every keystroke rather than letting the browser handle native input history.

This means the browser sometimes loses the input undo stack, especially if re-renders happen between updates.

🧪 What to do:

Test undo/redo behavior in all input fields before handing over to QA.
If using a state library or custom hooks, check whether it maintains the input history properly.
In cases where it fails and browser-native undo is critical (e.g., long text editors), consider using uncontrolled inputs or controlled wrappers that debounce updates.

🧰 Best Practices for React Forms

✅ Use controlled inputs when you need validation, dynamic control or shared state.
✅ Use uncontrolled inputs when performance and simplicity matter.
✅ For state libraries like Valtio, test cursor and undo behaviors thoroughly.
✅ Avoid unnecessary re-renders – use React.memo, useCallback or split components.
✅ Always test form UX — typing, cursor and undo/redo should feel native and fluid.

🎯 Final Thoughts

Controlled vs Uncontrolled Components seem like a small React concept, but they can deeply impact user experience, especially in complex form-heavy applications.

💡 Understanding these nuances (like cursor handling, undo/redo, and batching) helps you ship smoother, bug-free UIs that feel professional and reliable.

🚀 Whether you use plain React or libraries like Valtio, always test the real typing experience — that’s where users feel the difference.

💬 Have you ever faced cursor or undo issues in your React forms? Drop your experience or workaround below — let’s learn together! 👇

generateMetadata and Your Page Component Are Probably Querying the Same Data Twice

Sat, 19 Sep 2026 10:48:58 +0000

I wrote earlier about cache() fixing redundant database queries when multiple components in a tree independently call the same query function. There's one specific, extremely common place this exact pattern shows up that's worth calling out directly, because it's not an edge case, it's the default shape of nearly every dynamic page with SEO metadata.

The Setup Almost Every Dynamic Page Has

// app/blog/[slug]/page.tsx
import { getPost } from '@/lib/queries/posts';

export async function generateMetadata({ params }) {
  const { slug } = await params;
  const post = await getPost(slug); // query #1

  return {
    title: post.title,
    description: post.excerpt,
  };
}

export default async function BlogPost({ params }) {
  const { slug } = await params;
  const post = await getPost(slug); // query #2, same post, same request

  return (
    <article>
      <h1>{post.title}</h1>
      <div>{post.content}</div>
    </article>
  );
}

This is the completely standard pattern from the Next.js metadata setup covered in an earlier post, and it's genuinely necessary, generateMetadata needs the post's title and excerpt for the meta tags, the page component needs the post's full content to actually render it. Both run for every single request to this page. Both call getPost(slug) independently. Without a specific fix, that's two full database round trips for what is, semantically, the exact same piece of data, fetched twice, for every single page view.

Why This Specific Case Matters More Than It Might Seem

This isn't a rare pattern someone might occasionally write, it's the default shape of generateMetadata combined with any dynamic page, which means it's quietly doubling database load on nearly every content-driven page across an entire site, blog posts, product pages, any dynamically rendered detail page with SEO metadata. A site with meaningful traffic on these pages is paying for twice the database queries it actually needs, on some of its most frequently visited pages, for no functional reason.

The Fix: Wrap the Query in React's cache()

// lib/queries/posts.ts
import { cache } from 'react';
import { connectDB } from '@/lib/db';
import Post from '@/models/Post';

export const getPost = cache(async (slug: string) => {
  await connectDB();
  return Post.findOne({ slug }).lean();
});

That's the entire fix, no changes needed anywhere else. generateMetadata and the page component both still call getPost(slug) exactly as before, the code looks identical. What changes is that React's request memoization now recognizes both calls, same function, same argument, within the same render pass, and only actually executes the underlying query once, returning the cached result for the second call.

Why This Is Worth Checking Specifically, Not Just Generally

The earlier post on this topic covered the general pattern, multiple components in a tree calling the same query. This specific case is worth checking on its own because it's so easy to miss precisely because generateMetadata and the page component don't look like they're part of the same "component tree" in the way nested components obviously are. They're two separate exported functions in the same file, and it's genuinely easy to write both, each looking completely correct in isolation, without ever noticing they're duplicating a database call for the same request.

A Quick Way to Verify This Is Actually Happening

Add a temporary log inside the un-cached query function and load the page once:

export async function getPost(slug: string) {
  console.log('getPost called for:', slug); // temporary
  await connectDB();
  return Post.findOne({ slug }).lean();
}

If you see that log line twice for a single page load, that's this exact issue, confirmed directly rather than assumed. After wrapping the function in cache(), the same test should show it logging only once per request.

Where This Doesn't Apply

If generateMetadata and the page component genuinely need different data, metadata pulling from a lighter, separate summary endpoint while the page fetches full content from somewhere else entirely, there's no duplication to fix, since they're not actually calling the same underlying function with the same arguments in the first place. This fix specifically applies when both are calling the identical query function, which, for most blog and content-driven page setups, is exactly the common case.

The Actual Rule

Any query function called both inside generateMetadata and inside the page component it belongs to should be wrapped in cache(), as close to a universal default as a rule gets in this specific area. It costs nothing to add, changes no calling code, and directly eliminates a real, silent, doubled database query that exists by default on essentially every dynamically rendered page with SEO metadata.

Go check any page in your own project using generateMetadata alongside a matching page component, specifically whether the underlying query function is wrapped in cache(). If it's not, you're very likely double-querying on every single page load for that route. Drop what you find in the comments.

Get the templates: https://pixelanas.gumroad.com

Anas, full-stack Next.js developer building SaaS products and premium templates. X: @ASheikh69751

Code Fingerprinting: Detecting Duplicate Submissions Without Losing Your Mind (or Your API Budget)

Sat, 19 Sep 2026 07:49:18 +0000

While building LogicVisor's review system, I was faced with a question I hadn't thought about before: what would happen if users submitted duplicate algorithm solutions?

I decided to test it myself. I submitted a solution to a simple palindrome challenge. The request went through, my solution was reviewed correctly. Normal happy stuff. Then I submitted the exact same solution a second time. And the normal happy stuff happened again. But I was far from happy.

That duplicate submission meant more tokens spent, more database memory consumed, more remote procedure calls, more AI calls, and more waiting time. For a problem that had already been solved.

I shuddered at the thought of a bad actor effectively DOS-ing my site or running up my AI bill just by hammering the same solution repeatedly. Rate limiting controls the number of requests per client — but it has nothing to do with the content of those requests. A user could stay well within their rate limit and still abuse the system by submitting the same code over and over.

This wasn't going to work. I needed a way to identify duplicate submissions and return a cached response instead of processing everything from scratch. Which led me to a more interesting question: how do you actually detect identical code?

The Problem With "Identical"

Programming by its very nature allows us to solve the same problem in multiple ways. But even a single solution can be written in an infinite number of ways.

User A could create a function that takes and returns a string, and name it strawberries. User B could write the exact same function with the exact same logic and name it bananas. Different names, same implementation, same result.

For LogicVisor's purposes, these should be counted as duplicates. But how do you get a computer to see past the surface differences and recognize that two pieces of code are structurally the same?

I started thinking about it from first principles. At the end of the day, programming languages get compiled down to binary and then to electrical pulses. If two solutions solve the same problem the same way, they're ultimately firing the same set of pulses at the machine level. Variable names be damned.

So the question became: can I strip a solution down to its most fundamental representation and compare that?

Enter the Abstract Syntax Tree

More research led me to the Abstract Syntax Tree — something my OS professor had mentioned in school that I'd filed away as "probably never useful in web dev." No knowledge is wasted.

An Abstract Syntax Tree (AST) is a hierarchical, tree-shaped data structure that represents the syntactic structure of source code. It strips away superficial syntax — semicolons, curly braces, whitespace, comments — and keeps only the structural and semantic skeleton of the code. The shape of what you wrote, not the clothing it's wearing.

This was it. But I hit a wall almost immediately: I can't store and index an entire AST reliably in PostgreSQL. It's a complex nested structure, not a scalar value you can slap an index on.

Then I remembered an article about handling duplicate file uploads in Express with Multer. The trick was generating a SHA-256 hash of the file's content. A hash is a fixed-length fingerprint — the same input always produces the same output, and different inputs produce different outputs. Perfect.

The plan crystallized:

Parse the submitted code into an AST
Normalize the AST (rename variables, strip comments, etc.)
Hash the normalized output with SHA-256
Store and index that hash
On every new submission, compute the hash and check if it already exists

If the hash is in the database, return the cached review. No AI call. No wasted tokens. No waiting.

Two Parsers for the Price of One

LogicVisor supports eight languages: JavaScript, TypeScript, Python, Java, C, C++, Go, and Rust. Getting an AST for all of them required two different tools, each with their own approach to the world.

Babel Parser (for JavaScript and TypeScript)

Babel is something most JavaScript developers know as a transpiler — the thing that converts modern JS to something older browsers can run. But under the hood, Babel has a full-featured parser that produces a detailed AST of your JavaScript or TypeScript code. The @babel/parser package exposes this directly.

I'd used Babel's output for years but never touched the parser API directly. It's surprisingly clean. You hand it source code and options, and it gives you back a structured AST you can traverse and manipulate.

What made Babel the right choice for JS/TS specifically:

Native TypeScript support via a plugin — no separate toolchain
JSX support out of the box
The @babel/traverse package lets you walk the AST and modify nodes in place, which is exactly how I rename identifiers
@babel/generator converts the modified AST back to compact source code

The JS/TS canonicalization path: parse with Babel → traverse the AST and rename all non-builtin identifiers to generic names (v0, v1, v2...) → optionally sort top-level declarations → regenerate compact source → hash it.

Tree-sitter (for everything else)

Tree-sitter is a different beast. It's a parser generator — a tool for building fast, incremental, error-tolerant parsers for programming languages. You might have encountered it without knowing it: if you've used Neovim with syntax highlighting that actually understands your code structure (my first encounter with the tool), that's Tree-sitter. Same if you've used editors where renaming a variable highlights every reference correctly. It's doing real parsing, not just regex matching.

What makes Tree-sitter interesting for this use case is that it has community-maintained grammars for a huge range of languages — C, C++, Go, Rust, Java, Python, and many more. Each grammar is a separate package that teaches Tree-sitter how to parse that language. You install the grammar, set it on a parser instance, and Tree-sitter gives you a concrete syntax tree you can walk.

I hadn't really understood what Tree-sitter was before this. I'd seen it referenced in Neovim config files and assumed it was some Vim-specific thing. It's not — it's a general purpose parsing library with bindings for multiple languages including Node.js via the node-tree-sitter package.

The Tree-sitter path: parse the source → walk the resulting syntax tree with a DFS traversal → collect all identifier nodes by their byte ranges → build a name mapping (first-occurrence order) → apply replacements from the end of the file backwards (so byte indices don't shift) → normalize whitespace → hash it.

The backwards-replacement trick is worth calling out. When you're doing string replacements by index, if you replace something early in the file, every subsequent index is now wrong. By sorting replacements by start position descending and working from the end, you avoid that entire class of bug.

The Implementation

Here's how it all fits together.

The Simple Canonicalization (Pre-AST)

Before I landed on the AST approach, I had a simpler version that just handled whitespace and comments:

export function canonicalizeCode(code: string): string {
  // Remove multi-line comments
  let cleanedCode = code.replace(/\/\*[\s\S]*?\*\//g, "");

  // Remove single-line comments
  cleanedCode = cleanedCode.replace(/\/\/.*$/gm, "");

  // Normalize all whitespace to a single space
  cleanedCode = cleanedCode.replace(/\s+/g, " ").trim();

  return cleanedCode;
}

This catches the obvious cases — same code with different formatting or comments. But it falls apart the moment someone renames a variable. The AST approach handles that.

The Router

The main entry point checks the language and dispatches to the right parser:

export async function canonicalizeCodeAST(
  code: string,
  language: LanguageKey,
  options?: CanonicalizeOptions
): Promise<string> {
  const opts = { ...defaultOptions, ...(options || {}) };

  if (language === "javascript" || language === "typescript") {
    return canonicalizeJS(code, opts);
  }

  // C/C++/Go/Rust/Java/Python go through Tree-sitter
  return canonicalizeWithTreeSitter(code, language, null);
}

The Babel Path (JS/TS)

function canonicalizeJS(
  code: string,
  opts: Required<CanonicalizeOptions>
): string {
  const plugins: any[] = ["jsx"];
  if (opts) plugins.push("typescript");

  const ast = babelParser.parse(code, {
    sourceType: "module",
    plugins,
    allowReturnOutsideFunction: true,
  });

  // Rename all non-builtin identifiers to generic names
  const nameMap = new Map<string, string>();
  let counter = 0;

  traverse(ast as any, {
    Identifier(path) {
      if (!opts.normalizeVariableNames) return;

      const name = path.node.name;
      if (isBuiltinJsName(name)) return;

      if (!nameMap.has(name)) {
        nameMap.set(name, `v${counter++}`);
      }

      path.node.name = nameMap.get(name)!;
    },
  });

  // Optionally sort top-level declarations for order-independent comparison
  if (
    opts.sortTopLevel &&
    (ast as any).program &&
    Array.isArray((ast as any).program.body)
  ) {
    (ast as any).program.body.sort((a: any, b: any) => {
      const aStr = generate(a).code;
      const bStr = generate(b).code;
      return aStr.localeCompare(bStr);
    });
  }

  // Regenerate compact source from the modified AST
  return generate(ast as any, {
    comments: !opts.removeComments,
    compact: true,
  }).code;
}

The Tree-sitter Path (Everything Else)

function canonicalizeWithTreeSitter(
  code: string,
  language: LanguageKey,
  opts: Required<CanonicalizeOptions> | null
): string {
  const LangCtor = (treeSitterMap as any)[language];

  if (!LangCtor) {
    throw new Error(`Unsupported language for tree-sitter: ${language}`);
  }

  const parser = new Parser();
  parser.setLanguage(LangCtor);

  const tree = parser.parse(code);
  const root = tree.rootNode;

  const identifierNodeTypes = identifierTypeMap[language] || ["identifier"];
  const idNodes: { start: number; end: number; text: string }[] = [];
  const commentNodes: { start: number; end: number }[] = [];

  // DFS traversal to collect identifier and comment nodes
  const stack = [root];
  while (stack.length) {
    const node = stack.pop()!;
    const type = node.type;

    if (type.includes("comment") && opts?.removeComments) {
      commentNodes.push({ start: node.startIndex, end: node.endIndex });
    }

    if (identifierNodeTypes.includes(type) || /identifier|name/i.test(type)) {
      const text = code.slice(node.startIndex, node.endIndex);
      idNodes.push({ start: node.startIndex, end: node.endIndex, text });
    }

    for (let i = 0; i < node.childCount; i++) {
      stack.push(node.child(i) as SyntaxNode);
    }
  }

  // Build name mapping in first-occurrence order
  const nameMap = new Map<string, string>();
  let counter = 0;

  for (const n of idNodes) {
    const orig = n.text;
    if (isLanguageBuiltinName(orig, language)) continue;
    if (!nameMap.has(orig)) {
      nameMap.set(orig, `v${counter++}`);
    }
  }

  // Build replacement list
  const replacements: { start: number; end: number; replacement: string }[] = [];

  if (opts?.removeComments) {
    for (const c of commentNodes) {
      replacements.push({ start: c.start, end: c.end, replacement: "" });
    }
  }

  if (opts?.normalizeVariableNames) {
    for (const id of idNodes) {
      const mapped = nameMap.get(id.text);
      if (mapped) {
        replacements.push({ start: id.start, end: id.end, replacement: mapped });
      }
    }
  }

  // Apply replacements from end to start so byte indices stay valid
  replacements.sort((a, b) => b.start - a.start);

  let out = code;
  for (const r of replacements) {
    out = out.slice(0, r.start) + r.replacement + out.slice(r.end);
  }

  return out.trim();
}

Hashing the Canonical Form

Once we have the normalized source, hashing is straightforward:

export async function createCanonicalHash(canonicalCode: string): Promise<string> {
  const encoder = new TextEncoder();
  const data = encoder.encode(canonicalCode);
  const hashBuffer = await crypto.subtle.digest("SHA-256", data);
  const hashArray = Array.from(new Uint8Array(hashBuffer));
  return hashArray.map((b) => b.toString(16).padStart(2, "0")).join("");
}

SHA-256 gives us a fixed 64-character hex string regardless of how long the original code was. That's what gets stored and indexed in PostgreSQL.

Putting It All Together in the Request Handler

let canonicalCode;
try {
  canonicalCode = await canonicalizeCodeAST(solution, preferred_language);
} catch (error) {
  console.error("Canonicalization error:", error);
  return NextResponse.json(
    {
      success: false,
      error: {
        code: "BAD_USER_INPUT",
        message: "Unable to parse your code. Please check for syntax errors and try again.",
      },
    },
    { status: 400 }
  );
}

const canonicalHash = await createCanonicalHash(
  typeof canonicalCode === "string" ? canonicalCode : ""
);

// Check cache before touching the AI provider
const cachedReview = await getCachedAIReview(
  preferred_model.id + "-" + canonicalHash
);

if (cachedReview) {
  return NextResponse.json(
    { success: true, data: cachedReview },
    { status: 201 }
  );
}

// No cache hit — proceed with AI review and store the result

The model ID is included in the cache key because the same code reviewed by Gemini and Groq will produce different outputs. You don't want to serve a Gemini review to someone who selected Groq.

Does It Actually Work?

Here's the test that proved it out. Five TypeScript implementations of the same palindrome algorithm — different variable names, different formatting, different comments, and one with operand order swapped:

// Solution 1 — clean, descriptive names
function isPalindrome(x: number): boolean {
  let originalNum = x;
  let reversedNum = 0;
  while (originalNum > 0) {
    let lastDigit = originalNum % 10;
    reversedNum = (reversedNum * 10) + lastDigit;
    originalNum = Math.floor(originalNum / 10);
  }
  return x === reversedNum;
}

// Solution 2 — different variable names
function isPalindrome(n: number): boolean {
  let temp = n;
  let result = 0;
  while (temp > 0) {
    let digit = temp % 10;
    result = (result * 10) + digit;
    temp = Math.floor(temp / 10);
  }
  return n === result;
}

// Solution 3 — compressed formatting + comments
function isPalindrome(x: number): boolean{
  // Store original
  let originalNum=x;let reversedNum=0;
  while(originalNum>0){
    let lastDigit=originalNum%10; /* get digit */
    reversedNum=(reversedNum*10)+lastDigit;
    originalNum=Math.floor(originalNum/10);
  }
  return x===reversedNum;
}

// Solution 4 — addition operands swapped
function isPalindrome(x: number): boolean {
  let originalNum = x;
  let reversedNum = 0;
  while (originalNum > 0) {
    let lastDigit = originalNum % 10;
    reversedNum = lastDigit + (reversedNum * 10); // addition order changed
    originalNum = Math.floor(originalNum / 10);
  }
  return reversedNum === x; // comparison order changed
}

// Solution 5 — completely different algorithm
function isPalindrome(x: number): boolean {
  const str = x.toString();
  return str === str.split('').reverse().join('');
}

And the test harness:

export async function testDuplicateDetection() {
  const solutions = [ /* ...the five solutions above... */ ];

  console.log("Code hash comparison:");

  solutions.forEach(async (code, index) => {
    const canonicalized = await canonicalizeCodeAST(code, "typescript");
    const hash = await createCanonicalHash(canonicalized);
    console.log(`\nSolution ${index + 1}:`);
    console.log(`Hash: ${hash.substring(0, 12)}...`);
    console.log(
      `Canonical: ${
        typeof canonicalized === "string" ? canonicalized.substring(0, 100) : ""
      }...`,
    );
  });

  // Canonicalize before hashing — not after
  const hashes = await Promise.all(
    solutions.map(async (code) => {
      const canonicalized = await canonicalizeCodeAST(code, "typescript");
      return createCanonicalHash(canonicalized);
    }),
  );
  const uniqueHashes = new Set(hashes);

  console.log(`\nOriginal solutions: ${solutions.length}`);
  console.log(`Unique hashes: ${uniqueHashes.size}`);
  console.log(`Duplicates detected: ${solutions.length - uniqueHashes.size}`);
}

The output:

Code hash comparison:

Solution 1:
Hash: f7142f71e35a...
Canonical: function v0(v1:number):boolean{let v2=v1;let v3=0;while(v2>0){let v4=v2%10;v3=v3*10+v4;v2=Math.v5(v2...

Solution 2:
Hash: f7142f71e35a...
Canonical: function v0(v1:number):boolean{let v2=v1;let v3=0;while(v2>0){let v4=v2%10;v3=v3*10+v4;v2=Math.v5(v2...

Solution 3:
Hash: f7142f71e35a...
Canonical: function v0(v1:number):boolean{let v2=v1;let v3=0;while(v2>0){let v4=v2%10;v3=v3*10+v4;v2=Math.v5(v2...

Solution 4:
Hash: da8268fe24dd...
Canonical: function v0(v1:number):boolean{let v2=v1;let v3=0;while(v2>0){let v4=v2%10;v3=v4+v3*10;v2=Math.v5(v2...

Solution 5:
Hash: e3d2696010ad...
Canonical: function v0(v1:number):boolean{const v2=v1.v3();return v2===v2.v4('').v5().v6('');}...

Original solutions: 5
Unique hashes: 3
Duplicates detected: 2

Solutions 1, 2, and 3 collapse to the same hash — variable names, whitespace, and comments are all stripped away during canonicalization, leaving identical ASTs. Solution 4 gets a distinct hash because operand order (lastDigit + reversedNum 10 vs reversedNum 10 + lastDigit) is a syntactic difference the AST preserves — commutativity is math, not structure, and the canonicalizer doesn't evaluate expressions. Solution 5 is correctly identified as a genuinely different algorithm.
Two duplicates detected. Working as intended.

The Tradeoffs Worth Knowing

This approach isn't magic. A few things it doesn't catch:

Semantic equivalence across algorithms. Two completely different implementations that produce the same result — bubble sort vs. merge sort, for example — will have different hashes. This system detects structural duplicates, not algorithmic equivalence. That's actually the right behavior for a code review platform.

Heuristic identifier detection in Tree-sitter. The identifier node types vary between language grammars, and I'm using pattern matching (/identifier|name/i) as a fallback. It works well in practice but it's not exhaustive. If a grammar names its identifier nodes something unusual, you might miss some.

Builtin lists are incomplete. The isBuiltinJsName and isLanguageBuiltinName functions cover the most common builtins but aren't exhaustive. If a user names their variable fetch or setTimeout, those won't be normalized. Not a critical flaw, but worth knowing.

What I Took Away From This

The part I didn't expect: how much I'd enjoy it. This was one of those problems that pulled me away from the normal web dev loop of "call an API, render some data" and into something that felt closer to computer science. Parsers, ASTs, tree traversal — things I'd learned in school that felt abstract at the time suddenly had a real home in a real production system.

The other thing I took away: sometimes the right caching strategy isn't about time-to-live or cache invalidation. Sometimes it's about recognizing that two requests are asking for the same thing, even when they look different on the surface. That realization changed how I thought about a whole class of optimization problems.

If you're building anything that takes code as input — a linter, a reviewer, a plagiarism detector, an interview platform — the AST + hash pattern is worth having in your toolkit.

LogicVisor is an AI-powered tool that reviews your algorithmic code, breaks down time and space complexity, and gives you the kind of feedback you'd want before a technical interview. You can try it at logicvisor.vercel.app.

Finally Shipped st-core.fscss v2 — CSS Charts Without the 8-Point Ceiling

Sat, 19 Sep 2026 01:08:17 +0000

For a while, st-core.fscss charts had one annoying limit: every line, fill, and dot was locked to exactly 8 data points. --st-p1 through --st-p8, hardcoded. Want to plot 12 weeks of revenue? Tough. Want 5 points for a quick demo? You'd pad the array with zeros and hope nobody noticed.

v2 gets rid of that.

The core change: Array (`@arr`)

Instead of passing 8 raw numbers into @st-chart-points, you now declare a named array of any length:

@arr myData[50, 10, 97, 35, 66, 50, 80, 54, 70, 60]

@st-chart-fill(.chart-fill, myData)
@st-chart-line(.chart-line, myData)

st-core reads the array length at compile time, works out even X-spacing across the chart width, and builds the clip-path: polygon() shape from however many points you gave it. 5 points, 12 points, 50 points, doesn't matter. No more counting to 8.

The gotcha that took me a minute to internalize

Here's the part that actually tripped me up while building the multi-series examples: @st-chart-line(selector, array) and @st-chart-fill(selector, array) aren't what set the data on an element. They declare the renderer for that selector, the array argument just tells it how many points to expect.

The thing that actually writes --st-p1…--st-p{n} onto an element is @st-chart-points(array).

That distinction matters the moment you have more than one series on a chart. In a multi-line setup, if .line-b doesn't call @st-chart-points(seriesB) itself, it silently inherits whatever the parent container set, meaning it renders the wrong dataset. Looks fine at a glance, wrong shape in practice. Caught it while building a 3-series dashboard card where two lines were quietly drawing the same data.

.fill-b {
  @st-chart-points(seriesB)   /* required, not optional */
  opacity: 0.3;
  --st-accent: #4fffb0;
}

Once that clicked, the mental model got simple: renderer declarations are structural (once per selector), point data is per-element (as many times as you have distinct series).

How it feels to ship

Pure CSS charting was already a fun constraint to design inside of, no JS runtime, no SVG, no canvas. Making the dataset size flexible removes the last thing that made it feel like a toy rather than something you'd reach for on a real dashboard. Compiled output is still under a kilobyte. Still zero hydration.

Docs and full examples are up in the repo, including the legacy v1 reference for anyone still on fixed 8-point charts.

Repo: github.com/fscss-ttr/st-core.fscss

Open-source

with ♥️

The source code is totally math notebook 😅

Have been writing and debugging from morning till night, and finally released it. The usage is now as simple as CSS.

😊

Stop Crashing Your Database: A Developer's Guide to Connection Pooling

Wed, 16 Sep 2026 08:51:01 +0000

Database connections are the lifeblood of any web application, but they are also surprisingly expensive to create and destroy. When your application struggles under sudden traffic spikes, the database is often the first thing to break. This is where connection pooling comes in.

Connection pooling is a software development technique that creates and maintains a cache of active database connections, keeping them ready for immediate reuse. Instead of initiating and closing a brand-new connection every time a user requests data, your application borrows an already-established connection from this pre-allocated "pool." Once the query is complete, the connection is returned to the pool for the next request to use, rather than being discarded.

The Taxi Stand Analogy

To understand this clearly, think of a busy taxi stand outside a major hotel. If every guest who wanted to travel into town had to call a car dealership, buy a brand-new car, register it with the city, drive to their destination, and then crush the vehicle at a scrap yard when they arrived, transportation would be impossibly slow, expensive, and wasteful.

Instead, the hotel relies on a taxi stand. A small fleet of vehicles sits idling in the driveway. When a guest steps outside, they instantly hop into an available taxi, take their trip, and the driver returns to the stand to wait for the next passenger. The taxi is never destroyed; it is constantly reused. In this analogy, the guests are your incoming API requests, the taxis are your database connections, and the stand is your connection pool.

Why Connection Pooling Matters Daily

In real-world software engineering, establishing a database connection is not a simple task. It requires a multi-step process: initiating a TCP handshake over the network, negotiating security protocols (TLS/SSL), authenticating the database user's credentials, allocating memory on the database server, and setting up session variables. Doing this on every single HTTP request adds massive latency to your API responses.

Furthermore, databases have strict physical limits. Every active connection consumes a slice of the database server's RAM and CPU. If your Express server experiences a sudden burst of 500 concurrent users and tries to open 500 individual connections simultaneously, your MySQL database will quickly run out of memory. This leads to the dreaded "Too many connections" error, bringing your entire application to a grinding halt. Connection pooling caps the maximum number of connections, forcing extra requests to wait in a fast queue rather than crashing your database server.

Putting It into Code

Here is how you implement connection pooling in a Node.js application using the popular mysql2 driver. Instead of using createConnection, we use createPool to manage our resources efficiently.

const mysql = require('mysql2/promise');

// Configure and create the connection pool
const pool = mysql.createPool({
  host: 'localhost',
  user: 'root',
  password: 'your_secure_password',
  database: 'production_db',
  waitForConnections: true,    // Queue requests when pool is full
  connectionLimit: 10,         // Max number of simultaneous active connections
  queueLimit: 0                // No limit on waiting requests (0 means infinite)
});

// Helper function to fetch a user profile
async function getUserProfile(userId) {
  try {
    // The pool automatically checks out a connection, runs the query, and returns it
    const [rows] = await pool.execute(
      'SELECT id, username, email FROM users WHERE id = ?',
      [userId]
    );
    return rows[0];
  } catch (error) {
    console.error('Database query error:', error);
    throw error;
  }
}

// Example execution
getUserProfile(42).then(user => console.log('User retrieved:', user));

The Takeaway

Connection pooling is a quiet champion of system architecture, transforming database communication from a slow bottleneck into a reliable, high-performance system utility. By setting up a robust pool inside your application, you prevent database crashes, lower response times, and ensure that your backend can handle high-concurrency traffic without breaking a sweat.

Originally published on my blog. You can read the alternative breakdown here.

Core Web Vitals Benchmarks for Shopify Stores (2026 Data)

Wed, 16 Sep 2026 08:04:00 +0000

Shopify's own theme performance table still shows median pass rates in the 80–97% range for LCP, INP, CLS, and “all Core Web Vitals.” Open a multi-page Lighthouse sample on a real storefront and the story often flips: mobile Performance scores cluster near 50, lab LCP averages stretch well past Google's 2.5 s “good” line, and unused JavaScript shows up on almost every domain. Both views can be true. Theme-level CrUX aggregates describe shops that look like a given theme under field conditions. Portfolio monitoring needs URL-level and multi-page evidence, because apps, media, and collection templates decide what shoppers feel.

What follows separates those clocks, publishes a reproducible sample from Shopify leads we analyse in Apogee Watcher, and points at the tuning work that actually moves scores. For store-level fixes after you read the numbers, pair this with Shopify Speed Optimization: Core Web Vitals Guide for E-Commerce.

What counts as a reliable Shopify Core Web Vitals benchmark in 2026

For Core Web Vitals, the strongest public field source remains the Chrome UX Report (CrUX). Google's methodology is clear about inclusion: pages and origins must be publicly discoverable and popular enough, data comes from eligible Chrome users, and thresholds are read at the 75th percentile over a rolling window. That is why Search Console and the field section of PageSpeed Insights matter for ranking and real-user claims, while a single Lighthouse run remains a diagnostic tool.

Google's published thresholds are unchanged for the three metrics:

Metric	Good (p75)	Needs improvement	Poor
LCP	≤ 2.5 s	2.5 s – 4.0 s	> 4.0 s
INP	≤ 200 ms	200 ms – 500 ms	> 500 ms
CLS	≤ 0.1	0.1 – 0.25	> 0.25

An origin or URL group passes Core Web Vitals when all three sit in the good bucket at p75. A Lighthouse Performance score is a different artefact: it blends lab timings under a throttled environment and is useful for regression detection, not as a substitute for CrUX pass/fail. If you need a refresher on the three metrics themselves, start with What Are Core Web Vitals? A Practical Guide for 2026.

A useful Shopify benchmark therefore states four things up front: which clock (CrUX field versus Lighthouse lab), which URLs (homepage only versus collections, PDPs, cart), how scores are aggregated (single URL versus median or mean across pages), and the collection window. Platform marketing often quotes theme or “average store” figures without that scaffolding. Agencies managing twenty Shopify clients need the scaffolding more than another green screenshot.

Sources:

Shopify theme-level CrUX context from public data

Shopify's Performance team publishes aggregated Core Web Vitals performance by theme so merchants can compare popular themes under real Chrome traffic rather than a single lab screenshot. As of the 18 Aug 2026 update on their theme table, the median theme-level pass rates across listed themes were:

Metric	Median % of shops passing (theme table)
Largest Contentful Paint (LCP)	90.3%
Cumulative Layout Shift (CLS)	93.4%
Interaction to Next Paint (INP)	97.1%
Views passing all CWVs	82.0%

Those figures combine devices, cover recent theme versions with enough eligible shops, and still describe theme-level cohorts, not “every live Shopify store with every app enabled.” The same table shows wide floors and ceilings: all-CWV pass rates range from roughly 37% to 95% depending on the theme row, which is the first warning that platform averages do not replace store-specific monitoring.

Third-party research often cites a lower all-CWV pass rate for Shopify as a whole (for example ~45% in vendor CrUX summaries). The gap between ~45% store-wide claims and ~82% median theme-level “all CWVs” is not a mystery if samples, devices, and theme filters differ, so treat both as orientation and do not merge them into one scorecard column.

Source: Aggregated Core Web Vitals performance by Shopify Theme

Multi-page lab sample from Watcher's Shopify users

We keep Shopify-tagged storefronts in Watcher and run multi-page PageSpeed Insights analyses that store both Lighthouse lab metrics and CrUX field payloads when Google returns them. For this snapshot we took the latest domain report per Shopify-tagged lead with an aggregated payload (n = 64 stores; reports generated between April and September 2026). Each report averages Lighthouse metrics across the URLs selected for that domain (median 17 analysed pages per store; 1,070 page runs in total). Values below are store-level averages of lab runs, not single-URL CrUX p75s.

Mobile and desktop Lighthouse Performance

Slice	Mobile Performance (avg)	Desktop Performance (avg)
Minimum	26	35
25th percentile	37	57
Median	51	69
75th percentile	62	83
Maximum	86	99
Mean	51	69

Half of the sample sits at or below a mobile Performance average of about 51. Only 32 of 64 stores clear a mobile average of 50, and 9 of 64 clear 70; none of the 64 store averages reached 90 on mobile. Desktop is kinder in the same runs (median about 69), which matches what agencies see when a client forwards a desktop-only screenshot and assumes the site is fine.

Mobile lab Core Web Vitals timings (store averages)

Metric	Median	75th percentile	Notes in this sample
LCP (s)	9.8	14.5	0 / 64 store averages ≤ 2.5 s
INP (s)	0.30	0.41	15 / 64 store averages ≤ 0.20 s
CLS	0.02	0.07	55 / 64 store averages ≤ 0.1

CLS is the bright spot: layout stability is often already in a good lab band even when LCP is not. LCP is the structural problem. Multi-page mobile averages in the 8–15 s range do not mean every URL fails CrUX at that severity, but they do mean Lighthouse is consistently flagging heavy LCP candidates across templates. That is the signal agencies should take into a remediation backlog before arguing about a two-point Performance score change.

CrUX origin categories when field data exists

Not every lead has origin-level CrUX on the report. Where mobile origin overall category was present (n = 27), we saw 14 FAST, 8 AVERAGE, and 5 SLOW. Origin LCP was FAST on 22 of those stores, INP FAST on 22, and CLS FAST on 23. Field data is often healthier than multi-page lab averages on the same domains, especially for INP and CLS, because lab throttling and averaging across many templates punish media-heavy PDPs that may still pass origin-level CrUX when popular URLs are lighter.

Recurring Lighthouse opportunities

Across the 64 latest reports, the most common mobile opportunities were:

Opportunity	Stores where it appeared
Reduce unused JavaScript	61 / 64
Reduce unused CSS	52 / 64
Avoid multiple page redirects	26 / 64
Minify JavaScript	17 / 64
Minify CSS	10 / 64

Unused JavaScript is effectively the default Shopify finding in this sample. Theme code, app scripts, analytics, and personalisation layers compete for the same main thread that INP and Total Blocking Time care about. For ecommerce metric priorities beyond the three Core Web Vitals, see Performance Monitoring for E-Commerce: What Metrics Matter Most.

Why Shopify Core Web Vitals still vary store to store

Two shops on the same theme can land in different buckets for reasons that never appear in a theme marketing page:

App footprint. Review widgets, loyalty, chat, upsell drawers, and A/B tools add script weight on collection and PDP templates.
Media strategy. Hero video, large product galleries, and unprioritised LCP images dominate mobile LCP in lab runs.
Template mix. A clean homepage can hide a slow cart drawer or a script-heavy collection grid when you only test one URL.
Headless versus online store. Hydrogen or other custom storefronts change what source-level theme IDs reveal and how caching works.
Traffic shape and geography. CrUX inclusion and p75 values move with real visitor mix; lab runs do not.

Platform-level medians help set expectations in a roadmap review. They do not replace scheduled checks on the URLs that drive revenue. That is why we treat theme tables as context and multi-page samples as the operating view.

Global DTC Shopify cohort (lab and CrUX)

We also tracked a public high-visibility cohort tagged shopify-benchmark in Watcher (Allbirds, SKIMS, Kylie Cosmetics, Fashion Nova, Brooklinen, ColourPop, Steve Madden, Glossier, Netflix Shop, Mattel Creations). Six of those domains had multi-page Watcher lab reports in the lead pull above. On 7 Sep 2026 we also pulled mobile origin Chrome UX Report field data for the cohort via PageSpeed Insights (originLoadingExperience), so lab and field sit in separate tables rather than one mixed scorecard.

Lab multi-page averages (Watcher reports)

Brand	Domain	Mobile Performance (avg)	Desktop Performance (avg)	Mobile LCP avg (s)	Mobile CLS avg
Fashion Nova	fashionnova.com	37	60	8.6	0.05
Allbirds	allbirds.com	34	53	15.6	0.02
SKIMS	skims.com	33	52	9.1	0.04
Brooklinen	brooklinen.com	29	45	31.1	0.15
Kylie Cosmetics	kyliecosmetics.com	26	36	26.8	0.22
Netflix Shop	netflix.shop	26	41	15.7	0.15

These lab averages punish product-heavy template sets under throttling. They are not claims that every shopper sees a Performance score in the mid-20s.

Mobile CrUX at origin (PageSpeed Insights, Sep 2026)

Origin metrics are Chrome UX Report p75 values for phone traffic across the origin. CLS is shown on the usual 0–1 scale (PSI returns hundredths). “Passes all CWVs” means LCP, INP, and CLS are all in the good (FAST) bucket at p75.

Brand	Domain	Origin overall	LCP p75	INP p75	CLS p75	Passes all CWVs?
Allbirds	allbirds.com	Good	1.49 s (Good)	176 ms (Good)	0.00 (Good)	Yes
ColourPop	colourpop.com	Needs improvement	1.66 s (Good)	212 ms (Needs improvement)	0.15 (Needs improvement)	No
Kylie Cosmetics	kyliecosmetics.com	Needs improvement	2.02 s (Good)	319 ms (Needs improvement)	0.04 (Good)	No
Netflix Shop	netflix.shop	Needs improvement	2.06 s (Good)	221 ms (Needs improvement)	0.13 (Needs improvement)	No
Steve Madden	stevemadden.com	Needs improvement	2.09 s (Good)	245 ms (Needs improvement)	0.10 (Good)	No
Mattel Creations	creations.mattel.com	Needs improvement	2.10 s (Good)	314 ms (Needs improvement)	0.02 (Good)	No
Glossier	glossier.com	Needs improvement	2.17 s (Good)	239 ms (Needs improvement)	0.09 (Good)	No
Fashion Nova	fashionnova.com	Needs improvement	2.52 s (Needs improvement)	360 ms (Needs improvement)	0.20 (Needs improvement)	No
SKIMS	skims.com	Poor	2.64 s (Needs improvement)	204 ms (Needs improvement)	0.32 (Poor)	No
Brooklinen	brooklinen.com	Needs improvement	3.49 s (Needs improvement)	287 ms (Needs improvement)	0.13 (Needs improvement)	No

Homepage URL-level field rows can differ from origin (for example Fashion Nova’s homepage LCP was Good while origin LCP was Needs improvement), so use origin for brand-level scorecards and URL groups for template work.

Allbirds is the clearest split in this cohort: Watcher multi-page lab Performance averaged about 34, with lab LCP around 16 s, while mobile origin CrUX passed all three Core Web Vitals. Brand scale does not guarantee a field pass either, because only Allbirds cleared all three origin metrics in this snapshot and INP (and sometimes CLS) kept most of the other famous stores out of the good bucket even when LCP looked acceptable. If a client asks “are we slower than the big Shopify brands?”, compare their CrUX origin or Search Console URL groups to this table, and keep Lighthouse for regression hunting on the same money-path templates.

At the other end of our wider Watcher sample, smaller or carefully trimmed storefronts can clear mobile lab averages in the mid-70s to mid-80s (for example Beauty Connections, Valmont Wear, and Bäckerei Matzker in this pull). Those shops still run on Shopify. The gap versus the famous lab cohort is mostly implementation load: fewer competing scripts, lighter heroes, and fewer template surprises across the multi-page set.

Lab Lighthouse scores versus CrUX field data on Shopify

Use both clocks without mixing them in one column:

Question	Prefer
Does Google’s page experience view look healthy?	CrUX / Search Console / PSI field (p75)
Did last week’s theme or app change regress templates?	Scheduled Lighthouse (lab), same URL set
Are we comparing to Shopify’s theme table?	Theme CrUX aggregates (device mix, theme filter)
Are we comparing agency client A to client B?	Same tool, same URL roles, same aggregation

Watcher domain reports already separate lab score cards from field methodology notes (median of URL-level p75s where present, origin badge when available). Keep that separation in client decks. A green origin badge with a mobile Performance average of 40 is not a contradiction; it is a prompt to inspect which templates the lab average is punishing. For portfolio monitoring design, see Core Web Vitals Monitoring Checklist for Agencies and How to Set Up Automated PageSpeed Monitoring.

What to fix first when Shopify Lighthouse opportunities repeat

When unused JavaScript appears on nearly every store, a useful remediation order is usually:

Measure the money paths: homepage, top collections, top PDPs, cart, and checkout-adjacent templates, not a single marketing URL.
Cut or defer app scripts that are not required for first paint or first interaction on those templates.
Fix LCP candidates explicitly (image dimensions, priority hints, hero media weight) before chasing a Performance score vanity target.
Re-check CLS when banners, consent UI, or late-injected reviews shift layout after load.
Re-run the same URL set on a schedule so you can tell a real regression from run noise.

The detailed Shopify playbook for those steps lives in Shopify Speed Optimization: Core Web Vitals Guide for E-Commerce. Benchmark numbers stay honest when prioritisation stops here and the how-to lives in that companion post.

How to run a repeatable Shopify CWV cohort each month

A fixed process keeps each monthly refresh comparable:

Scope: mobile-first; LCP, INP, CLS; optional Lighthouse Performance as a lab companion metric.
URL roles: same template set each month (home, collection, PDP, cart at minimum).
Aggregation: decide in advance whether you report origin CrUX, URL-level field, or multi-page lab averages, and never mix them in one table column.
Guardrails: mark insufficient CrUX explicitly; record tool, date, and page count.
Reporting: publish median cohort values, share of stores passing field CWV when available, top gains and regressions, and hypotheses (apps, media, theme) labelled as hypotheses.

You can start wi

Core Web Vitals in 2026: What INP Means for Your Shopify and WordPress Rankings

Wed, 16 Sep 2026 07:42:43 +0000

Originally published at shehrozaftab.com.

Google's Core Web Vitals are still part of how it judges page experience, and the biggest recent change catches out a lot of stores: INP replaced FID. If your Shopify or WordPress site felt fine on the old metric but now shows warnings, this is usually why.

The three Core Web Vitals in 2026

LCP (Largest Contentful Paint): how fast the main content loads. Good is 2.5s or less.
INP (Interaction to Next Paint): how quickly the page responds to taps and clicks across the whole visit. Good is 200ms or less.
CLS (Cumulative Layout Shift): how much the layout jumps around. Good is 0.1 or less.

Why INP is stricter than the old FID

The retired metric, FID (First Input Delay), only measured the delay on your very first interaction, and only the input delay part. It was easy to pass even when a site felt laggy.

INP is tougher on purpose. It looks at interactions throughout the whole visit, not just the first, and it measures the full time until the page visually responds, not just the initial delay. So heavy JavaScript, bloated theme scripts and third-party tags that used to slide past FID now show up clearly.

What tends to hurt INP

On real Shopify and WordPress sites, the usual culprits are:

Too much JavaScript running on the main thread, especially from apps and plugins you installed and forgot.
Heavy third-party tags: chat widgets, popups, analytics, ad and tracking scripts all competing for the browser.
Page builders and app blocks loading scripts sitewide even on pages that do not use them.
Large, unbroken tasks that block the browser from responding while they run.

How I get sites back into the green

The approach is the same on both platforms, tuned to each:

Audit what is loading. Measure with real field data (Chrome UX Report / Search Console), not just lab tests, because INP is about real interactions.
Cut the JavaScript. Remove unused apps and plugins, defer non-critical scripts, and stop loading things on pages that do not need them.
Tame third-party tags. Load chat, popups and analytics after interaction or on idle, not on first paint.
Fix LCP and CLS alongside it. Optimise the hero image, reserve space for images and embeds, and use font-display: optional so text does not jump.
Re-measure in the field over the following weeks, since Core Web Vitals use a rolling 28-day window of real users.

The takeaway

INP rewards sites that stay responsive the whole time someone is using them, not just on the first tap. For most Shopify and WordPress stores the fix is less code, fewer competing scripts, and disciplined third-party tags. Get those right and page experience stops being a drag on your rankings.

Seeing INP or Core Web Vitals warnings in Search Console? Send me your site and I'll audit it and lay out exactly what to fix.

Using searchParams in a Next.js Page Silently Opts It Out of Static Rendering Too

Tue, 15 Sep 2026 09:33:11 +0000

I wrote earlier about a buried cookies() call silently forcing an entire page into dynamic rendering. searchParams causes the exact same category of behavior, through a completely different, and in some ways more surprising, mechanism, since using it doesn't look like reading anything server-specific at all.

Why This One Is Extra Surprising

cookies() and headers() at least sound like they're reaching into something request-specific, so forcing dynamic rendering feels intuitive once you think about it. searchParams just looks like reading a prop, a completely normal-looking destructure at the top of a page component. There's nothing about the syntax that signals "this is a server-specific value tied to this exact request."

// app/products/page.tsx
export default function ProductsPage({
  searchParams,
}: {
  searchParams: Promise<{ category?: string; page?: string }>;
}) {
  // reading searchParams here means this page can no longer be
  // statically generated, full stop, regardless of what else it does
}

The moment a page component accepts and uses searchParams, Next.js treats the entire page as dynamic. This makes complete sense once you think through why, the query string is part of the actual request URL, and a page reading it necessarily produces different output for /products?category=shoes versus /products?category=hats, which is inherently incompatible with generating one fixed, static HTML output ahead of time. It's correct, intentional behavior. It's just not obviously signaled by the code itself the way reading cookies() is.

Where This Actually Costs You

For a genuinely filterable page, a product listing with category and sort filters, this dynamic behavior is exactly correct and necessary, the whole feature depends on rendering different content per query string. This isn't a bug to fix in that case.

The actual cost shows up when searchParams gets accepted by a page component out of habit, or copied from a similar page, without the page actually using the specific query params in any way that changes its output.

// app/about/page.tsx
export default function AboutPage({
  searchParams, // accepted but never actually used for anything meaningful
}: {
  searchParams: Promise<{ [key: string]: string | undefined }>;
}) {
  return (
    <div>
      <h1>About Us</h1>
      <p>We build things.</p>
    </div>
  );
}

If this component's type signature includes searchParams but the actual rendered output never varies based on it, simply accepting the prop can still be enough to affect how Next.js treats the route, depending on whether the value is actually read and used within the render. A genuinely static about page, forced into unnecessary dynamic rendering because of a leftover parameter or copy-pasted type signature from a different page, loses real caching benefit for no actual functional reason.

How to Tell If This Is Actually Happening

Same check as the earlier cookies() post, run a real build and look at the route summary:

npm run build

A route marked dynamic (ƒ) that you'd expect to be static (○) is worth investigating. If the page genuinely needs to vary based on query parameters, that's correct and expected. If it doesn't, actually use those params to produce different output, the fix is simply not accepting or reading searchParams in that component at all.

The Distinction That Actually Matters

Does this specific page's rendered output genuinely need to differ based on the query string? If yes, dynamic rendering triggered by searchParams is correct, intentional, and not something to fight against, the feature depends on it.

Is searchParams present in the component's signature or destructured, but never actually used to change what gets rendered? That's the case worth fixing, either by removing the unused parameter entirely, or, if the params are genuinely needed for something that doesn't affect the main rendered content, handling that specific piece in a way that doesn't force the whole page dynamic, a client-side read via useSearchParams() inside a small Client Component, for something that only needs to react to params after the initial static render.

// A pattern that keeps the main page static while still reacting to query params
// for a small, specific piece of UI that genuinely needs them
export default function ProductsPage() {
  return (
    <div>
      <h1>Our Products</h1>
      <ProductGrid /> {/* genuinely static content */}
      <QueryAwareBanner /> {/* small client component reading useSearchParams() */}
    </div>
  );
}

The Broader Pattern

This is the same underlying lesson as the cookies() post, applied to a different, less obviously "server-specific" mechanism. Next.js's static-versus-dynamic decision is driven by what a page actually reads and depends on, not by how obviously server-related that code looks at a glance. searchParams, cookies(), and headers() all trigger the same category of behavior, and all three are easy to include without actually needing what they provide, especially when copying a page's structure from a similar one that genuinely did need it.

Go check your own build output for any page accepting searchParams that doesn't actually use it to change what gets rendered. If you find one, that's a free, easy static-rendering win sitting right there. Drop what you find in the comments.

Get the templates: https://pixelanas.gumroad.com

Anas, full-stack Next.js developer building SaaS products and premium templates. X: @ASheikh69751

Stop Overloading Your App: A Guide to Debouncing

Tue, 15 Sep 2026 09:03:36 +0000

What is Debouncing?

Debouncing is a programming practice used to ensure that a function only executes after a certain amount of time has passed since it was last triggered. In essence, it bundles a sequence of rapid-fire calls into a single execution.

The Elevator Analogy

Imagine you are standing in an elevator. As the doors begin to close, someone rushes up and hits the 'Open' button. The door reopens, and the timer to close the door resets. If people keep arriving and hitting that button, the elevator will never leave. The elevator only moves when there is a consistent, quiet pause of, say, 5 seconds without anyone pressing the button. Debouncing works the same way: it waits for the "noise" of rapid events to stop before finally firing your logic.

Why It Matters

In modern web development, we often attach functions to events like onScroll, onResize, or onKeyPress. If a user types into a search bar, an API request might fire for every single keystroke. Without debouncing, you might send 10 unnecessary requests to your database just because a user typed "JavaScript." Debouncing saves server resources, reduces database load, and creates a smoother experience for the user.

Implementation in React

Here is how you might implement a debounced search input in a React component:

import React, { useState, useEffect } from 'react';

const SearchBar = () => {
  const [query, setQuery] = useState('');

  useEffect(() => {
    const handler = setTimeout(() => {
      console.log('Fetching results for:', query);
      // Imagine an API call here
    }, 500);

    return () => clearTimeout(handler);
  }, [query]);

  return <input onChange={(e) => setQuery(e.target.value)} />;
};

Takeaway

Debouncing isn't just about saving bytes; it is about respecting the user's input and your system's limits. By introducing a small delay, you transform chaotic, repetitive input streams into clean, intentional actions, ensuring your backend only works when it truly needs to.

Originally published on my blog. You can read the alternative breakdown here.

How Farm.js compiles React components into direct DOM updates

Mon, 14 Sep 2026 16:21:03 +0000

title: How Farm.js compiles React components into direct DOM updates
published: true

tags: react, javascript, webdev, performance

I've been building Farm.js, a full-stack framework with an experimental ahead-of-time compiler for React (repo on GitHub). The short version: when the compiler can prove at build time exactly which DOM nodes a piece of state touches, it skips React's render cycle entirely for that update and writes to the DOM directly. When it can't prove it, the component stays on normal React. No new mental model, no different component syntax.

You can try everything in this article without installing anything. The playground runs the real framework in your browser via WebContainers: open it on StackBlitz.

This post is a deep dive into how the compiler works, with honest benchmark numbers (including the case where it does nothing), and then a tour of the rest of the framework.

The idea

Take the most ordinary component imaginable:

import { useState } from "react";

export function Counter(props: { initial: number }) {
  const [count, setCount] = useState(props.initial);
  return (
    <button
      className={count > 0 ? "active" : "idle"}
      onClick={() => setCount((value) => value + 1)}
    >
      Count: {count}
    </button>
  );
}

When you click, React re-runs the function, builds a new element tree, diffs it against the old one, and commits the differences. For this component that machinery is pure overhead: there is exactly one host element, and count can only ever affect two things, the text content and the className.

The Farm compiler analyzes the component at build time and, when the structure is provable, emits a definition like this instead:

createCompiledComponentWithFeatures(
  {
    initialize: (props) => [props.initial],
    render: (props, state) => (
      <button className={state[0].get() > 0 ? "active" : "idle"}>
        Count: {state[0].get()}
      </button>
    ),
    bindings: [
      {
        kind: "text",
        path: [],
        dependencies: [0],
        read: (_props, state) => ["Count: ", state[0].get()],
      },
      {
        kind: "attribute",
        path: [],
        dependencies: [0],
        name: "className",
        read: (_props, state) => (state[0].get() > 0 ? "active" : "idle"),
      },
    ],
  },
  [],
);

The interesting part is bindings. Each binding says: this state cell feeds this text node or this attribute, here is how to compute the value. render still exists and React still owns the first mount, so hydration, refs, and devtools all see a normal component. But after mount, calling the setter walks the dependency graph and applies the affected bindings straight to the DOM. No render, no reconciliation, no commit phase. There's a runtime test in the repo that renders the compiled and baseline versions side by side and asserts the compiled one updates with zero React renders.

The second argument, the empty array, is the component's capability list. Components that need more machinery (keyed lists, conditional ranges, nested compiled components) get feature modules in that array, and modules that don't use a feature never import it, so the unused structural runtimes are tree-shaken out of the production bundle. The measured runtime premium for direct-bindings-only output is 73.6% smaller than shipping the complete runtime.

The compiler refuses to guess

This is the part I care most about. The compiler is intentionally conservative: it only transforms a component when it can prove there's one stable host element tree and every supported state value maps to known text or attribute targets. Anything it can't prove falls back to the original React component. From the docs, things that always stay on React include:

unkeyed or index-keyed list rendering
effects, or hooks other than the supported useState shape
ref, dangerouslySetInnerHTML, JSX spreads
multiple or conditional returns
props whose updates carry identity (objects, arrays, functions, elements)
async components, setters called outside JSX event handlers

Fallback is silent by default, but you can turn on diagnostics:

[react-compiler] KeyedList: dynamic child structures require React reconciliation; using React.

And report: true writes .farm/react-compiler.json with exactly which components compiled and which fell back, with reasons and counts. So "is my hot component actually compiled" is a question with a checkable answer, not a vibe. You can also opt any component out with a "use no compiler" directive.

Enabling it is one flag on the renderer:

import { defineConfig } from "@farm.js/core";
import { react } from "@farm.js/react";

export default defineConfig({
  renderer: react({
    experimental: {
      compiler: true,
    },
  }),
});

Full compiler docs, including the complete eligibility table, are at farmjs.dev/docs.

Numbers, including the boring ones

The benchmark is a js-framework-benchmark-style keyed table (create 1,000 rows, update every 10th label, advance selection, swap two rows, clear), measured on production builds driven by Playwright Chromium. Latency is click dispatch to the MutationObserver callback for the resulting DOM writes, so it isn't quantized to frame boundaries the way rAF timing is. The runner installs the published npm packages rather than linking the workspace, and it fails hard unless the compiler report proves the workload component actually compiled, so it can't silently degrade into baseline vs baseline.

action	baseline p50	compiled p50	p50 speedup
create	2.70ms	2.30ms	1.17x
update	0.70ms	0.70ms	1.00x
select	0.60ms	0.50ms	1.20x
swap	3.40ms	0.50ms	6.80x
clear	3.80ms	0.70ms	5.43x

CPU work (script + style + layout) per full action cycle: baseline 18.71ms, compiled 9.05ms, about 2x less.

Notice update is 1.00x. Row label updates were already cheap in React, and the compiled path doesn't magically beat the DOM write itself. The wins come where reconciliation is the cost: swaps, clears, structural churn on keyed lists. Methodology, correctness controls, and raw per-sample data live in benchmarks/compiler in the repo if you want to pick it apart or rerun it.

The rest of the framework

The compiler is the flashy part, but I've spent most of the time on everything around it. Farm is a full-stack framework: Vite dev server, file-based app directory routing, streaming SSR, and Nitro production output. Here's what I think is worth showing.

Routes are typed, end to end

The app directory looks like what you already know (page.tsx, layout.tsx, loading.tsx, error.tsx, dynamic [param] and catch-all [[...slug]] segments). The difference is that Farm generates route types from it:

export type RoutePath =
  | "/"
  | "/about"
  | `/repos/${string}/${string}`
  | `/users/${string}`;

Link hrefs are checked against this union, so a typo'd internal link is a type error, and the types regenerate on dev start and whenever routes change.

Server functions with schemas, not just "use server"

Server actions take zod schemas for input and output, so the wire boundary is validated in both directions:

import { createServerFn } from "@farm.js/core/server-fn";
import { z } from "zod";

export const submitMessage = createServerFn({
  input: z.object({
    name: z.string().trim().min(1, "Name is required"),
    message: z.string().trim().min(1, "Message is required"),
  }),
  output: z.object({ success: z.literal(true), message: publicMessageSchema }),
  async handler({ input }) {
    // runs on the server, even when called from a client component
  },
});

On the client, useServerFn gives you formAction for progressive-enhancement forms plus optimistic updates with automatic rollback:

const action = useServerFn(submitMessage, {
  optimistic({ formData }) {
    return { success: true as const, message: { id: -1, /* ... */ } };
  },
  rollbackOnError: true,
});

<form action={action.formAction}>...</form>

And server queries are the read-side sibling, with cache keys, stale times, and server-side invalidation that reaches into client caches:

export const productQuery = createServerQuery({
  input: z.object({ id: z.string() }),
  key: ({ input }) => ["product", input.id],
  staleTime: "30s",
  async handler({ input }) { /* ... */ },
});

// in a mutation:
invalidate(["product", input.id]);

Integrations are typed capabilities, not SDK glue

This is the piece I haven't seen elsewhere. An integration (Stripe, Supabase, WorkOS, Polar, Unkey, and so on) mounts as a typed capability in config:

export const appIntegrations = {
  billing: stripe({
    products: stripeProducts,
    instance: stripeInstance,
    webhookSecret: process.env.STRIPE_WEBHOOK_SECRET,
    async onWebhook(event) { /* typed webhook events */ },
  }),
} as const;

Then four lines give your whole app a typed client for every mounted integration:

import { createIntegrations } from "@farm.js/core/client";
import type { AppIntegrations } from "./integrations";

export const { api, apiClient } = createIntegrations<AppIntegrations>();

From a client component, await apiClient.billing.checkout({ body: { productId } }) is fully typed. The same call from a server component stays in-process instead of making an HTTP round trip back into your own app. The vendor SDK instance never leaves server-only modules.

Rendering modes as directives

Instead of exporting config objects, a route can declare its rendering mode in one line:

"use ssg; 60";

export default function BlogPage() {
  return <main>Blog</main>;
}

That's static generation with 60-second revalidation. use dynamic and use ppr; 60 (partial prerendering with a static shell and streamed dynamic holes) work the same way. The classic export const revalidate = 60 style works too.

Hydration islands with click replay

Client components can defer their own hydration:

"use client";

export const island = "interaction";

export function CopyButton({ value }: { value: string }) {
  return <button onClick={() => navigator.clipboard.writeText(value)}>Copy</button>;
}

interaction means the component ships zero hydration work until the first click, and Farm replays that click after hydration so the user's first interaction isn't swallowed. There are also visible and idle strategies, with load as the compatibility default.

Small things that add up

URL state hooks, nuqs-style: useQueryState('page', asInteger.withDefault(1)) with history and throttling control.
Chainable middleware: middleware().use(...).redirect('/old', '/new').when(ctx => ctx.pathname.startsWith('/api'), ...) with matcher/exclude config, per-segment middleware supported.
Storage helpers for KV across local files, SQLite, Postgres, MySQL, Redis, and MongoDB from one import, and a pluggable shared cache adapter (Redis-backed ISR/PPR cache for multi-instance deploys).
Typed API routes with createEndpoint + zod, and a generated RPC-style client for them.
farm preview: a public tunnel URL for your running local dev server, no build or deploy involved.
farm doctor and farm explain <path>: the second one tells you exactly how a URL maps to a route and which runtime serves it.
A docs site in your app with docs: { enabled: true }: markdown under src/app/docs becomes a searchable docs site with llms.txt, sitemap, and robots handling out of the box.
Skew protection: when a deploy happens while a user has a tab open, stale clients get a structured 409 with a "refresh before trying again" contract instead of mystery failures.

Not just React

The compiler is React-specific, but the framework isn't. Renderers are one line in config, and Preact, Solid, Vue, and Svelte are first-class, with native bindings for each (useServerQuery in Solid and Vue, createServerQuery in Svelte, and so on). Same routes, same server functions, same integrations. Deployment targets cover Vercel, Cloudflare, Netlify, and plain Node, with any other Nitro preset accepted as pass-through.

Try it

The fastest path is the browser playground: StackBlitz. Locally:

pnpm create @farm.js/app@beta my-app

Everything is beta (0.1.0-beta.61 as I write this) and the compiler is explicitly experimental and opt-in. The fallback behavior is the contract I'm most committed to: if the compiler can't prove a component is safe to transform, you get exactly the React you wrote.

The website is farmjs.dev, docs at farmjs.dev/docs, and the repo is github.com/farming-labs/farm.js. I'd genuinely love feedback, especially skeptical feedback, on the compiler's eligibility model and the typed integration approach. If you break something in the playground, an issue with a repro makes my day.

{% embed https://github.com/farming-labs/farm.js %}

Stop Overworking Your Code: A Guide to Debouncing

Mon, 14 Sep 2026 09:21:17 +0000

What is Debouncing?

Debouncing is a programming practice that ensures a specific task is only executed after a set amount of time has passed since the last time it was triggered. Essentially, it waits for a "pause" in activity before performing an action, preventing a function from running too many times in quick succession.

The Elevator Analogy

Imagine you are standing in an elevator in a busy office building. As you step in, the door starts to close. Suddenly, a coworker runs up and taps the "open door" button. The door reopens, resetting the timer. This happens again and again—every time someone arrives, the door's automatic closing process is canceled and restarted. The elevator only actually begins its journey once there has been a lull in arrivals for, say, five seconds. That pause is your debounce window.

Why It Matters

In modern web development, users perform actions like typing into a search bar or resizing a browser window. If your application sends a network request to a database every single time a user presses a key, you are going to overwhelm your server and waste bandwidth. By using debouncing, we ensure the search request only triggers once the user has finished typing, making the experience smoother and the infrastructure much more stable.

Code Example

Here is a simple JavaScript function that "debounces" an input event:

function debounce(func, delay) {
  let timer;
  return (...args) => {
    clearTimeout(timer);
    timer = setTimeout(() => func(...args), delay);
  };
}

// Usage: Only runs 500ms after the user stops typing
const search = debounce(() => console.log("Fetching results..."), 500);

Takeaway

Debouncing is less about doing less work and more about doing work at the right time. By strategically choosing when to execute expensive operations, you bridge the gap between user intent and system performance, creating a highly responsive application that doesn't buckle under the weight of constant, redundant events.

Resources

GitHub Repository: react-hook-lab
react-hook-lab: npm package
Connect with me on LinkedIn: Saurav Pandey

Originally published on my blog. You can read the alternative breakdown here.

Redis Caching Strategies for High-Performance APIs

Sat, 12 Sep 2026 22:30:40 +0000

Redis Caching Strategies for High-Performance APIs

The Cache That Made My API Faster—and Introduced a New Problem.

When I first started thinking seriously about API performance, my instinct was simple:

If the database is slow, optimize the database.

Add indexes. Improve the query. Tune the connection pool.

And sometimes, that's exactly what you should do.

But eventually, you run into a different problem.

The database isn't necessarily slow.

You're just asking it the same question too many times.

Imagine an endpoint that returns product information.

GET /api/products/123

The product changes once every few hours, but the endpoint gets thousands of requests every minute.

Without caching, every request goes straight to PostgreSQL:

Request
   ↓
API
   ↓
PostgreSQL
   ↓
Response

The database is doing the same work over and over again.

That's where Redis comes in.

The Simple Idea Behind Caching

Instead of asking the database every time, we store frequently requested data temporarily.

Request
   ↓
Redis
   ↓
  HIT ─────→ Response
   │
  MISS
   ↓
Database
   ↓
Redis
   ↓
Response

The first request pays the database cost.

The next requests don't.

In a Node.js API, a simple cache-aside implementation looks like this:

const key = `product:${productId}`;

const cached = await redis.get(key);

if (cached) {
  return JSON.parse(cached);
}

const product = await prisma.product.findUnique({
  where: { id: productId },
});

if (!product) {
  throw new Error("Product not found");
}

await redis.set(
  key,
  JSON.stringify(product),
  { EX: 300 }
);

return product;

That's the basic pattern:

Check Redis → cache miss → query database → store result → return.

Simple.

But this is where caching stops being just a performance optimization.

Because now we have two copies of the data.

And they can disagree.

The Stale Data Problem

Suppose Redis contains:

product:123
price: ₦45,000

Then someone updates the product:

Database
price: ₦50,000

Redis still contains:

₦45,000

If the cache has a five-minute TTL, users could receive the old price for another five minutes.

This is the real challenge with caching.

Making data fast to access is easy. Keeping it correct is harder.

That's why TTL alone isn't enough.

TTL vs Invalidation

TTL answers:

"How long should this cache entry exist?"

Invalidation answers:

"When should I remove it because the underlying data changed?"

For example:

await prisma.product.update({
  where: { id: productId },
  data: updateData,
});

await redis.del(`product:${productId}`);

Now the next request becomes a cache miss and retrieves the latest value from the database.

For many APIs, cache-aside + TTL + delete-on-write is a very practical starting point.

You don't need a complicated caching architecture on day one.

Then There's the Cache Stampede

Here's where things get interesting.

Imagine a popular cache entry expires:

product:123

At the exact same time, 1,000 requests arrive.

They all check Redis:

MISS
MISS
MISS
MISS
...

And suddenly all 1,000 requests hit PostgreSQL.

The cache that was supposed to protect the database has just created a database traffic spike.

This is commonly called a cache stampede or thundering herd.

For extremely hot keys, you can prevent this by allowing only one request to rebuild the cache while the others wait.

Redis can help here through distributed locking.

But I wouldn't introduce locking everywhere.

If a query takes 10ms, the complexity may not be worth it.

If rebuilding the cache takes several seconds under heavy traffic, it's a different conversation.

Optimize for the workload you actually have.

What Should You Cache?

This is probably the most important question.

I wouldn't cache something simply because Redis is available.

Good candidates usually have three characteristics:

Frequently requested
+
Expensive to retrieve
+
Safe to be slightly stale

For example:

Product details
Public profiles
Search results
Configuration
Reference data
Expensive aggregations

I'd be much more careful with:

Account balances
Payment state
Authorization decisions
Inventory
Transaction state

Especially in financial systems.

If a wallet contains ₦100,000 and Redis says ₦100,000 while the database says ₦70,000 after a transaction, using that cached value for a financial decision is dangerous.

Fast wrong data is still wrong data.

For critical state, the database should remain authoritative.

Your Cache Key Is Part of Your Architecture

A cache key isn't just a random string.

This:

is a terrible cache key.

This is better:

user:123

And in a multi-tenant system:

tenant:456:user:123

For filtered results:

products:category:phones:page:1

Every parameter that changes the response needs to be represented appropriately in the key.

Otherwise, you can end up returning the wrong cached response to the right request.

That's a correctness bug—not a performance bug.

Redis Should Accelerate Your System, Not Become Your System

One of the most important design decisions is what happens when Redis fails.

For a cache-aside architecture, I'd rather have:

Redis available
    ↓
Use cache

Redis unavailable
    ↓
Use database

The API might become slower.

But it remains functional.

That's a much better failure mode than:

Redis unavailable
    ↓
Entire API unavailable

If your application completely depends on Redis just to retrieve data that already exists in PostgreSQL, you've accidentally turned your cache into a critical dependency.

Measure the Difference

Finally, don't add Redis and declare victory.

Measure it.

Before caching:

Average latency: 400ms
P95 latency:     800ms
DB queries/sec:  2,000

After caching:

Average latency: 70ms
P95 latency:     150ms
DB queries/sec:  500
Cache hit rate:  85%

Now you have evidence.

The goal isn't to make Redis busy.

The goal is to make the entire system do less unnecessary work.

The Bigger Lesson

Redis isn't what makes an API fast.

Avoiding unnecessary work makes an API fast.

Redis is simply one of the tools that helps you do that.

The moment you introduce a cache, you're making a trade:

More memory
+
More complexity
+
Potentially stale data

in exchange for:

Lower latency
+
Less database load
+
Higher throughput
+
Better scalability

The best caching strategy isn't the one that puts the most data into Redis.

It's the one that eliminates the most unnecessary work while keeping the system correct.

That's the difference between simply using Redis and actually designing with Redis.

GPU-Accelerating MSCRED with CUDA, im2col, GEMM, and a Custom PyTorch Extension

Fri, 11 Sep 2026 18:07:01 +0000

Introduction

During my M.Tech in Data Science and Artificial Intelligence at IIT Bhilai (2021–2023), I conducted thesis research on multivariate time-series anomaly detection and GPU acceleration using industrial process signals.

My thesis, titled “Anomaly Detection of Multivariate Time Series Data and Acceleration Using GPUs,” explored an MSCRED—Multi-Scale Convolutional Recurrent Encoder-Decoder—and ConvLSTM-based pipeline for modelling relationships between signals and detecting abnormal temporal patterns.

Although the model produced useful anomaly-detection results, its convolution-heavy encoder created a significant CPU bottleneck. I therefore implemented GPU-accelerated operations using CUDA, transformed convolution into an im2col plus GEMM workflow, investigated global-memory and shared-memory implementations, and integrated the accelerated operations with PyTorch.

In the measured workload, encoder runtime decreased from 300 seconds on the CPU to 23.664 seconds on the GPU, representing an approximately 12.68× encoder speed-up.

Research attribution: MSCRED and ConvLSTM are established architectures introduced by the researchers cited in the References section. This article describes how I used, adapted, profiled, and accelerated selected operations as part of my M.Tech thesis.

Data confidentiality: The original research involved industrial process data. This article discusses the architecture, optimisation methodology, and aggregate benchmark results without publishing proprietary signals, internal process details, confidential datasets, or restricted source code.

The Anomaly-Detection Problem

Industrial systems often generate many related sensor and process-control signals at the same time. A single signal may appear normal when viewed independently, while the relationship between several signals may indicate an abnormal operating condition.

This makes multivariate time-series anomaly detection more difficult than analysing one signal at a time.

The model must learn:

Relationships between multiple signals
Changes across different time windows
Temporal dependencies between successive observations
Deviations from normal operating patterns
Reconstruction differences that may indicate anomalies

The goal of the research was to detect unusual multivariate patterns while preserving the temporal and spatial relationships present in the data.

MSCRED and ConvLSTM Architecture

MSCRED models relationships between multiple time-series signals by constructing signature matrices over different temporal windows.

At a high level, the pipeline contains:

Signature-matrix generation from multivariate signals
A convolutional encoder for spatial feature extraction
ConvLSTM layers for temporal modelling
A decoder that reconstructs expected signal relationships
Reconstruction-error analysis for anomaly scoring

The encoder processes the signature matrices through convolutional operations, while ConvLSTM captures how the encoded representations evolve over time.

Multivariate signals
        ↓
Signature matrices
        ↓
CNN encoder
        ↓
ConvLSTM
        ↓
Decoder
        ↓
Reconstruction error
        ↓
Anomaly score

Why Signature Matrices Are Useful

A signature matrix represents pairwise relationships between signals over a selected time interval.

By generating these matrices at multiple scales, the model can observe both short-term and longer-term relationships. This helps identify anomalies that may not be visible in an individual signal.

Role of the Encoder

The convolutional encoder extracts spatial patterns from the signature matrices. It reduces the raw matrix representation into compact feature maps that capture important relationships between signals.

Role of ConvLSTM

ConvLSTM models how those spatial representations change over time.

Unlike a standard fully connected LSTM, ConvLSTM preserves the spatial structure of feature maps while learning temporal dependencies.

Role of the Decoder

The decoder reconstructs the expected signature matrices.

When the reconstructed output differs significantly from the observed input, the reconstruction error can indicate an anomalous pattern.

Anomaly-Detection Evaluation

The purpose of the MSCRED and ConvLSTM pipeline was to detect abnormal relationships and temporal behaviour across multiple industrial process signals.

The model learned normal multivariate patterns by reconstructing signature matrices. During evaluation, the difference between the observed signature matrices and the reconstructed outputs was used to calculate a reconstruction error.

Observed signature matrix
            ↓
MSCRED reconstruction
            ↓
Reconstruction error
            ↓
Thresholding
            ↓
Normal or anomalous

A larger reconstruction error indicated that the observed relationships differed from patterns learned during normal operation and could therefore represent an anomaly.

The thesis evaluated anomaly-detection quality using the F1 score. Under the reported experimental setup, the best recorded F1 score was 0.941.

This result represents model-quality evaluation and should be interpreted separately from the CUDA runtime measurements presented later. The GPU memory optimisations were designed to reduce computation time, not to improve predictive accuracy.

Where the Performance Bottleneck Appeared

The encoder repeatedly applied convolutional operations to signature matrices.

On the CPU, these operations involved substantial nested computation across:

Input channels
Output filters
Spatial positions
Kernel dimensions
Batch elements

This made the encoder a strong candidate for GPU acceleration because many convolution calculations could be performed in parallel.

The objective was not to redesign the anomaly-detection model. Instead, it was to preserve its expected computational behaviour while accelerating selected expensive operations.

Transforming Conv2D into `im2col` and GEMM

A convolution repeatedly applies a kernel to overlapping regions of an input feature map.

The im2col, or image-to-column, transformation rearranges those local regions into columns of a matrix.

After this transformation, convolution can be expressed as a general matrix multiplication:

Output matrix = Weight matrix × im2col(Input)

This conversion is useful because matrix multiplication is highly parallel and can be executed efficiently on a GPU.

Conceptual Transformation

Input feature map
        ↓
Extract overlapping kernel windows
        ↓
Arrange windows as matrix columns
        ↓
Multiply by reshaped filter matrix
        ↓
Reshape result into output feature map

Why Use GEMM?

GEMM stands for General Matrix-Matrix Multiplication.

Modern GPU architectures are designed to perform large numbers of multiply-and-accumulate operations in parallel. Expressing convolution as GEMM allows the workload to use that parallelism.

Trade-Off

im2col introduces additional memory usage because overlapping input regions are copied into an intermediate matrix.

The method is most effective when the computational benefit of parallel matrix multiplication outweighs the cost of creating and storing the intermediate representation.

CUDA Kernel Design

The CUDA implementation divided the computation across many GPU threads.

Each thread was responsible for part of the output, while thread blocks grouped related work.

Important kernel-design considerations included:

Mapping output elements to threads
Selecting thread-block dimensions
Coalescing memory access where possible
Reducing unnecessary global-memory operations
Avoiding excessive synchronisation
Preserving correct tensor indexing
Managing intermediate buffers efficiently

The implementation required careful validation because incorrect indexing in GPU code can produce plausible-looking but incorrect outputs.

Global Memory and Shared Memory

The first GPU implementation relied primarily on global memory.

Global memory provides access to the complete input, but its latency is considerably higher than that of on-chip shared memory.

The optimised implementation staged reusable tiles inside shared memory. Threads within the same block could then reuse nearby values without repeatedly loading them from global memory.

The shared-memory optimisation was intended to:

Reduce repeated global-memory reads
Improve data locality
Increase data reuse within each thread block
Reduce memory-related kernel overhead
Improve matrix-multiplication efficiency

Conceptual Data Flow

Global memory
      ↓
Load reusable tile
      ↓
Shared memory
      ↓
Threads perform repeated calculations
      ↓
Write result to global memory

Shared memory is limited in capacity, so tile dimensions and thread-block organisation must be selected carefully.

Poorly selected configurations can:

Reduce occupancy
Increase synchronisation overhead
Create bank conflicts
Use too much shared memory per block
Limit the number of active thread blocks

Integrating the CUDA Implementation with PyTorch

To use the accelerated operations inside the existing model pipeline, I connected the CUDA implementation to PyTorch through a custom extension.

The integration enabled PyTorch tensors to be passed to the CUDA implementation and the computed results to be returned to the model pipeline.

Important integration concerns included:

Tensor dimensions
Device placement
Data types
Contiguous-memory assumptions
Compilation and linking
Error handling
CPU-versus-GPU output comparison

Correctness validation was essential. A faster kernel is not useful if it changes tensor shapes, produces incorrect indexing, introduces unacceptable numerical differences, or behaves inconsistently for the tested configurations.

The custom extension made it possible to retain PyTorch for model development while using lower-level CUDA operations for selected performance-critical components.

Experimental Setup

The benchmark used the following environment:

GPU: NVIDIA RTX A6000, 48 GB
CPU: 16-core Intel Xeon Gold 5218
System memory: 64 GB
Data points: 36,000
Number of features: 30
Batch size: 128
CPU threads: 32

Timing Methodology

The timings reported here are measurements from my thesis experiments using the specified hardware and workload. The CPU and GPU implementations used the same input dimensions and batch configuration.

The results should be treated as comparative experimental measurements rather than universal library benchmarks. Performance may vary with implementation details, CUDA configuration, tensor dimensions, data-transfer costs, compiler settings, and hardware.

Performance Benchmark Results

Component	CPU	GPU: global memory	GPU: shared memory
Encoder	300 s	27.492 s	23.664 s
ConvLSTM	1,200 s	909.208 s	847.260 s
Total runtime	2,700 s	1,594 s	1,481 s

Results at a Glance

Anomaly-detection quality: best recorded F1 score of 0.941
Encoder acceleration: approximately 12.68×
End-to-end acceleration: approximately 1.82×
Total runtime reduction: approximately 45.1%

The shared-memory implementation reduced encoder runtime from 300 seconds to 23.664 seconds, producing an approximately 12.68× encoder speed-up.

For the complete measured pipeline, runtime decreased from 2,700 seconds to 1,481 seconds. The total-runtime measurement represents the complete measured pipeline rather than only the encoder and ConvLSTM components shown separately.

Why the Encoder Improved More Than ConvLSTM

The convolutional encoder benefited strongly from GPU execution because its operations provided substantial data parallelism.

ConvLSTM achieved a smaller improvement because of:

Recurrent dependencies
Sequential temporal processing
Memory movement
Repeated state updates
More limited parallelism across time steps

Consequently, the 12.68× encoder acceleration did not translate into a 12.68× end-to-end speed-up.

After the encoder was accelerated, ConvLSTM became the dominant runtime bottleneck.

This is an important performance-engineering lesson: accelerating one component can move the bottleneck to another part of the system.

Correctness Validation

The CUDA implementation was compared with the reference implementation for the tested tensor shapes and configurations.

The optimisation work focused on reducing execution time while preserving the expected computational behaviour. Model-quality evaluation and runtime evaluation were treated as separate concerns.

Engineering Challenges

Several engineering challenges appeared during the implementation.

1. Correct Tensor Indexing

The CUDA kernel had to reproduce the same logical output as the reference convolution operation.

A small indexing error could generate incorrect feature maps without necessarily causing a runtime failure.

2. Memory Management

The im2col transformation created additional intermediate data.

This required careful allocation, reuse, and release of GPU memory.

3. Synchronisation

Threads sharing data through shared memory had to synchronise at appropriate points.

Missing or unnecessary synchronisation could cause incorrect results or reduced performance.

4. Kernel Configuration

Thread-block dimensions and tile sizes affected:

Occupancy
Shared-memory usage
Memory access
Work distribution
Overall kernel performance

5. PyTorch Integration

The extension had to account for tensor shapes, data types, device placement, memory layout, compilation, linking, and output comparison.

6. Measuring the Correct Bottleneck

Improving one component changed the relative cost of other components.

Profiling therefore had to be repeated after major optimisations.

Engineering Lessons

1. Optimising One Component Does Not Guarantee an Equivalent End-to-End Improvement

Once the encoder was accelerated, ConvLSTM represented a larger proportion of the remaining runtime.

2. `im2col` Makes Convolution GPU-Friendly but Introduces Memory Overhead

The transformation enables GEMM-based execution while creating a larger intermediate representation.

3. Shared Memory Is Valuable When Threads Repeatedly Reuse Nearby Data

Its benefit depends on access patterns, tile sizes, synchronisation, and thread-block configuration.

4. Profiling Must Guide Optimisation

Performance work should focus on measured bottlenecks rather than assumptions about which component is slow.

5. Correctness Is as Important as Speed

GPU output should be compared against a trusted reference implementation before performance improvements are accepted.

6. Custom PyTorch Extensions Offer Flexibility at the Cost of Additional Complexity

Device handling, tensor shapes, data types, compilation, numerical validation, and fallback behaviour all require careful attention.

Frequently Asked Questions

What is MSCRED?

MSCRED stands for Multi-Scale Convolutional Recurrent Encoder-Decoder.

It is an architecture designed to model relationships between multiple time-series signals and detect abnormal behaviour through reconstruction error.

What does `im2col` do?

im2col rearranges overlapping convolution windows into matrix columns, allowing convolution to be expressed as matrix multiplication.

Why did shared memory help?

Shared memory allowed threads in the same block to reuse data without repeatedly loading it from slower global memory. This reduced memory-access overhead for reusable tiles.

Did GPU memory optimisation improve the F1 score?

That was not the purpose of the optimisation, and this article does not claim that it did.

The best recorded F1 score describes anomaly-detection quality in the thesis experiments. The CPU and GPU timing results describe computational performance. They should be interpreted separately.

Why was the encoder speed-up much larger than the total speed-up?

After the encoder became faster, ConvLSTM represented a larger share of total runtime. The overall system was therefore limited by the remaining bottleneck.

Is a Custom CUDA Kernel Always Faster Than PyTorch Operations?

No.

Optimised PyTorch and CUDA libraries are already highly efficient for many operations. A custom kernel is most useful when:

The workload has a specialised access pattern
Existing operations create unnecessary overhead
Profiling identifies a clear bottleneck
Correctness and maintenance costs are justified

Can the Original Industrial Dataset Be Shared?

No.

The original dataset contains industrial process information. A public reproduction would require synthetic or appropriately anonymised multivariate time-series data.

Are These Benchmark Results Universal?

No.

Performance depends on GPU architecture, CPU configuration, tensor dimensions, batch size, kernel implementation, memory layout, CUDA and PyTorch versions, compiler settings, and workload characteristics.

The reported numbers apply to the specific experimental setup described in this article.

Limitations and Data-Privacy Considerations

The reported results were obtained on a specific hardware configuration and workload.

Different GPUs, tensor dimensions, batch sizes, memory layouts, CUDA versions, and kernel configurations may produce different results.

The original dataset cannot be published because it contains industrial process information. A public reproduction would therefore require synthetic or appropriately anonymised multivariate time-series data.

This article presents aggregate benchmark measurements and technical methodology rather than the complete industrial dataset or restricted implementation.

Future Work

Future work could focus on:

Profiling and optimising the ConvLSTM component
Reducing intermediate memory allocation
Reducing host-to-device and device-to-device memory movement
Comparing the custom implementation with optimised CUDA libraries
Evaluating additional tile and thread-block configurations
Testing across different GPU architectures
Reproducing the workflow using synthetic public data
Adding automated correctness and regression tests
Measuring kernel-level execution using GPU profiling tools

Conclusion

This work was completed as part of my M.Tech thesis research at the Indian Institute of Technology Bhilai.

It demonstrated how low-level GPU optimisation can be integrated into a deep-learning anomaly-detection pipeline.

The convolutional encoder benefited substantially from:

CUDA parallelisation
im2col-based lowering
GEMM
Shared-memory data reuse
Custom PyTorch integration

Encoder runtime decreased from 300 seconds to 23.664 seconds in the measured workload.

The broader result also illustrated an important systems principle: once one stage is accelerated, another stage may become the new performance bottleneck.

In this case, ConvLSTM limited the end-to-end improvement even after the encoder achieved a substantial speed-up.

The work reinforced that successful performance engineering requires both low-level optimisation and system-level measurement.

References

Zhang, C., Song, D., Chen, Y., Feng, X., Lumezanu, C., Cheng, W., Ni, J., Zong, B., Chen, H., and Chawla, N. V. (2019). A Deep Neural Network for Unsupervised Anomaly Detection and Diagnosis in Multivariate Time Series Data. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01), 1409–1416. https://doi.org/10.1609/aaai.v33i01.33011409
Shi, X., Chen, Z., Wang, H., Yeung, D.-Y., Wong, W.-K., and Woo, W.-C. (2015). Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting. Advances in Neural Information Processing Systems 28, 802–810. https://proceedings.neurips.cc/paper_files/paper/2015/hash/07563a3fe3bbe7e3ba84431ad9d055af-Abstract.html
NVIDIA. CUDA C++ Programming Guide. https://docs.nvidia.com/cuda/cuda-c-programming-guide/
PyTorch. Custom C++ and CUDA Operators. https://docs.pytorch.org/tutorials/advanced/cpp_custom_ops.html

Citation and Feedback

If this article informs academic or technical work, please cite the original MSCRED and ConvLSTM papers above.

To refer specifically to my thesis implementation, optimisation process, or reported benchmarks, you may cite this article as:

Vutnoor, R. (2026). “GPU-Accelerating MSCRED with CUDA, im2col, GEMM, and a Custom PyTorch Extension.” DEV Community.

If you found the article useful, a reaction, comment, or constructive technical question on DEV would be appreciated.

I am Ranjith Vutnoor, an AI/ML Software Engineer and IIT Bhilai alumnus working on production RAG systems, LLM evaluation, model fine-tuning, PyTorch, and GPU-accelerated machine learning.

Stop Recreating Database Connections: An Illustrated Guide to Connection Pooling

Fri, 11 Sep 2026 02:43:51 +0000

If you have ever built an application that talks to a database, you have used a database connection. A database is simply a structured storage system for digital information, and a connection is the secure, active pipeline established between your application's code and that database.

However, constantly opening and closing these connections for every single user request is incredibly slow and resource-heavy. To solve this, developers use a technique called connection pooling. Connection pooling is a performance optimization method where a group (or "pool") of database connections is kept open and idle, waiting to be reused by whoever needs them, rather than being destroyed and recreated on demand.

The Analogy: The Pizza Delivery Fleet

To understand why connection pooling is so vital, imagine you run a busy, high-volume pizza restaurant. Every time an order comes in, you need to deliver it to a customer.

Imagine if, for every single delivery, you had to hire a brand-new driver, run a background check, sign a legal employment contract, set up their vehicle registration, and purchase a GPS. Once that single delivery was complete, you immediately fired them, canceled their insurance, and sold the car. It sounds absurd, right? The administrative overhead of hiring and firing would take three times longer than the actual delivery, and your restaurant would go bankrupt in a week.

Instead, you hire a permanent, dedicated fleet of five delivery drivers. They sit in the break room, fully certified and ready. When a pizza is boxed, the first available driver grabs the box, drives to the house, delivers it, and returns to wait in the break room for the next order.

In this analogy, the drivers are the "connection pool," the deliveries are your database queries (requests for information), and the hiring process is the network handshake required to open a new database connection.

Why It Matters in Daily Software Engineering

In real-world software, opening a database connection is highly expensive in terms of computing power. The application must perform a security handshake, authenticate its username and password, allocate memory on the database server, and establish a network route. This process can easily take 50 to 100 milliseconds.

If your website experiences a surge in traffic—say, 1,000 users clicking a button at the exact same moment—your server would try to open 1,000 separate connections simultaneously. The database server would quickly run out of memory, slow down to a crawl, and eventually crash, displaying a "database connection error" to your users.

Connection pooling prevents this complete system collapse. It sets a strict limit on the maximum number of connections allowed (e.g., capping it at 20). If 1,000 users arrive, they politely share those 20 open connections. Because each connection is already established, queries execute in 1 to 2 milliseconds. Once a query is done, the connection is instantly recycled for the next user in line, protecting your database from crashing.

A Simple Code Example in Node.js

Below is a simple JavaScript example using a popular PostgreSQL database driver. It demonstrates how we initialize a pool and query the database without manually opening and closing connections.

const { Pool } = require('pg');

// 1. Create a pool with a maximum capacity of 10 connections
const pool = new Pool({
  user: 'db_user',
  host: 'database.server.com',
  database: 'mydb',
  password: 'securepassword',
  port: 5432,
  max: 10, // Maximum connections in the pool
});

async function fetchUserData(userId) {
  // 2. Grab an already-open connection from the pool
  const client = await pool.connect();
  
  try {
    // 3. Execute the database query
    const result = await client.query('SELECT * FROM users WHERE id = $1', [userId]);
    return result.rows[0];
  } catch (err) {
    console.error('Database query error:', err);
  } finally {
    // 4. Crucial: Release the connection back to the pool
    // This does NOT close the connection; it just makes it available for others!
    client.release();
  }
}

The Takeaway

Connection pooling turns a heavy, repetitive administrative chore into a fast, shared utility. By keeping a smart, limited buffer of active connections ready to work, software systems can handle tens of thousands of requests smoothly, ensuring high-speed page loads while protecting underlying servers from sudden traffic spikes.

Resources

GitHub Repository: react-hook-lab
react-hook-lab: npm package
Connect with me on LinkedIn: Saurav Pandey

Originally published on my blog. You can read the alternative breakdown here.

Speed Up Your Queries: A Plain-English Guide to Database Indexing

Fri, 11 Sep 2026 02:43:19 +0000

What is Database Indexing?

Database indexing is a technique used by software systems to speed up the retrieval of data from a database table. By creating a separate, organized pointer structure, the database can locate specific records without having to search through every single row of data. Think of it as a specialized shortcut map for your digital filing cabinet.

The Library Card Catalog Analogy

Imagine walking into a massive metropolitan library containing 100,000 books. If you want to find a book titled "The Midnight Mystery" and there is no catalog or organization system, you would have to start at the first shelf in the corner and look at every single book cover one by one until you find it. This slow, painful process is what a database does during a "full table scan."

Now, imagine the library has an alphabetical card catalog. You walk straight to the drawer marked "M," find the index card for "The Midnight Mystery," and read the card. It tells you exactly which aisle, shelf, and position the book is on. You walk directly there and grab it in seconds. In this scenario, the card catalog is the index, the books are the rows of data in your table, and the physical location is the memory address of the database record.

Why It Matters Daily in Tech

In the tech industry, engineers use database indexing to prevent application lag and catastrophic server crashes. Without indexes, as a company's database grows from thousands of rows to millions, simple daily actions like logging in, searching for an item, or pulling up a user profile would take seconds or even minutes instead of milliseconds.

Slow database queries hog the system's central processing unit (CPU) and memory, causing other requests to back up and eventually freeze the entire application. By adding appropriate indexes, developers keep application programming interfaces (APIs) lightning-fast, reduce cloud hosting bills (because servers do not have to work as hard), and ensure a smooth user experience. However, engineers must find a balance: every index takes up storage space and slows down write operations (like adding or updating data), because the database has to update both the main table and the index every single time a change is made.

How It Works in Practice

Here is a simple example in SQL (Structured Query Language). Imagine a system trying to look up a user by their email address.

-- Without an index, the database must scan every row in the users table
SELECT * FROM users WHERE email = 'developer@example.com';

-- To speed this query up, we create an index on the email column
CREATE INDEX idx_users_email ON users(email);

-- Now, the database uses the index to jump directly to the correct user
SELECT * FROM users WHERE email = 'developer@example.com';

The Takeaway

Database indexing is the ultimate balancing act of software performance optimization. It turns grueling, resource-heavy data searches into near-instant lookups, keeping applications snappy as they scale up to handle millions of users. Just remember that indexes are not free; treat them like directory signs in a physical store—use them where visitors frequently get lost, but do not clutter every wall with them, or you will slow down the inventory restocking process.

Resources

GitHub Repository: react-hook-lab
react-hook-lab: npm package
Connect with me on LinkedIn: Saurav Pandey

Originally published on my blog. You can read the alternative breakdown here.

Coder Legion - Recent posts tagged performance

What Is Idempotency?

🔍 What Is Idempotency?

🛠️ How Do We Implement It in Ruby on Rails?

💎 Why Is Idempotency Important?

⚠️ One Important Detail

RubyOnRails #API #Idempotency #BackendDevelopment #SoftwareArchitecture #DistributedSystems #WebDevelopment #SoftwareEngineering

Redux Refactoring Playbook: From Action-Types Soup to RTK Query — Without Rewriting the App

Rule 1 — Strangle, don't rewrite

Rule 2 — Selectors are the API. Freeze them first.

Rule 3 — Kill loading flags with RTK Query, not with better thunks

Rule 4 — Polling moves from timers to tags

What stayed in Redux core

The honest ledger

What I'd do differently

Your refactoring checklist

You Upgraded to Next.js 15 and Your Site Got Slower. The Caching Defaults Flipped

What Changed

Why It Is Easy to Miss

How to See It in 30 Seconds

Fix 1: Cache One Request on Purpose

Fix 2: Cache With an Expiry

Fix 3: Set It Once for the Whole Page

Fix 4: Route Handlers Need Their Own Opt-In

The Part People Forget: Your Database Was Never Covered

Should You Cache Everything Again?

The Rule

Seventeen Seconds to Parse One Name: A Security Scan Across Eleven PHP Projects

What the scans found, and what is fixed

Bugs that needed no attacker

Bugs a server, a file, or a document could trigger

The other high and medium findings

memory_limit stopped stopping things

The re-release day

Refusing vulnerable PHP releases

pdo_duckdb 0.9.0: DuckDB 2.0

Performance, with the hardware attached

Breaking changes worth reading before you upgrade

What a scan is worth

Mastering React Performance: Optimizations That Actually Matter 🚀

Introduction

1️⃣ Understanding React’s Rendering Behavior

2️⃣ Optimize Component Re-Renders with React.memo

3️⃣ Minimize Re-Renders with Derived State

4️⃣ Optimize Function Props with useCallback

5️⃣ Optimize Expensive Computations with useMemo

6️⃣ Optimize Lists & Large Data Rendering

7️⃣ Reduce Bundle Size with Code Splitting & Lazy Loading

Conclusion: Build Fast & Scalable React Apps

Finding the slow part of a WooCommerce page with Query Monitor

Install it on staging first

Read the queries by component

Check outgoing HTTP calls

Time your own code

Look at a page as a guest

When you are done

Shipping a 4K Wallpaper App Without Running Out of Memory: ImageIO Downsampling and Core Image

1. The memory maths everyone forgets

2. Downsampling with ImageIO

3. A grid that stays smooth

4. The detail screen: full resolution, exactly once

5. Create mode: blur that keeps clean edges

6. Saving with the minimum permission

7. The honest part: you can't set the wallpaper for the user

8. Real-world example: Wallpapers HD: Exotic 4K

9. Before you ship an image-heavy app: a checklist

Wrapping up

PS4 Pro Belize DDR3 patch v2

Image SEO in 2026: Alt Text, Modern Formats and Fast Loading

Key takeaways

Why Does a Website Become Slow, and How Can We Fix It?

The Problem: A Website That Loads Too Slowly

Step 1: Find the Real Cause

Step 2: Optimize Images and Static Files

Step 3: Improve Database Queries

Step 4: Use Caching Where It Makes Sense

Step 5: Reduce Unnecessary JavaScript

Step 6: Test the Changes

What If the Website Is Still Slow?

My Main Takeaway

2️⃣ Optimize Component Re-Renders with `React.memo`

4️⃣ Optimize Function Props with `useCallback`

5️⃣ Optimize Expensive Computations with `useMemo`

3. `.pth` import lines become `.start` files: auditable, not safe

Option A — `data-value` + CSS `::after`

2. `useMemo`: Cache an Expensive Calculation

3. `useCallback`: Cache a Function Reference

4. `React.memo`: Skip Unnecessary Component Renders