Coder Legion - Recent posts tagged nodejs

Building a static blog with Sitelo, one function per page

Fri, 30 Oct 2026 14:35:56 +0000

In my last post on DEV I explained why I built Sitelo: most websites are documents, and documents shouldn't need a JavaScript application to reach the browser. That post was the argument. This one is the practice.

We'll build a small blog from an empty folder: a home page, post pages generated from JSON files, and a comments section that stays fresh on an otherwise static page. Along the way I'll point out what Sitelo is doing at each step, and what it deliberately doesn't do.

The whole framework fits in one sentence, and it's worth keeping in mind as we go:

Files are routes. Functions return HTML. Browser code is explicit.

Everything below is that sentence, applied.

Step 1: one page is one function

Start with an empty folder and install Sitelo. It needs Node 20.19+ or 22.12+, and Vite is bundled, so there's nothing else to install or configure.

npm init -y
npm pkg set type=module
npm install -D sitelo

Add two scripts to package.json:

"scripts": {
  "dev": "sitelo",
  "build": "sitelo build"
}

Now the first page. Any file in src/ ending in .ht.js is a page, and its default export returns HTML:

// src/index.ht.js
export default () => `
  <html lang="en">
    <head>
      <title>My blog</title>
      <link rel="stylesheet" href="/styles.css">
    </head>
    <body>
      <h1>Hello world</h1>
    </body>
  </html>
`

Create a src/styles.css with a few rules, then run npm run build and you get:

dist/
  index.html   ← your HTML, with <!DOCTYPE html> added
  styles.css   ← bundled and minified, because the page linked it
  404.html     ← a default one, until you write src/404.ht.js

The function ran once, at build time, in Node. The browser gets the string it returned and nothing else: no runtime, no hydration, no bundle of page code. If you want JavaScript in the browser, you add a <script> tag, the same as you would in a hand-written HTML file.

Template literals aren't the only option. A page can also return JSX from a .ht.jsx or .ht.tsx file, which is rendered to a static string with react-dom/server. Or you can use my companion library javascript-to-html, where every element is a function. I'll stick with template literals here because they need no explanation.

Step 2: a layout is just another function

A blog needs the same <head> and header on every page. There's no layout system to learn for this, because a layout is a function that takes some HTML and wraps it:

// src/lib/layout.js
export const layout = ({ title, children }) => `
  <html lang="en">
    <head>
      <meta charset="utf-8">
      <meta name="viewport" content="width=device-width, initial-scale=1">
      <title>${title} · My blog</title>
      <link rel="stylesheet" href="/styles.css">
    </head>
    <body>
      <header><a href="/">My blog</a> · <a href="/blog">Posts</a></header>
      <main>${children}</main>
    </body>
  </html>
`

// src/index.ht.js
import { layout } from './lib/layout.js'

export default () => layout({
  title: 'Home',
  children: '<h1>Hello world</h1><p>Welcome to my blog.</p>',
})

layout.js doesn't end in .ht.js, so it isn't a page. It also never ends up in dist/. Sitelo only emits a JavaScript or TypeScript file when your HTML references it with a <script src>; a module you only import from page code stays on the build side. That rule matters more later, when the helpers you import hold API keys or database code. They can't leak into the browser by accident.

The same rule works in the other direction. Write <script type="module" src="/main.ts"></script> and Sitelo bundles src/main.ts with esbuild, emits main.js, and rewrites the tag to match. Browser code exists only where you asked for it.

Step 3: posts from JSON files

Now for the posts. I'll keep them in the repo as JSON, one file per post, outside src/ because they're content rather than code:

// data/posts/hello-world.json
{
  "title": "Hello, world",
  "date": "2026-10-01",
  "body": "<p>My first post.</p>"
}

The post page lives at src/blog/[slug].ht.js. The square brackets make it a dynamic route, and it exports two more functions next to the page itself:

generateStaticParams() lists the pages to build, one per post.
data() loads what one page needs. Whatever it returns arrives in the page function as data.

// src/blog/[slug].ht.js
import { readJsonCollection } from 'sitelo/data'
import { layout } from '../lib/layout.js'

const posts = () => readJsonCollection('data/posts', { sort: '-date' })

export async function generateStaticParams() {
  return (await posts()).map((post) => ({ slug: post.slug }))
}

export async function data({ params }) {
  return (await posts()).find((post) => post.slug === params.slug)
}

export default ({ data: post }) => layout({
  title: post.title,
  children: `
    <article>
      <h1>${post.title}</h1>
      <time datetime="${post.date}">${post.date}</time>
      ${post.body}
    </article>
  `,
})

readJsonCollection reads every .json file in the folder and takes each slug from its file name. It parses each file once per build, however many pages ask for it, and two posts with the same slug fail the build instead of one silently overwriting the other.

The index page is the same idea without the brackets:

// src/blog/index.ht.js
import { readJsonCollection } from 'sitelo/data'
import { layout } from '../lib/layout.js'

export const data = () => readJsonCollection('data/posts', { sort: '-date' })

export default ({ data: posts }) => layout({
  title: 'Posts',
  children: `
    <h1>Posts</h1>
    <ul>
      ${posts.map((post) => `<li><a href="/blog/${post.slug}">${post.title}</a></li>`).join('')}
    </ul>
  `,
})

A build now writes dist/blog/index.html and one dist/blog/<slug>/index.html per post. In the dev server (npm run dev) pages render on request through Vite, so editing a JSON file reloads the browser, and an error points at the exact line of your page in the terminal.

If your posts live in a CMS instead, data() can fetch them. Sitelo exports fetchWithCache, a drop-in for fetch that caches responses during the build. A 500-page site then doesn't make 500 identical requests to your API.

Step 4: fresh comments on a static page

Comments are the classic reason static blogs reach for a client-side widget or give up and go server-rendered. Sitelo's answer is a server island: the page stays static, and one marked region is rendered on the server when someone views it.

An island is a module in src/islands/ whose default export returns HTML. Islands are fragments rather than pages, so the file is plain .js, not .ht.js:

// src/islands/comments.js
import { getComments } from '../lib/comments.js'

const escape = (s) => String(s).replace(/[&<>"']/g, (c) => `&#${c.charCodeAt(0)};`)

export default async function comments({ props }) {
  const list = await getComments(props.postId)
  if (!list.length) return '<p>No comments yet.</p>'
  return `<ul>${list.map((c) => `<li><b>${escape(c.author)}</b> ${escape(c.text)}</li>`).join('')}</ul>`
}

getComments is whatever your comment store is: a database query, a call to a hosted service. Comments are user input, hence the escape.

Place the island in the post page with island(). The third argument is the fallback HTML that ships in the static build, and when: 'visible' waits until the reader scrolls near the comments before making the request:

// src/blog/[slug].ht.js (the page function)
import { island } from 'sitelo/islands'

export default ({ params, data: post }) => layout({
  title: post.title,
  children: `
    <article>…</article>
    ${island('comments', { postId: params.slug }, '<p>Loading comments…</p>', { when: 'visible' })}
    <script type="module" src="/islands.js"></script>
  `,
})

The script tag loads a two-line client that finds the placeholders and fetches them:

// src/islands.js
import { mountIslands } from 'sitelo/islands/client'

mountIslands()

In npm run dev and sitelo preview this already works: Sitelo serves islands at /_sitelo/islands/<name>. In production your static host serves the pages, and you mount a small handler wherever you can run server code. createIslandsHandler from sitelo/islands/server takes a Web Request and returns a Response, so it fits a serverless or edge function. The repo has Netlify and Vercel versions.

A few details I cared about getting right:

Props are untrusted. They're embedded in the page and sent back on the request, so anyone can edit them. Set SITELO_ISLANDS_SECRET and the build signs every placeholder with HMAC-SHA256, and the handler rejects edited props with a 403.
Islands are server-only. The rule from Step 2 applies: nothing references comments.js from HTML, so it never reaches dist/, and neither does your database code.
Failure is graceful. If the island endpoint is down or slow (10 seconds by default), the fallback HTML stays where it was. The page still works.

Step 5: let the build check your work

The blog works. The last step is a config file, and it's where Sitelo earns its keep on a real site:

// sitelo.config.js
export default {
  site: 'https://myblog.example',
  rss: {
    site: 'https://myblog.example',
    title: 'My blog',
    description: 'Latest posts',
    routePrefix: '/blog',
  },
  images: true,
  linkCheck: 'error',
  lighthouse: {
    thresholds: { accessibility: 100, 'best-practices': 95, seo: 100 },
  },
}

Here's what each line buys you:

site generates dist/sitemap.xml from every route, including the ones generateStaticParams produced.
rss writes dist/rss.xml with an item for every page under /blog.
images turns each <img> into a resized, WebP srcset with width and height set, so a 3000px hero photo no longer ships to phones. It needs sharp installed, and it works in the dev server too, so what you see is what you ship.
linkCheck: 'error' resolves every internal <a href> against the files the build actually wrote, the way a static host would. A typo like /blgo fails the build with the page it's on.
lighthouse gives sitelo lighthouse thresholds to enforce. Any score under its threshold exits non-zero, so it can gate CI. Like sharp, Lighthouse is an optional install.

I left performance out of the thresholds on purpose. It depends on the machine running the audit, and a page that scores 100 on a laptop can score around 60 on a shared CI runner. Accessibility, best practices and SEO are properties of the HTML itself, so they score the same anywhere.

One check needs no config. If a page links a script or stylesheet that doesn't exist in src/ or public/, the build fails and lists the paths it tried. That's why Step 1 asked you to create styles.css first.

Every build ends with a report of what you're shipping: file counts and sizes per type, the largest files, and how long each phase took. On a static site, "how much JavaScript am I sending?" should be a number you see every time, not something you go looking for.

Building it with a coding agent

I designed Sitelo with coding agents in mind, and the small surface area matters even more for them than for people. Ask an agent to add a page to an unfamiliar framework and it reaches for whatever it has seen most: React components, Next.js conventions, Astro frontmatter. With Sitelo there's less to get wrong, because there's less of everything. A page is a function, a route is a file, and browser code is a <script> tag.

To close the remaining gap, Sitelo ships two Agent Skills: one for the framework (pages, routing, data, islands, deployment) and one for its component library. Install them into your project and Claude Code, Cursor, Codex and other agents load them when a task needs them:

npx skills add Paul-Browne/sitelo

The skills also ship inside the npm package, matching the version you installed. Sitelo's test suite checks every API the skills mention against the package's real exports, so a renamed or removed API fails the tests before it reaches a release. For agents that prefer to read a URL, the docs publish sitelo.dev/llms.txt, plus a single-file llms-full.txt with every guide in Markdown.

The build checks from Step 5 help here too. An agent that invents a stylesheet path or links to a page that doesn't exist gets a failed build naming the problem, not a site that quietly ships broken.

What we built, and when not to use it

In five steps we went from an empty folder to a blog with:

a home page, a post index and one page per JSON file
comments rendered on the server at view time, on otherwise static pages
a sitemap, an RSS feed and responsive images
a build that fails on broken links and missing assets, plus a Lighthouse gate for accessibility and SEO

The browser receives HTML, one stylesheet and a 1.7 kB island loader. That's all, because those are the only things we asked for.

Sitelo isn't the right tool for everything. JSX pages render to static HTML only, so onClick handlers and hooks do nothing in the browser. If you're building an application, with client-side state and routing, use an application framework. Astro, Next.js and Eleventy are all excellent at what they do. Sitelo is for docs, blogs, marketing sites and content sites, where "view source" should show what you wrote.

If that sounds like your next project:

Docs: sitelo.dev/docs
Code (MIT licensed): github.com/Paul-Browne/sitelo
Package: npmjs.com/package/sitelo

The repo includes a complete blog example that goes further than this post, with Markdown posts. And if you find something that makes Sitelo more complicated than it needs to be, please open an issue. Keeping it boring is the whole point.

Demystifying the Facade Pattern in Node.js: Clean APIs Made Simple

Wed, 28 Oct 2026 10:10:50 +0000

What Is the Facade Pattern?

The facade pattern is a structural software design pattern that provides an uncomplicated, unified interface in front of a complex collection of classes, libraries, or APIs. Instead of forcing your application code to interact with dozens of intricate internal functions, the facade exposes a single, high-level method that coordinates the messy work behind the scenes. In short, it gives a sprawling, complicated subsystem a clean and welcoming front door.

The Real-World Analogy: The Home Theater Remote

Imagine you want to watch a movie on a high-end home entertainment setup. To make that happen naturally, you need to execute half a dozen distinct tasks:

Turn on the television.
Set the video input to HDMI 2.
Turn on the surround sound receiver.
Adjust the audio amplifier to optical audio.
Dim the smart living room lights.
Power on the media streaming console.

You could juggle four different remote controls and flip wall switches yourself, learning the exact quirks of each manufacturer. Alternatively, you could pick up a modern universal remote and press a single button: Watch Movie.

That universal remote is a facade. It does not replace the television, receiver, or lights; they still exist with all their specialized knobs and settings. Instead, the remote hides the tedious orchestration behind one single, convenient touchpoint.

Why It Matters in Daily Engineering

In everyday software development, projects tend to grow through accretion. A feature starts out simple, but soon requires five external dependencies: an authentication provider, a payment gateway, a cloud storage bucket, an email delivery service, and an internal logging pipeline.

Without a facade, engineers frequently write "glue code" directly inside their HTTP request handlers (controllers). If five different routes need to charge a customer, each route handler winds up importing and configuring the payment library, checking inventory, calculating sales tax, and firing off analytics calls. This creates two immediate crises:

High Coupling: If the payment provider updates its software development kit (SDK) or changes its function signatures, you must find and update twenty different files across your project.
Cognitive Overload: Engineers reviewing or writing features are forced to read through hundreds of lines of low-level plumbing when they only care about whether an order succeeded.

By placing a facade between your core business logic and external subsystems, engineers prevent third-party updates from leaking across the entire codebase. When an underlying vendor changes, you update one facade file, leaving the rest of the application completely untouched.

Practical Implementation in Node.js

Consider an e-commerce checkout flow in an Express and Node.js application. Completing an order touches an inventory service, a payment processor, and a notification service.

Without a facade, your Express route handler would have to instantiate and coordinate every service manually. Here is how you encapsulate that complexity using a Facade:

// Subsystem 1: Inventory Management
class InventoryService {
  checkStock(productId) {
    // Simulate querying database stock levels
    console.log(`Checking stock for item: ${productId}`);
    return true;
  }
  reserve(productId, quantity) {
    console.log(`Reserved ${quantity} unit(s) of ${productId}`);
  }
}

// Subsystem 2: Payment Processing
class PaymentProcessor {
  charge(amount, paymentToken) {
    console.log(`Charging $${amount} using token: ${paymentToken}`);
    return { status: 'success', transactionId: 'txn_987654' };
  }
}

// Subsystem 3: Customer Notifications
class NotificationService {
  sendReceipt(email, transactionId) {
    console.log(`Receipt sent to ${email} for transaction ${transactionId}`);
  }
}

// The Facade: OrderFacade
class OrderFacade {
  constructor() {
    this.inventory = new InventoryService();
    this.payment = new PaymentProcessor();
    this.notifications = new NotificationService();
  }

  processOrder({ productId, quantity, price, paymentToken, customerEmail }) {
    const isAvailable = this.inventory.checkStock(productId);
    if (!isAvailable) {
      throw new Error('Product is out of stock.');
    }

    this.inventory.reserve(productId, quantity);
    const paymentResult = this.payment.charge(price * quantity, paymentToken);
    this.notifications.sendReceipt(customerEmail, paymentResult.transactionId);

    return { success: true, transactionId: paymentResult.transactionId };
  }
}

// Express Route Usage
const express = require('express');
const app = express();
app.use(express.json());

const orderFacade = new OrderFacade();

app.post('/api/checkout', (req, res) => {
  try {
    const orderConfirmation = orderFacade.processOrder(req.body);
    return res.status(200).json(orderConfirmation);
  } catch (error) {
    return res.status(400).json({ error: error.message });
  }
});

The route handler does not know or care how inventory reservation works or what parameters the payment gateway demands. It invokes processOrder() and receives a clear outcome.

Key Takeaway

A facade is not an abstraction that conceals how a system works to make it private; it is an ergonomic shield that keeps your application's day-to-day feature code clean, readable, and insulated from volatile third-party tools.

Enjoyed this breakdown? Support my open-source work on Buy Me a Coffee or explore react-hook-lab!

Originally published on my blog. You can read the alternative breakdown here.

Understanding Database Replication: How to Keep Your Apps Fast and Reliable

Tue, 27 Oct 2026 09:21:52 +0000

What is Database Replication?

Database replication is the automated process of keeping multiple exact copies of a database synchronized across different server instances. When data is added, updated, or removed in the primary database, those changes are copied to one or more replica databases in near real-time. This setup ensures that if one database fails or experiences high traffic, the overall application remains operational, responsive, and secure against data loss.

A Real-World Analogy: The Restaurant Kitchen Whiteboard

Imagine a high-volume restaurant kitchen with a single Head Chef and five Line Cooks. Every time a new order arrives or a special menu item changes, the Head Chef writes it down on a master whiteboard in the center of the kitchen.

If all five Line Cooks had to walk up to that single whiteboard every few seconds to read the latest order, a massive bottleneck would form around the Head Chef. Cooking would grind to a halt because everyone is waiting in line just to read information.

To solve this, the kitchen hires a runner whose sole job is to copy whatever the Head Chef writes on the master whiteboard onto five individual clipboards stationed at each Line Cook's workstation. The Head Chef remains the only person writing new orders (the Primary database), while the Line Cooks read from their dedicated station clipboards (the Read Replicas). Orders flow instantly, cooks stay productive, and the head chef is never overwhelmed.

Why Database Replication Matters in Modern Engineering

In real-world web applications, user activity is overwhelmingly biased toward reading data rather than writing it. For example, millions of people scroll through social media posts or view e-commerce products for every one person who creates a new post or completes a purchase. If every read request hit the exact same database server handling write operations, that single database would quickly exhaust its memory and processing power.

Engineers use database replication to solve several critical challenges:

High Availability and Fault Tolerance: A hardware failure or crash on the primary database server does not bring down the entire website. System administrators can quickly promote a replica to become the new primary writer with minimal downtime.
Performance Scaling: Read traffic is distributed across multiple read-only replica servers. This practice, known as load balancing, prevents any single server from becoming a CPU bottleneck.
Heavy Reporting Offloading: Running complex analytical queries (like generating monthly revenue reports) takes immense computing resources. Performing these queries on a replica ensures transactional write operations for live customers remain fast and unaffected.

Implementing Read/Write Splitting in Node.js

To take advantage of database replication, backend application code must route write commands (INSERT, UPDATE, DELETE) to the primary database, while directing read queries (SELECT) to replica nodes. A common database driver used in Node.js applications is mysql2.

Here is how you can set up dual connections in a Node.js module to execute read/write splitting:

// db.js - Database connection module for Primary and Replica instances
const mysql = require('mysql2/promise');

// Primary connection reserved strictly for write operations
const primaryPool = mysql.createPool({
  host: process.env.DB_PRIMARY_HOST || 'primary.db.example.com',
  user: 'admin',
  password: 'securepassword',
  database: 'store_db',
  connectionLimit: 10
});

// Replica connection pool used for read-only queries
const replicaPool = mysql.createPool({
  host: process.env.DB_REPLICA_HOST || 'replica-1.db.example.com',
  user: 'read_only_user',
  password: 'readpassword',
  database: 'store_db',
  connectionLimit: 20
});

// Function to execute write operations on the primary DB
async function executeWrite(sqlQuery, params) {
  const [result] = await primaryPool.execute(sqlQuery, params);
  return result;
}

// Function to execute read operations on the replica DB
async function executeRead(sqlQuery, params) {
  const [rows] = await replicaPool.execute(sqlQuery, params);
  return rows;
}

module.exports = {
  executeWrite,
  executeRead
};

In this example, calling executeWrite() executes the SQL query on the primary database node, ensuring that structural data modifications land safely on the source authority. Calling executeRead() queries the replica node, taking performance stress off the writer server.

Key Takeaway

Database replication shifts application architecture from a fragile single point of failure into a redundant, horizontally scalable infrastructure. By separating write transactions from high-volume read requests, development teams guarantee that their software stays online and lightning-fast, even during massive user traffic surges.

Enjoyed this breakdown? Support my open-source work on Buy Me a Coffee or explore react-hook-lab!

Originally published on my blog. You can read the alternative breakdown here.

Stop Inventing Custom API Errors: A Guide to RFC 7807 Problem Details

Mon, 26 Oct 2026 17:41:23 +0000

RFC 7807 Problem Details is an official technical specification that defines a standard JSON format for describing errors returned by HTTP web APIs. Instead of having every software engineer create their own custom fields like message, err_code, or reason, this standard establishes predictable key names such as type, title, status, detail, and instance. This gives both computer programs and human developers a uniform way to understand exactly what went wrong during an API request.

The Everyday Analogy: Car Diagnostic Reports

Imagine taking your car to a mechanic when the check engine light comes on.

Option A: The mechanic hands you a piece of paper that simply says: "Car is broken. Cost: $300."

Option B: The mechanic hands you a standardized vehicle diagnostic report that lists the exact sensor code (e.g., P0300), a short title ("Random Cylinder Misfire"), the severity level, specific details ("Cylinder 3 failed 14 times in the past 10 miles"), and a reference link to the manufacturer's repair guide.

Option A leaves you guessing and forces you to ask ten follow-up questions. Option B gives you instant, structured clarity. RFC 7807 acts as that standardized diagnostic report for your web applications.

Why RFC 7807 Matters for Engineers

In modern software engineering, web applications are built by connecting multiple services together. Front-end user interfaces (like React apps) communicate with back-end servers running Express.js, which in turn communicate with databases or external payment systems.

When something fails—such as an invalid credit card or an out-of-stock product—the back-end server must notify the front-end application. Without a standard like RFC 7807, team A might structure errors as { "error": "Invalid payload" }, while team B uses { "statusCode": 400, "description": "Bad input" }.

This inconsistency causes real operational pain:

Fragile UI Code: Front-end developers must write endless conditional if/else checks to parse different error formats from different endpoints.
Wasted Debugging Hours: Software engineers spend extra time reading through server logs trying to figure out what a vague error message actually means.
Poor User Experience: Applications fail silently or show generic "An unknown error occurred" popups because the front-end couldn't parse the unexpected error structure.

By enforcing RFC 7807, teams agree on a single contract. Front-end apps can use one central error-handling mechanism to extract the exact field that failed and display a meaningful message to the user.

Code Example: Express.js Error Handler

Here is how you can implement an RFC 7807 compliant error handler in a Node.js and Express server. The custom HTTP content type for this standard is application/problem+json.

const express = require('express');
const app = express();

app.use(express.json());

// Route that triggers a validation error
app.post('/api/users', (req, res, next) => {
  const { email } = req.body;
  if (!email) {
    const error = new Error('Email field is required.');
    error.status = 400;
    error.type = 'https://api.example.com/errors/missing-field';
    error.title = 'Invalid Request Payload';
    error.instance = '/api/users';
    return next(error);
  }
  res.status(201).json({ message: 'User created' });
});

// Central RFC 7807 Error Handling Middleware
app.use((err, req, res, next) => {
  const status = err.status || 500;
  
  const problemDetails = {
    type: err.type || 'https://api.example.com/errors/internal-error',
    title: err.title || 'Internal Server Error',
    status: status,
    detail: err.message || 'An unexpected error occurred on the server.',
    instance: err.instance || req.originalUrl
  };

  // Set the standard RFC 7807 header
  res.setHeader('Content-Type', 'application/problem+json');
  res.status(status).json(problemDetails);
});

app.listen(3000, () => console.log('Server running on port 3000'));

Key Takeaway

RFC 7807 transforms unpredictable API errors into predictable, structured data that both machines and humans can process seamlessly. Adopting this standard eliminates entire classes of error-handling bugs, simplifies frontend integration, and establishes clear debugging contracts across your entire software ecosystem.

Enjoyed this breakdown? Support my open-source work on Buy Me a Coffee or explore react-hook-lab!

Originally published on my blog. You can read the alternative breakdown here.

Your Pagination Is Showing Users the Same Item Twice (and Skipping Others). Here's Why Offset Breaks

Mon, 26 Oct 2026 11:11:24 +0000

A user scrolls through your feed. Page 1 ends with "Post 20". They tap "Next", and page 2 starts with... "Post 20" again.

Another user pages through an admin table and never sees order #4817. It exists. It is in the database. It just never showed up on any page.

Nobody changed any code. The data changed underneath the pagination, and your skip and limit could not cope.

The Code Everyone Writes

app.get('/api/posts', async (req, res) => {
  const page = Number(req.query.page) || 1;
  const limit = 20;

  const posts = await Post.find()
    .sort({ createdAt: -1 })
    .skip((page - 1) * limit)
    .limit(limit);

  res.json(posts);
});

It is simple, it works in development, and ?page=2 is easy to understand. That is why it is everywhere.

Why It Breaks

Offset pagination says "skip the first N rows, then give me the next 20". It counts positions, and positions move when the data does.

Duplicates. Someone publishes a new post while a reader is on page 1. Every existing post shifts down one position. When the reader requests page 2, the last item of page 1 is now the first item of page 2.

Skips. The reverse: a post is deleted while the reader is on page 1. Everything shifts up one, and the first item of what would have been page 2 slides into page 1's range. The reader never sees it.

Before:  [1 2 3 4 5 | 6 7 8 9 10]    page 1 = 1-5, page 2 = 6-10
New item arrives at the top:
After:   [0 1 2 3 4 | 5 6 7 8 9]     page 2 now starts with 5, which they already saw

The Second Problem: It Gets Slower on Deep Pages

skip(100000) does not jump to row 100,000. The database has to walk past 100,000 documents and throw them away before returning your 20. Page 1 is fast. Page 5,000 is slow, and the cost grows with the page number.

This is why a feed that feels fine with 1,000 rows slows down quietly at a million.

The Fix: Cursor Pagination

Instead of "skip N", say "give me the 20 items after this one". The anchor is a cursor: a value that identifies where the last page ended.

A new item arriving at the top cannot shift your place, because you are not counting positions. You are standing on a specific item.

Step 1: Sort by something stable, with a tie-breaker

createdAt alone is not enough, because two documents can share the same timestamp. Add _id as a tie-breaker so the order is always exact:

.sort({ createdAt: -1, _id: -1 })

Step 2: Support the query with an index

PostSchema.index({ createdAt: -1, _id: -1 });

With this index, the database jumps straight to the cursor and reads 20 documents, whether you are on page 1 or page 5,000.

Step 3: Query "after the cursor"

app.get('/api/posts', async (req, res) => {
  const limit = 20;
  const cursor = req.query.cursor ? decodeCursor(req.query.cursor) : null;

  const filter = cursor
    ? {
        $or: [
          { createdAt: { $lt: cursor.createdAt } },
          { createdAt: cursor.createdAt, _id: { $lt: cursor._id } },
        ],
      }
    : {};

  const rows = await Post.find(filter)
    .sort({ createdAt: -1, _id: -1 })
    .limit(limit + 1); // fetch one extra to know if there is a next page

  const hasMore = rows.length > limit;
  const items = hasMore ? rows.slice(0, limit) : rows;
  const last = items[items.length - 1];

  res.json({
    items,
    nextCursor: hasMore ? encodeCursor(last) : null,
  });
});

The $or says: "older than the cursor, or the same timestamp but a smaller _id". That second branch is what stops items with identical timestamps from being skipped or repeated.

Step 4: Encode the cursor so clients do not depend on it

const encodeCursor = (doc) =>
  Buffer.from(JSON.stringify({ createdAt: doc.createdAt, _id: doc._id })).toString('base64url');

const decodeCursor = (str) => {
  const { createdAt, _id } = JSON.parse(Buffer.from(str, 'base64url').toString());
  return { createdAt: new Date(createdAt), _id: new mongoose.Types.ObjectId(_id) };
};

Clients treat it as an opaque string and send it back as it is. You can change what is inside later without breaking anyone. In production, validate the decoded values and handle a malformed cursor with a 400, not a crash.

What You Give Up

Cursor pagination has a trade-off, and you should know it before switching:

No "jump to page 47". You can only go forward (or back, with a second cursor). For feeds and infinite scroll, that is perfect. For a numbered page bar, it is not.
No cheap total count. countDocuments on a large collection is not free either way, so many apps drop the "page 3 of 500" display.

When Offset Is Still Fine

Do not rewrite everything. Offset pagination is a reasonable choice for small, slow-changing tables, like an admin list with a few hundred rows, where users really do want page numbers.

Situation	Use
Feed, timeline, infinite scroll	Cursor
Large collection, deep pages	Cursor
Data changes while people browse	Cursor
Small admin table with page numbers	Offset is fine

The Rule

Offset pagination counts positions, and positions change. Cursor pagination anchors to a specific item, so it stays correct when data is added or removed. Always sort by a unique, indexed combination, and fetch one extra row to know whether there is a next page.

Quick audit: search your project for .skip(. For each hit, ask whether the collection can grow past a few thousand documents, or change while someone is paging through it.

If you work with MongoDB, I collect more patterns like this, around indexes and queries, in my MongoDB and Mongoose patterns guide, and you can browse my Next.js templates on pixelanas.com.

Get the templates: https://pixelanas.gumroad.com

Have you ever been bitten by duplicate or missing items in a paginated list? Tell me how you noticed in the comments.

Anas, full-stack Next.js developer building SaaS products and premium templates. X: @ASheikh69751

Your 'Pay Now' Button Can Charge Twice. Idempotency Keys Are the Fix Nobody Adds Until It Happens

Sun, 25 Oct 2026 11:09:21 +0000

A customer clicks "Pay now". Nothing happens for three seconds, so they click again. Or their phone loses signal for a moment and the app quietly retries the request. Or your load balancer times out and sends the request again.

Your server received the same order twice, and it did exactly what you told it to do: it created two orders.

This is not a rare edge case. It is the default behavior of every POST endpoint you have ever written.

The Endpoint That Looks Fine

app.post('/api/orders', requireAuth, async (req, res) => {
  const order = await createOrder(req.user.id, req.body);
  await chargeCard(order);
  res.status(201).json({ orderId: order.id });
});

Nothing is wrong with this code in isolation. The problem is what it cannot know: whether this request is new, or a repeat of one that already succeeded.

Why This Is Hard to Catch

You test with one request at a time. The bug needs two requests with the same intent, close together.
Timeouts are ambiguous. When a client times out, it does not know if the server never got the request, or did the work and the response got lost. Retrying is the only safe-looking choice, and it is the one that duplicates.
Disabling the button is not enough. It helps with double clicks but not with network retries, mobile apps, or another service calling your API.

The Idea: Same Key, Same Result

An idempotent operation is one where doing it twice has the same effect as doing it once. POST is not idempotent by default, but you can make it behave that way:

The client generates a unique idempotency key for each intent and sends it in a header.
The server remembers the key together with the result.
If the same key arrives again, the server returns the saved result and does not do the work again.

Payment providers such as Stripe use exactly this pattern with an Idempotency-Key header.

A Working Version With Express and MongoDB

First, a unique index, plus an expiry so old keys clean themselves up:

await keys.createIndex({ userId: 1, key: 1 }, { unique: true });
await keys.createIndex({ createdAt: 1 }, { expireAfterSeconds: 86400 });

Then the endpoint:

app.post('/api/orders', requireAuth, async (req, res) => {
  const key = req.get('Idempotency-Key');
  if (!key) return res.status(400).json({ error: 'Idempotency-Key header required' });

  const userId = req.user.id;
  const bodyHash = hash(stableStringify(req.body));

  // 1. Try to claim the key. Only one request can win this insert.
  try {
    await keys.insertOne({ userId, key, bodyHash, status: 'processing', createdAt: new Date() });
  } catch (err) {
    if (err.code !== 11000) throw err; // 11000 = duplicate key

    const saved = await keys.findOne({ userId, key });
    if (saved.bodyHash !== bodyHash) {
      return res.status(422).json({ error: 'Key was used with a different request' });
    }
    if (saved.status === 'processing') {
      return res.status(409).json({ error: 'Request is still being processed' });
    }
    return res.status(saved.statusCode).json(saved.response); // replay
  }

  // 2. We own the key. Do the work once.
  try {
    const order = await createOrder(userId, req.body);
    await chargeCard(order);

    const response = { orderId: order.id };
    await keys.updateOne(
      { userId, key },
      { $set: { status: 'done', statusCode: 201, response } }
    );
    return res.status(201).json(response);
  } catch (err) {
    await keys.deleteOne({ userId, key }); // let the client retry
    throw err;
  }
});

The Race Condition Most Examples Miss

A common first attempt is:

const existing = await keys.findOne({ userId, key });
if (existing) return res.json(existing.response);
// ... do the work ...
await keys.insertOne({ userId, key, response });

Two simultaneous requests both run findOne, both see nothing, and both do the work. You added a check and still got a duplicate.

The fix is in the version above: claim the key first with an insert that can only succeed once, and let the unique index decide the winner. The database settles the race, not your code.

Four Details That Matter

1. Scope keys per user. Two users who happen to generate the same key must not see each other's responses. That is why the index is on { userId, key }.

2. Compare the request body. If the same key arrives with a different body, that is a bug or an attack, not a retry. Return an error instead of replaying a response that does not match the request. Use a stable stringify (sorted keys) for the hash, or two identical bodies with different property order will hash differently.

3. Generate the key per intent, not per attempt. On the client, create the key when the user starts the action and reuse it for every retry of that action. A new key per retry brings the duplicate right back.

const key = crypto.randomUUID(); // once, when the checkout starts
await fetch('/api/orders', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json', 'Idempotency-Key': key },
  body: JSON.stringify(cart),
});

4. Pass the key on to your payment provider too. If your server crashes after charging the card but before saving the result, your own table cannot help. A key sent to the provider protects that gap, and the expiry index clears any stuck processing record.

The Rule

Any request that creates something or moves money must be safe to send twice. Do not rely on disabled buttons or "the network is reliable". Give the client a key, claim it atomically on the server, and replay the saved result on a repeat.

Quick audit: list your POST endpoints that create orders, payments, invoices, emails or bookings. For each, ask what happens if the request arrives twice, a second apart. If the answer is "two of them", you have found your next ticket.

If you work with MongoDB, my MongoDB and Mongoose patterns guide has more on using the database to enforce rules, and you can browse my Next.js templates on pixelanas.com.

Get the templates: https://pixelanas.gumroad.com

Have you ever shipped a double-charge or a duplicate-order bug? Tell me how you found out, in the comments.

Anas, full-stack Next.js developer building SaaS products and premium templates. X: @ASheikh69751

Building the MyZubster Metaverse: From Prototype to Secure Realtime Infrastructure

Fri, 23 Oct 2026 05:22:14 +0000

Building the MyZubster Metaverse: From Prototype to Secure Realtime Infrastructure
The MyZubster Metaverse is evolving from a simple interactive experience into a distributed platform designed around secure identity, realtime communication, reliability, privacy, and verifiable collaboration.
During the latest development phase, we focused on the infrastructure required to operate the metaverse safely under real production conditions.
This article explains what we implemented, how the architecture works, what we verified, and what remains before the system can be considered production-ready.
The Current Architecture
The metaverse now follows a layered architecture:
Authenticated user

↓

MyZubster session

↓

Short-lived realtime token

↓

Authorized realtime channel

↓

Distributed gateway

↓

Redis coordination layer

↓

Presence, rate limits and observability
The objective is to prevent clients from connecting directly to unrestricted realtime resources.
Every connection must originate from an authenticated MyZubster session and receive only the permissions required for its specific room or operation.
Authenticated Realtime Sessions
We introduced a dedicated token exchange for realtime connections.
The server:

verifies the current MyZubster session;
checks whether the user may access the requested resource;
generates a short-lived realtime token;
restricts that token to authorized channels;
rejects anonymous or malformed requests.
This limits the damage caused by leaked tokens and prevents clients from subscribing to arbitrary rooms.
A client cannot simply invent a room identifier and connect to it. Authorization is evaluated server-side before realtime access is granted.
Channel Authorization
Realtime channels are treated as protected resources.
The authorization layer validates:
user identity;
session validity;
resource ownership;
room membership;
token expiration;
permitted operations.
This establishes a clear trust boundary between the browser and the realtime gateway.
The client requests access, but the server remains responsible for deciding whether that access is legitimate.
Distributed Presence with Redis
In-memory presence works only while a single application instance is running. It becomes unreliable when multiple instances or serverless functions are involved.
We therefore introduced Redis-backed presence coordination.
Redis is used to track:
active participants;
room membership;
temporary presence state;
expiration of abandoned sessions;
coordination between separate gateway instances.
Presence entries have a limited lifetime. If a client disappears without completing a normal disconnection, the corresponding state can expire automatically instead of remaining indefinitely.
Graceful Redis Failure Handling
Redis is now considered an explicit infrastructure dependency.
The system distinguishes between:
the application process being alive;
the realtime gateway being operational;
Redis being reachable;
the complete service being ready to accept traffic.
Failures are handled deliberately rather than silently ignored. This is important because returning a successful health response while the coordination layer is unavailable would provide misleading information to deployment systems and operators.
Distributed Rate Limiting
Local counters cannot reliably enforce limits across multiple runtime instances.
We implemented a distributed rate-limiting foundation backed by Redis so that requests are evaluated against shared state.
The rate-limiting system is designed to protect:
realtime token generation;
connection attempts;
room joins;
message publication;
presence updates;
abuse-sensitive API routes.
This provides a foundation for defending the metaverse against spam, accidental request loops and automated abuse.
Future work will refine quotas by route, identity, IP risk and operation type.
Privacy-Safe Observability
Reliability requires visibility, but observability must not expose private user information.
The realtime infrastructure now supports structured operational signals for:
authorization failures;
connection lifecycle events;
Redis availability;
rate-limit decisions;
degraded states;
health and readiness checks.
The goal is to make infrastructure problems diagnosable without logging raw credentials, session tokens or unnecessary personal information.
Health and Readiness Endpoints
A process being alive does not mean it is ready to serve realtime traffic.
We extended the health model to include Redis readiness. The service can now report a degraded state when Redis is unavailable instead of returning an unconditional success response.
Conceptually:
Process alive + Redis unavailable
```
       ↓
```
Service is not fully ready
```
       ↓
```
Health response reports degradation
This distinction is essential for:
deployment validation;
load balancers;
automated monitoring;
incident diagnosis;
rolling releases;
recovery procedures.
Shared Redis Client
A reusable Redis client layer was introduced to avoid creating duplicated connections across services and requests.
The shared client centralizes:
configuration;
connection reuse;
error handling;
readiness checks;
shutdown behavior;
instrumentation.
Centralizing this behavior reduces connection pressure and gives every Redis-dependent feature a consistent operational model.
Preview Infrastructure
The new realtime changes were deployed to an isolated Vercel Preview environment.
This allows us to validate infrastructure modifications before they affect the public production deployment.
The preview environment provides a safe place to test:
application builds;
serverless deployment behavior;
environment configuration;
readiness responses;
Redis connectivity;
authentication flows;
realtime regressions.
Production remains unchanged while the current pull request is still under review.
Testing and Continuous Integration
The implementation is covered by automated backend tests, including readiness behavior and Redis degradation scenarios.
The principal validation pipeline successfully completed:
backend tests;
frontend tests;
linting;
application builds;
Seller Free policy verification;
continuous evidence generation;
preview deployment.
This provides strong evidence that the new infrastructure integrates correctly with the existing application.
However, passing tests alone do not make the service production-ready. Runtime drills, dependency remediation and real multi-instance load testing are still required.
Current Development Status
The realtime readiness work is currently available in a draft pull request:
MyZubster realtime readiness PR #40
The implementation includes four focused commits:
feat(realtime): expose Redis readiness
test(realtime): cover Redis readiness
feat(realtime): gate health on Redis
test(realtime): cover Redis health degradation
Keeping the commits focused makes review and rollback easier.
Current Security Blocker
The security audit currently reports vulnerabilities in the transitive ip-address dependency for versions up to 10.7.3.
The available patched version is 10.7.4.
Before merging the realtime work, we need to:
identify which direct dependency introduces ip-address;
update the dependency chain safely;
regenerate the lockfile;
run the complete test suite;
repeat the security audit;
confirm that no high or critical vulnerability remains.
The pull request should remain in draft until this has been addressed and the runtime verification has been completed.
What Still Needs to Be Done
Dependency remediation
Upgrade the affected dependency chain and obtain a clean security audit.
Direct runtime verification
Verify the deployed preview response from:
/api/realtime/health
The response should explicitly show whether Redis is ready and whether the service is operating normally or in a degraded state.
Redis failure drill
Temporarily simulate Redis unavailability in an isolated environment and verify that:
readiness fails correctly;
the service does not claim to be healthy;
errors remain controlled;
recovery occurs after Redis reconnects.
Multi-instance load testing
Run concurrent tests against multiple gateway instances and verify:
shared rate limits;
consistent presence;
room synchronization;
connection cleanup;
acceptable latency;
stable Redis connection usage.
Restart and recovery testing
Restart application and Redis instances while clients are connected.
We need evidence that the platform can recover without leaving permanent ghost sessions or inconsistent presence records.
End-to-end acceptance testing
The complete flow must be tested:
Login
→ request realtime token
→ authorize channel
→ connect
→ join room
→ publish event
→ update presence
→ disconnect
→ clean up state
Unauthorized variations must also be tested.
Production rollout strategy
The production release should be gradual:
deploy to Preview;
complete runtime verification;
run load and failure drills;
review monitoring signals;
merge after approval;
deploy production;
observe error rates and Redis health;
retain a documented rollback procedure.
Longer-Term Metaverse Roadmap
Once reliability and security are validated, development can move toward the experience layer.
Planned areas include:
WebXR support;
immersive rooms;
interactive avatars;
spatial collaboration;
verified professional profiles;
Zorgax integration;
verified knowledge inside virtual environments;
community-managed spaces;
portable identity;
privacy-safe analytics;
independent interoperable nodes.
The long-term objective is not merely to build another virtual world.
MyZubster aims to combine identity, knowledge, collaboration and transparent open-source development inside an interoperable environment.
Lessons Learned
The most important lesson from this phase is that realtime development is not simply about sending messages quickly.
A reliable realtime platform must answer several harder questions:
Who is allowed to connect?
Which channels may they access?
How is state shared across instances?
What happens when Redis fails?
How is abuse limited globally?
Can operators distinguish “alive” from “ready”?
Can the system recover cleanly after a restart?
Can failures be investigated without exposing private data?
These concerns are now becoming first-class parts of the MyZubster architecture.
Conclusion
The MyZubster Metaverse has progressed beyond an isolated prototype.
We now have the foundation for:
authenticated realtime access;
server-controlled channel authorization;
Redis-backed presence;
distributed rate limiting;
privacy-aware observability;
dependency-aware health checks;
isolated preview deployments;
automated tests and evidence generation.
The next milestone is to prove that this architecture remains reliable under dependency failures, restarts, concurrency and realistic traffic.
Only after those checks—and after resolving the current security advisory—should the realtime infrastructure be promoted to production.
MyZubster remains an open-source project, and technical review, testing and constructive contributions are welcome.
Project Links
MyZubster repository
MyZubster Metaverse
Realtime readiness pull request
Repository GitHub: https://github.com/danieldirimini-myzubster/myzubster
Metaverso MyZubster: https://www.myzubster.com/metaverse
Pull request Realtime Readiness #40: https://github.com/danieldirimini-myzubster/myzubster/pull/40
Roadmap Linear MYZ-83: https://linear.app/myzubster/issue/MYZ-83/b6-realtime-observability-reliability-and-load-tests
Organizzazione GitHub MyZubster Ecosystem: https://github.com/MyZubster-Ecosystem
Profilo GitHub MyZubster: https://github.com/myzubster
Sito principale: https://www.myzubster.com
Knowledge Explorer: https://www.myzubster.com/knowledge
Profilo Zorgax: https://www.myzubster.com/zorgax-profile-onboarding

How to Decide If a Node.js App Belongs Behind SafeLine

Fri, 23 Oct 2026 03:05:28 +0000

A Node.js app is not protected by installing a library that these SafeLine documents never mention. The supplied pages describe a self-hosted web application firewall that filters and monitors HTTP traffic between a web application and the Internet. They do not document a Node module, an npm package, or framework middleware. The supported decision is whether the process can sit behind the documented reverse proxy. I will not invent a runtime agent to fill that gap.

That boundary is the whole plan. Validation, sessions, and secrets stay in the application. SafeLine sits on the network path in front of it. Clients reach the WAF first. Only traffic it accepts should be forwarded to the Node process that still serves the original app.

What the pages actually specify

The introduction says deploying a WAF in front of a web application places a shield between that application and the Internet. A forward proxy hides a client. SafeLine is the other shape: a reverse proxy that reduces direct exposure of the server, because clients pass through it before they reach the origin. It filters, monitors, and blocks malicious HTTP and HTTPS traffic, and it is also described as preventing unauthorized data from leaving the app by applying policies that separate malicious traffic from safe traffic. Listed core capabilities are defenses for web attacks, proactive bot abuse defense, HTML and JS code encryption, IP-based rate limiting, and a web access control list. https://docs.waf.chaitin.com/en/home

The application guide is the operational half of that statement. SafeLine is developed based on nginx. It receives traffic for the original website, cleans malicious attack traffic, and forwards safe and reliable traffic to the original application. https://docs.waf.chaitin.com/en/GetStarted/AddApplication

Nothing on those pages ties the origin to one language. A Node HTTP server qualifies only as an upstream address. Express, Fastify, or the built-in http module do not receive a special connector in this material. The checklist is how I decide the proxy is appropriate, then how I fill only the fields the console documents.

Five checks before DNS changes

Host fit is first. The deploy guide requires Linux, x86_64 or arm64, Docker 20.10.14 or above, and Docker Compose 2.0.0 or above. The minimum environment is 1 core CPU, 1 GB memory, and 5 GB disk. The CPU must support ssse3. The same page lists uname -m, docker version, docker compose version, free -h, df -h, and an lscpu check for ssse3 so the baseline can be confirmed before install. https://docs.waf.chaitin.com/en/GetStarted/Deploy

Edition fit is second, and it matters on small Node hosts. SafeLine installed on ARM requires a Pro license to work properly. The Personal Edition is not supported on ARM. An arm64 image is therefore a license decision, not a footnote after the container starts. https://docs.waf.chaitin.com/en/GetStarted/Deploy

Upstream design is third. Log into the web admin console, open Applications, and choose Add Application. Domain may be a domain name, a hostname, or an IP address. Port is what SafeLine listens on, such as 80 or 443. HTTPS applications need the SSL option checked. Upstream is the real address of the original app, and that is where cleaned traffic is forwarded. After the entry is saved, resolve the entered name to the IP of the server running SafeLine. https://docs.waf.chaitin.com/en/GetStarted/AddApplication

I do not publish the Node listen port as the client entry. If the upstream name resolves back to SafeLine, the forward path loops. The docs do not ship a Node bind tutorial, so the rule stays the documented one: upstream must be the real application address, reachable from the WAF.

Proof is fourth. The application guide gives sample cases for SQL injection, XSS, path traversal, code injection, and XXE. Replace the example host with the protected domain. A blocking page means that case was defended. The web console attack list, then the detail view, shows whether the event was recorded. https://docs.waf.chaitin.com/en/GetStarted/AddApplication

Release track is fifth. The LTS version will no longer be maintained starting from 9.1.0-LTS (14 August 2025). LTS users are strongly recommended to upgrade to the latest version. If there are concerns, the deploy guide recommends waiting one month after a new release before upgrading. https://docs.waf.chaitin.com/en/GetStarted/Deploy

Controls that match a public Node service

Attack blocking is the wide net. The feature list names SQL injection, XSS, code injection, OS command injection, CRLF injection, XXE, SSRF, and path traversal. The introduction also names LDAP injection, XPath injection, RCE, backdoor, bruteforce, HTTP flood, and bot abuse. https://docs.waf.chaitin.com/en/home

Rate limiting is documented against DoS attacks, bruteforce attempts, traffic surges, and other abuse, by throttling traffic that exceeds defined limits. IP-based rate limiting is part of the core list. A captcha challenge is documented so human users can continue while crawlers and bots are blocked. When the authentication challenge is on, visitors must enter the password or they are blocked. When dynamic protection is on, HTML and JS from the web server are dynamically encrypted on each visit. https://docs.waf.chaitin.com/en/home

I do not apply every control as if every Node app were a browser site. Dynamic HTML and JS encryption matches a page that returns markup. A JSON API should not be described as if that encryption were its main defense. For an API, the documented controls I plan around are attack filtering, rate limits, the bot challenge, and the access control list. The authentication challenge is a visitor password gate. It is not application login, and these pages do not say it replaces tokens or sessions inside Node.

The home page also publishes an effect evaluation with 33669 samples in each column. SafeLine Balance is listed at 71.65 percent detection, 0.07 percent false positive, and 99.45 percent accuracy. SafeLine Strict is listed at 76.17 percent detection, 0.22 percent false positive, and 99.38 percent accuracy. The same table includes ModSecurity Level 1 and CloudFlare Free. I treat those cells as published comparison figures, not as a measurement I ran against a Node route. https://docs.waf.chaitin.com/en/home

Scale and commercial notes, kept apart from setup

The introduction says SafeLine is production-ready. It reports over 180,000 installations worldwide, protection for over 1,000,000 websites, and more than 30,000,000,000 HTTP requests handled daily. https://docs.waf.chaitin.com/en/home

As of September 30, 2026, SafeLine surpassed 700,000 installations worldwide. SafeLine Pro annual promotional pricing is $650-$880 versus the $1,000 regular price. I keep those planning figures next to the older published installation count, and I do not let either number skip the proxy checks.

Install steps stay inside the deploy guide. Automatic deploy runs the official manager script, then uses the web UI. Manual deploy creates a directory with at least 5 GB free, downloads compose.yaml, writes the .env file, generates the documented secrets, restricts permissions, and starts the stack with docker compose up -d. The initial admin account is printed by docker exec safeline-mgt resetadmin. The console is opened at https://<safeline-ip>:9443/. https://docs.waf.chaitin.com/en/GetStarted/Deploy

I do not extend that into an unsupported Node story. These pages do not document WebSocket handling, gRPC mode, sticky sessions, or a process manager integration. If the app is ordinary HTTP or HTTPS, add the application, set domain, port, SSL, and upstream, and point DNS at SafeLine. If the traffic is not that, leave the claim unsupported instead of inventing a connector.

Does SafeLine document a Node.js plugin?

No. The supplied home, deploy, and application pages describe a reverse proxy and console fields for domain, port, SSL, and upstream. They do not document an npm package or a Node agent. Protect the app by placing that proxy in front of the process.

Which address should clients use?

Clients should use the domain that resolves to the SafeLine server and the listen port on the application entry, such as 80 or 443. The Node address belongs in upstream. Leaving the Node port open on the public interface bypasses the shield described in the introduction.

How can I verify a block without guessing?

Use the documented SQL injection, XSS, path traversal, code injection, and XXE cases against your own domain. A blocking page is the expected sign that the case was stopped. Then open the console attack list and check detail for that event. https://docs.waf.chaitin.com/en/GetStarted/AddApplication

When does architecture force a Pro license?

When the install is ARM. The deploy guide says ARM requires a Pro license to work properly, and that the Personal Edition is not supported on ARM. Confirm uname -m before you treat a cheap arm64 host as ready. https://docs.waf.chaitin.com/en/GetStarted/Deploy

What should stay inside the Node process?

Route validation, account authentication, and secret handling. SafeLine is documented to clean traffic and forward the safe portion to the original application. It does not take over application logic. Keep those checks even after the sample attacks show a blocking page.

SafeLine has surpassed 700,000 installations worldwide as of September 30, 2026. SafeLine Pro annual promotional pricing is $650-$880, compared with the regular $1,000 per year.

SafeLine website: https://ly.cyberserval.tech/MqhYdmI
Demo: https://demo.waf.chaitin.com:9443/statistics
Official docs: https://docs.waf.chaitin.com/en/home
GitHub: https://github.com/chaitin/SafeLine
Pro offer: https://cyberserval.tech/landing/lto

How to Protect a Node.js App with a SafeLine Fit Checklist

Fri, 23 Oct 2026 03:04:56 +0000

Protecting a Node.js app with SafeLine starts as a fit decision, not as a search for an in-process library. The product documents describe SafeLine as a self-hosted web application firewall that filters and monitors HTTP traffic between a web application and the Internet. Clients pass through it before they reach the server. It is a reverse proxy, not a module loaded by your runtime (https://docs.waf.chaitin.com/en/home). The application guide says the same thing in operational language: SafeLine receives traffic for the original website, cleans malicious attack traffic, and forwards the safe remainder to the original application (https://docs.waf.chaitin.com/en/GetStarted/AddApplication). Those pages do not document a Node.js SDK, Express middleware, Fastify plugin, or runtime agent. If that is what the keyword suggested, do not invent the integration. Keep Node as the upstream and decide whether the documented front door fits.

The console model is small enough to checklist. You add an application with a domain, hostname, or IP, a listen port such as 80 or 443, the SSL option for an HTTPS app, and an upstream address. Upstream is the real address of the original app. For this keyword, that is the host and port the Node process already serves. After the application is saved, you resolve the domain to the SafeLine server so visitors reach the proxy first (https://docs.waf.chaitin.com/en/GetStarted/AddApplication). A Node port that is still open on the public internet is a bypass. The checklist is not done until that bypass is closed.

Confirm the protocol before you argue about frameworks. SafeLine is documented for HTTP and HTTPS web traffic. A Node process that only speaks some other protocol is outside this guide. Do not treat a reverse-proxy WAF as a generic socket filter just because the service happens to be written in JavaScript.

Map risk to the published attack list. The home page names SQL injection, XSS, code injection, OS command injection, CRLF injection, LDAP injection, XPath injection, RCE, XXE, SSRF, path traversal, backdoor, brute force, HTTP flood, and bot abused traffic (https://docs.waf.chaitin.com/en/home). Those classes matter when a Node route builds SQL, reflects input into HTML, shells out, parses XML, fetches a user-supplied URL, or reads a file path from a request. They do not retire bugs that never become a policy decision at the proxy, such as broken authorization inside a route or a secret committed to the repo. Say that limit out loud before the DNS cutover.

Then pick only controls the docs name. Core capabilities are defenses for web attacks, proactive bot abused defense, HTML and JS code encryption, IP-based rate limiting, and a web access control list (https://docs.waf.chaitin.com/en/home). Rate limiting is described as throttling traffic that exceeds defined limits, aimed at DoS, brute force, traffic surges, and other abuse. CAPTCHA is described as allowing human users while blocking crawlers and bots. The authentication challenge blocks visitors who do not enter the password. Dynamic protection encrypts HTML and JS on each visit (https://docs.waf.chaitin.com/en/home). A server-rendered Node site can use that HTML and JS control as documented. A JSON API should not be described as encrypted by it. For an API, the documented primary controls are attack blocking, rate limiting, and the access control list, with CAPTCHA or the password challenge only if you accept the visitor friction.

Check the host against the deploy guide, not against your Node version. Requirements are Linux, x86_64 or arm64, Docker 20.10.14 or above, Docker Compose 2.0.0 or above, at least 1 CPU core, 1 GB of memory, and 5 GB of disk, plus a CPU with ssse3 (https://docs.waf.chaitin.com/en/GetStarted/Deploy). SafeLine installed on ARM requires a Pro license to work properly. The Personal Edition is not supported on ARM. The LTS version is no longer maintained starting from 9.1.0-LTS on 14 August 2025, and the guide strongly recommends the latest version. None of that is an npm constraint. Meeting it does not mean SafeLine inspects your package manifest.

Prove blocking with the published cases, then read the console. The application guide lists SQL injection, XSS, path traversal, code injection, and XXE samples and tells you to replace the example domain with yours. A blocking page means the defense fired. The attack list and its detail view are how you confirm what was stopped (https://docs.waf.chaitin.com/en/GetStarted/AddApplication). Replay legitimate Node routes in the same pass. Encoded queries, file uploads, and webhook bodies can look hostile without being attacks against your app.

Use the published evaluation as a comparison, not as your error budget. On 33,669 samples, the home page lists SafeLine Balance at 71.65% detection, 0.07% false positives, and 99.45% accuracy, and SafeLine Strict at 76.17% detection, 0.22% false positives, and 99.38% accuracy. The same table lists ModSecurity Level 1 at 69.74% detection, 17.58% false positives, and 82.20% accuracy, and CloudFlare Free at 10.70% detection, 0.07% false positives, and 98.40% accuracy (https://docs.waf.chaitin.com/en/home). Stage your own traffic anyway. Those rows do not mention your routes.

As of September 30, 2026, SafeLine surpassed 700,000 installations worldwide. SafeLine Pro annual promotional pricing is $650-$880 versus the $1,000 regular price. Treat that range as a licensing input when you need Pro, including the ARM case in the deploy guide. Do not treat it as proof that every Node.js app needs that edition. The home page also calls SafeLine production-ready, protecting over 1,000,000 websites and handling over 30,000,000,000 HTTP requests daily (https://docs.waf.chaitin.com/en/home). Scale still does not replace an upstream health check. If the proxy cannot be reached, Node never sees the client. If Node is down, the proxy cannot write your application response for you.

The console is https://<safeline-ip>:9443/ unless the documented management port is changed, and the initial administrator account comes from docker exec safeline-mgt resetadmin (https://docs.waf.chaitin.com/en/GetStarted/Deploy). If you enable CAPTCHA or the password challenge, test them in a browser, because both change what happens before Node receives the request. The fit decision is complete only when clients use SafeLine, the Node upstream is private to that proxy path, attack probes show a blocking page and a console detail, and normal traffic still arrives.

Does SafeLine provide a Node.js integration package?

The install guide and the application guide do not document an SDK, middleware, or runtime agent. The supported pattern is to add the Node process as the upstream and send clients to SafeLine first (https://docs.waf.chaitin.com/en/GetStarted/AddApplication).

Which fields define the Node upstream?

The documented fields are domain, port, the SSL option for HTTPS, and upstream. Upstream is the real address of the original app, meaning the host and port your Node process already serves (https://docs.waf.chaitin.com/en/GetStarted/AddApplication).

Will dynamic protection encrypt JSON API responses?

Dynamic protection is documented as encrypting HTML and JS on each visit (https://docs.waf.chaitin.com/en/home). Judge a JSON API on attack blocking, IP-based rate limiting, the web access control list, and, if you accept the extra step, CAPTCHA or the authentication challenge.

How do I confirm a probe was stopped by SafeLine?

Run the published SQL injection, XSS, path traversal, code injection, and XXE cases against your domain. A blocking page plus an attack-list detail in the console is the documented confirmation (https://docs.waf.chaitin.com/en/GetStarted/AddApplication).

When does Pro pricing matter for this setup?

ARM installs require a Pro license, and the Personal Edition is not supported on ARM (https://docs.waf.chaitin.com/en/GetStarted/Deploy). As of September 30, 2026, SafeLine Pro annual promotional pricing is $650-$880 versus the $1,000 regular price. That is a license input, not a claim that every x86_64 Node deployment must be Pro.

SafeLine has surpassed 700,000 installations worldwide as of September 30, 2026. SafeLine Pro annual promotional pricing is $650-$880, compared with the regular $1,000 per year.

SafeLine website: https://ly.cyberserval.tech/MqhYdmI
Demo: https://demo.waf.chaitin.com:9443/statistics
Official docs: https://docs.waf.chaitin.com/en/home
GitHub: https://github.com/chaitin/SafeLine
Pro offer: https://cyberserval.tech/landing/lto

# Node.js Brotli RCE (CVE-2026-111497): What Happened and How to Patch

Thu, 22 Oct 2026 15:32:04 +0000

Node.js Brotli RCE (CVE-2026-111497): What Happened and How to Patch

Node.js · Security · npm

On October 21, 2026 the Node.js team released security fixes for the 22.x, 24.x and 26.x lines, covering 10 CVEs. The most serious, CVE-2026-111497, is an out-of-bounds write in the Brotli decoder that the advisory says permits remote code execution. The advisory lists no workaround, so the only fix is to update Node.

This post covers what was fixed, how to check if you're exposed, and how to patch your projects and keep an unpatched Node from coming back.

TL;DR

Update to Node 22.23.4, 24.22.1 or 26.12.1 (or newer).
There is no workaround for the Brotli bug.
The advisory doesn't mention Node 20, but it says end-of-life versions are always affected. If you're on one, plan a move to a supported line.
Add an engines range to package.json so an unpatched Node gets flagged.

What was fixed

Four High, three Medium and three Low severity issues:

CVE	Severity	What it is	Lines
CVE-2026-111497	High	Brotli decoder out-of-bounds write permits remote code execution	22, 24, 26
CVE-2026-105482	High	`dns.resolve` truncates names containing a NUL character	22, 24, 26
CVE-2026-105483	High	TLS hostname bypass when an IP is written with fullwidth digits	24, 26
CVE-2026-106069	High	TLS hostname bypass when a host loosely resolves to an IP	22, 24, 26
CVE-2026-105481	Medium	Keep-alive socket processes a response written before the request was sent	22, 24, 26
CVE-2026-106072	Medium	Non-canonical IPv4 hostnames let DNS SANs authenticate IP endpoints	22, 24, 26
CVE-2026-106074	Medium	`crypto.randomUUID()` / `randomInt()` entropy caches saved into startup snapshots repeat the same values	22, 24, 26
CVE-2026-106070	Low	`ZipFile` returns attacker-controlled prefix bytes after failed header validation	26
CVE-2026-106071	Low	Permission Model filesystem over-grant via wildcard radix lookup	26
CVE-2026-106073	Low	Permission Model bypass with `fs.realpathSync`	22, 24, 26

The same releases also update bundled dependencies (c-ares, OpenSSL, Undici and others).

Why the Brotli bug is the one to care about

Brotli is a compression format. Node decodes it in several places you may not think about:

fetch() decodes responses sent with Content-Encoding: br.
Express 5's body parser (body-parser 2.x) decodes Brotli request bodies by default.
axios asks for Brotli by default.
Any direct zlib Brotli decompression call.

Browser JavaScript uses the browser's own decoder, and code that only compresses with Brotli isn't the entry point. The trigger is described as a malformed Brotli stream. Reports on public proof-of-concepts disagree about whether it works against a standard build, and one tracker says no exploitation in the wild had been reported as of October 22. I wouldn't wait to find out: update.

Step 1: check what you're running

node -v

You want 22.23.4+, 24.22.1+ or 26.12.1+. There's also a direct check. The fix adds a decoder error constant, so this prints true only on a patched Node (tested on 24.21.0 → false and 24.22.1 → true):

node -p "'BROTLI_DECODER_ERROR_FORMAT_BLOCK_SWITCH' in require('zlib').constants"

Check every place Node runs: your laptop, CI, Docker base images, and your hosting runtime.

Step 2: update

With nvm:

nvm install 24          # installs the newest 24.x (24.22.1 or later)
nvm alias default 24
nvm use 24

Two traps to watch for:

An already-open terminal keeps the old Node until you run nvm use 24.
Docker and CI need a rebuild on a patched base image (docker build --pull --no-cache .). Managed runtimes (Firebase, Vercel and similar) choose the patch version for you, so check your provider rather than assuming.

Step 3: stop an unpatched Node from sneaking back in

Updating your machine fixes your machine. To protect the project, declare which Node versions it supports with an engines field in every package.json (root, server, workspaces):

{
  "engines": {
    "node": ">=22.23.4 <23.0.0 || >=24.22.1 <25.0.0"
  }
}

npm also mirrors it into the lockfile's root entry (packages[""].engines). Here's what that range does:

Node version	Result
22.23.3	flagged
22.23.4	allowed
24.21.0	flagged
24.22.1	allowed
26.12.1	flagged, because it isn't in this range

If you run the 26 line, add || >=26.12.1 <27.0.0.

By default npm only warns (EBADENGINE) and keeps installing:

npm warn EBADENGINE Unsupported engine {
npm warn EBADENGINE   required: { node: '>=22.23.4 <23.0.0 || >=24.22.1 <25.0.0' },
npm warn EBADENGINE   current: { node: 'v24.21.0', npm: '12.2.0' }
npm warn EBADENGINE }

To make it a hard failure, add this to .npmrc:

engine-strict=true

Step 4: make sure your dependencies accept the patched Node

Before tightening the range, check that no installed package refuses the patched version. Save this as scan-engines.cjs in your project root (it needs semver, which most projects already have in node_modules; otherwise npm i -D semver):

// node scan-engines.cjs 24.22.1
const fs = require("fs");
const path = require("path");
const semver = require("semver");

const target = process.argv[2];
const rejected = [];
let checked = 0;

function scan(dir) {
  if (!fs.existsSync(dir)) return;
  for (const name of fs.readdirSync(dir)) {
    const full = path.join(dir, name);
    if (name.startsWith("@")) { scan(full); continue; }
    const manifest = path.join(full, "package.json");
    if (!fs.existsSync(manifest)) continue;
    const { name: pkg, version, engines } = JSON.parse(fs.readFileSync(manifest, "utf8"));
    if (!engines?.node) continue;
    checked++;
    if (!semver.satisfies(target, engines.node)) rejected.push(`${pkg}@${version} needs ${engines.node}`);
  }
}

scan(path.resolve("node_modules"));
console.log(`${checked} packages declare engines.node`);
console.log(rejected.length ? rejected.join("\n") : `none reject Node ${target}`);

If it lists any packages, check whether a newer version of each supports the patched Node before you tighten the range.

Checklist

[ ] node -v shows a patched version on every machine, CI runner and container
[ ] Docker images rebuilt with --pull --no-cache
[ ] Hosting runtime confirmed with your provider
[ ] engines.node set in every package.json (and mirrored in the lockfile)
[ ] Optional: engine-strict=true in .npmrc
[ ] Dependencies checked against the patched Node

Sources

How to Scrape Threads with Node.js: Profiles, Posts, Replies, and an Engagement Tracker (2026)

Wed, 21 Oct 2026 09:05:56 +0000

Threads is an awkward target. The web app is a React client from Meta, most of the interesting data loads through internal GraphQL calls, and a logged-out visitor sees only a slice of what a logged-in one does. Still, a lot of public data ships inside the first HTML response, and for a modest job like "track likes and replies on one account's recent posts" that slice is enough.

This tutorial builds a small threads scraper in Node.js 18+ with nothing but native fetch. It reads a public profile, pulls its recent posts with like, reply, repost, and quote counts, and appends a timestamped snapshot to a JSON Lines file every time it runs. At the end there's a short report script that shows which posts gained the most engagement between runs.

I'll also be upfront about where this breaks, because on Threads it breaks often.

What a logged-out visitor can reach

Short version, as of writing:

Profile pages (https://www.threads.com/@handle): name, bio, follower count, verification, and the first batch of recent posts.
Single post pages (https://www.threads.com/@handle/post/<code>): the post itself and usually a first batch of replies.
Search, feeds, follower lists, "load more": these either show a login prompt or depend on GraphQL calls that need session tokens. Treat them as off-limits for a no-login scraper.

Threads moved its main domain from threads.net to threads.com in 2025. Old threads.net links redirect, so let fetch follow redirects and normalize URLs to threads.com in your own storage so the same post doesn't get two keys.

Where the data actually lives

If you open the page source of a profile, you won't find the posts as readable HTML. What you will find are many <script type="application/json"> tags, several marked with a data-sjs attribute. These are server-side preloaded data for Meta's Relay client, and some of them contain the same post objects the app renders: code, taken_at, like_count, caption.text, and a text_post_app_info object with direct_reply_count, repost_count, and quote_count.

The structure is deeply nested and changes without notice, so don't hard-code a path like result.data.mediaData.edges[0].... A sturdier approach is to parse every JSON script and walk the whole tree looking for anything that looks like a post. That costs a few milliseconds and survives most reshuffles.

Setup

mkdir threads-scraper && cd threads-scraper
npm init -y
npm pkg set type=module

No dependencies. Node 18 or newer gives you global fetch, and everything else comes from the standard library.

Step 1: Fetch a page politely

// fetchPage.js
const UA =
  "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 " +
  "(KHTML, like Gecko) Chrome/129.0 Safari/537.36";

const sleep = (ms) => new Promise((r) => setTimeout(r, ms));

export async function fetchPage(url, { retries = 3 } = {}) {
  for (let attempt = 0; attempt <= retries; attempt++) {
    const res = await fetch(url, {
      headers: {
        "user-agent": UA,
        "accept": "text/html,application/xhtml+xml",
        "accept-language": "en-US,en;q=0.9",
      },
      redirect: "follow",
    });

    if (res.ok) {
      const html = await res.text();
      // A login redirect sometimes comes back as 200 with a tiny shell page.
      if (res.url.includes("/login") || html.length < 20_000) {
        throw new Error(`Looks like a login wall for ${url} (final URL ${res.url})`);
      }
      return html;
    }

    if (res.status === 429 || res.status >= 500) {
      const wait = 2 ** attempt * 5_000 + Math.random() * 2_000;
      console.warn(`HTTP ${res.status} on ${url}, retrying in ${Math.round(wait / 1000)}s`);
      await sleep(wait);
      continue;
    }
    throw new Error(`HTTP ${res.status} for ${url}`);
  }
  throw new Error(`Gave up on ${url} after ${retries + 1} attempts`);
}

The 20,000-character threshold is a heuristic, not a documented rule. A real profile page is far larger than that, while the shell you get when Meta wants you to sign in is much smaller. Tune it if you see false positives.

Step 2: Pull the embedded JSON out of the HTML

// extract.js
const SCRIPT_RE = /<script type="application\/json"[^>]*>([\s\S]*?)<\/script>/g;

export function jsonBlobs(html) {
  const blobs = [];
  for (const m of html.matchAll(SCRIPT_RE)) {
    try {
      blobs.push(JSON.parse(m[1]));
    } catch {
      // Some blobs are not valid standalone JSON; skip them.
    }
  }
  return blobs;
}

// Walk any JSON tree and collect objects that look like Threads posts.
export function findPosts(root) {
  const found = new Map();
  const stack = [root];
  while (stack.length) {
    const node = stack.pop();
    if (!node || typeof node !== "object") continue;

    if (
      typeof node.code === "string" &&
      typeof node.taken_at === "number" &&
      "like_count" in node &&
      node.user?.username
    ) {
      found.set(node.code, node);
    }
    for (const v of Array.isArray(node) ? node : Object.values(node)) {
      if (v && typeof v === "object") stack.push(v);
    }
  }
  return [...found.values()];
}

// Same idea for the profile header.
export function findProfile(root, handle) {
  const stack = [root];
  while (stack.length) {
    const node = stack.pop();
    if (!node || typeof node !== "object") continue;
    if (node.username === handle && "follower_count" in node) return node;
    for (const v of Array.isArray(node) ? node : Object.values(node)) {
      if (v && typeof v === "object") stack.push(v);
    }
  }
  return null;
}

The Map keyed by code matters: the same post often appears in more than one blob, and you don't want to count it twice. Using an explicit stack instead of recursion avoids blowing the call stack on Meta's very deep payloads.

Step 3: Normalize a post

Raw post objects carry dozens of fields you don't need. Flatten them into something you'd actually store:

// normalize.js
export function normalizePost(p) {
  const info = p.text_post_app_info ?? {};
  return {
    code: p.code,
    url: `https://www.threads.com/@${p.user.username}/post/${p.code}`,
    author: p.user.username,
    postedAt: new Date(p.taken_at * 1000).toISOString(),
    text: p.caption?.text ?? "",
    likes: p.like_count ?? null,
    replies: info.direct_reply_count ?? null,
    reposts: info.repost_count ?? null,
    quotes: info.quote_count ?? null,
    isReply: Boolean(info.is_reply),
  };
}

Two caveats. First, authors can hide like counts, so like_count may be missing or zero even on a popular post; store null rather than guessing. Second, the profile page mixes in the author's own continuation posts (a multi-part thread shows up as a post plus replies to itself). isReply lets you filter those out if you only want top-level posts.

Step 4: The threads scraper core (a profile)

// profile.js
import { fetchPage } from "./fetchPage.js";
import { jsonBlobs, findPosts, findProfile } from "./extract.js";
import { normalizePost } from "./normalize.js";

export async function scrapeProfile(handle) {
  handle = handle.replace(/^@/, "");
  const html = await fetchPage(`https://www.threads.com/@${handle}`);
  const blobs = jsonBlobs(html);

  let profile = null;
  const posts = new Map();
  for (const blob of blobs) {
    profile ??= findProfile(blob, handle);
    for (const p of findPosts(blob)) {
      if (p.user.username === handle) posts.set(p.code, normalizePost(p));
    }
  }

  return {
    handle,
    followers: profile?.follower_count ?? null,
    verified: profile?.is_verified ?? null,
    bio: profile?.biography ?? null,
    posts: [...posts.values()].sort((a, b) => b.postedAt.localeCompare(a.postedAt)),
  };
}

if (import.meta.url === `file://${process.argv[1]}`) {
  const data = await scrapeProfile(process.argv[2] ?? "zuck");
  console.log(data.followers, data.posts.length);
  console.table(data.posts.slice(0, 5).map(({ code, likes, replies, reposts }) => ({ code, likes, replies, reposts })));
}

Run it with node profile.js someuser. Expect a first page of posts, roughly what you'd see before scrolling. If you get zero posts but a valid profile, Meta has probably changed where the posts are preloaded, or this request was served a stripped page. Save the HTML to disk and search it for a post code you know exists; that tells you quickly which case you're in.

Step 5: Replies on a single post

The post permalink page works the same way. The main post plus the first batch of replies are usually in the preloaded JSON:

// post.js
import { fetchPage } from "./fetchPage.js";
import { jsonBlobs, findPosts } from "./extract.js";
import { normalizePost } from "./normalize.js";

export async function scrapePost(url) {
  const clean = url.replace("threads.net", "threads.com").split(/[?#]/)[0];
  const code = clean.match(/\/post\/([^/]+)/)?.[1];
  if (!code) throw new Error(`Not a Threads post URL: ${url}`);

  const all = new Map();
  for (const blob of jsonBlobs(await fetchPage(clean))) {
    for (const p of findPosts(blob)) all.set(p.code, normalizePost(p));
  }
  const post = all.get(code) ?? null;
  all.delete(code);
  return { post, replies: [...all.values()] };
}

"Replies" here means everything else on the page that looks like a post. On some pages that includes the parent post (if you opened a reply) or related posts. If you need strict parent/child relationships, inspect the surrounding thread_items structure in the blob rather than relying on the flat list.

Why not just call the GraphQL API?

You'll see older tutorials that POST to https://www.threads.com/api/graphql with a doc_id and an lsd token copied from the page. It can work for a while, but plan for three problems:

doc_id churn. Each query is identified by a numeric document ID that changes when Meta ships a new client build. Hard-coded IDs go stale, sometimes within weeks.
Token coupling. The lsd token and related headers are tied to the page you loaded. You end up scraping the HTML anyway just to call GraphQL.
Pagination is where login walls live. "Load more posts" and deep reply threads are exactly the calls that tend to need a session. Using a logged-in account for scraping risks that account, and it's clearly against Meta's terms.

For a no-login threads scraper, the embedded JSON is the more stable source, even though you only get the first page.

Rate limits, IP blocks, and legal notes

Meta doesn't publish limits for the public web pages. From what people commonly report, a residential IP fetching a few pages a minute is fine, while datacenter IPs get challenged or redirected to login much sooner. I'd stick to these rules:

One request every few seconds, with random jitter, and back off on 429.
Cache HTML during development so you're not hitting Threads every time you tweak a parser.
Watch for soft blocks. A 200 with a login shell is the usual symptom, not a clean 403.

On the legal side: Meta's terms prohibit automated collection without permission, and courts in different countries have treated public-data scraping differently. Only collect public data, don't collect more personal data than your use case needs, respect GDPR if you store data about EU users, and talk to a lawyer before building anything commercial. This post is a tutorial, not legal advice.

The official Threads API

Meta has an official Threads API (graph.threads.net). It's the right tool when you're working with your own account: publishing, reading your posts and their insights, managing replies. Meta has since added keyword search and some public profile lookups, but they require app review, specific permissions, and an access token tied to a Threads user, and they come with their own rate limits. Check the current docs before relying on them, because the scope has been changing.

What it won't give you is an easy "give me any public account's posts" call with no app review. That gap is why people write scrapers in the first place.

Project: a Threads engagement tracker

Now the practical bit. This script snapshots a few accounts and appends one line per post per run to snapshots.jsonl. JSON Lines keeps it dependency-free; swap in SQLite (better-sqlite3, or node:sqlite on recent Node versions) when the file gets large.

// track.js
import { appendFile } from "node:fs/promises";
import { scrapeProfile } from "./profile.js";

const HANDLES = process.argv.slice(2);
if (!HANDLES.length) {
  console.error("usage: node track.js handle1 handle2 ...");
  process.exit(1);
}
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));

const runAt = new Date().toISOString();
for (const handle of HANDLES) {
  try {
    const { followers, posts } = await scrapeProfile(handle);
    const lines = posts
      .filter((p) => !p.isReply)
      .map((p) => JSON.stringify({ runAt, handle, followers, ...p }));
    if (lines.length) await appendFile("snapshots.jsonl", lines.join("\n") + "\n");
    console.log(`${handle}: ${lines.length} posts, ${followers ?? "?"} followers`);
  } catch (err) {
    console.error(`${handle}: ${err.message}`);
  }
  await sleep(4_000 + Math.random() * 3_000);
}

Schedule it with cron every few hours:

0 */4 * * * cd /path/to/threads-scraper && /usr/bin/node track.js handle1 handle2 >> track.log 2>&1

And a report that compares each post's first and latest snapshot:

// report.js
import { readFile } from "node:fs/promises";

const rows = (await readFile("snapshots.jsonl", "utf8"))
  .split("\n")
  .filter(Boolean)
  .map((l) => JSON.parse(l));

const byPost = new Map();
for (const r of rows) {
  const s = byPost.get(r.code) ?? { first: r, last: r };
  if (r.runAt < s.first.runAt) s.first = r;
  if (r.runAt > s.last.runAt) s.last = r;
  byPost.set(r.code, s);
}

const delta = (a, b) => (a == null || b == null ? null : b - a);
const report = [...byPost.values()]
  .map(({ first, last }) => ({
    post: last.url,
    snapshots: `${first.runAt.slice(0, 16)} -> ${last.runAt.slice(0, 16)}`,
    likes: last.likes,
    likeGain: delta(first.likes, last.likes),
    replyGain: delta(first.replies, last.replies),
    repostGain: delta(first.reposts, last.reposts),
  }))
  .sort((a, b) => (b.likeGain ?? -1) - (a.likeGain ?? -1));

console.table(report.slice(0, 15));

One thing to keep in mind when reading this report: because you only see the first page of posts, older posts drop out of the snapshots once newer ones push them off. A post that stops appearing hasn't stopped growing; you just can't see it anymore. If you need to keep following specific posts, feed their URLs through scrapePost() on each run instead.

The managed alternative

If maintaining the parser becomes a chore, or you need more than the first page from datacenter IPs, ScrapeAtlas* has Threads endpoints that return the same post objects (like_count, taken_at, text_post_app_info.direct_reply_count, and so on). Authentication is an x-api-key header:

const atlas = (path, params) =>
  fetch(`https://api.scrapeatlas.com/v1/threads/${path}?${new URLSearchParams(params)}`, {
    headers: { "x-api-key": process.env.SCRAPEATLAS_API_KEY },
  }).then((r) => r.json());

const profile = await atlas("profile", { handle: "zuck" });
console.log(profile.username, profile.follower_count, profile.is_verified);

const feed = await atlas("user/posts", { handle: "zuck", since: "2026-10-01" });
console.log(feed.posts.map((p) => [p.url, p.like_count, p.text_post_app_info.direct_reply_count]));
console.log(feed.coverage); // e.g. { complete: false, warnings: ["posts_first_page_only"], ... }

const thread = await atlas("post", { url: "https://www.threads.com/@zuck/post/<code>" });
console.log(thread.post.like_count, thread.comments.length);

To be clear about the limits: user/posts is also first page only, which the response tells you in coverage.warnings, so the tracker's blind spot is the same. post takes include_comments=false if you only want the post, and every endpoint accepts trim=true for a compact projection. There are also user/replies, post/comments, search (keyword search with recent, start_date, and end_date), search/users, and search/tags. Search is the part you can't do logged-out on your own.

Wrapping up

Read the preloaded JSON, not the DOM. Walk every application/json script and match on post shape, not a fixed path.
Dedupe by code and normalize threads.net to threads.com.
Expect the first page only without logging in, and design your tracker around that.
Skip GraphQL doc_id calls unless you're ready to re-capture them every time Meta ships a new build.
Go slow and cache, and treat a small 200 response as a block.
For your own account, use the official Threads API.

If the extractor stops finding posts for you, leave a comment with what the page source looks like now and I'll update the post.

*I build it.

Stop Battling JavaScript Date Bugs: A Complete Guide to the Temporal API

Wed, 21 Oct 2026 08:53:20 +0000

Stop Battling JavaScript Date Bugs: A Complete Guide to the Temporal API

The Temporal API is a modern, built-in JavaScript solution designed to overhaul and replace the legacy Date object. It introduces a comprehensive suite of distinct, immutable objects for managing dates, times, durations, and time zones with exact precision. By cleanly separating wall-clock times from global timestamps, Temporal eliminates decades of common date-handling pitfalls in web applications.

The Analogy: Mechanical Wall Clocks vs. Modern Atomic Smartwatches

To understand why we need the Temporal API, picture a traditional mechanical wall clock hanging in a busy train station. If you manually turn the clock's minute hand forward to adjust for a delay, the internal gears might accidentally shift the hour hand, slip off track, or completely lose alignment with the sun outside. Worse yet, if two people try to adjust the clock at the same time, the clock's physical state gets corrupted.

This is how JavaScript's legacy Date object operates: it is mutable (meaning its value can be modified after creation) and constantly tries to guess your local time zone behind the scenes.

The Temporal API, on the other hand, acts like a synchronized atomic smartwatch network. When you want to see what time it will be in three hours, the smartwatch does not alter your current time display. Instead, it generates a brand-new, unchangeable reading specifically for that future moment. It keeps local time, calendar dates, and exact global timestamps in separate, dedicated digital registers, ensuring that adjusting one setting never corrupts another.

Why It Matters in Daily Web Development

Every day, software engineers write code that schedules notifications, processes monthly recurring subscriptions, or logs critical system events. Using traditional JavaScript Date handling, these operations frequently trigger subtle, costly bugs in production systems:

Accidental Mutations: Passing a standard date object into a utility function might accidentally alter the original date variable elsewhere in your codebase.
Time Zone Confusions: A traditional date object automatically converts timestamps based on the local machine's system settings. This causes a server running in UTC (Coordinated Universal Time, the primary time standard by which the world regulates clocks) to interpret a user's midnight entry as 4:00 PM the previous day.
Daylight Saving Time (DST) Distortions: Calculating 24 hours from a given moment during a DST transition can result in events firing an hour early or late.

The Temporal API solves these daily problems by enforcing immutability and introducing specific types like Temporal.PlainDate (for calendar dates without times) and Temporal.ZonedDateTime (for exact moments anchored to a specific geographic region).

Code Example: Node.js Date Calculations

Below is a comparison showing how legacy Node.js date handling leads to unwanted side effects compared to the clean, safe approach provided by the Temporal API.

// Legacy JavaScript Date Approach (Prone to bugs)
const eventDate = new Date('2025-05-01T10:00:00Z');

// We want to calculate a follow-up date 7 days later
const followUpDate = eventDate;
followUpDate.setDate(eventDate.getDate() + 7);

// BUG: Mutating followUpDate unexpectedly altered eventDate too!
console.log(eventDate.toISOString()); // Output: 2025-05-08T10:00:00.000Z


// Modern Temporal API Approach (Safe and Predictable)
// Note: Temporal is available in modern JS runtimes or via the @js-temporal/polyfill package
const { Temporal } = require('@js-temporal/polyfill');

// Create an exact zonal date-time for an event in Tokyo
const originalEvent = Temporal.ZonedDateTime.from({
  timeZone: 'Asia/Tokyo',
  year: 2025,
  month: 5,
  day: 1,
  hour: 10,
  minute: 0
});

// Add 7 days cleanly without modifying the original object
const reminderEvent = originalEvent.add({ days: 7 });

// The original object remains entirely untouched
console.log(originalEvent.toString()); // 2025-05-01T10:00:00+09:00[Asia/Tokyo]
console.log(reminderEvent.toString());  // 2025-05-08T10:00:00+09:00[Asia/Tokyo]

Key Takeaway

Transitioning to the Temporal API shifts date manipulation from a fragile, error-prone chore into a robust, type-safe development experience. By cleanly isolating distinct concepts like calendar dates, wall-clock times, and regional time zones into immutable objects, Temporal guarantees that your application processes time consistently across every server and browser on the planet.

Enjoyed this breakdown? Support my open-source work on Buy Me a Coffee or explore react-hook-lab!

Originally published on my blog. You can read the alternative breakdown here.

MyZubster Dev Update: Local AI, GitHub Onboarding & Decentralized Collaboration 🚀

Wed, 21 Oct 2026 05:38:10 +0000

MyZubster Dev Update: Local AI, GitHub Onboarding & Decentralized Collaboration 🚀

We're building MyZubster, an open-source ecosystem connecting developers, independent projects, services, and communities.

Today we made significant progress on Zorgax, our locally hosted AI assistant.

Our goal is simple: make it easier for anyone — including people with no programming experience — to join GitHub, contribute to projects, and become part of a collaborative ecosystem.

🔧 What We Built and Tested

1. Vercel ↔ VPS Connectivity

We restored communication between our Vercel Production deployment and the VPS backend using a signed HTTPS gateway.

Production assistant endpoint responding successfully
HTTP 502 connectivity issue resolved in our tests
Node.js gateway operational
MongoDB connected
Local Ollama inference available

2. GitHub Contributor Onboarding

We implemented a deterministic onboarding guide in Zorgax.

The assistant can now:

Guide new users to GitHub registration
Recognize completion messages such as FATTO
Ask for a GitHub username
Recognize that username within the conversation
Advance to Git and GitHub CLI setup
Avoid claiming an account has been verified without evidence

3. Real Production Testing

Our regression tests passed: 4/4.

We also tested the public Zorgax chat and confirmed that the assistant correctly advanced from GitHub registration to the username and tooling stages.

4. Local AI Architecture

Our system currently combines:

Node.js backend
Vercel frontend and gateway
VPS-hosted services
Ollama with qwen2.5:3b
Deterministic onboarding rules
MongoDB

This approach helps us keep sensitive onboarding workflows predictable while using local AI for more flexible assistance.

🐛 What Still Needs Work

Our tests also revealed that Zorgax can lose track of the onboarding sequence when it falls back to Ollama.

For example, after Git and GitHub CLI version checks, it may answer a different MyZubster-related question instead of continuing with GitHub authentication.

We're working toward a complete flow:

GitHub signup → username → Git/CLI checks → authentication → repository → first pull request

We want every step to be clear, verifiable, and beginner-friendly.

🌍 Beyond a Traditional Marketplace

MyZubster is also exploring ways to connect:

Independent developer projects
Skills and services through a marketplace
Contributor profiles and verifiable contributions
Local community initiatives
Decentralized infrastructure and autonomous nodes
Privacy-conscious payments and collaboration

Our longer-term vision is a federated ecosystem where contributors can develop their own projects while remaining connected to a broader network.

These are ongoing development goals, not claims that every feature is already production-ready.

🤝 Looking for Contributors

We're particularly interested in developers who want to help with:

AI-assisted GitHub onboarding
Node.js and API testing
UX/UI simplification
GitHub automation
Marketplace workflows
VPS and decentralized infrastructure
Open-source documentation

You don't need to build everything. Even a small, well-tested contribution can move the project forward.

Explore MyZubster: https://www.myzubster.com

Try Zorgax: https://www.myzubster.com/zorgax

GitHub: https://github.com/MyZubster-Ecosystem

Let's collaborate

We're interested in connecting independent developers and projects rather than requiring everyone to work inside a single centralized system.

Question for the community: How would you design an AI onboarding assistant that guides a complete beginner from creating a GitHub account to opening their first pull request?

OpenSource #AI #GitHub #NodeJS #Decentralization #DeveloperCommunity

Demystifying MongoDB Write Concern and Read Concern in Node.js

Mon, 19 Oct 2026 09:22:10 +0000

What Are Write Concern and Read Concern?

In MongoDB, write concern and read concern are configuration settings that specify the level of guarantee you require when saving or retrieving data across a distributed cluster. A write concern determines how many database nodes must successfully record an update—and whether that data must be written to disk—before MongoDB reports success to your application. A read concern determines how fresh and consistent the data returned by a query must be, protecting your application from reading temporary data that might later be reversed.

Together, these two settings let developers choose where their application sits on the classic spectrum between maximum execution speed and rock-solid data safety.

The Real-World Analogy: Certified Mail vs. Kitchen Whiteboards

Imagine you run a bustling catering kitchen with multiple head chefs.

When taking an order, you have two choices for your write concern. You could scribble the customer's allergy request on a loose scrap of paper and immediately yell "Order placed!" (a fast write, but if you drop the paper, the order is lost). Alternatively, you could walk the order ticket over to the main order board, watch two sous-chefs sign off on it, and wait for them to file it in the metal lockbox before confirming the order with the customer. The second route takes several seconds longer, but you are guaranteed that the kitchen will never forget the order even if one chef walks out.

Now consider the read concern. If a waiter needs to know the soup of the day, they can either glance at a whiteboard where someone started drafting ideas with an erasable marker, or they can refuse to serve anything until they see the official menu stamped and approved by the head kitchen manager. Glancing at the draft whiteboard is instantaneous, but you might serve an experimental soup that the manager ultimately rejects five minutes later.

Why It Matters in Daily Software Engineering

Modern web architectures rarely run on a single database server. Instead, they use replica sets—groups of connected servers where one "primary" node handles writes and multiple "secondary" nodes replicate that data for redundancy.

If you leave MongoDB on default settings during high-traffic moments, subtle race conditions emerge:

Rollback Catastrophes: If an API endpoint confirms an e-commerce order after writing only to the primary node, and that primary node suddenly experiences a hardware failure before syncing to secondary nodes, the cluster elects a new primary. Your customer's order is wiped out, yet your payment processor already charged their credit card.
Dirty Reads: A background worker updates a bank balance. A microservice queries the secondary replica before the primary finishes replicating. If that transaction aborts or fails over, the microservice has operated on data that never officially existed.

By tuning write and read concerns, software engineers intentionally shield critical operations (payments, account transfers, credential updates) from data loss while letting low-risk operations (logging, view counters, user analytics) run with blazing speed.

Implementation in Node.js

Here is how you apply write and read concerns when working with the official MongoDB Node.js driver:

import { MongoClient } from 'mongodb';

const client = new MongoClient('mongodb://localhost:27017,localhost:27018/?replicaSet=rs0');

async function processPayment(orderId, paymentDetails) {
  await client.connect();
  const db = client.db('store');

  // Configure high-durability options for financial transactions
  const ordersCollection = db.collection('orders', {
    writeConcern: {
      w: 'majority',      // Must be acknowledged by more than half of the replica set nodes
      j: true,            // Must be written to the on-disk journal before confirming
      wtimeout: 5000      // Abort if replicas do not respond within 5 seconds
    },
    readConcern: {
      level: 'majority'   // Only read data committed by a majority of nodes
    }
  });

  // Step 1: Safely write with acknowledgment
  const writeResult = await ordersCollection.updateOne(
    { _id: orderId },
    { $set: { status: 'PAID', payment: paymentDetails } }
  );

  // Step 2: Read guaranteed, durable data
  const confirmedOrder = await ordersCollection.findOne({ _id: orderId });

  console.log('Payment processed safely:', confirmedOrder._id);
  return confirmedOrder;
}

In this example, specifying w: 'majority' and j: true ensures that the write cannot be rolled back if the primary server abruptly crashes. Setting level: 'majority' on the read ensures your application reads only finalized records.

Key Takeaway

Database speed and data reliability exist on a continuous slider: write concern dictates how hard your cluster works to preserve your updates, while read concern dictates how strictly your cluster verifies the stability of what you fetch. Mastering both allows you to protect financial transactions and vital data without unnecessarily throttling high-volume, low-risk endpoints across your stack.

Enjoyed this breakdown? Support my open-source work on Buy Me a Coffee or explore react-hook-lab!

Originally published on my blog. You can read the alternative breakdown here.

Stop Hardcoding Secrets: Master Environment Variables and Dotenv in Node.js

Sat, 17 Oct 2026 08:03:00 +0000

Stop Hardcoding Secrets: Master Environment Variables and Dotenv in Node.js

When building modern web applications, keeping sensitive information secure is just as critical as writing functional code. One of the most fundamental practices every developer must learn is using environment variables and dotenv configuration.

What Are Environment Variables and Dotenv Configuration?

Environment variables are dynamic key-value pairs stored at the operating system or runtime level outside of an application's codebase. In Node.js development, dotenv configuration refers to utilizing a lightweight library called dotenv that automatically loads settings from a plain .env file into Node's runtime environment (process.env). This mechanism allows developers to customize application behavior and supply secret credentials without modifying source code files.

A Real-World Analogy: The Wallet Index Card

Imagine you own a custom leather wallet. Instead of permanently embroidering your house address, garage passcode, and emergency contact numbers onto the outer leather where every passerby can read them, you write those sensitive details on a small index card and slide it into an interior pocket.

When you move to a new apartment or change your passcode, you do not buy a brand-new wallet or destroy the leather stitching. You simply pull out the old index card and slide in an updated one. In this analogy, your wallet is your core application codebase, while the index card inside the pocket acts as your local .env file holding temporary, site-specific configuration details.

Why Environment Variables Matter Every Single Day

In professional software engineering, committing raw passwords, secret encryption keys, or API tokens directly into a Git repository is a dangerous vulnerability. Automated web scraping bots constantly crawl public code repositories like GitHub, scanning for exposed database passwords or cloud provider credentials within seconds of a commit. Exposing these keys can lead to compromised database instances, stolen customer data, or massive financial liability from unauthorized cloud usage.

Furthermore, modern applications run across multiple environments throughout their lifecycle, including local development machines, staging servers for testing, and live production servers handling real user traffic. Environment variables allow engineers to deploy the exact same application code across all these distinct targets. The application simply reads its configuration from whatever local environment variables exist on that specific host server.

Code Example: Configuring Express with Dotenv

Below is a standard Node.js and Express server setup using the dotenv package to securely access server configuration values and third-party API credentials.

// Step 1: Import dependencies
// Ensure you have installed express and dotenv via npm: npm install express dotenv
require('dotenv').config();
const express = require('express');

const app = express();

// Step 2: Read configuration values from process.env
const PORT = process.env.PORT || 3000;
const API_KEY = process.env.PAYMENT_GATEWAY_KEY;
const DATABASE_URL = process.env.DATABASE_URL;

// Step 3: Define an Express route utilizing the loaded variables
app.get('/api/status', (req, res) => {
  if (!API_KEY) {
    return res.status(500).json({ error: 'Missing critical API configuration.' });
  }
  
  res.json({
    status: 'Server is running securely',
    connectedDatabase: DATABASE_URL ? 'Connected' : 'Not Configured',
    environment: process.env.NODE_ENV || 'development'
  });
});

// Step 4: Start the server using the dynamic PORT
app.listen(PORT, () => {
  console.log(`Server listening on port ${PORT}`);
});

Along with this JavaScript file, you create a file named .env in the root folder of your project:

PORT=5000
NODE_ENV=development
DATABASE_URL=mysql://user:password@localhost:3306/dev_db
PAYMENT_GATEWAY_KEY=secret_live_key_987654321

Crucially, you must add .env to your project's .gitignore file so it is never tracked by version control or pushed to remote repositories.

Key Takeaway

Decoupling application code from environment configuration is a foundational pillar of modern, secure software architecture. By adopting environment variables and dotenv configuration early in your development pipeline, you shield your systems from catastrophic credential leaks and build resilient applications ready to run anywhere without friction.

Enjoyed this breakdown? Support my open-source work on Buy Me a Coffee or explore react-hook-lab!

Originally published on my blog. You can read the alternative breakdown here.

A very Simple Rate Limiting App ( Node.JS )

Thu, 15 Oct 2026 13:40:19 +0000

enter code here

const express = require("express");

const app = express();

const requests = new Map();

const LIMIT = 5;
const WINDOW = 60 * 1000; // 1 minute

app.use((req, res, next) => {
  const ip = req.ip;
  const now = Date.now();

  let data = requests.get(ip);

  // First request or expired time window
  if (!data || now - data.startTime >= WINDOW) {
    data = {
      count: 1,
      startTime: now,
    };

    requests.set(ip, data);
    return next();
  }

  // Rate limit exceeded
  if (data.count >= LIMIT) {
    return res.status(429).json({
      message: "Too many requests. Try again later.",
    });
  }

  data.count++;
  next();
});

app.get("/", (req, res) => {
  res.json({ message: "Request successful!" });
});

app.listen(3000, () => {
  console.log("Server running on http://localhost:3000");
});

Demystifying Greedy Algorithms: How Local Choices Solve Global Problems in Node.js

Thu, 15 Oct 2026 08:35:56 +0000

What is a Greedy Algorithm?

A greedy algorithm is an algorithmic strategy that solves a problem by making the most optimal decision at each individual step. Instead of looking ahead to long-term consequences or evaluating every possible combination, it focuses exclusively on immediate, short-term gain. If local choices logically compound into the ideal final result, the greedy strategy succeeds with maximum computational efficiency.

The Cashier’s Coin Trick

Imagine you are a cashier at a grocery store, and you need to give a customer 41 cents in change using the fewest coins possible. You look at your cash register, which contains quarters (25¢), dimes (10¢), nickels (5¢), and pennies (1¢).

Instead of calculating every conceivable combination of coins, you naturally make the "greedy" choice at each step:

Grab the largest coin that does not exceed 41 cents: a Quarter (25¢). You now owe 16 cents.
Grab the largest coin that does not exceed 16 cents: a Dime (10¢). You now owe 6 cents.
Grab the largest coin that does not exceed 6 cents: a Nickel (5¢). You now owe 1 cent.
Grab the largest coin that does not exceed 1 cent: a Penny (1¢). You now owe 0 cents.

You solved the problem instantly by picking the biggest immediate payoff at every step. That is precisely how a greedy algorithm operates in software.

Why Greedy Algorithms Matter in Tech

In software engineering, computing the mathematically perfect solution for massive datasets can sometimes take hours—or even years—if an application tries to test every possible outcome. Software developers intentionally use greedy algorithms when a guaranteed quick answer that is optimal saves massive amounts of CPU memory and processing time.

For instance, network routing protocols use greedy choices to route internet data packets along the shortest immediate paths between routers. Database query engines, file compression utilities like ZIP tools, and job schedulers in operating systems all rely on greedy strategies to keep web software responsive and performant under heavy traffic loads.

Code Example: Making Change in Node.js

Here is a plain Node.js implementation showing how a greedy approach quickly calculates coin change for any given amount:

// Function to give change using the minimum number of coins
function makeChange(amount, coins = [25, 10, 5, 1]) {
  const result = {};
  let remaining = amount;

  // Iterate through available coins sorted from highest to lowest
  for (const coin of coins) {
    if (remaining <= 0) break;

    // Calculate how many coins of this denomination fit
    const count = Math.floor(remaining / coin);
    if (count > 0) {
      result[`${coin}c`] = count;
      remaining -= count * coin;
    }
  }

  return result;
}

// Example usage: Change for 87 cents
const change = makeChange(87);
console.log('Change breakdown:', change);
// Output: Change breakdown: { '25c': 3, '10c': 1, '1c': 2 }

In this code, the algorithm loops through sorted coin values and takes as many of the highest denomination as possible before moving to smaller coins.

The Takeaway

Greedy algorithms trade exhaustive foresight for immediate speed, proving that making the best local decision at every step can often yield the best global outcome. While they do not work for every single problem—especially when early sacrifices yield better long-term rewards—understanding when to apply a greedy strategy is a fundamental skill for building high-performance backend systems.

Enjoyed this breakdown? Support my open-source work on Buy Me a Coffee or explore react-hook-lab!

Originally published on my blog. You can read the alternative breakdown here.

Demystifying Graph Databases: Beyond the Table and Column

Tue, 13 Oct 2026 11:07:58 +0000

What Is a Graph Database?

A graph database is a specialized storage engine designed to treat relationships between data points with the exact same importance as the data points themselves. Instead of storing information in rigid rows, columns, and isolated tables, it organizes data into distinct items called nodes and explicit connections called edges. This architectural design means finding how two items connect requires following a direct pointer rather than calculating expensive match-ups across disparate tables.

The Real-Life Analogy: The Train Route Map

Imagine planning a trip across Europe using a subway or train route map. Each city station is an actual physical location (a node), and the train tracks running directly between them are the routes (the edges).

If you want to travel from Lisbon to Vienna, you trace your finger directly along the tracks that link each connecting station. You do not need to scan an enormous phone directory listing every citizen living in Madrid, Paris, or Berlin just to see if their tracks align. You follow the direct physical lines already drawn on the map. A traditional relational database acts like that giant phone directory, forcing you to cross-reference multiple massive lists to deduce who connects to whom. A graph database simply follows the drawn train tracks.

Why It Matters Daily in Software Engineering

In standard relational systems like MySQL, finding indirect relationships—such as "friends of friends" or "fraudulent transactions linked by shared IP addresses"—requires combining tables using operations called JOINs. When you join a table to itself three, four, or five levels deep, query execution time degrades exponentially. The database engine must scan and compare millions of rows across foreign keys, causing CPU spikes, connection pool exhaustion, and slow API responses.

Engineers reach for graph databases (or graph data modeling patterns) to prevent these multi-hop queries from bringing production services to a halt. When building recommendation feeds, access control lists (RBAC/ABAC), or fraud rings, graph structures guarantee index-free adjacency. That means traversing from one record to its neighbor takes constant time, regardless of whether your database holds ten thousand records or ten billion.

Code Demonstration: Graph Traversal in Node.js

To see how naturally relationships can be navigated compared to complex SQL recursive joins, here is a pure Node.js in-memory graph implementation showing how to find second-degree network connections (friends of friends):

// A lightweight graph implementation using Node.js Maps and Sets
class SocialGraph {
  constructor() {
    this.adjacencyList = new Map();
  }

  addUser(userId) {
    if (!this.adjacencyList.has(userId)) {
      this.adjacencyList.set(userId, new Set());
    }
  }

  // An edge represents a bidirectional friendship
  addFriendship(userA, userB) {
    this.addUser(userA);
    this.addUser(userB);
    this.adjacencyList.get(userA).add(userB);
    this.adjacencyList.get(userB).add(userA);
  }

  // Find recommendations: friends of friends who are not direct friends
  getFriendRecommendations(userId) {
    if (!this.adjacencyList.has(userId)) return [];

    const directFriends = this.adjacencyList.get(userId);
    const recommendations = new Set();

    for (const friend of directFriends) {
      const friendsOfFriend = this.adjacencyList.get(friend);
      for (const candidate of friendsOfFriend) {
        // Exclude the user themselves and existing direct friends
        if (candidate !== userId && !directFriends.has(candidate)) {
          recommendations.add(candidate);
        }
      }
    }

    return Array.from(recommendations);
  }
}

// Usage example
const network = new SocialGraph();
network.addFriendship('Alice', 'Bob');
network.addFriendship('Bob', 'Charlie');
network.addFriendship('Charlie', 'Diana');

const suggestionsForAlice = network.getFriendRecommendations('Alice');
console.log('Suggestions for Alice:', suggestionsForAlice); 
// Output: [ 'Charlie' ]

Notice that this requires zero nested scans over an entire table of users. We simply jump along existing references.

Key Takeaway

Relational databases excel at structured transactions and financial ledgers, but when your application's core value depends on discovering how entities connect, forcing that data into two-dimensional tables creates severe architectural friction. Choosing a graph model shifts the focus from data isolated in rows to the living connections between them, turning sluggish multi-table bottlenecks into instant pointer hops.

Enjoyed this breakdown? Support my open-source work on Buy Me a Coffee or explore react-hook-lab!

Originally published on my blog. You can read the alternative breakdown here.

Fix pnpm Migration Phantom Dependency Errors

Sat, 10 Oct 2026 20:36:36 +0000

Migrating a Node.js project from package managers that use hoisted node_modules structures to strict content-addressable package managers often exposes hidden import assumptions that break continuous integration runners.

Quick solution:

pnpm add sharp

When a Node script fails with Error [ERR_MODULE_NOT_FOUND] for a package like sharp imported from scripts/medium-package.mjs after a clean continuous integration install, add the missing package to your manifest using pnpm add sharp for runtime dependencies or pnpm add -D sharp for build tools. Prerequisites include running a clean environment with a lockfile and running pnpm install --frozen-lockfile to ensure accurate dependency trees after committing updates.

Diagnosing Module Resolution Versus Build Failures

Traditional package managers often rely on node_modules layouts where dependencies of dependencies are hoisted to the root directory, allowing scripts or build tools to import packages without explicitly declaring them in the project manifest. When transitioning to pnpm, its strict symlinked and isolated structure removes this hoisting behavior. Undeclared imports that worked locally due to cached modules or ambient hoisting suddenly fail in clean continuous integration environments.

Symptom to Resolution Guide

Option	When to Use It	Trade-off or Failure Mode	Recommendation
`pnpm add <package>`	Runtime dependency is missing from the manifest	Increases direct dependency count	Explicitly declare all runtime packages in dependencies
`pnpm add -D <package>`	Build or script tool is missing from the manifest	Might bloat devDependencies if misused	Use for local build scripts and automation tasks
Frozen Lockfile Update	Pipeline throws lockfile mismatch errors	Requires committing workspace changes	Always commit lockfile updates before pushing to remote CI

Auditing and Fixing Missing Direct Imports

To prevent continuous integration failures, audit all custom scripts, build hooks, and automation tools located in directories like scripts or tools. Ensure every package imported via ES module syntax or CommonJS require statements is explicitly listed under dependencies or devDependencies in your project configuration file.

{
  "devDependencies": {
    "sharp": "*"
  }
}

After declaring the missing dependencies, update and commit the correct workspace lockfile, rerun a clean frozen install in a fresh checkout or clean workspace, and then rerun the exact failing script to verify that module resolution succeeds without relying on residual global state.

Verification and Execution Steps

Run pnpm install --frozen-lockfile in a fresh checkout or clean workspace to simulate a clean installation and verify that the expected result is a successfully generated dependency tree without unexpected lockfile modifications.
Locate the specific importing workspace or package manifest and verify that any required module is explicitly declared under dependencies or devDependencies.
Execute the exact failing script locally using node scripts/medium-package.mjs to confirm that the previous module resolution error no longer occurs.
Review continuous integration pipeline logs after pushing your updated lockfile and manifest changes to ensure artifact generation completes successfully.

Caching Strategies Demystified: Cache-Aside, Write-Through, and Write-Behind

Fri, 09 Oct 2026 10:55:06 +0000

In software development, caching is the practice of storing frequently used data in a high-speed, temporary storage layer so future requests can be served faster. Caching strategies are the architectural patterns that dictate how your application coordinates the movement of data between this high-speed temporary memory (the cache) and your permanent, slower storage system (the database). Choosing the right strategy ensures your application stays incredibly fast while keeping data accurate and consistent.

The Notebook and the Library Analogy

Imagine you are a researcher studying in a massive university library (the database) containing millions of books.

Cache-Aside (Lazy Loading): You keep a blank personal notebook (the cache) on your desk. When you need a fact, you look in your notebook first. If it is there (a cache hit), you read it instantly. If not (a cache miss), you walk into the library archives, search the shelves, find the fact, write it down in your notebook, and then read it.
Write-Through: Every time you discover a new piece of information, you write it in your personal notebook and immediately send a copy to the library's archiving desk to ensure both records are updated simultaneously before you proceed.
Write-Behind (Write-Back): When you are brainstorming rapidly, you scribble your notes exclusively in your notebook. At the end of the day, you hand your notebook to a library assistant who updates the main archives in the background while you sleep.

Why Caching Strategies Matter in the Tech Industry

Without these strategies, modern platforms would collapse. If a viral social media post forces millions of users to fetch the same user profile directly from a database, the database will exhaust its connection pool, slow down, and eventually crash.

Software engineers use these patterns strategically to balance speed and data integrity:

Cache-Aside prevents databases from being choked by heavy read traffic while keeping implementation simple.
Write-Through avoids data discrepancies, ensuring that critical data (like bank account balances) is never out of sync between the memory and the disk.
Write-Behind is utilized in write-heavy systems (like logging user activity or IoT sensor tracking) to group thousands of database writes into batch updates, protecting disk-based storage from wearing out or bottlenecking.

Implementing Cache-Aside in Node.js & Express

Here is how you can implement a standard Cache-Aside strategy using Express.js and MySQL. In this scenario, we check an in-memory cache object before querying our relational database.

const express = require('express');
const mysql = require('mysql2/promise');
const app = express();

// Mocking our fast memory cache (e.g., Redis)
const memoryCache = new Map();

// MySQL connection pool configuration
const dbPool = mysql.createPool({
  host: 'localhost',
  user: 'root',
  database: 'production_db'
});

// Route utilizing Cache-Aside
app.get('/api/users/:id', async (req, res) => {
  const userId = req.params.id;
  const cacheKey = `user:${userId}`;

  try {
    // 1. Check if the data exists in the high-speed cache
    if (memoryCache.has(cacheKey)) {
      console.log('Cache Hit! Serving from memory.');
      return res.json(memoryCache.get(cacheKey));
    }

    // 2. Cache Miss: Retrieve data from the slow MySQL database
    console.log('Cache Miss. Fetching from MySQL database...');
    const [rows] = await dbPool.query('SELECT id, name, email FROM users WHERE id = ?', [userId]);

    if (rows.length === 0) {
      return res.status(404).json({ error: 'User not found' });
    }

    const userData = rows[0];

    // 3. Write data to the cache so the next request is instant
    memoryCache.set(cacheKey, userData);

    return res.json(userData);
  } catch (error) {
    return res.status(500).json({ error: 'Internal Server Error' });
  }
});

app.listen(3000, () => console.log('Server running on port 3000'));

Key Takeaway

Caching is not a one-size-fits-all solution; it is a careful compromise between performance and accuracy. While caching can make your application feel instantaneous, selecting the wrong update strategy can lead to stale data displays or lost records, making it vital to match your architectural pattern to your application's specific read-and-write ratios.

Originally published on my blog. You can read the alternative breakdown here.

🧠 Coder Legion Dev Log — Building the MyZubster Metaverse from the infrastructure up

Thu, 08 Oct 2026 02:32:06 +0000

🧠 Coder Legion Dev Log — Building the MyZubster Metaverse from the infrastructure up

I’ve been working on the MyZubster Metaverse with a very specific engineering approach:

do not start from the 3D layer. Start from identity, authorization, state, lifecycle and failure handling.

The current stack already includes several production-oriented building blocks.

Identity and session model

The Metaverse supports both:

guest characters
authenticated, account-linked identities

For authenticated users, canonical character data is resolved server-side so the client cannot freely override trusted identity state.

This matters because virtual-world logic becomes much harder to secure once identity, permissions and session state are mixed together.

Realtime authorization

One of the more important backend changes was introducing a fail-closed authorization path for community realtime channels.

If the membership authority is unavailable, access is denied instead of waiting on buffered database operations or leaving socket authorization in an ambiguous state.

I added coverage for:

disconnected MongoDB state
active persisted membership
membership query failures
unauthorized access attempts

This is a small change in code, but a meaningful system-design decision.

Room lifecycle

The room layer already supports:

public / authenticated / private access policies
invitations
session creation
participant join/leave
host controls
stage requests
speaker approval/revocation
participant moderation
message reporting
bounded moderation history

The important point is that the room is treated as a stateful system, not just a visual container.

Frontend state hardening

I’ve also been cleaning up React lifecycle behavior.

Recent fixes include:

reloading room state when authentication changes
stabilizing async data loading with useCallback
tightening effect dependencies
keeping UI state aligned with server-side session state

Example recent commits:

0b5e7fa — refresh Metaverse room when auth state changes
fa94a88 — stabilize frontend loading callback
0347827 — make paid analytics explicit instead of automatic

Auth and observability testing

I also aligned tests around:

authenticated Metaverse flows
server-backed sessions
auth expiry
same-origin authenticated requests
health endpoint behavior
storage failure semantics

A previous checkpoint also hardened realtime membership authorization and added test coverage around failure states:

0af298e — fail-closed realtime auth
9ac9f53 — auth session test alignment
5bf365d — auth + observability test alignment

Deployment discipline

Another part of the work was CI/CD hygiene.

I removed automatic activation of paid Vercel Web Analytics from the deployment workflow.

That means deployment stays deterministic and billing-related features remain explicit operational decisions.

Current architecture direction

The development sequence I’m following is:

identity
→ authorization
→ state
→ room lifecycle
→ moderation
→ observability
→ realtime rendering

This is intentional.

A visually impressive world with weak session handling, unclear authorization or fragile failure modes is still a fragile system.

So the current priority is to make the platform behavior coherent before connecting the final immersive realtime rendering layer.

Skills this work is exercising

This project has been a practical exercise in:

React hooks and state synchronization
Node.js backend architecture
REST API design
realtime authorization
authentication and session handling
MongoDB / Mongoose
fail-closed security patterns
moderation systems
automated testing
observability
CI/CD
Vercel deployment workflows
production debugging
incremental system hardening

The immersive layer is still the next milestone.

But the system underneath it is becoming much more real.

Build the world after the rules of the world are reliable.

CoderLegion #SoftwareEngineering #React #NodeJS #MongoDB #Realtime #SystemDesign #DevOps #WebDevelopment #Metaverse #OpenSource

Hardening a Contributor Verification VPS Without Breaking Production

Wed, 07 Oct 2026 11:17:48 +0000

Building a Safer Contributor Verification VPS Without Breaking Production

Hardening a live VPS is rarely about applying a single security checklist.

The harder problem is this:

How do you reduce attack surface on a server that is already running production services, contributor verification workloads, Docker environments, databases, reverse proxies, and experimental integrations — without breaking any of them?

That was the situation on a MyZubster VPS we are evolving into a more structured Contributor Verification Node.

The objective was deliberately conservative:

identify what is actually running
reduce unnecessary public binds
preserve all live workloads
distinguish repository architecture from runtime architecture
document every conclusion with reproducible technical evidence

No mass restarts. No destructive Git operations. No speculative rewiring.

Just one dependency at a time.

The VPS was doing too many jobs at once

The server was already hosting a mix of services:

nginx
MyZubster production gateway
PM2 applications
Docker contributor environments
MongoDB
Monero services
Bitcoin verification services
IPFS
Qdrant
Ollama
n8n
contributor bridges and pilot services

This creates a common infrastructure trap.

A port may look unnecessary.

A process may look duplicated.

A service may appear safe to restart.

But if you do not first understand the dependency graph, a cleanup command can become an outage.

That immediately ruled out broad actions such as:

pm2 restart all

or:

git reset --hard
docker system prune
ufw reset

The working rule became:

observe
→ identify ownership
→ map dependencies
→ change one thing
→ verify
→ persist

Start from sockets, not assumptions

The first meaningful artifact was a listener inventory.

Using:

ss -lntp

we classified services into two groups.

Some were already correctly restricted:

127.0.0.1:5003   MyZubster Gateway
127.0.0.1:8787   BTC verifier
127.0.0.1:27017  MongoDB
127.0.0.1:6333   Qdrant
127.0.0.1:5678   n8n
127.0.0.1:8092   contributor bridge

Others were bound globally:

*:5002
0.0.0.0:5005
0.0.0.0:5173

A global bind does not automatically mean “publicly exposed.”

A firewall may still block traffic.

But it does mean the application is willing to accept traffic on every interface, which increases ambiguity and weakens least-privilege design.

So the next question was not:

Can we close this port?

It was:

Who actually needs to reach this service?

Hardening port 5173

The frontend process was listening on:

0.0.0.0:5173

But nginx already proxied to it locally:

location / {
    proxy_pass http://localhost:5173;
}

That gave us the actual runtime graph:

Internet
   ↓
nginx :443
   ↓
127.0.0.1:5173

There was no need for the Node process itself to listen on every interface.

The original code contained:

server.listen(PORT, '0.0.0.0', () => {

We changed only the bind address:

server.listen(PORT, '127.0.0.1', () => {

Then checked syntax:

node --check server-static.js

and restarted only that PM2 process:

pm2 restart myzubster-web

Not the entire PM2 stack.

Verify before persisting

After the restart:

ss -lntp | grep ':5173'

showed:

127.0.0.1:5173

The local service still returned 200.

Then nginx HTTP returned the expected redirect.

Finally HTTPS was tested directly against the local nginx instance using the real virtual host:

curl -k -sS -o /dev/null \
  --resolve myzubster.com:443:127.0.0.1 \
  -w 'HTTPS nginx -> %{http_code}\n' \
  https://myzubster.com/

Result:

HTTPS nginx -> 200

Only after that did we run:

pm2 save

This sequencing matters.

You do not persist a new runtime state before proving that the change is healthy.

Hardening port 5002

Port 5002 belonged to another Node application.

It was listening globally because the server was started without an explicit host:

httpServer.listen(PORT, () => {

Before changing it, we checked:

nginx references
frontend references
active TCP connections
WebSocket usage
service routes

The frontend already targeted the service internally:

const API_TARGET = 'http://127.0.0.1:5002';

There was no nginx route sending external traffic directly to 5002.

No active external dependency was visible.

So we changed the listener to:

httpServer.listen(PORT, '127.0.0.1', () => {

Again:

backup
→ syntax check
→ restart one process
→ verify socket
→ verify health

The service remained reachable locally.

The frontend proxy returned a 404 for one test route rather than 502.

That difference was useful.

A 404 meant:

upstream reachable, route missing

A 502 would have meant:

upstream unreachable

That distinction prevented us from treating a valid hardening step as a regression.

The interesting part: a broken TAZ contract

During the dependency review, we found a frontend WebSocket client that constructed:

`${protocol}//${window.location.host}/api/taz/live`

The TAZ interface also expected:

GET /api/taz/dashboard
GET /api/taz/xmr/summary
WS  /api/taz/live

At first this looked like something that might have been affected by our network changes.

So we tested the production gateway directly.

The results were:

404 Cannot GET /api/taz/dashboard
404 Cannot GET /api/taz/xmr/summary
404 Cannot GET /api/taz/live

This was important.

The feature was already incomplete.

The hardening work had not broken it.

That leads to one of the most useful lessons from the whole exercise:

A failure discovered during infrastructure work is not automatically caused by the infrastructure change.

Always reproduce the behavior against the actual upstream service.

Nginx was doing exactly what it should

The production nginx configuration routed:

location / {
    proxy_pass http://localhost:5173;
}

location /api/ {
    proxy_pass http://localhost:5003;
}

So /api/taz/live already reached the production gateway.

The gateway returned the 404.

That meant blindly adding WebSocket proxy headers such as:

proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";

would not solve the real problem.

A reverse proxy cannot invent an application endpoint.

There was already a realtime subsystem

Repository inspection revealed something more interesting.

MyZubster already contains a realtime implementation based on Socket.IO.

Its server path is:

/realtime

and supporting HTTP routes are mounted under:

/api/realtime

The realtime code exposes an explicit attachment function:

attachRealtimeServer(httpServer)

This changed the architecture question completely.

Instead of asking:

How do we build a new WebSocket service for TAZ?

the better question became:

Can TAZ reuse the existing realtime stack?

Maintaining two separate realtime systems would add unnecessary complexity unless there is a strong protocol reason.

Source code and production runtime were not the same thing

There was another subtle mismatch.

The repository's normal backend startup path includes:

attachRealtimeServer(server)

But the active systemd service starts:

/root/myzubster/scripts/start-gateway-systemd.js

and that startup path currently creates the server with:

const server = app.listen(port, host, ...)

without obviously attaching the realtime server.

So the repository may support realtime while production does not actually expose it.

This is a recurring infrastructure lesson:

Repository architecture is not runtime architecture.

A feature being present in source code does not prove that the active service manager actually executes it.

The authoritative chain is closer to:

systemd / PM2
→ executable
→ startup script
→ imported modules
→ bound socket
→ reverse proxy
→ externally observed response

That chain matters more than a repository grep.

Do not fabricate telemetry to make a dashboard look alive

The TAZ frontend expects values like:

drinksServed
xmrReceived
robot.status
transactions

During repository searches, we found unrelated simulation and robot data.

It would have been technically easy to wire some of that into the dashboard.

We deliberately did not.

A live interface must not display simulation values as if they were production evidence.

Our rule was:

no authoritative source → no authoritative metric

A degraded dashboard is more honest than a beautiful dashboard with fabricated telemetry.

What changed

The hardening work produced two concrete improvements:

MyZubsterWeb
0.0.0.0:5173
→
127.0.0.1:5173

and:

Urban Lab
*:5002
→
127.0.0.1:5002

The main gateway was already:

127.0.0.1:5003

nginx HTTPS continued to work.

Docker verification environments were not disturbed.

PM2 state was persisted only after verification.

No destructive Git operation touched the live checkout.

What we deliberately did not do

A large part of safe infrastructure work is knowing what not to touch.

We did not:

restart every PM2 application
reset firewall configuration
expose additional ports
modify nginx before proving nginx was the problem
connect TAZ to an unrelated WebSocket server
feed simulation values into production telemetry
reset or pull the live Git checkout
treat code presence as runtime proof

Those non-actions were part of the engineering result.

The Contributor Verification Node model

The broader goal is to evolve the VPS toward a structured Contributor Verification Node.

That should not mean giving every contributor unrestricted shell access.

A safer pattern looks like:

SSH identity
   ↓
dedicated Unix account
   ↓
dedicated workspace
   ↓
container / sandbox
   ↓
explicit mounts
   ↓
explicit network permissions
   ↓
reproducible verifier
   ↓
immutable evidence

The node should prove that a technical checkpoint can be reproduced.

It should not silently become a shared production administration box.

Current state

At the current checkpoint:

5173  localhost-only  ✅
5002  localhost-only  ✅
5003  localhost-only  ✅
nginx HTTPS           ✅
PM2 persistence       ✅
TAZ HTTP endpoints    missing
TAZ live WebSocket    missing
realtime subsystem    present in source
production realtime   wiring under review

This is not a flashy infrastructure milestone.

But it is a useful one.

We now know more precisely:

what is actually listening
which services need external exposure
which assumptions in the frontend are currently false
which realtime infrastructure already exists
where production startup differs from repository design

That is enough to make the next change smaller and safer.

Final takeaway

The most valuable part of production hardening is often not the final configuration.

It is the discipline of reducing uncertainty.

For every service, ask:

What process owns it?

Who actually needs to reach it?

Does it need to bind globally?

Which reverse proxy reaches it?

What does production really execute?

How will I prove the change did not break anything?

The result is not just a smaller attack surface.

It is a better mental model of the system.

And once the runtime model is accurate, contributor verification becomes much easier to isolate, reproduce, and trust.

The Developer's Safety Net: Understanding CI/CD Pipelines

Wed, 07 Oct 2026 10:36:33 +0000

A CI/CD pipeline (Continuous Integration and Continuous Delivery or Continuous Deployment) is an automated sequence of steps that software code passes through on its way to production. It acts as an automated quality-control assembly line, taking raw code written by a developer, testing it for bugs, compiling it, and safely delivering the finished product to the end users. By removing human error from the deployment process, it ensures software is delivered rapidly, frequently, and reliably.

The Analogy: The Modern Car Assembly Line

Imagine building cars in a high-tech factory. Instead of having a single mechanic manually build a car from scratch, test-drive it on open roads, and paint it by hand, you use a highly optimized, automated assembly line.

When an engineer designs a new steering wheel (analogous to a developer writing a new feature), they place it on the conveyer belt. Robotic arms instantly run physical stress tests on the wheel to ensure it doesn't break under pressure (automated testing). Next, other machines check if the wheel fits the steering column perfectly without disrupting the dashboard electronics (integration testing). If any test fails, the assembly line immediately halts, alarms ring, and the faulty wheel is rejected before it can ever be installed in a completed car. If everything passes, the car is automatically painted and loaded onto a transport truck, ready for the dealership (automated deployment).

Why It Matters Daily in the Tech Industry

In the daily grind of software development, engineering teams use CI/CD pipelines to prevent "integration hell"—the nightmarish scenario where multiple developers work on separate features for weeks, only to find that everything breaks when they try to combine their work at the last minute.

Without a pipeline, a developer might accidentally upload code that contains a syntax error or a broken database query, instantly crashing the live website for thousands of customers. A CI/CD pipeline prevents this disaster by intercepting the code changes. Before the new code ever touches the live production server, the pipeline automatically spins up an isolated test environment, runs hundreds of checks in seconds, and rejects the upload if even a single test fails. This keeps the application online, protects the business from costly downtime, and saves developers from stressful midnight firefighting sessions.

Putting It Into Code: An Automated Test Script

At the heart of any CI/CD pipeline is the test phase. Below is a simple Node.js and Express.js test using a popular testing framework called Jest. This script automatically checks if our web server is running and responding correctly. The CI/CD pipeline will execute this code every time a developer tries to push an update.

// server.js - Our simple Express server
const express = require('express');
const app = express();

app.get('/api/health', (req, res) => {
  res.status(200).json({ status: 'healthy', database: 'connected' });
});

module.exports = app;

// server.test.js - The test script run by our CI/CD pipeline
const request = require('supertest');
const app = require('./server');

describe('GET /api/health', () => {
  it('should return a 200 OK status and verify backend health', async () => {
    const response = await request(app).get('/api/health');
    
    // Verify the response code is exactly 200 (Success)
    expect(response.statusCode).toBe(200);
    
    // Verify the payload contains the expected health details
    expect(response.body.status).toBe('healthy');
  });
});

When a developer pushes code, the pipeline automatically runs npm test. If our code fails to return a 200 status code, the pipeline aborts, preventing the broken server from being deployed to production.

The Takeaway

CI/CD pipelines turn software shipping from a stressful, high-stakes manual event into a quiet, routine background process. By automating the repetitive steps of testing and deployment, they provide developers with an immediate feedback loop and a reliable safety net, allowing businesses to innovate rapidly without sacrificing system stability.

Originally published on my blog. You can read the alternative breakdown here.

From Contributor Research to a Verifiable Open-Source Node

Mon, 05 Oct 2026 10:14:47 +0000

From Contributor Research to a Verifiable Open-Source Node

How we built the MyZubster Open Period Care pilot with Node.js, Nginx, deterministic SHA-256, provenance, Evidence Payloads and a public read-only manifest

Over the last few days I’ve been working on a MyZubster pilot called Open Period Care.

The original objective was relatively simple: take a contributor’s research package and make it accessible inside the MyZubster ecosystem.

But the project quickly evolved into something more interesting.

Instead of treating the contributor’s work as static documentation, we started building a small verifiable research node: a service capable of exposing Knowledge Cards, provenance, evidence references, validation states and deterministic cryptographic fingerprints through a read-only API.

The result is now a working Node.js service running on a VPS behind Nginx, with the application itself bound only to localhost.

The architecture is intentionally conservative:

Contributor research
→ Knowledge Cards
→ provenance
→ evidence references
→ deterministic canonical JSON
→ SHA-256 fingerprints
→ Evidence Payload v1
→ private Node.js service
→ Nginx HTTPS reverse proxy
→ public read-only node manifest

The goal is not to claim that decentralization, blockchain or hashing magically makes research “true”.

The goal is much narrower and more useful:

make the origin, structure, version and state of evidence easier to inspect and reproduce.

Starting from real contributor work

The node is based on an actual Open Period Care research contribution produced inside the MyZubster contributor workflow.

The contribution includes two Knowledge Cards:

KC-OPC-001 — Multi-Layer Biomaterial Architecture for Reusable Textile Absorbents
KC-OPC-002 — Contributor Privacy, Data Minimization & Clinical Claim Boundaries

The research package also includes an evidence matrix, scientific and technical references, product requirements and explicit evidence states.

An important part of the design is that we do not flatten everything into a generic “verified” label.

Each piece of information maintains an epistemic state such as:

PROPOSED
SUPPORTED
TESTED
VERIFIED

For the current cards, the state is:

SUPPORTED

This distinction matters.

A research document supported by literature is not the same thing as a laboratory-tested prototype.

A laboratory-tested prototype is not the same thing as a clinically validated product.

A merged GitHub contribution is evidence of contribution provenance, not automatic scientific certification.

That boundary is intentionally preserved in the node.

Why a node instead of just Markdown files?

Markdown documentation is useful, but it becomes harder to integrate when multiple systems need to consume the same information.

A node gives us a structured API layer.

For example, the service currently exposes endpoints for:

health status
service metadata
Knowledge Cards
full-text search
contributor provenance
evidence references
content hashes
Evidence Payloads
node manifest

This means another tool does not need to scrape a Markdown file to understand the contribution.

It can ask directly:

Which cards exist?

Who contributed them?

What evidence state do they have?

Which sources support them?

What is the SHA-256 of the current content?

Has the payload changed?

Which API routes does this node support?

Node.js service design

The service is intentionally small.

It runs on Node.js and currently does not depend on a large external framework.

The application is bound to:

127.0.0.1:3101

not:

0.0.0.0:3101

This means the Node.js process itself is not directly reachable from the public Internet.

That is an important design decision.

The VPS already hosts other MyZubster infrastructure, so we wanted the Open Period Care pilot to remain isolated from existing services.

The existing MyZubster bridge continues to use its own local service and port.

Open Period Care has its own process, its own systemd service, its own data directory and its own application port.

The application also runs under a dedicated system user rather than root.

That gives us a clean separation between:

the operating system
Nginx
the existing MyZubster bridge
the Open Period Care node
the research data

Read-only by design

The current Open Period Care node is deliberately read-only.

That is not a limitation we are trying to hide.

It is part of the security model.

The node is currently meant to publish sanitized evidence and structured knowledge, not accept arbitrary public modifications.

Nginx only exposes approved read routes.

Write attempts are blocked at the reverse proxy.

For example, a POST request to the public API returns:

HTTP 403

This gives us defense in depth.

Even if application behavior changes later, the reverse proxy still enforces the public read-only boundary.

Nginx as the public boundary

The Node.js service remains private on localhost.

Nginx provides the public HTTPS interface.

The public path is exposed under the existing MyZubster bridge domain:

/open-period-care/

The request flow looks like this:

Internet
→ HTTPS port 443
→ Nginx
→ /open-period-care/
→ localhost port 3101
→ Open Period Care Node

The private application port remains inaccessible directly from the Internet.

This is much better than simply opening port 3101 in the firewall.

It lets us centralize TLS, request restrictions, headers and routing in Nginx.

Structured Knowledge Cards

The first card focuses on reusable textile absorbent architecture.

It contains structured information about:

contact layer materials
absorbent core materials
waterproof barrier
fasteners
durability targets
circular-economy targets
product requirements
evidence references
provenance
validation state

The second card focuses on governance and privacy.

It contains rules around:

prohibited sensitive-data collection
anonymized feedback
epistemological classification
clinical claim boundaries
medical claim restrictions
privacy-by-design principles
GDPR-related references

This is important because Open Period Care is health-adjacent.

The software should help structure evidence without pretending to replace professional, laboratory or regulatory review.

Provenance as a first-class field

One of the main features of the node is contributor provenance.

Each card contains explicit provenance information such as:

contributor identifier
source contribution
source branch
pull request reference
evidence origin
merge status

For the current research package, the contributor is:

khongten124

and the source contribution is connected to the MyZubster Open Period Care pull request.

This means provenance is not an afterthought or a comment hidden inside documentation.

It is machine-readable.

Another node or reviewer can retrieve it directly.

Deterministic SHA-256

One of the most useful technical milestones was implementing deterministic SHA-256 fingerprints.

Hashing JSON sounds simple until object key ordering becomes relevant.

Two JSON objects can represent the same information while having different key order.

If you hash their raw serialized representation, you may get different hashes.

To avoid that, the node recursively sorts object keys before serialization.

The process is:

structured object
→ recursive key sorting
→ canonical JSON
→ SHA-256

This means repeated requests for the same unchanged card produce the same hash.

For example, the current hash for KC-OPC-001 is:

360b98cce04f34f56f479dd51122c8e46abeacb1b467ff602cb5d7c51dc213b0

The second card currently has:

6a3c346501205328a00f4aa3f5d01e8dc3753d4001a7c7869f58556c22b47e47

If the underlying card changes, the hash changes.

That gives us a simple way to detect content drift between nodes or over time.

Evidence Payload v1

The next step was to create a portable Evidence Payload.

Instead of returning only the Knowledge Card, the node can build a normalized payload containing fields such as:

{
  "schema": "myzubster.circular-care.evidence.v1",
  "pilotId": "open-period-care",
  "projectId": "open-period-care-research",
  "knowledgeCardId": "KC-OPC-001",
  "actorRef": "khongten124",
  "actorRole": "researcher",
  "messageType": "EVIDENCE_SUBMISSION",
  "evidenceState": "SUPPORTED",
  "privacyClass": "PUBLIC_SANITIZED",
  "payloadSha256": "73d864cd49295408859f197bba7f170b3722a978909a27e90a82d04683eb3d85"
}

The payload also includes:

evidence references
claim references
summary notes
review references
creation timestamp
privacy classification

The payload itself has its own deterministic SHA-256.

For KC-OPC-001, the current payload hash is:

73d864cd49295408859f197bba7f170b3722a978909a27e90a82d04683eb3d85

For KC-OPC-002:

a576fc507109a2db4507d8abdafbce6dac151bb5b57cebccba8a331dcf1f5ae7

Repeated requests produce the same hashes as long as the payload content remains unchanged.

Why hash the payload separately from the card?

The Knowledge Card and the Evidence Payload serve different purposes.

The card represents structured knowledge.

The payload represents a transportable evidence record.

A future node may not need every internal field from the original Knowledge Card.

It may only need:

project
contributor
evidence state
evidence references
claims
privacy class
hash

By hashing both independently, we can verify:

the exact Knowledge Card content

and

the normalized evidence record derived from it.

This creates a cleaner interoperability boundary.

A public node manifest

The latest milestone was adding a public node manifest.

The manifest describes the node itself.

It includes:

node identifier
service name
version
operating mode
scope
evidence schema
privacy class
public base path
available endpoints
Knowledge Cards
card hashes
payload hashes
contributor references
pull request references

The manifest schema is currently:

myzubster.node.manifest.v1

The service version is:

0.9.0

The node mode is:

read-only

The manifest itself is also hashed.

Current manifest SHA-256:

a173b0c4253643b68f692fb35b0cf49ee60f41c9c4d741de389fc26ecc52c43d

We tested the manifest locally, through HTTPS and through repeated requests.

The hash remained identical.

That means the manifest can act as a compact fingerprint of what the node currently publishes.

Why the manifest matters

Imagine two independent nodes.

Node A publishes an Open Period Care manifest.

Node B retrieves it.

Node B can check:

service version
supported schema
available cards
expected card hashes
expected payload hashes
manifest hash

If one of those values changes, Node B can detect that the state changed.

This does not prove the scientific truth of the content.

It proves something more limited:

the published digital state is identifiable and reproducible.

That is a useful primitive for decentralized research infrastructure.

Privacy boundaries

Privacy is especially important for this pilot.

The node is not designed to collect menstrual histories, clinical records or intimate participant data.

The current payload classification is:

PUBLIC_SANITIZED

The research workflow explicitly separates product and usability evidence from identifiable personal health information.

The governance Knowledge Card prohibits or restricts collection of information such as:

clinical diagnoses
medication histories
intimate photographs
personal identifiers
IP or location metadata where inappropriate
unsupported medical claims

The goal is to make research and product requirements traceable without turning a public repository into a health-data database.

Scientific boundaries

Another key principle is that a cryptographic hash is not scientific validation.

A SHA-256 digest proves that some bytes match some other bytes.

It does not prove:

material safety
product effectiveness
laboratory performance
regulatory compliance
clinical validity
physical recycling
professional credentials

Likewise, Git provenance proves that a contribution exists in a repository history.

It does not automatically certify the contributor as a medical or scientific authority.

The current system therefore keeps explicit states such as:

SUPPORTED

rather than automatically promoting research to:

VERIFIED

Future transitions to TESTED or VERIFIED should require actual evidence supporting those states.

Reproducibility over hype

One of the goals of MyZubster is to experiment with decentralized and contributor-driven infrastructure.

But I think it is important not to confuse decentralization with trustlessness or automatic truth.

For this pilot, the practical focus has been:

Can another person inspect the source?

Can another node retrieve the same payload?

Can we reproduce the same hash?

Can we identify who submitted the research?

Can we distinguish literature support from laboratory testing?

Can we expose data without exposing the VPS?

Can we keep sensitive data out of the public evidence layer?

Those questions are more useful to me than simply saying “this is decentralized”.

What worked well

A few architectural decisions have worked well so far.

First, keeping the Node.js process private on localhost made deployment much safer.

Second, using Nginx as the public boundary let us expose only a controlled read-only interface.

Third, separating data from application code made it much easier to update Knowledge Cards without changing server logic.

Fourth, deterministic canonical JSON gave us stable hashes.

Fifth, separating card hashes, payload hashes and manifest hashes created clear integrity layers.

And finally, keeping explicit evidence states prevents the API from overstating the certainty of the underlying research.

Current architecture

The current Open Period Care architecture can be summarized like this:

Contributor
    │
    ▼
Research contribution
    │
    ▼
Knowledge Cards
    │
    ├── provenance
    ├── evidence references
    ├── validation state
    └── requirements
    │
    ▼
Canonical JSON
    │
    ▼
SHA-256
    │
    ├── Card hash
    └── Evidence Payload hash
    │
    ▼
Node.js service
127.0.0.1:3101
    │
    ▼
Nginx reverse proxy
    │
    ▼
HTTPS read-only API
    │
    ▼
Public Node Manifest

The application port remains private.

Only the Nginx HTTPS endpoint is public.

What comes next

The next step is interoperability.

The idea is to let independent contributors or reviewers run their own nodes and exchange sanitized evidence records through a documented interface.

A possible future workflow looks like:

Researcher node
→ Evidence Payload
→ MyZubster bridge
→ reviewer node
→ review response
→ updated evidence state
→ Knowledge Graph

Possible message types could include:

EVIDENCE_SUBMISSION
REVIEW_REQUEST
REVIEW_RESPONSE
SOURCE_CHALLENGE
CORRECTION

The goal would be to preserve provenance and state transitions without giving external contributors access to the VPS itself.

No SSH credentials.

No server passwords.

No database credentials.

No wallet keys.

No health records.

Only sanitized evidence payloads.

Possible future attestations

The deterministic hashes also make optional external attestations possible.

For example, a payload hash could eventually be anchored to an external timestamping or blockchain system.

But the distinction must remain clear:

A blockchain anchor can prove that a hash existed at a certain point.

It cannot prove that the associated physical event actually happened.

If someone claims that a material was recycled, tested or clinically validated, the hash alone is not evidence of that physical claim.

The supporting evidence still needs to exist independently.

That distinction is critical.

Lessons from this build

This pilot reinforced a few ideas for me.

1. Provenance should be structured, not implied.

If provenance matters, put it into the data model.

2. Evidence states should be explicit.

Do not make consumers guess whether something is proposed, supported, tested or verified.

3. Cryptographic integrity and scientific validity are different concepts.

Both matter, but they solve different problems.

4. Public APIs should expose less than the internal infrastructure.

The VPS does not need to become part of the contributor interface.

5. Read-only is a useful first interoperability milestone.

Before accepting external writes, prove that sanitized retrieval and validation work correctly.

6. Small services can be easier to reason about.

For this pilot, a lightweight Node.js service behind Nginx has been enough to validate the architecture without adding unnecessary complexity.

Technology used

The current implementation uses:

Node.js
native Node.js HTTP APIs
Node.js crypto
JSON-based Knowledge Cards
deterministic recursive JSON key sorting
SHA-256
systemd
Nginx
HTTPS
Linux VPS
GitHub contribution provenance
MyZubster Evidence Payload v1
MyZubster Node Manifest v1

No public database is required for the current read-only pilot.

Current milestone

At the current milestone, the node can:

serve real contributor Knowledge Cards
search card contents recursively
expose evidence references
expose contributor provenance
expose explicit validation states
calculate deterministic card hashes
generate deterministic Evidence Payloads
publish a verifiable node manifest
operate behind HTTPS
reject public write requests
keep its application port private
keep sensitive health information outside the public data model

It is still a pilot.

It is not a production clinical platform.

It is not a medical device.

It does not establish regulatory compliance.

It does not prove laboratory validation.

But it is now a working example of how contributor research can move from static documentation toward a structured, verifiable and interoperable evidence layer.

Why I think this is interesting

Open-source projects are very good at tracking code provenance.

Git already gives us commits, authors, diffs and history.

Research-oriented projects often need something similar for knowledge.

Where did this claim come from?

Which source supports it?

Who structured it?

Has it been tested?

Has it been reviewed?

Did the payload change?

Can another machine reproduce the same state?

The Open Period Care node is a small attempt to explore that problem.

Instead of replacing peer review, laboratories or domain experts, software can make the evidence path easier to inspect.

That is the direction we are experimenting with in MyZubster.

I’d be very interested in feedback from developers and researchers working on:

Node.js infrastructure
reproducible research
knowledge graphs
provenance systems
decentralized architectures
content-addressed data
cryptographic attestations
DevOps
open-source hardware
evidence-based product development

In particular, I’m interested in how others would design the next interoperability layer between independent contributor and reviewer nodes while keeping the security boundary small.

MyZubster Open Period Care Node — current service version: 0.9.0

Current manifest SHA-256:

a173b0c4253643b68f692fb35b0cf49ee60f41c9c4d741de389fc26ecc52c43d

OpenSource #NodeJS #DevOps #SHA256 #Provenance #Research #Reproducibility #KnowledgeGraph #CyberSecurity #MyZubster

From GitHub Contributions to a Public Knowledge Graph: What We're Building with MyZubster

Fri, 02 Oct 2026 17:17:25 +0000

From GitHub Contributions to a Public Knowledge Graph: What We're Building with MyZubster

How we connected independent contributors, public development evidence, Knowledge Cards, and an interactive graph — and what we learned along the way.

By Daniel Ioni | MyZubster | October 2026

Topics: Open Source, JavaScript, Node.js, GitHub, Knowledge Graphs, Developer Tools, AI

What if a GitHub contribution could become reusable knowledge?

A pull request contains more than code.

It can represent hours of debugging, architectural decisions, technical research, new testing methods, and practical experience. Yet much of that knowledge remains scattered across repositories, commits, discussions, and documentation.

At MyZubster, we have been experimenting with a system that connects these different sources without losing their original context.

Our idea is relatively simple:

Every documented contribution should have the opportunity to become part of a discoverable knowledge network.

Instead of automatically converting repository activity into claims about someone's abilities, we want to preserve the original evidence and give contributors the option to build their own knowledge records around it.

This experiment started with our collaboration with Nicola and has progressively expanded to other open-source contributors.

Our latest development milestone is now available through the public MyZubster Knowledge Graph.

Explore it here: MyZubster Interactive Knowledge Graph

In this article, we'll explain our implementation, the problems we encountered, and how other developers can apply similar principles to their own projects.

1. Starting with a real contributor: Nicola

Our initial experiment involved working with Nicola, also known as N4K48.

We wanted to explore how an independent contributor could document technical work, preserve ownership of their own development environment, and connect their knowledge to a broader ecosystem.

Our collaboration has involved software development, Docker, GitHub workflows, and the documentation of technical experiments.

An important part of this process was learning to distinguish between an experiment that works locally and a fully validated integration.

For example, we prepared experimental infrastructure for a MyZubster Node Bridge, including HTTPS endpoints and Docker-related workflows.

However, preparing an accessible server is not the same as proving that an independent contributor's machine has completed every stage of a distributed integration.

This distinction influenced the entire knowledge system.

We began asking: what exactly does each piece of evidence prove?

A repository commit might document a code change. A test result can support a claim about specific behavior. A deployment record shows that a deployment occurred.

None of these records automatically proves every broader claim about a project's maturity.

We carried this principle into the design of our Knowledge Cards.

Technical references:

2. Building Knowledge Cards with supporting evidence

A Knowledge Card is a structured record that describes a specific piece of knowledge or experience.

Instead of storing a generic statement such as "I know backend development," a card can describe a particular technical activity and link to supporting evidence.

Consider this illustrative example:

{
  "title": "API Integration Testing",
  "domain": "Backend Engineering",
  "description": "Developed tests for an API integration",
  "evidence": [
    {
      "label": "Public GitHub Pull Request",
      "url": "https://github.com/example/project/pull/123"
    }
  ],
  "verificationNote": "Public source linked; broader skill claims have not been independently verified."
}

This is a simplified example rather than the complete MyZubster production schema.

The important elements are the relationship between a claim, its source, and its verification status.

We also wanted contributors to control publication.

Our backend therefore distinguishes between private drafts and public Knowledge Cards.

The public catalog is served through:

GET /api/knowledge-evidence/public

The relevant backend query is based on two explicit conditions:

const cards = await KnowledgeDraft.find({
  status: 'published',
  visibility: 'public'
});

This design avoids exposing private drafts simply because they exist in the database.

It also means that creating a draft and publishing a card are separate operations.

The public catalog is not a complete inventory of every contributor's private knowledge. It contains only records that satisfy the publication requirements.

3. Turning individual records into an interactive graph

Once we had a public knowledge catalog, another problem became apparent.

A list of cards provides useful information, but it does not immediately show how everything connects.

One contributor might work across several domains. Multiple cards might reference related public evidence. Different contributors might work on separate aspects of the same technical problem.

We introduced an interactive graph to make those relationships easier to explore.

Pull Request #1438 implemented the initial public graph.

It connects four principal node types:

Knowledge Cards
Contributors
Knowledge domains
Public evidence sources

Conceptually, the initial graph can be represented as:

              Knowledge Domain
                     |
                     |
Contributor ---- Knowledge Card
                     |
                     |
               Public Source

The frontend retrieves public cards, builds the node and relationship structures, and renders them as an interactive visualization.

Users can search, filter, select nodes, and inspect the related information.

The implementation also preserves verification notes, which help communicate the limitations of the evidence displayed.

Try it: Interactive Knowledge Graph

4. The problem we discovered: our contributors were missing

After introducing the graph, we discovered an important limitation.

We already had several contributors with documented public GitHub activity. But many of them did not appear in the Knowledge Graph.

The reason was straightforward: the graph was built exclusively from personally published Knowledge Cards.

A public pull request was not enough to create a graph node.

We had two possible approaches.

The first was to automatically generate personal Knowledge Cards for contributors based on their repository activity.

We decided against that approach.

A public pull request does not necessarily authorize another organization to create a personal knowledge profile, publish private information, or assign qualifications.

Instead, we implemented a second evidence layer.

Our solution: separate GitHub contributions from personal Knowledge Cards

On October 2, 2026, we integrated Pull Request #1453.

The update adds selected, publicly documented GitHub contributions to the Knowledge Graph.

These contributions are represented separately from owner-published Knowledge Cards.

The expanded model now looks conceptually like this:

                    Knowledge Domain
                      /        \
                     /          \
           Knowledge Card    Public GitHub PR
                 |                |
                 |                |
            Contributor       GitHub Account
                 |
                 |
           Public Evidence

This separation matters because the two types of records have different meanings.

A Knowledge Card is a record explicitly published by its owner.

A GitHub contribution record documents public activity associated with a repository.

The second record does not automatically become the first.

Preserving pull request status

We also distinguish between different contribution states.

For example:

const workStatus = {
  merged: 'PR integrated',
  submitted: 'PR submitted / not integrated',
  closed_without_merge: 'PR closed without integration'
};

This helps prevent a common documentation mistake: presenting submitted or abandoned work as a completed production feature.

Our initial implementation uses a curated selection of public GitHub contributions. It is not an unrestricted or continuously synchronized GitHub crawler.

Records must be reviewed and updated as their underlying status changes.

The change has been merged, and the corresponding Vercel production deployment reached the ready state.

View the Implementation — PR #1453

5. Real examples from our open-source community

We began the new contribution layer with documented activities from several MyZubster contributors.

These examples demonstrate how a knowledge graph can connect completely different technical domains.

NFC development and documentation

Contributor @jdjioe5-cpu has public contributions related to our Animal Registry.

The associated work covers browser-based NFC simulation, physical tag design, and API documentation.

Rather than combining these into one generic skill label, the individual contributions can be represented separately.

Sources:

Geospatial development

Contributor @leanworld7-netizen has a merged contribution addressing garden geolocation and area-based search.

This connects backend engineering with geographic data and community-garden applications.

Garden Geolocation — PR #57

Frontend and telemetry

Contributor @laurentketterle-hub has a merged contribution involving a Space Station telemetry dashboard.

It provides an example of how frontend development and monitoring can be represented within a larger knowledge network.

Telemetry Dashboard — PR #399

Automated testing

Contributor @foxxx009 has a merged contribution involving automated tests for GitHubMonitor.

Test automation is an interesting knowledge domain because the associated evidence can include both the implemented tests and their documented execution.

GitHubMonitor Tests — PR #259

Other selected records represent work involving security, documentation, and metaverse integration.

Their individual states are preserved rather than automatically presenting every submission as an accepted result.

6. From Knowledge Cards to Contributor Passports

Our next question was how to connect multiple contributions made by the same person.

That led us to introduce the Contributor Passport framework.

The idea is to organize evidence-backed knowledge records into a coherent contributor history.

The conceptual sequence is:

Contribution
     |
     v
Public Evidence
     |
     v
Knowledge Card
     |
     v
Contributor Passport
     |
     v
Optional Independent Pilot Node

We integrated the framework through PR #1445.

Contributor Passports are not automatic professional certifications.

They are intended to organize documented contributions, declared knowledge, and associated evidence.

Participation is also optional.

A contributor should not be required to publish private personal information or enroll in a separate program merely to participate in open-source development.

Read the Contributor Pilot Node Framework

7. Independent nodes and the future of distributed collaboration

An interesting extension of the Contributor Passport is the Independent Contributor Pilot Node.

The concept explores whether contributors can maintain their own development scope and technical identity while connecting relevant knowledge to MyZubster.

Potential applications include independent software projects, robotics, documentation, testing, and environmental monitoring.

However, publishing the framework is only one step.

Actually deploying independent nodes requires reproducible onboarding, clearly defined technical interfaces, and end-to-end validation.

We are treating these as separate milestones.

8. Paid Bounties and transparent contributor agreements

We have also introduced a public Paid Bounties page.

Explore MyZubster Paid Bounties

The system distinguishes proposed tasks from explicitly reserved work.

This is another example of why precise status information matters.

An interesting technical proposal should not be confused with a funded contract.

Likewise, completing a Knowledge Card should not automatically authorize a contributor's enrollment in a separate development program.

We are experimenting with connecting accepted contributions, documented evidence, optional knowledge records, and separately agreed rewards.

Our implementation is documented in PR #1444.

9. What other developers can learn from this experiment

Our development process has highlighted several principles that are useful beyond MyZubster.

First, keep the original source accessible.

A knowledge claim becomes easier to examine when people can follow its supporting evidence back to an original repository, commit, pull request, or document.

Second, model different evidence states explicitly.

Private drafts, published cards, submitted pull requests, merged contributions, and reviewed scientific results should not share a single generic verification label.

Third, separate evidence from interpretation.

A merged pull request provides evidence of integration. Interpreting the skills required to produce it may require additional context.

Fourth, respect contributor consent.

Public development activity does not automatically authorize the creation of a personal profile or the disclosure of unrelated personal information.

Fifth, test the relationships, not just individual features.

A knowledge system depends on the integrity of the connections between contributors, claims, sources, and publication states.

Our public contributor graph update introduced regression tests designed to preserve the distinction between GitHub evidence and personally published Knowledge Cards.

The change passed our CI, security, and evidence-gate workflows before integration.

10. Explore the project and contribute

We are developing MyZubster openly because we believe the knowledge generated during development should be useful to other people.

Our next challenges include expanding contributor participation, improving the accessibility of the Knowledge Graph, maintaining accurate public contribution records, and further validating independent development environments.

We welcome technical reviews, reproducible bug reports, documentation improvements, and contributions that help make the system more transparent.

Project links:

If you maintain an open-source project, we'd also be interested in discussing how you document the knowledge created by your contributors.

Do you currently rely on README files, documentation websites, GitHub discussions, contributor profiles, or something else?

Our experiment is built around one idea: preserve the evidence, document the knowledge, and make the connections visible.

MyZubster is an evolving open-source project. Public GitHub evidence, contributor-declared knowledge, independently verified results, and experimental infrastructure are represented as distinct concepts.

# Building a Trustworthy Developer Portfolio with GitHub, Node.js, and Structured Evidence

Fri, 02 Oct 2026 05:25:24 +0000

Building a Trustworthy Developer Portfolio with GitHub, Node.js, and Structured Evidence

How we built Knowledge Cards in MyZubster, integrated GitHub evidence, protected profile README updates, and learned from a real open-source documentation proposal.

By Daniel Ioni — Founder & Product Builder, MyZubster

Introduction

What does a GitHub profile actually tell us about a developer?

We can examine repositories, commits, pull requests, documentation, and contribution activity. However, those artifacts don't always explain what problems a developer investigated, what knowledge they acquired, or how their work relates to specific technical subjects.

At the same time, a traditional portfolio might contain a long list of declared skills without providing any supporting references.

While developing MyZubster, we started experimenting with a different model.

Our idea was to connect structured descriptions of technical activities with supporting GitHub references while preserving an important distinction:

The existence of a technical artifact is not the same thing as independent verification of someone's expertise.

This article explains how we implemented that idea using Knowledge Cards, our Zorgax integration, GitHub APIs, and a controlled README synchronization workflow.

I'll also describe the bugs we encountered and how an unmerged Monero Docs pull request helped us improve our approach to documenting open-source activities.

1. Designing the Knowledge Card model

We wanted to move beyond traditional skill lists.

Instead of displaying a skill such as Node.js: Expert, we wanted developers to explain their activities and attach relevant technical references.

We created Knowledge Cards containing five main elements:

Title: What activity or area of knowledge does the card describe?
Domain: Which technologies or technical subjects are involved?
Description: What does the owner declare they worked on?
Evidence: Which external sources support or document that description?
Verification note: What can readers actually establish from the available evidence?

For example, one of our published cards describes the development of digital identity and privacy-related modules for MyZubster Space Station.

It references a GitHub commit containing implementation work, technical documentation, tests, and GitHub Actions configuration.

View the Space Station Knowledge Card

Inspect the associated GitHub commit

The card distinguishes the owner's description from the public source reference.

This design also helps readers investigate an activity without interpreting every statement as an independently certified qualification.

2. GitHub URL validation: syntax is not verification

One of our first practical problems involved a malformed GitHub commit URL.

We initially added a commit reference manually. It looked correct but was missing the final character of its SHA identifier.

Consequently, our README importer excluded it.

We subsequently added the complete reference using the evidence attachment workflow.

This debugging experience highlighted two different responsibilities:

Validate that a GitHub URL has an acceptable structure.
Independently check whether the resource is publicly accessible.

For README import, we use a validation function similar to the following:

const publicGithubSourceUrl = value => {
  const url = String(value || '').trim();

  return /^https:\/\/github\.com\/[a-z\d_.-]+\/[a-z\d_.-]+\/(?:pull\/[1-9]\d*|commit\/[a-f\d]{40})\/?$/i.test(url)
    ? url
    : '';
};

The regular expression accepts GitHub commit URLs with complete 40-character hexadecimal SHA identifiers and GitHub pull request URLs.

However, this validation does not establish that the commit or pull request exists.

For that, our backend performs a separate GitHub API request.

For example:

GET https://api.github.com/repos/OWNER/REPOSITORY/commits/SHA

If GitHub does not return an accessible resource, our application must not label it as publicly verified.

This distinction is especially important when working with private repositories, inaccessible forks, and authenticated API sessions.

A resource visible to one authenticated account might not be publicly accessible.

Engineering lesson: never confuse format validation with external evidence verification.

3. Linking evidence without automatically certifying expertise

We implemented a workflow that allows developers to attach public GitHub evidence to existing Knowledge Cards.

The owner selects a card, provides a supported GitHub URL, and explicitly confirms the operation.

Our backend checks whether the referenced artifact is publicly accessible before attaching it.

When the check succeeds, the reference can be described as an accessible public GitHub source.

However, we deliberately avoid describing the associated activity as independently certified.

A GitHub commit contains useful information, including its changes, timestamps, commit metadata, and potentially an associated GitHub account.

That information can support an investigation.

It does not independently prove every claim a developer makes about their expertise or individual responsibility.

Our verification notes communicate those limitations directly.

This approach allows structured portfolios to become more transparent without pretending that an automated system can accurately measure developer competence from GitHub metadata alone.

4. Documenting an unmerged Monero Docs proposal

The most interesting test of our evidence model involved Monero Docs.

Through my GitHub account DanielIoni-creator, I prepared a documentation clarification in a fork of the Monero Docs repository.

The proposed change involved the pending parameter of the Wallet RPC method get_transfers.

The original documentation stated:

Include pending transfers.

Our proposed clarification was:

Include pending, unconfirmed outgoing transfers.

The objective was to clarify that the parameter concerns pending outgoing transfers that have not yet been confirmed.

The proposal was submitted to the upstream project through pull request #389.

The pull request was closed without being merged.

Another complication emerged during verification: the original fork and pull request were not accessible without authentication.

Although our connected GitHub account could retrieve information about the activity, we could not treat that access as equivalent to independent public verification.

We had two options.

We could omit the activity entirely because the original evidence wasn't publicly accessible.

Or we could document it transparently, explicitly explaining the limitations.

We chose the second approach.

We prepared a publicly accessible technical document containing the original wording, the proposed modification, references to the original commit and pull request, and a clear statement about the outcome.

Read our Monero Docs contribution dossier

We then created a dedicated Knowledge Card.

View the Monero Knowledge Card

Crucially, we label the supporting document as self-declared documentation that has not been independently verified.

We don't describe the proposal as an accepted upstream contribution, an official Monero collaboration, or an independently certified achievement.

What other developers can learn from this

Not every meaningful engineering activity results in merged code.

Developers investigate problems, prepare proposals, explore unfamiliar codebases, and sometimes submit changes that are not accepted.

These experiences can be documented and discussed.

But the outcome is part of the engineering record.

A transparent portfolio should distinguish a proposed change from a merged change rather than treating both as identical achievements.

5. Using separate evidence categories

The Monero example revealed a limitation in systems that treat every external URL as equivalent evidence.

We needed to distinguish between publicly accessible GitHub commits and public documents authored by the card owner.

These are different categories.

A publicly accessible commit allows readers to inspect a specific repository change.

An owner-authored document allows readers to inspect the owner's account of an activity.

Both can be useful, but they support different conclusions.

We updated our README importer to track them separately.

Conceptually:

let githubSourceCount = 0;
let declaredDocumentCount = 0;

for (const source of evidenceItems) {
  if (source.isDocument) {
    declaredDocumentCount++;
  } else {
    githubSourceCount++;
  }
}

This simplified illustration reflects the separation implemented in our importer.

Instead of making every reference look like a verified GitHub contribution, the generated README displays distinct labels.

This makes the resulting portfolio more informative and reduces the risk of accidentally overstating the evidence.

6. Safely synchronizing a GitHub profile README

Once the Knowledge Cards were publicly available, we wanted users to include them in their GitHub profiles.

However, automatic README modification introduces a serious risk.

Developers often maintain their GitHub profile descriptions manually. An automated system should not erase existing biographies, project references, or other user-authored content.

We introduced an explicitly managed section:

<!-- MYZUBSTER-KNOWLEDGE-CARDS:START -->

## Knowledge Cards

<!-- Reviewed Knowledge Card content -->

<!-- MYZUBSTER-KNOWLEDGE-CARDS:END -->

During synchronization, Zorgax prepares the updated Knowledge Card content inside these boundaries while preserving unrelated README sections.

If the managed block already exists, it is replaced instead of duplicated.

This makes the operation repeatable.

But we added another important restriction.

If Zorgax cannot retrieve the current GitHub README, the update stops.

During our testing, we encountered this condition when the existing README was temporarily unavailable to the application.

The workflow refused to prepare a potentially unsafe update.

This is preferable to guessing the previous state of an external document.

User-controlled publication

We also separated content generation from publication.

The user must load the existing profile, generate the updated content, inspect the preview, approve it, and authorize the GitHub update.

Knowledge Card publication does not automatically rewrite the user's GitHub profile.

This explicit approval process provides an additional layer of control over public identity information.

7. Idempotence matters for generated text, too

Idempotence is usually discussed in relation to APIs, infrastructure, and database operations.

We discovered that it matters just as much when generating public descriptions.

Our backend automatically appends a standard verification statement when public GitHub evidence is attached to a card.

Initially, a normalization function could remove the final punctuation from an existing custom note.

The result was grammatically incorrect:

... does not independently certify expertise At least
one public GitHub source has been linked ...

We corrected the text-normalization logic so it preserves or restores punctuation.

However, fixing new updates wasn't enough.

We also needed to repair notes that had already been published.

We therefore supported an explicit refresh operation for existing cards.

One requirement was especially important: refreshing an already corrected note should not append the same verification statement again.

Conceptually, our regression test checks:

const first = refreshVerificationNote(existingNote);

const second = refreshVerificationNote(first);

expect(second).toBe(first);

This example illustrates the behavior rather than reproducing the complete backend test suite.

We deployed the correction, refreshed the affected Knowledge Card, synchronized the GitHub profile again, and verified the resulting README.

Engineering lesson: generated text deserves the same attention to repeatability and regression testing as other application data.

8. What we actually published

After completing the workflow, we verified the public GitHub profile README.

It contained three published Knowledge Cards, three public GitHub commit references, and one separately labeled public document relating to the Monero Docs proposal.

It also contained exactly one managed Knowledge Card section.

Our published cards cover:

MyZubster ecosystem development and supporting repository activity.
Space Station security, digital identity, and privacy-related modules.
A Monero Wallet RPC documentation proposal that was not merged upstream.

Explore the MyZubster GitHub profile

The resulting README is not a certificate of technical expertise.

Instead, it provides a structured overview of activities and references that readers can inspect and evaluate for themselves.

9. Lessons for developers building similar systems

Several principles emerged from this work.

Keep claims and evidence separate. A user description and its supporting source have different roles. Your data model should preserve that distinction.

Validate both structure and availability. A correctly formatted GitHub URL doesn't guarantee that the resource exists or is public.

Don't inflate contribution status. A proposed documentation improvement can be worth discussing even when the upstream pull request is closed without a merge.

Preserve existing user content. When updating external documents, define the exact area your application controls and stop when the original document cannot be retrieved.

Make publication an explicit decision. AI-assisted generation and external publication should be separate operations.

Test repeated operations. Re-running synchronization or refreshing verification text should not introduce duplicate sections or statements.

Treat documentation quality as engineering quality. Small punctuation errors can undermine the clarity of public verification disclosures.

10. What's next for MyZubster?

Our Knowledge Card implementation is an evolving foundation.

We're continuing to explore ways to improve source management, clean up outdated evidence, clarify verification states, and connect structured technical activities with other parts of the MyZubster ecosystem.

The broader goal is to make documented knowledge easier to discover without introducing misleading automated expertise scores or unsupported claims.

We believe developers should be able to explain not only what they successfully delivered, but also the investigations, proposals, experiments, and technical decisions behind their work.

That history becomes especially valuable when the supporting evidence and its limitations are clearly presented.

Conclusion

This project started as an attempt to improve our GitHub profile.

It evolved into an exploration of evidence modeling, API validation, safe synchronization, idempotent operations, and transparent open-source documentation.

The most valuable result isn't simply having three Knowledge Cards displayed on GitHub.

It's having a workflow that encourages developers to document their activities while preserving the distinction between personal declarations, accessible technical artifacts, and independently established facts.

A useful developer portfolio should make technical work discoverable, supporting evidence inspectable, and verification limitations visible.

That's what we're working toward with MyZubster and Zorgax.

Project references

Author: Daniel Ioni
Project: MyZubster

The Monero Docs proposal discussed in this article was not merged into the upstream project. The linked public dossier records the proposal and explicitly discloses its verification limitations.

Engineering an Evidence-First Knowledge Graph: From GitHub Artifacts to On-Chain Proofs in MyZubster

Tue, 29 Sep 2026 02:03:45 +0000

Certo. Per CoderLegion lo farei più tecnico del post DEV, con più attenzione ad architettura, data provenance, canonicalizzazione, hashing e al motivo per cui dopo il test con Nicola stiamo valutando standard esistenti invece di continuare ad aggiungere smart contract custom.
Engineering an Evidence-First Knowledge Graph: From GitHub Artifacts to On-Chain Proofs in MyZubster
Building and testing a complete provenance chain with a real contributor, canonical payloads, SHA-256, Ethereum Sepolia, RAG, and a Knowledge Graph
Tonight we ran one of the most complete end-to-end experiments so far on MyZubster.
The test involved a real contributor, Nicola / N4K48, rather than synthetic fixtures.
What started as a relatively simple test of the MyZubster Knowledge Profile evolved into a much larger architecture experiment involving:
Authentication

↓

Knowledge Profile

↓

Private Knowledge Cards

↓

Publication

↓

Real software contribution

↓

GitHub evidence

↓

RAG / Qdrant

↓

Knowledge Graph

↓

Canonical payload

↓

SHA-256

↓

Ethereum Sepolia

↓

Publicly inspectable provenance

The most important result was not the blockchain transaction.
It was discovering where the boundaries of the MyZubster architecture should probably be.

The problem we are trying to solve
Most professional profiles fundamentally contain claims.
For example:
{
"person": "N4K48",
"skill": "Docker"
}

This tells us what somebody says about themselves.
It tells us very little about why another person should trust that statement.
We wanted to experiment with a different model:
Person
↓
Claim
↓
Activity
↓
Artifact
↓
Evidence
↓
Cryptographic commitment
↓
Attestation

The objective is not to magically transform a claim into truth.
The objective is to make its provenance inspectable.

Testing with a real contributor
Nicola created a MyZubster Knowledge Profile representing different parts of his real activity.
We used three Knowledge Cards during the experiment:
N4K48
│
├── Truck driving and transport organization
│
├── Docker / AI experiments on myzubster-mvp
│
└── Learning and collaboration in MyZubster

This immediately forced us to distinguish two concepts that are easy to accidentally merge:
CLAIM

and:
EVIDENCE

A professional statement is not automatically evidence.
A GitHub commit is not automatically proof of every professional statement either.
Therefore MyZubster needs to preserve the semantic boundary between them.

Authentication failed before cryptography did
The first failures had nothing to do with Ethereum.
Nicola encountered an expired JWT during the onboarding flow.
The UI indicated an authenticated state, but a subsequent request produced:
jwt expired

Another issue appeared in the conversational onboarding flow: information already supplied by the user was not always being reused correctly.
We also discovered a routing issue where the Knowledge Profile Builder URL displayed the MyZubster homepage instead of the Builder.
These problems were fixed before continuing.
The authentication behavior was changed so that an expired session could trigger re-authentication while preserving the locally prepared onboarding information.
This matters because provenance starts before hashing.
If users lose their work because authentication expires, the cryptographic layer does not save the product.

Private-first Knowledge Cards
The next test covered persistence.
Nicola created three cards and saved them privately.
The tested state transition was approximately:
NEW
↓
PRIVATE DRAFT
↓
RELOAD
↓
EDIT
↓
PREVIEW
↓
PUBLISH

The important property here is that creation does not imply publication.
A Knowledge Card can exist privately while the owner reviews:
claims
activities
sources
evidence
verification state

Only an explicit publication action makes the card public.
Published cards can also be withdrawn before being modified.
That gives us a lifecycle closer to:
DRAFT
│
▼
PUBLIC
│
▼
WITHDRAWN / DRAFT

rather than treating the knowledge database as an append-only public feed.

Turning software work into evidence
The next step was intentionally practical.
Instead of adding another manually entered skill, Nicola performed a real change in his myzubster-mvp repository.
The work involved:
scripts/ingest_knowledge.py

and improvements to the chunking/knowledge-ingestion process.
The resulting test reported:
46 knowledge chunks loaded into Qdrant
1 empty document skipped

The Git commit associated with that work could then become a source attached to the relevant Knowledge Card.
We now had something structurally different from:
N4K48
↓
"knows software development"

We had:
N4K48
↓
Knowledge Card
↓
documented software activity
↓
GitHub repository
↓
specific commit
↓
inspectable code change

This is an important distinction.
MyZubster does not need to decide that the commit proves a particular competence.
It can state something narrower:
This artifact is evidence associated with this activity and this Knowledge Card.

RAG became part of the evidence trail
The test also exercised the knowledge retrieval stack.
The local project involved:
Docker Compose
│
├── Ollama
│
├── Qdrant
│
├── API
│
└── Open WebUI

During retrieval testing, the context being returned was insufficient.
The experiment therefore changed:
AI_CONTEXT_LIMIT=1

to:
AI_CONTEXT_LIMIT=5

and repeated the retrieval tests.
That work itself became documentable evidence.
This creates a useful recursive property.
The system can represent knowledge about the process used to improve the system itself:
engineering work

↓

Git artifact

↓

Knowledge Card

↓

Knowledge Graph

Building the Knowledge Graph
Once the cards were public, we started connecting them into a graph.
Conceptually:

            N4K48
              │
 ┌────────────┼────────────┐
 │            │            │
 ▼            ▼            ▼

Transport Learning Docker / AI
Card Card Card

                             │
                             ▼
                         GitHub repo
                             │
                             ▼
                           commit

Sources become traversable relationships instead of text buried inside a profile.
During this phase we found another real-world infrastructure issue.
The visual Knowledge Graph deployment was protected by Vercel, preventing Nicola from accessing it.
After adjusting deployment accessibility, testing continued, including from mobile.
We later corrected another graph/catalog connection that produced a "Catalog unavailable" result.
These may look like small frontend/infrastructure bugs, but they matter for an evidence system.
Evidence that technically exists but cannot be navigated is significantly less useful.

Proof v1: hashing the wrong thing
Then we introduced Ethereum Sepolia.
The first MyZubsterProof experiment anchored a SHA-256 digest related to the Knowledge Card.
However, the digest represented the Knowledge Card URL.
Conceptually:
Knowledge Card URL
```
↓
```
SHA-256
```
↓
```
bytes32
```
↓
```
Ethereum Sepolia

This works cryptographically.
But semantically it proves much less than we wanted.
Suppose:
https://example.com/card/123

continues to exist while the content served at that URL changes.
Hashing the URL does not detect the content change.
The first experiment therefore became useful precisely because it exposed this weakness.
We explicitly documented it rather than treating an on-chain hash as stronger evidence than it actually was.

Proof v2: canonicalize before hashing
Proof v2 changed the architecture.
Instead of hashing a location, we created a deterministic representation of the Knowledge Card itself.
Conceptually:
Knowledge Card
↓
canonical representation
↓
exact bytes
↓
SHA-256
↓
bytes32
↓
Ethereum Sepolia

For the tested card, the resulting digest was:
6097e05866bafceec24663d2638cb1dae5742ac78284abbfd45cc9c3b0bfb845

The corresponding on-chain value was:
0x6097e05866bafceec24663d2638cb1dae5742ac78284abbfd45cc9c3b0bfb845

Reading knowledgeHash() from the deployed MyZubsterProof contract returned that same value.
The canonical payload and the verification procedure were also committed to the repository.
That gives us a reproducible verification path:
canonical payload

   │
   ▼

SHA-256(payload)

   │
   ▼

expected digest

   │
   ▼

on-chain knowledgeHash()

   │
   ▼

comparison

If all values match, we can establish that the committed canonical representation corresponds to the digest stored by the contract.

Why canonicalization matters
This is a subtle but critical engineering detail.
Consider two JSON objects:
{
"name": "N4K48",
"activity": "Docker"
}

and:
{
"activity": "Docker",
"name": "N4K48"
}

Semantically, an application may consider them equivalent.
At the byte level, they are different.
And therefore:
SHA256(payloadA) != SHA256(payloadB)

can occur even when both payloads represent the same logical information.
The same issue appears with:
whitespace
field ordering
Unicode normalization
line endings
optional values
number serialization
dates

So an evidence system cannot simply say:
hash(JSON.stringify(object))

and assume the problem is solved forever.
The canonicalization algorithm effectively becomes part of the protocol.
A durable proof therefore needs something like:
payload format
canonicalization version
hash algorithm
digest
attestation reference

Proof v2 pushed us toward that realization.

Integrity is not truth
This is probably the most important architectural rule from the experiment.
Assume the canonical payload contains:
{
"claim": "N4K48 is an expert in Docker"
}

and suppose we perfectly canonicalize it, hash it and anchor the digest on Ethereum.
We have demonstrated something about the integrity of the statement.
We have not demonstrated that the statement itself is true.
Formally:
integrity(claim) ≠ truth(claim)

Likewise:
existence(GitHub commit) ≠ proof(skill)

and:
blockchain timestamp ≠ professional certification

Different evidence supports different propositions.
MyZubster therefore needs to model not only evidence, but also what each piece of evidence is capable of establishing.

Provenance as a graph
This leads to a more interesting representation.
Instead of:
Person → Skill

we can represent:
Person
│
▼
Claim
│
▼
Activity
│
▼
Knowledge Card
│
├──────────────► GitHub artifact
│
├──────────────► external source
│
▼
Canonical Payload
│
▼
Digest
│
▼
Attestation

Each edge has semantics.
For example:
Person --DECLARES--> Claim

Claim --SUPPORTED_BY--> Evidence

Activity --PRODUCED--> Artifact

KnowledgeCard --CANONICALIZED_AS--> Payload

Payload --HASHED_AS--> Digest

Digest --ATTESTED_BY--> Attestation

This is much more expressive than simply attaching a blockchain transaction ID to a user profile.

The architectural question that emerged
After completing Proof v2, Nicola compared the experiment with existing systems and standards.
The comparison included:
- Talent Protocol / Builder Score;
- Ethereum Attestation Service;
- cheqd;
- Open Badges 3.0 and Verifiable Credentials.
  That comparison raised a very useful question.
  Why should MyZubster implement every cryptographic primitive itself?
  The ecosystem already contains mature approaches for:
  identity
  credentials
  attestations
  reputation
  issuers
  verification
  revocation
  cryptographic proofs

Creating:
MyZubsterProofV3.sol
MyZubsterProofV4.sol
MyZubsterProofV5.sol

may therefore be less valuable than making MyZubster interoperable with existing primitives.

Separating the layers
The experiment suggests that the architecture could eventually be separated into several layers.
Identity layer
Person
Wallet
GitHub identity
MyZubster account

Knowledge layer
Knowledge Card
Claim
Activity
Skill
Project

Evidence layer
GitHub commit
Pull request
Repository
Document
External source
Artifact

Integrity layer
canonical payload
hash algorithm
digest
version

Attestation layer
Potentially:
Ethereum Attestation Service

rather than a growing family of MyZubster-specific contracts.
Credential layer
Potential interoperability with:
W3C Verifiable Credentials
Open Badges

Relationship layer
This is where MyZubster's Knowledge Graph becomes particularly important.
Person
↓
Claim
↓
Activity
↓
Evidence
↓
Artifact
↓
Attestation
↓
Credential

MyZubster may be the graph, not the protocol
This was probably the biggest architectural insight of the evening.
Originally it is tempting to think:
MyZubster
=
our smart contract

But after this experiment, another model looks more interesting:
MyZubster
=
evidence/provenance graph
+
workflow
+
verification semantics
+
human-readable navigation

with standards handling lower-level primitives where appropriate.
That means MyZubster could potentially consume:
GitHub evidence
EAS attestations
Verifiable Credentials
Open Badges
blockchain transactions
traditional documents
other public evidence

without pretending that all evidence has the same meaning.

A possible future architecture
The architecture we are now exploring looks closer to this:

               PERSON
                 │
                 ▼
               CLAIM
                 │
                 ▼
              ACTIVITY
                 │
                 ▼
          KNOWLEDGE CARD
                 │
    ┌────────────┴────────────┐
    │                         │
    ▼                         ▼
EVIDENCE                   ARTIFACT
    │                         │
    └────────────┬────────────┘
                 │
                 ▼
        CANONICAL PAYLOAD
                 │
                 ▼
              SHA-256
                 │
     ┌───────────┴───────────┐
     │                       │
     ▼                       ▼

ATTESTATION CREDENTIAL
e.g. EAS VC / Badge

     │                       │
     └───────────┬───────────┘
                 ▼
         MYZUBSTER GRAPH

Notice that the blockchain is no longer the center of the system.
The relationships are.

Why this differs from a reputation score
Consider a reputation system that produces:
Score = 87

That number may be useful for ranking or filtering.
But a provenance graph asks a different question:
Why?

A MyZubster-style path could eventually allow someone to inspect:
N4K48
↓
Docker-related Knowledge Card
↓
specific development activity
↓
GitHub commit
↓
source code
↓
canonical representation
↓
cryptographic digest
↓
attestation

The user is not forced to trust only the final number.
They can inspect the path that produced the evidence.

Evidence types should not collapse into one trust score
This is another consequence of the experiment.
Imagine three pieces of evidence:
A: self-declared professional experience

B: public GitHub commit

C: credential issued by a university

All three may belong in the graph.
But they should not automatically become:
A = B = C

Their semantics are different.
A useful graph therefore needs metadata such as:
evidence type
issuer
subject
source
verification method
verification status
created timestamp
attestation reference
revocation state

The graph should preserve those differences rather than hide them behind one universal score.

What the experiment actually demonstrated
By the end of the session we had exercised a chain approximately like this:
Real contributor
↓
authenticated MyZubster account
↓
private Knowledge Cards
↓
public Knowledge Cards
↓
real software activity
↓
GitHub evidence
↓
RAG / Qdrant test
↓
Knowledge Graph
↓
canonical card payload
↓
SHA-256
↓
Sepolia Proof v2
↓
public verification information

This is more useful to us than an isolated smart-contract demo because every layer exposed different failure modes.
We encountered:
expired authentication
routing problems
deployment access restrictions
catalog/navigation problems
retrieval-context limitations
evidence-modeling questions
URL-vs-content hashing semantics
canonicalization requirements
truth-vs-integrity boundaries

That is exactly what an end-to-end proof of concept should reveal.

What we deliberately did not prove
It is equally important to document what the experiment does not establish.
Proof v2 does not prove that every statement in a Knowledge Card is true.
It does not automatically prove professional competence.
It does not establish that GitHub activity is equivalent to mastery of a skill.
It does not turn blockchain data into a trusted credential issuer.
What it does establish is a mechanism for connecting an exact representation of information with a reproducible digest and a public cryptographic reference.
That is an integrity primitive.
Trust requires additional layers.
The next engineering experiment
Instead of immediately creating another custom proof contract, the next useful experiment is likely to take the exact Proof v2 data model and map it onto existing standards.
Something like:
Knowledge Card
↓
canonical payload
↓
current Proof v2
│
├────► EAS schema mapping
│
└────► Open Badges / VC mapping

Then we can compare:
custom MyZubster contract
vs.
standardized attestation

across:
schema flexibility
revocation
references
issuer semantics
verification
interoperability
developer complexity
gas cost
long-term maintainability

Only after that comparison should we decide what belongs inside MyZubster itself.

The broader engineering lesson
Tonight's work started with a very ordinary product task:
Can a contributor create a knowledge profile?

It progressed into:
Can the profile contain inspectable evidence?

Then:
Can that evidence point to real engineering work?

Then:
Can the exact knowledge representation be cryptographically committed?

And eventually:
Should MyZubster even own the attestation protocol?

That progression matters.
A proof of concept should not merely prove that the original architecture works.
Sometimes its most valuable outcome is discovering that part of the original architecture should not exist.
For MyZubster, that may mean allowing established standards to handle credentials and attestations while concentrating our engineering effort on the part we actually care about:
human claim

↓

real activity

↓

inspectable evidence

↓

artifact

↓

verification

↓

provenance

and making that entire chain navigable.
That's the direction we're exploring next.
CoderLegion title
Engineering an Evidence-First Knowledge Graph: GitHub, SHA-256, Ethereum and MyZubster
Subtitle
How a real contributor test evolved from Knowledge Cards and RAG into canonical payloads, on-chain proofs, provenance graphs, and a rethink of our attestation architecture.
Short description / excerpt
We tested MyZubster end-to-end with a real contributor: authentication, private and public Knowledge Cards, GitHub evidence, Qdrant/RAG, a Knowledge Graph, canonical payloads, SHA-256 and Ethereum Sepolia. The most important result wasn't the smart contract—it was realizing that MyZubster may be better positioned as an evidence and provenance graph built on interoperable attestation standards.

Building Verifiable Bounty Payments and Revocable Account Sessions at MyZubster Over the latest MyZu

Mon, 28 Sep 2026 14:14:58 +0000

Engineering MyZubster: Verifiable Crypto Settlements, Refresh-Token Rotation, Revocable Sessions, and Cross-Mongoose Runtime Hardening

MyZubster recently completed a substantial infrastructure iteration spanning two systems that initially appear unrelated: contributor bounty settlement and authentication/session security.

They share an important engineering requirement:

external or client-side state must not be treated as authoritative without server-side verification and persistence.

For bounty payments, a blockchain transaction needs to become a verified, replay-safe treasury record.

For authentication, possession of a cryptographically valid JWT should not permanently imply that a session is still authorized.

This development cycle introduced infrastructure addressing both problems.

This article goes through the implementation, security properties, failure modes we discovered during integration testing, and the validation performed before pushing the final changes.

1. The Systems We Changed

The work covered two major areas.

Treasury and bounty settlement

We implemented infrastructure for recording verified cryptocurrency conversions associated with bounty settlement.

The flow now supports:

external transaction
        │
        ▼
blockchain reference
        │
        ▼
conversion verification
        │
        ▼
normalized treasury object
        │
        ▼
dry-run validation
        │
        ▼
idempotency check
        │
        ▼
persistent settlement record

Authentication and account security

MYZ-71 introduced a more substantial authentication lifecycle:

authentication
      │
      ▼
persistent server session
      │
      ├── short-lived access token
      │
      └── rotating opaque refresh token
                    │
                    ▼
             replay detection
                    │
                    ▼
              session revocation

The frontend then gained automatic renewal, account/device controls, terminal-expiry recovery, and CSRF protection for cookie-authenticated mutations.

Part I — Verifiable Cryptocurrency Bounty Settlement

2. Moving Beyond “A Transaction Was Sent”

Sending cryptocurrency is only one part of a bounty system.

From an engineering perspective, the application needs to answer several additional questions:

Why was the transaction made?
Which asset left the treasury?
Which asset arrived?
Which network was used?
What external provider participated?
Which blockchain transaction proves the operation?
Has the conversion been independently verified?
Has this exact operation already been recorded?
Can the dashboard consume the result without directly coupling itself to persistence?

For the settlement flow implemented during this cycle, the normalized record contains information such as:

{
  "purpose": "BOUNTY_SETTLEMENT",
  "provider": "SimpleSwap",
  "source": {
    "asset": "BTC",
    "amount": "0.000464",
    "txId": "2ea0bc209a2d5bcfbf520f830f1a39dee0691e47c461afc316e5aeb2e83cdd2d"
  },
  "target": {
    "asset": "ETH",
    "network": "ethereum-mainnet",
    "amount": "0.014355678390291923",
    "txId": "0x62b58e69931ef6046d8f68df39c118fe0b1fc7234b88c780d26a8b53fdcddcc6"
  },
  "state": "CONVERSION_COMPLETED",
  "verification": {
    "status": "VERIFIED"
  }
}

This represents the BTC → ETH treasury conversion as application state rather than leaving the operation as an external exchange receipt or manually interpreted transaction history.

3. Separating Verification From Persistence

A financial script should not immediately mutate production data simply because it successfully parsed its input.

We therefore implemented a dry-run path.

The sequence becomes:

construct record
     │
     ▼
validate input
     │
     ▼
verify external reference
     │
     ▼
print normalized result
     │
     ▼
STOP

The operator sees:

DRY-RUN OK — no database write performed

Only explicit commit mode proceeds to persistence.

This creates a useful operational boundary:

verification != mutation

That distinction matters for treasury operations because malformed metadata can be caught before it becomes part of the persistent accounting history.

4. Idempotent Conversion Recording

Financial scripts are frequently rerun.

A terminal disconnect, uncertain command result, deployment retry, or operator mistake should not produce a second logical settlement record.

The first committed execution recorded the conversion:

RECORDED
{
  "id": "6ab96bcb4c201256912e837a",
  "purpose": "BOUNTY_SETTLEMENT",
  "state": "CONVERSION_COMPLETED",
  "verificationStatus": "VERIFIED"
}

Repeating the operation returned:

IDEMPOTENT REPLAY — conversion already recorded

instead of creating a duplicate.

The desired invariant is:

same external conversion
        +
same transaction identity
        =
same internal logical record

This is much safer than relying on the operator to remember whether a previous command completed.

5. Provider Boundary for Settlement Observability

The settlement dashboard should not need to understand database internals.

We introduced a conversion-provider boundary so the dashboard can receive preloaded conversion information.

Conceptually:

buildDashboard({
  conversionProvider: () => conversion
});

This provides two useful properties.

First, the dashboard can consume treasury state without performing hidden database I/O.

Second, provider failure does not need to crash the entire dashboard.

If the repository fails, the conversion layer can instead return:

configured: false
items: []
reason: conversion repository unavailable

and emit an explicit warning.

That is deliberate degraded behavior.

For operational dashboards, an unavailable subsystem should be visible as unavailable — not silently represented as empty valid state and not allowed to crash unrelated observability.

Part II — Revocable Authentication Sessions

6. Why JWT Validation Alone Was Not Enough

A signed JWT answers an important question:

Was this token issued by a trusted signer and has its cryptographic validity expired?

It does not necessarily answer:

Does the server still want this browser session to exist?

Those are different questions.

A user may need to revoke a stolen device.

A refresh token may have been replayed.

An administrator or security flow may invalidate a session.

A user may explicitly log out.

For these cases, authentication needs authoritative server-side state.

We introduced persistent AuthSession records.

The resulting model is conceptually:

User
 │
 ├── AuthSession A
 │      ├── sessionId
 │      ├── expiry
 │      ├── device metadata
 │      ├── lastSeenAt
 │      ├── refreshTokenHash
 │      └── revocation state
 │
 ├── AuthSession B
 │
 └── AuthSession C

Access JWTs contain a session identifier.

JWT verification can therefore be followed by server-session validation.

A valid signature no longer necessarily means an active session.

7. Refresh Tokens Became Opaque, Persistent, and Rotating

The new refresh credential is an opaque token.

Its raw value is not persisted.

Instead, the server stores an HMAC-SHA256 representation.

The lifecycle looks like this:

refresh token R1
      │
      ▼
hash / identify
      │
      ▼
validate session
      │
      ▼
atomically consume R1
      │
      ▼
issue R2
      │
      ▼
store hash(R2)

R1 is now consumed.

The server maintains bounded history for previously consumed refresh-token hashes.

The bounded part matters: security metadata must not create an indefinitely growing session document.

8. Atomic Rotation

Refresh rotation introduces a concurrency problem.

Imagine two requests attempting to use R1 at nearly the same time:

Request A ── R1 ──┐
                  ├── session
Request B ── R1 ──┘

A naive implementation could allow both requests to observe R1 as valid before either update becomes visible.

Rotation therefore needs an atomic state transition.

Conceptually:

UPDATE AuthSession
WHERE
    session = S
AND currentRefreshHash = hash(R1)

SET
    currentRefreshHash = hash(R2)
    consumed += hash(R1)

Only the request that successfully matches the expected current state should rotate the token.

This is effectively compare-and-swap behavior applied to authentication state.

9. Refresh Replay Detection

Single-use refresh tokens provide another security signal.

If R1 was already consumed and later appears again, something unusual happened.

Possibilities include:

credential theft;
duplicated stale browser state;
client bugs;
copied authentication material;
an actual replay attempt.

Our implementation treats reuse as a security event.

The tested behavior is:

consumed refresh token reused
            │
            ▼
AUTH_REFRESH_REPLAY
            │
            ▼
revoke complete session
            │
            ▼
revokedReason = refresh-token-replay

The response is therefore stronger than simply rejecting the stale token.

The associated session itself becomes invalid.

10. Frontend Refresh Deduplication

Server-side single-use credentials require corresponding client-side concurrency control.

Suppose a page launches five API requests and the access token expires.

All five may receive 401.

Without coordination:

401 A → refresh
401 B → refresh
401 C → refresh
401 D → refresh
401 E → refresh

Only one of those refresh attempts can legitimately consume the current refresh token.

So the browser authentication client shares an in-flight refresh promise:

401 A ─┐
401 B ─┤
401 C ─┼──► one refresh operation
401 D ─┤
401 E ─┘
              │
              ▼
          new access state
              │
              ▼
         retry requests

This aligns frontend concurrency semantics with backend single-use token semantics.

Requests are retried only once.

The client does not enter an infinite 401 → refresh → retry loop.

Part III — Account Security

11. Session Management API

The session architecture is exposed through account-security endpoints:

POST   /api/auth/refresh
POST   /api/auth/logout
GET    /api/auth/me
GET    /api/auth/me/sessions
DELETE /api/auth/me/sessions/:sessionId

These endpoints provide the foundation for user-controlled session management.

12. Active Device Management

The frontend now contains an account-security surface.

Users can inspect session information including:

whether the session is the current one;
creation time;
last activity;
expiry;
available revocation actions.

A remote session can be revoked without destroying every other login.

The current session can also be terminated.

When that happens, server-side state is revoked and browser authentication state is cleared.

This turns logout from a mostly client-side token deletion into an actual server-side authorization transition.

13. Session Metadata and Privacy

Session observability is useful, but it should not require retaining unnecessary sensitive network data.

The session implementation therefore does not store raw client IP addresses as plain values.

Instead, IP information can be represented through an HMAC-SHA256 digest.

This allows certain correlation/security use cases without turning the session collection into a direct historical database of raw IP addresses.

Session lifetime is also bounded.

Part IV — Authentication Recovery in the Metaverse

14. A Renewable Authenticated Fetch Layer

The Metaverse API was moved onto a reusable authenticated request layer.

The high-level flow is:

API request
    │
    ▼
access accepted?
 ┌──┴──┐
yes    no
 │      │
return  ▼
     renewable auth failure?
          │
       ┌──┴──┐
       no    yes
       │      │
      fail    ▼
           refresh
              │
              ▼
          retry once

Not every 401 is renewable.

An explicitly revoked session, for example, should not cause endless attempts to resurrect authentication.

The client distinguishes terminal authentication state from renewable access-token failure.

15. Terminal Expiry Events

If refresh fails terminally, the browser:

clears migration authentication state;
emits an authentication-expired event;
redirects through the login flow.

This behavior was connected to:

Neon Plaza;
Metaverse room pages;
room creation.

Multiple concurrent failures are deduplicated so the application does not perform several competing redirects.

16. Preserving the Intended Destination

Authentication expiry should not destroy user navigation context.

The system preserves a local return path so a user can authenticate and return to the intended Metaverse location.

That can include:

/metaverse/rooms/example?invite=...#section

But return paths are also security-sensitive.

Protocol-relative destinations are rejected.

The application falls back to a known local Metaverse path when the return target is unsafe.

This prevents the convenience mechanism from becoming an open-redirect primitive.

Part V — CSRF Protection

17. Cookies Changed the Threat Model

Once authentication relies on HttpOnly cookies, the browser automatically attaches those credentials to matching requests.

That is useful for protecting credentials from direct JavaScript access, but it also means CSRF must be addressed explicitly.

The server now distinguishes:

Authorization: Bearer ...

from ambient cookie authentication.

Unsafe cookie-authenticated browser mutations require trusted request-origin information.

18. Origin, Referer, and Fetch Metadata

For applicable unsafe methods, the server evaluates browser request context.

Cross-site Fetch Metadata can be rejected before session lookup:

Sec-Fetch-Site: cross-site
        │
        ▼
      reject

Trusted Origin or Referer information is checked for cookie-authenticated mutations.

Rejected requests return a structured error:

AUTH_CSRF_REJECTED

Trusted production origins can be configured rather than hardcoded.

Bearer clients remain supported during the migration because explicit bearer credentials have different ambient-credential behavior from browser cookies.

Part VI — Structured Authentication Errors

19. Error Codes Instead of Raw Cryptographic Errors

Raw JWT/library exceptions are not a good public API.

The authentication layer now uses structured application errors and request IDs.

Conceptually:

{
  "error": {
    "code": "AUTH_...",
    "request_id": "..."
  }
}

This creates a stable contract between server and frontend.

The UI can distinguish:

access expiry;
revoked sessions;
refresh replay;
terminal refresh failure;
CSRF rejection;

without parsing arbitrary exception strings.

Request IDs also improve operational debugging.

Part VII — The Mongoose/BSON Failure We Found During Testing

20. The Initial Symptom

One of the most useful results of the final integration test was not a passing test.

It was a hang.

The social-identity suite stopped at:

RUNS tests/socialIdentityService.test.js

Eventually it failed with:

MongooseError:
Operation `metaversecharacters.findOne()` buffering timed out after 10000ms

The same thing happened during test cleanup:

metaversecharacters.deleteMany()
buffering timed out

At first glance, this looked like a MongoMemoryServer or asynchronous-test problem.

It wasn't.

21. Two Mongoose Runtimes in One Process

The root application used:

Mongoose 7.8.12

while the nested backend used:

Mongoose 8.24.3

User and AuthSession came from the root runtime.

MetaverseCharacter came from:

backend/src/models/MetaverseCharacter

and that model resolved its own:

require('mongoose')

against the backend installation.

So the test had effectively created:

MongoMemoryServer
       │
       ▼
root Mongoose 7
       │
       ├── User
       └── AuthSession


backend Mongoose 8
       │
       └── MetaverseCharacter
            X
       no connected test DB

The timeout suddenly made sense.

22. Connecting Both Runtimes Revealed the Deeper Problem

As a diagnostic step, both Mongoose instances were connected to the same in-memory MongoDB.

The timeout disappeared.

But the next error was more informative:

BSONVersionError:
Unsupported BSON version,
bson types must be from bson 6.x.x

An ObjectId produced by one Mongoose/BSON ecosystem was being passed into a model using another BSON implementation.

So simply connecting both instances was not a sound fix.

It exposed the deeper boundary problem.

23. Model Factory as the Boundary Repair

We did not solve this by increasing Jest timeouts.

We did not keep --forceExit.

We did not normalize every ObjectId manually and hope no other BSON type crossed the boundary.

Instead, MetaverseCharacter was changed to expose a model factory.

Conceptually:

function createMetaverseCharacterModel(mongoose) {
  const schema = new mongoose.Schema({
    // ...
  });

  return mongoose.models.MetaverseCharacter ||
    mongoose.model('MetaverseCharacter', schema);
}

Backend consumers can continue using the backend Mongoose runtime.

Root consumers can instantiate the same schema using the root runtime.

That changes the topology to:

Root application
       │
       ▼
root Mongoose
       │
       ├── User
       ├── AuthSession
       └── MetaverseCharacter

and:

Backend application
       │
       ▼
backend Mongoose
       │
       └── MetaverseCharacter

The schema definition can be shared without forcing two incompatible Mongoose runtime objects into the same database operation.

24. Why This Matters Beyond Tests

It would have been easy to classify the failure as “just a Jest problem.”

That would have been incorrect.

The test exposed an actual architectural boundary:

shared model source
≠
shared Mongoose runtime

Node's module resolution can load different versions of the same dependency from different package trees.

ODM objects such as ObjectIds, models, connections, and schemas are not automatically interchangeable simply because they ultimately represent MongoDB concepts.

The lesson is broader:

When a Node.js process contains multiple ODM/library runtimes, pass primitives or explicitly bind shared schemas to the owning runtime. Do not assume runtime-specific objects are portable across package boundaries.

Part VIII — Repairing Source Corruption

25. UTF-8 and Control-Character Validation

During the MYZ-71 repair we also discovered corrupted frontend source content.

The affected surfaces included authentication/security and Metaverse pages.

We explicitly checked for:

Unicode replacement characters;
unexpected control characters;
malformed JSX;
corrupted localized strings.

The affected source was repaired and then checked again before running the frontend suite.

This was important because some apparent React/parser failures were not behavioral regressions at all — they were malformed source bytes or broken string literals.

26. JavaScript String Parsing Failure

Another backend gate exposed a malformed JavaScript string containing an apostrophe:

Autorizzazione GitHub non associata all'account ...

The single quote terminated the surrounding single-quoted JavaScript string.

This prevented Babel/Jest from parsing the controller.

The string was repaired and syntax validation was performed before rerunning the backend suites.

This is a simple bug, but it reinforces why syntax/build gates belong in the final validation pipeline even after behavioral tests have been added.

Part IX — Validation

27. Backend Final Gate

After repairing the Mongoose boundary and JavaScript syntax, the targeted backend gate completed successfully:

Test Suites: 3 passed, 3 total
Tests:       18 passed, 18 total
Time:        4.478 s

The suites covered the relevant session controller, social authentication callback, and social identity behavior.

The previously hanging social identity tests now completed normally.

28. Frontend Final Gate

The targeted frontend gate also completed successfully:

Test Suites: 4 passed, 4 total
Tests:       13 passed, 13 total

Coverage included the authentication-expiry redirect, account-security page, auth-session API, and Metaverse API behavior.

29. Production Build

We then generated an optimized React production build:

Creating an optimized production build...
Compiled with warnings.

The build completed successfully.

The remaining ESLint warnings were non-blocking and outside the scope of the repair.

They were deliberately not mixed into the security fix simply to produce an artificially warning-free diff.

30. Git Integrity Gate

Before committing:

git diff --cached --check

completed cleanly.

Generated build artifacts were removed from the commit.

An unrelated untracked directory was also deliberately kept outside the MYZ-71 commit.

The final repair was committed as:

1878e02a
fix: harden revocable account sessions

and pushed to the MYZ-71 branch.

Part X — What Is Operational Now

31. Treasury

The implemented treasury infrastructure now supports:

bounty settlement purpose
        │
        ▼
crypto source transaction
        │
        ▼
asset conversion
        │
        ▼
target-chain transaction
        │
        ▼
verification
        │
        ▼
dry-run
        │
        ▼
idempotent persistence
        │
        ▼
dashboard provider

The recorded BTC → ETH operation reached:

CONVERSION_COMPLETED
VERIFIED

and replaying the recorder did not create a duplicate conversion.

32. Authentication

The implemented authentication/session infrastructure now provides:

login
 │
 ▼
persistent AuthSession
 │
 ├── access JWT bound to session
 │
 └── opaque rotating refresh credential
          │
          ├── atomic rotation
          ├── bounded consumed-token history
          └── replay detection
                   │
                   ▼
              session revocation

The frontend adds:

authenticated request
       │
       ▼
renew if allowed
       │
       ▼
retry once
       │
       ▼
terminal failure?
       │
       ▼
clear state + auth-expired event
       │
       ▼
login
       │
       ▼
restore safe local destination

Part XI — Security Properties Achieved

The resulting implementation has several concrete security properties.

Server authority over sessions

A cryptographically valid JWT can still be rejected when its server session has been revoked or removed.

Single-use refresh credentials

Refresh credentials rotate instead of remaining indefinitely reusable.

Replay detection

Reuse of consumed refresh state causes session revocation.

No raw refresh-token persistence

Refresh credentials are represented server-side through cryptographic hashes.

Client/server concurrency alignment

The frontend shares concurrent refresh work while the backend performs atomic token rotation.

Remote session revocation

A user can invalidate another authenticated device without necessarily destroying every active session.

Unsafe cookie-authenticated browser operations are protected by trusted-origin/Fetch Metadata checks.

Structured auth failures

Clients receive stable authentication semantics rather than raw JWT/library errors.

Authentication recovery preserves useful local navigation without accepting arbitrary protocol-relative redirect destinations.

Explicit ODM runtime ownership

The same Metaverse schema can be bound to the Mongoose runtime that actually owns the surrounding connection and models.

Replay-safe treasury persistence

Reexecuting the same verified conversion recorder does not produce duplicate settlement state.

Part XII — What We Deliberately Did Not Claim

Engineering reports are more useful when they clearly distinguish completed infrastructure from future work.

This iteration does not mean the entire MYZ-71 roadmap is finished.

Remaining work includes:

passkeys and/or magic-link authentication;
step-up authentication for sensitive operations;
extending automatic renewal to authenticated clients outside the current Metaverse integration;
completing migration away from legacy localStorage/bearer compatibility;
strict production server-session enforcement after the migration window;
production trusted-origin verification;
broader OAuth production verification;
production MongoDB stability validation.

On the treasury side, a verified conversion record is also not the final form of a contributor payment platform.

Future settlement work can connect:

bounty
  ↓
contributor
  ↓
approved amount
  ↓
payment instruction
  ↓
source transaction
  ↓
optional conversion
  ↓
destination transaction
  ↓
verification
  ↓
notification
  ↓
settled bounty state

into one auditable lifecycle.

Conclusion

The most valuable part of this development cycle was not the number of endpoints or React components added.

It was the strengthening of system boundaries.

For payments:

blockchain activity
      ↓
verified application state
      ↓
idempotent treasury record

For authentication:

client credential
      ↓
server-authoritative session
      ↓
rotating credentials
      ↓
replay detection
      ↓
revocation

And for the Node/MongoDB architecture:

shared schema
      ↓
explicit runtime ownership
      ↓
consistent Mongoose connection

The final repair was validated with 18/18 targeted backend tests, 13/13 targeted frontend tests, a successful optimized frontend build, and a clean Git diff check.

More importantly, integration testing exposed a real Mongoose 7/Mongoose 8 BSON boundary that would have been easy to hide with larger timeouts. Fixing the ownership boundary instead of suppressing the symptom left the architecture in a better state.

MyZubster now has stronger primitives for two areas where correctness matters disproportionately: moving value and maintaining identity.

That gives us a much safer base for the next layer of automation.

Securing the MyZubster Metaverse: From Stateless JWTs to Revocable Sessions

Mon, 28 Sep 2026 12:31:55 +0000

Securing the MyZubster Metaverse: From Stateless JWTs to Revocable Sessions
We have completed another major security milestone for the MyZubster Metaverse.
The original authentication system depended mainly on browser-stored JWT access tokens. That solution was simple, but it could not provide the session lifecycle controls required by a platform with persistent identities, virtual characters, private rooms, realtime interactions and future Marketplace or payment operations.
Under MYZ-71 — Authentication, Sessions & Account Security, we have now implemented:

persistent server-side sessions;
session-bound JWT access tokens;
real logout;
active-device management;
remote session revocation;
rotating opaque refresh tokens;
refresh-token replay detection;
automatic access-token renewal for the Metaverse;
concurrent refresh deduplication;
origin-based CSRF protection;
structured authentication errors.
The implementation is currently available in a Draft pull request:
https://github.com/danieldirimini-myzubster/myzubster/pull/10
It has not been merged or declared production-ready yet.
Why stateless JWTs were no longer enough
A server can validate a JWT without loading a database session, but a completely stateless design has significant limitations.
A token may remain valid even after:
the user logs out;
the device is no longer trusted;
the password changes;
the account owner wants to disconnect another device;
a security event requires immediate revocation.
Long-lived tokens also increase the impact of credential theft.
If they are stored in localStorage, they are accessible to frontend JavaScript. This makes an XSS vulnerability significantly more dangerous.
We moved toward a hybrid architecture:
User authentication
```
  ↓
```
Persistent server-side session
```
  ↓
```
Short-lived JWT access token
```
  ↓
```
Rotating opaque refresh token
```
  ↓
```
Automatic renewal
```
  ↓
```
Replay detection and revocation
The JWT is still useful, but the persistent session is now the final source of authority.
Persistent server-side sessions
Each authenticated device receives its own MongoDB AuthSession.
A session records:
the session identifier;
the owning user;
creation and expiration timestamps;
last activity;
limited device metadata;
a privacy-preserving network digest;
revocation time and reason;
the current refresh-token hash;
previously consumed refresh-token hashes;
refresh-token rotation metadata.
MongoDB TTL indexing is used to remove expired sessions automatically.
A valid JWT is rejected when its corresponding session is missing, expired or revoked.
Binding access tokens to sessions
New access tokens include a server-generated sid claim.
A simplified token looks like this:
{
"sub": "user-id",
"sid": "session-uuid",
"jti": "access-token-uuid",
"iat": 1790540000,
"exp": 1790540900
}
Request authentication now consists of two steps:
Verify the JWT
```
↓
```
Validate the server-side session
This provides both efficient request authentication and immediate revocation.
If a device is removed from the account-security interface, its JWT stops working even if it has not reached its expiration time.
Real logout and active-device management
The React frontend now includes:
/account/security
Users can inspect:
active sessions;
the current device;
creation time;
last activity;
expiration time;
limited device information.
They can disconnect another device or terminate their current session.
The supporting endpoints are:
POST /api/auth/logout
GET /api/auth/me
GET /api/auth/me/sessions
DELETE /api/auth/me/sessions/:sessionId
Logout is idempotent.
Even when the access token is expired or malformed, the backend clears the authentication cookies. When possible, the refresh token is used to identify and revoke the persistent session.
Opaque refresh tokens
The refresh token is deliberately not a JWT.
Its conceptual format is:
myzr.<session-id>.<cryptographically-random-secret>
The raw token is never stored in MongoDB.
The backend calculates:
HMAC-SHA256(server secret, raw refresh token)
Only the resulting digest is persisted.
If the session database is exposed, an attacker cannot directly use the stored digest to renew a session.
Atomic refresh-token rotation
Refresh tokens are single-use.
When the browser calls:
POST /api/auth/refresh
the server:
1. reads the HttpOnly refresh cookie;
2. extracts the session identifier;
3. hashes the submitted token;
4. creates a new random refresh token;
5. atomically compares the submitted hash with the current stored hash;
6. replaces the current hash;
7. stores the consumed hash in a bounded history;
8. issues a new access token;
9. sends new authentication cookies.
  Only one request can successfully consume the current token.
  This prevents ordinary concurrent requests from rotating the same credential more than once.
  Refresh-token replay detection
  The server maintains a bounded history of consumed refresh-token hashes.
  If a previously consumed token appears again, the session is treated as potentially compromised:
  Old refresh token submitted
  ↓
  Replay detected
  ↓
  Entire session revoked
  ↓
  Cookies cleared
  ↓
  New login required
  The revocation reason is recorded as:
  refresh-token-replay
  The history is limited so the MongoDB session document cannot grow indefinitely.
  Automatic renewal in the Metaverse
  We introduced a reusable frontend client:
  authenticatedFetch(input, options)
  The client handles the access-token lifecycle:
  Protected request
  ↓
  401 access-token response
  ↓
  POST /api/auth/refresh
  ↓
  Receive rotated access token
  ↓
  Retry original request once
  The complete Metaverse API module now uses this client.
  It covers operations involving:
profiles;
rooms;
invitations;
sessions;
participants;
stage access;
messages;
moderation;
reports;
landmarks;
realtime world actions.
The client retries only recognized access-token errors and never enters an infinite retry loop.
Deduplicating concurrent refresh operations
A Metaverse page can perform several API requests simultaneously.
When an access token expires, multiple requests may receive a 401 at approximately the same time.
Without coordination, every request could call /api/auth/refresh using the same single-use refresh token. This could produce an accidental replay event.
The client therefore shares one in-flight refresh operation:
Request A → 401 ┐
Request B → 401 ├─→ One refresh
Request C → 401 ┘
```
               ↓
         New access token
               ↓
        Retry A, B and C
```
Tests confirm that two simultaneous protected requests produce exactly one refresh operation and two successful retries.
Handling terminal authentication failures
The client does not silently renew an explicitly revoked session.
For example:
AUTH_SESSION_REVOKED
is returned directly to the application.
If the refresh token is no longer valid, the client:
clears the temporary browser credentials;
emits:
myzubster:auth-expired
The UI can use this event to return the user to authentication without every page implementing its own renewal logic.
CSRF protection
Cookie authentication introduces a different security risk: browsers attach cookies automatically.
State-changing requests must therefore be protected against Cross-Site Request Forgery.
We added an origin-based CSRF guard for unsafe cookie-authenticated methods:
POST
PUT
PATCH
DELETE
The guard evaluates:
Origin;
Referer;
Sec-Fetch-Site;
configured application origins;
the actual request target.
A browser request marked as cross-site is rejected before the backend performs a session lookup:
Sec-Fetch-Site: cross-site
The API returns a structured response:
{
"success": false,
"request_id": "request-uuid",
"error": {
"code": "AUTH_CSRF_REJECTED",
"message": "Origine della richiesta non autorizzata"
}
}
Which routes are protected?
The trusted-origin policy is applied to:
password registration;
password login;
OAuth ticket verification;
social-login ticket exchange;
refresh-token rotation;
logout;
Gmail verification exchange;
protected mutations authenticated through cookies.
Bearer-token clients remain compatible during the migration because they do not depend on automatically attached cookies.
CLI and server-to-server clients that do not send browser origin metadata also remain supported.
Configuring trusted origins
Production and preview origins can be configured with:
AUTH_TRUSTED_ORIGINS=https://www.myzubster.com,https://preview.myzubster.com
The middleware also recognizes:
FRONTEND_URL
PUBLIC_APP_URL
GATEWAY_PUBLIC_URL
Local development accepts:
http://localhost:3000
http://127.0.0.1:3000
These values still need to be verified in the real deployment before the platform switches to cookie-only authentication.
Cookie protections
The authentication cookies use:
HttpOnly
SameSite=Lax
Secure in production
The refresh cookie is restricted to:
Path=/api/auth
This prevents it from being attached to unrelated application requests.
SameSite=Lax provides another security layer, but it is not treated as the only CSRF defence.
Privacy considerations
Raw client IP addresses are not stored.
The server stores an HMAC-derived digest instead. This allows limited correlation for security analysis without retaining the original address as ordinary application data.
Device metadata is also intentionally limited.
A session-security feature should help users recognize their devices without becoming an invasive tracking mechanism.
Validation results
The focused backend security scope passes:
30/30 tests
The focused frontend and Metaverse scope passes:
19/19 tests
The test coverage includes:
persistent session creation;
session-bound JWT validation;
real logout;
remote revocation;
refresh-token hashing;
atomic token rotation;
refresh replay detection;
replay-driven session revocation;
automatic access-token renewal;
exactly-once retry;
concurrent refresh deduplication;
structured authentication failures;
same-origin CSRF acceptance;
configured frontend-origin acceptance;
cross-site CSRF rejection;
rejection before database lookup;
bearer-token compatibility;
non-browser compatibility.
The React production build also completes successfully.
GitHub Actions status
The affected backend suites are green:
tests/authSessionService.test.js
tests/authSessionController.test.js
tests/authMiddlewareSessions.test.js
tests/socialIdentityService.test.js
tests/csrfProtection.test.js
The complete repository currently reports:
142 test suites passed
42 test suites failed

746 tests passed
44 tests failed
The remaining failures belong to existing repository-wide problems involving Zorgax, Marketplace, seller policies and legacy Metaverse expectations.
The new authentication implementation is green in its affected scope, but the pull request remains a Draft because the repository as a whole is not yet CI-clean.
What is completed?
Persistent server-side sessions DONE
Session-bound access tokens DONE
Real logout DONE
Active-device management DONE
Remote session revocation DONE
Rotating opaque refresh tokens DONE
Refresh-token replay detection DONE
Password and social authentication DONE
Account-security interface DONE
Metaverse automatic renewal DONE
Concurrent refresh deduplication DONE
Origin-based CSRF protection DONE
Structured authentication errors DONE
Focused automated testing DONE
What is still missing?
Renewal outside the Metaverse
Some Marketplace, Zorgax, profile and administrative pages still use direct fetch calls and manually constructed bearer headers.
They need to be migrated to authenticatedFetch.
Removal of localStorage authentication
Browser storage remains as a temporary migration bridge.
The final architecture should avoid exposing reusable credentials to frontend JavaScript.
Before removing it, we must migrate:

all authenticated API clients;
protected React pages;
realtime connections;
embedded or mobile clients;
legacy user sessions.
Production trusted-origin validation
The public deployment must verify:
exact frontend origins;
preview domains;
reverse-proxy host and protocol forwarding;
cookie paths;
cross-site rejection;
same-origin refresh;
session revocation;
deliberate replay detection.
Strict session enforcement
After legacy tokens have expired, we can enable:
REQUIRE_SERVER_SESSION=true
JWTs without a valid persistent session will then be rejected.
Passkeys and magic links
MYZ-71 still includes passwordless authentication.
Passkeys require complete WebAuthn support for:
challenge generation;
relying-party configuration;
allowed origins;
credential counters;
credential revocation;
recovery.
Magic links require short-lived, single-use verification tokens.
Step-up authentication
Sensitive operations should require recent stronger authentication.
Possible examples include:
changing an email address;
changing a password;
registering or deleting passkeys;
deleting an account;
wallet operations;
payment approval;
sensitive Marketplace changes.
Production OAuth and database validation
Google, GitHub and Facebook authentication must still be tested using the production callback URLs and secrets.
MongoDB Atlas must be checked for:
TTL-index creation;
refresh-token indexes;
document growth;
expired-session cleanup;
concurrent rotation;
rolling-deployment compatibility.
Recommended next steps
1. Review the Draft pull request
2. Configure production trusted origins
3. Deploy with migration compatibility enabled
4. Verify cookies and database indexes
5. Test automatic Metaverse renewal
6. Test concurrent refresh requests
7. Test cross-site rejection
8. Test deliberate refresh-token replay
9. Migrate the remaining frontend API clients
10. Allow legacy tokens to expire
11. Remove tokens from localStorage
12. Enable strict session enforcement
13. Remove bearer migration support
14. Add passkeys, magic links and step-up authentication
  Final thoughts
  Authentication security is not just about issuing a JWT.
  A complete system must manage the entire lifecycle:
  Authenticate
  Create
  Validate
  Renew
  Observe
  Revoke
  Recover
  Monitor
  MyZubster now has the foundations for secure multi-device sessions, refresh-token rotation, replay detection, automatic Metaverse renewal and browser-origin protection.
  The next challenge is deployment and migration: validate the public environment, move the remaining clients to the shared authentication layer and eliminate browser-accessible credentials.
  Draft pull request:
  https://github.com/danieldirimini-myzubster/myzubster/pull/10

MyZubster #Metaverse #CyberSecurity #Authentication #NodeJS #ReactJS #MongoDB #JWT #CSRF #WebSecurity #SoftwareArchitecture #OpenSource

How We Built Revocable Sessions and Refresh-Token Replay Protection for the MyZubster Metaverse

Mon, 28 Sep 2026 01:35:52 +0000

Ecco la versione in inglese pronta per CoderLegion.
How We Built Revocable Sessions and Refresh-Token Replay Protection for the MyZubster Metaverse
Authentication in a Metaverse platform is more than a login form.
An authenticated identity may control characters, social relationships, private spaces, Marketplace activity and, eventually, payment-related operations. For MyZubster, relying only on long-lived JSON Web Tokens was no longer a sufficient security model.
Under MYZ-71 — Authentication, Sessions & Account Security, we started replacing stateless browser authentication with persistent, revocable and observable server-side sessions.
This article describes what we implemented, how refresh-token rotation works, how we detect token replay, what we tested and what still needs to be completed before production deployment.
The implementation is currently in a Draft pull request. It has not been merged or declared production-ready.

Why stateless JWT authentication was not enough
JWTs are useful because a server can validate them without loading a database record for every request.
However, a completely stateless model creates important limitations:

logout cannot reliably invalidate an existing token;
users cannot see which devices are authenticated;
administrators cannot revoke one compromised device;
a stolen token remains valid until expiration;
password changes do not automatically terminate sessions;
long-lived tokens increase the potential impact of credential theft;
tokens stored in localStorage can be accessed by malicious JavaScript after an XSS compromise.
We therefore introduced a hybrid architecture:
Authentication
```
↓
```
Persistent server-side session
```
↓
```
Short-lived JWT access token
```
↓
```
Rotating opaque refresh token
```
↓
```
Replay detection and session revocation
JWTs are still used for request authentication, but the server-side session is now the final source of authority.
Persistent server-side sessions
We added a MongoDB AuthSession model.
A separate session is created for each authenticated device. The record contains:
a unique session identifier;
the user who owns the session;
creation and expiration timestamps;
the last activity timestamp;
revocation state and reason;
limited device metadata;
a privacy-preserving client-IP digest;
the current refresh-token hash;
a bounded history of consumed refresh-token hashes;
refresh-token rotation metadata.
MongoDB TTL indexing is used to remove expired session records automatically.
This enables real session lifecycle management. A valid JWT is rejected whenever its associated session is expired, missing or revoked.
Session-bound access tokens
New JWT access tokens include a server-generated sid claim.
A simplified payload looks like this:
{
"sub": "user-id",
"sid": "session-uuid",
"jti": "access-token-uuid",
"iat": 1790540000,
"exp": 1790540900
}
Authentication now involves two checks:
1. Verify the JWT signature and expiration
2. Validate the persistent session referenced by sid
  This architecture preserves the advantages of JWTs while restoring the ability to revoke access immediately.
  If the user removes a device from the account-security page, the session becomes invalid even if its most recent JWT has not expired.
  Opaque refresh tokens
  The refresh token is not a JWT and does not contain trusted user data.
  Its conceptual format is:
  myzr.<session-id>.<cryptographically-random-secret>
  The random section is produced using a cryptographically secure random-number generator.
  The complete refresh token is sent to the authenticated browser, but it is never stored directly in MongoDB.
  Instead, the backend calculates:
  HMAC-SHA256(server secret, raw refresh token)
  Only the resulting digest is saved.
  This prevents a database leak from immediately exposing usable refresh credentials.
  Protected authentication cookies
  Refresh tokens are stored in cookies configured with:
  HttpOnly
  SameSite=Lax
  Secure in production
  Path=/api/auth
  HttpOnly prevents frontend JavaScript from reading the credential.
  The restricted path reduces the number of requests carrying the refresh cookie.
  Access tokens can also be received through an HttpOnly cookie. Bearer-token authentication remains temporarily supported to avoid breaking clients during the migration.
  Atomic refresh-token rotation
  A refresh token is single-use.
  When the browser calls:
  POST /api/auth/refresh
  the backend:
3. reads the refresh cookie;
4. extracts the session identifier;
5. calculates the HMAC digest of the submitted token;
6. generates a new opaque token;
7. atomically compares the submitted hash with the current stored hash;
8. replaces the stored hash;
9. records the consumed hash;
10. issues a new access token;
11. sends new access and refresh cookies.
  The database update only succeeds if:
the session exists;
the session has not expired;
the session has not been revoked;
the submitted hash is still the current refresh-token hash.
This is effectively a compare-and-swap operation.
If two requests try to use the same refresh token simultaneously, only one can rotate it successfully.
Refresh-token replay detection
Rotation prevents normal reuse, but the system must also detect when an old token appears again.
After every successful rotation, the consumed token hash is placed in a bounded history.
If a token from that history is submitted again, the server treats it as a possible credential compromise.
Previously consumed token submitted
```
          ↓
```
Replay detected
```
          ↓
```
Entire server-side session revoked
```
          ↓
```
Authentication cookies cleared
```
          ↓
```
A new login is required
The session receives the revocation reason:
refresh-token-replay
The history is limited to the latest 50 hashes so the MongoDB document cannot grow indefinitely.
Replay does not necessarily prove malicious activity. It could also indicate an outdated client or an unusual concurrency condition. Revoking the session is nevertheless the safest response.
New authentication endpoints
The current implementation provides the following endpoints.
Refresh the session
POST /api/auth/refresh
This endpoint rotates the refresh token and issues a new access token.
The raw refresh token is never included in the JSON response.
Log out
POST /api/auth/logout
Logout is idempotent.
Authentication cookies are cleared even when the access token is missing, malformed, expired or already revoked.
When possible, the refresh token is used to find and revoke the persistent session.
Retrieve the authenticated account
GET /api/auth/me
Returns the authenticated user and safe session metadata.
List active devices
GET /api/auth/me/sessions
Returns the user’s active sessions and indicates the current device.
Revoke a device
DELETE /api/auth/me/sessions/:sessionId
Allows users to terminate another session remotely.
Revoking the current session also clears local authentication state and returns the user to the login flow.
Password and social authentication
Password registration and login now create persistent sessions and issue both access and refresh cookies.
Verified social-login flows use the same architecture.
The raw refresh token is not placed inside:
redirect URLs;
OAuth query parameters;
signed callback tickets;
public API responses.
This reduces the risk of credentials leaking through browser history, analytics systems, proxy logs or referrer headers.
Account-security interface
The React frontend now includes:
/account/security
The page displays:
active sessions;
the current device;
session creation time;
last activity;
expiration time;
limited device information.
Users can remotely revoke another device or terminate the current session.
The page is linked from the authenticated account panel and Neon Plaza.
The deployment configuration also supports the route as a single-page application path and applies:
X-Robots-Tag: noindex, nofollow
Account-security pages should not be indexed by search engines.
Privacy-preserving session metadata
The system does not store raw client IP addresses.
Instead, it saves an HMAC-derived digest. This provides a limited correlation signal for security purposes without retaining the original IP address as normal application data.
Device metadata is also intentionally limited.
Session security should help users recognize their devices without becoming an invasive tracking mechanism.
Legacy migration support
The existing frontend still contains API clients that expect a token in localStorage or use:
Authorization: Bearer
Removing this immediately would break existing routes and clients.
The migration is therefore divided into stages:
Persistent sessions and cookie support
```
          ↓
```
Rotating refresh tokens
```
          ↓
```
Centralized access-token renewal
```
          ↓
```
CSRF protection
```
          ↓
```
Cookie-only authentication
```
          ↓
```
Expiration of legacy browser tokens
```
          ↓
```
Strict server-session enforcement
```
          ↓
```
Removal of localStorage and bearer migration
An environment option is available for strict enforcement:
REQUIRE_SERVER_SESSION=true
It should only be activated after existing clients and legacy credentials have been migrated.
Testing the implementation
The focused authentication implementation currently passes:
Backend: 30/30 tests
Frontend: 12/12 tests
The React production build also completes successfully.
Real MongoDB integration coverage verifies the complete replay lifecycle:
Create persistent session
```
  ↓
```
Use current refresh token
```
  ↓
```
Rotate successfully
```
  ↓
```
Submit the old token again
```
  ↓
```
Reject the replay
```
  ↓
```
Revoke the server-side session
```
  ↓
```
Persist refresh-token-replay as the reason
The following authentication suites also pass in GitHub Actions:
tests/authSessionService.test.js
tests/authSessionController.test.js
tests/socialIdentityService.test.js
Repository-wide CI status
The complete repository pipeline is not yet green.
The latest full backend execution reported:
141 test suites passed
42 test suites failed

738 tests passed
44 tests failed
The failing tests mainly concern pre-existing behavior in:

Zorgax payments and payment intents;
Marketplace flows;
seller-membership policies;
older Metaverse authentication expectations;
Metaverse observability;
legacy source-code assertion tests.
The Security Audit and MYZ-164 policy workflows also remain red because of previously identified repository-wide conditions.
The new MYZ-71 session and refresh-token suites are green, but the pull request must remain a Draft until the wider baseline and the remaining authentication work are addressed.
What still needs to be implemented
Centralized access-token renewal
The frontend needs a shared API layer that can:
1. identify an expired access token;
2. call /api/auth/refresh;
3. retry the original request;
4. prevent simultaneous refresh calls;
5. redirect to login when renewal fails.
  Without a centralized implementation, individual pages may respond differently when the access token expires.
  CSRF protection
  Cookie-based authentication requires explicit protection for state-changing operations.
  The final design should combine appropriate controls such as:
origin validation;
CSRF tokens;
double-submit cookies;
carefully selected SameSite policies;
rejection of unsafe cross-origin requests.
SameSite=Lax should not be considered the only defence.
Removal of browser-stored tokens
The current frontend still updates the legacy localStorage token as a temporary compatibility bridge.
The final architecture should not expose reusable authentication credentials to frontend JavaScript.
Before removing localStorage, we must migrate:
every authenticated API client;
protected React routes;
realtime and websocket authentication;
mobile or embedded clients;
old sessions and legacy JWTs.
Passkeys and magic links
The MYZ-71 roadmap includes passwordless authentication based on flows such as:
/auth/start
/auth/verify
Passkey support will require correct WebAuthn management for:
relying-party configuration;
allowed origins;
challenge creation and expiration;
credential counters;
credential revocation;
account recovery.
Magic links will require short-lived, single-use verification tokens and replay protection.
Step-up authentication
Sensitive operations should require recent stronger verification.
Examples include:
changing the primary email;
changing a password;
registering or deleting passkeys;
deleting the account;
accessing wallet or payment functionality;
modifying high-impact Marketplace settings.
A valid session should not automatically authorize every sensitive action indefinitely.
Password-change session policy
The application must define what happens after a password change:
revoke every session;
keep only the current session;
allow the user to choose;
require step-up authentication.
This policy must be enforced on the server and tested with real session records.
Production OAuth validation
Google, GitHub and Facebook login still need production verification for:
exact callback URLs;
public frontend and API origins;
provider secrets;
redirect allowlists;
account-linking behavior;
missing-email cases;
cookie behavior across domains.
MongoDB Atlas validation
The production deployment must verify:
TTL-index creation;
refresh-token indexes;
cleanup of expired sessions;
document growth;
rolling-deployment compatibility;
session behavior under concurrent refresh requests.
Rate limiting
Authentication endpoints need dedicated abuse protection.
This includes limits for:
password login;
refresh requests;
password recovery;
magic-link generation;
WebAuthn challenge creation;
social-auth callbacks.
Security monitoring
Structured security events should be produced for:
login success and failure;
session creation;
session revocation;
remote device removal;
refresh-token replay;
password changes;
passkey registration;
suspicious authentication activity.
Logs must never contain passwords, raw access tokens, raw refresh tokens, OAuth authorization codes or provider secrets.
Recommended deployment sequence
The safest rollout order is:
1. review the Draft pull request;
2. deploy with legacy compatibility enabled;
3. verify MongoDB indexes;
4. validate cookies using the real public domains;
5. test login, refresh, logout and remote revocation;
6. execute a controlled refresh-token replay test;
7. implement centralized frontend renewal;
8. add and validate CSRF protection;
9. monitor session and replay events;
10. allow legacy credentials to expire;
11. activate strict server-session enforcement;
12. remove localStorage and legacy bearer authentication;
13. introduce passkeys, magic links and step-up authentication.
  Current progress
  Persistent server-side sessions DONE
  Session-bound JWT access tokens DONE
  Real logout DONE
  Active-device listing DONE
  Remote session revocation DONE
  Opaque rotating refresh tokens DONE
  Refresh-token replay detection DONE
  Password and social-login integration DONE
  Account-security interface DONE
  Focused backend and frontend testing DONE

Centralized access-token renewal PENDING
CSRF hardening PENDING
Cookie-only migration PENDING
Removal of localStorage authentication PENDING
Passkeys and magic links PENDING
Step-up authentication PENDING
Production OAuth validation PENDING
MongoDB Atlas production validation PENDING
Repository-wide green CI PENDING
Final thoughts
Secure authentication is not just about generating a token.
It requires a complete lifecycle:
Issue
Verify
Renew
Observe
Revoke
Recover
Monitor
MyZubster now has the foundation for persistent multi-device sessions, secure refresh-token rotation and automatic replay response.
The next major objective is completing the browser migration safely through centralized renewal, CSRF protection and the removal of JavaScript-accessible authentication credentials.
Only after those controls are validated should strict session enforcement be activated in production.
The current implementation can be reviewed in the Draft pull request:
https://github.com/danieldirimini-myzubster/myzubster/pull/10

From Stateless JWTs to Revocable Sessions: Securing the MyZubster Metaverse

Sun, 27 Sep 2026 18:29:37 +0000

From Stateless JWTs to Revocable Sessions: Securing the MyZubster Metaverse
MyZubster is developing a persistent open-source digital ecosystem that connects verified identities, metaverse characters, community missions, knowledge, marketplaces, and contributor activity.
As the platform grows, authentication can no longer mean simply issuing a JSON Web Token and trusting it until it expires.
A persistent metaverse requires a stronger security model:

real server-side logout;
session and device management;
remote revocation;
privacy-preserving activity metadata;
secure browser cookies;
structured authentication errors;
compatibility with existing clients;
a path toward passkeys and rotating refresh tokens.
We have now implemented the first major security slice of MYZ-71 — Authentication, sessions and account security.
The implementation is available in the following draft pull request:
MYZ-71: Add revocable sessions and device security controls
The pull request is still a draft. It has not been merged or deployed to production.
Why stateless JWTs were no longer enough
The previous authentication architecture issued a signed JWT containing information such as:
{
"userId": "USER_ID",
"username": "daniel",
"role": "user",
"iat": 1789970000,
"exp": 1790574800
}
The server could validate the signature and expiration time, but it could not immediately answer several important security questions:
Did the user log out?
Was the device remotely revoked?
Is the token associated with a known active login?
Was the session disabled after the token was created?
Which browser created the session?
Should this token be rejected before its cryptographic expiration?
A purely stateless JWT remains valid until it expires. Logging out in the browser may remove a local token, but it does not necessarily invalidate a copied token.
That limitation becomes especially important when an account controls:
a persistent metaverse character;
private rooms;
creator permissions;
marketplace activity;
verified social identities;
community moderation;
contributor or reward-related actions.
The new hybrid architecture
We retained JWTs, but every newly issued JWT is now connected to a persistent server-side session.
The new token includes two identifiers:
{
"userId": "USER_ID",
"username": "daniel",
"role": "user",
"sid": "4cb9c3a1-8d39-4f2d-8a91-82c9556abc91",
"jti": "48ab2bd5-b4fa-426a-af42-3e24758e0434",
"iat": 1789970000,
"exp": 1790574800
}
The sid claim identifies the persistent session. The jti uniquely identifies the issued JWT.
Authentication now follows this sequence:
Request
↓
Extract token from secure cookie or Authorization header
↓
Verify JWT signature and expiration
↓
Read the sid claim
↓
Find the corresponding server-side session
↓
Verify ownership, expiration and revocation state
↓
Allow or reject the request
A valid JWT is no longer sufficient when its server-side session has been revoked or expired.
Persistent AuthSession records
A new MongoDB model stores the authentication state.
Conceptually, the model contains:
{
sessionId: String,
userId: ObjectId,
userAgent: String,
ipHash: String,
createdAt: Date,
lastSeenAt: Date,
expiresAt: Date,
revokedAt: Date,
revokedReason: String
}
Session creation
Every password or social login creates a cryptographically random session identifier:
const sessionId = crypto.randomUUID();
A JWT is then signed with that identifier:
const token = jwt.sign(
{
userId: String(user._id),
username: user.username,
role: user.role,
sid: sessionId
},
jwtSecret,
{
expiresIn: sessionTtl,
jwtid: crypto.randomUUID()
}
);
The session document and token are created as part of the same authentication flow.
Expiration enforcement
The session contains an explicit expiration timestamp and a MongoDB TTL index.
However, MongoDB TTL cleanup is asynchronous. An expired document may remain in the collection temporarily.
For this reason, authentication never relies only on physical deletion. The validation query requires:
{
sessionId: decoded.sid,
userId: decoded.userId,
revokedAt: null,
expiresAt: { $gt: new Date() }
}
The session becomes unusable immediately after expiration, even if MongoDB has not deleted it yet.
Revocation
Revocation does not delete the session immediately. Instead, the server records when and why it was revoked:
{
revokedAt: new Date(),
revokedReason: "user-revoked"
}
Keeping this state is useful for security analysis and makes the revocation operation auditable.
Privacy-preserving IP metadata
The server does not store raw IP addresses.
When an address is available, it is converted into an HMAC-SHA256 digest:
const ipHash = crypto
.createHmac("sha256", SESSION_IP_HASH_SECRET)
.update(ipAddress)
.digest("hex");
This allows the application to identify meaningful changes between login contexts without creating a database of clear-text IP addresses.
The HMAC secret prevents the value from behaving like a simple unsalted hash that could be easily precomputed.
Last-activity tracking without excessive writes
The session records lastSeenAt, but it is not updated on every request.
Updating the database for every authenticated API call would create unnecessary write volume.
Instead, the timestamp is refreshed only when the previous update is older than five minutes:
if (Date.now() - lastSeenAt > FIVE_MINUTES) {
await AuthSession.updateOne(
{
```
sessionId: decoded.sid,
revokedAt: null
```
},
{
```
$set: {
  lastSeenAt: new Date()
}
```
}
);
}
This provides useful account activity information while reducing database pressure.
Secure cookies and compatibility mode
The authentication token can now be received from either:
Authorization: Bearer TOKEN
or the following browser cookie:
myzubster_session
The cookie uses these properties:
{
httpOnly: true,
secure: production,
sameSite: "lax",
path: "/",
maxAge: sessionTtl
}
Why use HttpOnly?
Browser JavaScript cannot directly read an HttpOnly cookie.
This limits simple token extraction through code such as:
document.cookie
It does not solve every XSS scenario, but it substantially reduces the exposure of authentication secrets.
Why bearer tokens are still supported
MyZubster already has clients that use tokens stored by older authentication flows.
Removing bearer support immediately would invalidate existing sessions and potentially break multiple parts of the frontend.
The system therefore supports a migration period:
Legacy client
→ Authorization: Bearer

New browser session
→ HttpOnly myzubster_session cookie
Older JWTs may also lack an sid claim. They remain temporarily compatible unless strict mode is enabled:
REQUIRE_SERVER_SESSION=true
Once existing tokens have expired, strict mode can reject every JWT that is not connected to a server session.
New authentication endpoints
Four important endpoints were added.
Current account
GET /api/auth/me
Returns the authenticated user and associated public character information.
Active devices and sessions
GET /api/auth/me/sessions
Example response:
{
"success": true,
"request_id": "bf7a957e-1669-4432-a9a4-e686cb70cc07",
"sessions": [

{
  "id": "4cb9c3a1-8d39-4f2d-8a91-82c9556abc91",
  "current": true,
  "device": "Mozilla/5.0 ...",
  "createdAt": "2026-09-27T08:00:00.000Z",
  "lastSeenAt": "2026-09-27T09:00:00.000Z",
  "expiresAt": "2026-10-04T08:00:00.000Z"
}

]
}
Only sessions belonging to the authenticated user are returned.
Revoke a session
DELETE /api/auth/me/sessions/:sessionId
The update is scoped by both the user and session identifier:
{
userId: authenticatedUserId,
sessionId,
revokedAt: null
}
This prevents one account from revoking another user’s session.
Logout
POST /api/auth/logout
Logout is deliberately idempotent.
The cookie is cleared even when the submitted token is:

expired;
malformed;
already revoked;
disconnected from a server session.
The logout route is intentionally not protected by the standard authentication middleware. Otherwise, an expired token could be rejected before the handler had an opportunity to clear the invalid cookie.
When a valid session is available, logout also revokes it server-side.
Structured authentication errors
Authentication errors now use stable public codes instead of exposing raw JWT library messages.
Example:
{
"success": false,
"request_id": "5fe11522-c804-4907-a4d1-0f459f94ab2a",
"error": {
"code": "AUTH_SESSION_REVOKED",
"message": "Session expired or revoked"
}
}
Current error categories include:
AUTH_TOKEN_MISSING
AUTH_TOKEN_INVALID
AUTH_TOKEN_EXPIRED
AUTH_SESSION_REVOKED
Each response contains a request ID that can be correlated with server logs.
This is useful for debugging while avoiding the disclosure of stack traces, signing details, or internal JWT parsing errors.
Integration with social authentication
Persistent sessions are now created for:
registration;
password login;
Google login;
GitHub login;
Facebook login.
The social OAuth ticket exchange now creates the same server-side session used by password authentication and sets the secure cookie.
During integration testing, we also corrected two existing issues.
GitHub profile persistence
An undefined nested GitHub public-profile snapshot could cause a Mongoose casting error.
The new implementation updates nested data explicitly and writes the public snapshot only when a defined value is available.
Facebook accounts without email
Not every Facebook account exposes an email address.
MyZubster already supported a deterministic private relay identity for this situation. The integration tests were aligned with that intended behavior instead of incorrectly requiring every account to provide an email.
The relay value is treated as an internal identifier, not as a verified personal email.
The account-security interface
A new React page is available at:
/account/security
It displays:
the authenticated account;
all active devices;
the current device;
session creation time;
last activity;
session expiration;
controls for remote revocation;
current-session termination;
logout.
The page is linked from:
the authenticated account panel on the MyZubster home interface;
the Neon Plaza metaverse top bar.
Frontend API module
The React application uses a dedicated session API:
getCurrentAccount();
getAuthSessions();
revokeAuthSession(sessionId);
logoutCurrentSession();
clearBrowserAuth();
Requests include same-origin cookies:
fetch(path, {
credentials: "same-origin",
headers: {
...bearerMigrationFallback
}
});
Structured server errors are preserved so the UI can display a support request ID when an operation fails.
Remote device revocation
When the user revokes another device:
1. the UI requests confirmation;
2. a session-specific DELETE request is sent;
3. the server verifies session ownership;
4. the session receives a revocation timestamp;
5. the frontend removes it from the active list;
6. the revoked device is rejected on its next request.
  Current-device termination
  Terminating the current session:
7. revokes the server-side session;
8. clears the secure cookie;
9. removes legacy browser credentials;
10. redirects the user through authentication again.
  Legacy browser keys removed during logout include:
  myzubster-token
  token
  accessToken
  myzubster-user
  myzubster-identity-provider
  myzubster-metaverse-character-id
  Protected production routing
  The new account page is configured as an explicit Vercel SPA route.
  It includes:
  X-Robots-Tag: noindex, nofollow
  Account-security pages should not be indexed, cached as search snippets, or treated as public content.
  Both the root and standalone frontend Vercel configurations were updated and validated as JSON.
  Testing and verification
  Backend
  Five focused backend suites pass:
  authSessionService
  authMiddlewareSessions
  authSessionController
  socialAuthCallback
  socialIdentityService
  Results:
  5 suites passed
  24 tests passed
  Coverage includes:
persistent session creation;
bearer and cookie extraction;
active-session validation;
revoked-session rejection;
token-expiration handling;
structured errors;
session listing;
remote revocation;
valid logout;
malformed-token logout;
Google authentication;
GitHub authentication;
Facebook authentication;
persistent metaverse characters.
The social identity tests use MongoDB Memory Server so the persistence behavior is exercised rather than completely mocked.
Frontend
Three focused frontend suites pass:
authSessions API
AccountSecurityPage
MetaversePage
Results:
3 suites passed
11 tests passed
The tests cover:
same-origin credentials;
bearer-token migration;
safe session-ID encoding;
structured server errors;
local credential cleanup;
account and session rendering;
current-device labeling;
remote revocation;
removal of revoked devices;
safe date formatting;
existing Neon Plaza mission-state behavior.
The React production bundle also compiles successfully.
Repository-wide CI status
The pull request remains a draft because repository-wide CI still contains failures outside this authentication change.
A previous complete run reported:
149 test suites passed
34 test suites failed

774 tests passed
44 tests failed
The sampled failures were primarily related to existing:

Zorgax payment code;
cultural modules;
Kefir workflows;
Marketplace contract assertions;
realtime behavior;
observability source assertions.
The dependency audit also reports vulnerabilities from the unchanged lockfile. This authentication work does not modify package.json or package-lock.json.
These failures should remain visible, but unrelated systems should not be rewritten inside an authentication PR without appropriate scope and review.
What remains to be implemented
This pull request does not complete MYZ-71.
Rotating refresh tokens
The final architecture should use:
Short-lived access token
```
  +
```
Rotating refresh token
The refresh token should:
be cryptographically random;
be stored only as a hash;
belong to a token family;
be consumed exactly once;
rotate after every refresh;
detect reuse;
revoke the family after suspected replay.
A conditional database operation will be necessary to prevent two simultaneous requests from consuming the same token successfully.
Passkeys and WebAuthn
The planned API includes:
POST /api/auth/start
POST /api/auth/verify
Passkey verification must validate:
the challenge;
origin;
relying-party ID;
credential ID;
public-key signature;
authenticator data;
user presence;
user verification;
authenticator signature counters.
The account UI will also need credential listing, naming and revocation.
Magic-link authentication
Magic links require:
single-use random challenges;
hashed challenge storage;
short expiration;
rate limiting;
account-enumeration resistance;
safe redirect validation;
protection against email link scanners;
invalidation after first use.
Step-up authentication
Sensitive operations should require recent stronger authentication.
Examples include:
changing an email address;
changing payment information;
adding or removing passkeys;
exporting sensitive data;
changing privileges;
deleting an account.
The server will need to distinguish between a valid session and a recently re-verified session.
CSRF protection
SameSite=Lax is a useful baseline, but it is not the complete strategy for a cookie-authenticated application.
Before the system becomes cookie-only, state-changing requests should add explicit protection through techniques such as:
origin validation;
synchronizer tokens;
signed double-submit cookies;
required custom headers.
Removing browser-readable tokens
Bearer compatibility and local-storage cleanup exist for migration purposes.
The intended final browser architecture is:
Credentials:
HttpOnly secure cookies

Frontend JavaScript:
no direct access to reusable authentication secrets
Once the migration period ends, MyZubster should stop returning long-lived authentication tokens to frontend JavaScript.
Production validation
The following checks remain:

secure cookie verification on the public HTTPS domain;
remote revocation between two physical devices;
GitHub OAuth callback verification in production;
Google and Facebook regression checks;
MongoDB TTL-index verification;
investigation of intermittent Atlas ReplicaSetNoPrimary events;
strict-session migration testing;
request-ID propagation through production logs and proxies.
Why this is important for the metaverse
Authentication is not separate from the metaverse. It is the foundation of persistent identity.
The MyZubster identity chain is becoming:
MyZubster account
↓
Revocable server session
↓
Verified character
↓
Neon Plaza participation
↓
Missions and contributions
↓
Knowledge, marketplace and community activity
If a token is stolen, revocation must also protect the connected metaverse character and its permissions.
This session infrastructure prepares the platform for:
private metaverse rooms;
verified contributors;
creator permissions;
moderation roles;
university pilots;
community governance;
marketplace actions;
development requests;
contribution evidence.
Current status
Implemented:
persistent authentication sessions;
JWT-to-session binding;
secure cookies;
bearer-token migration;
session expiration;
device listing;
remote revocation;
server-side logout;
privacy-preserving IP hashing;
structured errors and request IDs;
social-login integration;
account-security UI;
Neon Plaza integration;
protected Vercel routing;
focused backend and frontend tests.
Still required:
refresh-token rotation;
replay detection;
passkeys;
magic links;
step-up authentication;
CSRF hardening;
removal of browser-readable credentials;
production OAuth verification;
Atlas stability verification;
repository-wide CI cleanup.
This is an important security milestone, but it is not a declaration that authentication is finished.
The goal is to make every security claim testable, observable and reversible during rollout.
In MyZubster, trust should not depend on promises. It should be built through verifiable behavior.
Suggested Coder Legion tags:
Cybersecurity
Node.js
React
Web Development
Authentication
Metaverse
Suggested excerpt:
How MyZubster is moving from stateless JWT authentication to persistent, revocable sessions with secure cookies, device management, structured errors, social-login integration and a React account-security dashboard.

Database Concurrency 101: Optimistic vs. Pessimistic Locking

Sun, 27 Sep 2026 09:32:31 +0000

In modern software development, we often have to handle multiple users interacting with our application at the exact same time. Optimistic and pessimistic locking are two distinct strategies used to manage concurrency, which is the technical term for when multiple users or systems try to update the exact same database record simultaneously. While optimistic locking assumes conflicts are rare and verifies changes only at the very end of an operation, pessimistic locking assumes conflicts are highly likely and preemptively blocks others from accessing the data until the current operation is finished.

A Relatable Everyday Analogy

To understand the difference, imagine a collaborative digital workspace where you and your colleagues edit text files.

Pessimistic locking is like an old-school file server where only one person can open a document for editing at a time. The moment you open the file, the server locks it. If your colleague tries to open it, they receive a "Read-Only" warning and are completely blocked from making edits until you save, close, and release your lock on the file.

Optimistic locking, on the other hand, is like a modern version control system like Git. Both you and your colleague can open, read, and edit the exact same document simultaneously. However, when you click "Save," the system checks if anyone else has saved a new version of the document since you opened it. If no one has, your save succeeds. If your colleague managed to save their changes first, the system blocks your save, alerts you to a "conflict," and asks you to review and merge the changes before trying again.

Why It Matters in the Tech Industry

In the tech industry, failing to implement a proper locking strategy leads to a highly destructive bug known as a "lost update" or a "race condition"—where the outcome of a process depends on the unpredictable sequence or timing of other events.

Imagine a financial application where two family members try to withdraw $50 from a shared $100 bank account at the exact same millisecond. Without locking, both transactions will read the balance as $100 simultaneously. Both transactions will approve the $50 withdrawal, and both will attempt to write the new balance of $50 back to the database. The bank is now down $100, but the database incorrectly shows a remaining balance of $50 because the second write silently overwrote the first one. By using database locking, software engineers prevent this double-spending, protect system integrity, and ensure transactions are processed safely.

Implementation in Node.js and MySQL

Here is a simple example of how to implement optimistic locking in a Node.js application using a MySQL database client. We do this by adding a version column to our database table and verifying it during our update query.

// A helper function to update a user's bank balance using Optimistic Locking
async function updateAccountBalance(connection, accountId, withdrawAmount, currentVersion) {
  // We attempt to update the balance and increment the version number,
  // but ONLY if the version in the database matches the version we originally read.
  const query = `
    UPDATE bank_accounts 
    SET balance = balance - ?, version = version + 1 
    WHERE id = ? AND version = ?
  `;

  const [result] = await connection.execute(query, [withdrawAmount, accountId, currentVersion]);

  // If no rows were affected, it means the version changed in the background
  // because another transaction updated it first. We must roll back and retry.
  if (result.affectedRows === 0) {
    throw new Error("Concurrency Conflict: The account was updated by another process. Please retry.");
  }

  return "Balance updated successfully!";
}

The Strategic Takeaway

Choosing between these two approaches is always a trade-off between speed and strict prevention. You should default to optimistic locking for applications with low-to-medium write conflicts because it keeps your application fast, highly available, and free of database bottlenecks. However, when dealing with highly critical operations where conflicts are frequent and data corruption is unacceptable—such as reservation systems or payment gateways—pessimistic locking is the superior choice, despite the performance hit of making users wait in a digital queue.

Originally published on my blog. You can read the alternative breakdown here.

From Broken Google OAuth to a Stable Sepolia E2E Environment: Engineering MyZubster on a VPS

Sun, 27 Sep 2026 00:54:50 +0000

From Broken Google OAuth to a Stable Sepolia E2E Environment: Engineering MyZubster on a VPS

Over the last development cycle, we worked on stabilizing the MyZubster Sepolia environment and turning what was initially a collection of partially working services into a much more predictable end-to-end infrastructure.

The work involved several layers of the stack:

Node.js / Express backend
Google OAuth 2.0
React frontend
Ethereum Sepolia testing
MongoDB
PM2 process management
Cloudflare tunneling
VPS deployment
Environment configuration
Urban Lab / I-ECO-01 services
Git branch and deployment management

This article documents what we fixed, how we diagnosed the problems, what is working now, and what still needs to be completed.

1. The Starting Point

The Sepolia environment was running on a VPS with several services managed by PM2.

The main components included:

MyZubster Sepolia backend
MyZubster Sepolia frontend
MyZubster Social
MyZubster Web
Urban Lab / I-ECO-01
Cloudflared

The environment was operational, but two important areas were unstable.

The first was Google authentication.

Users could start the OAuth flow and Google returned the authorization callback, but the backend failed during the authorization-code exchange.

The second problem was the Urban Lab service.

It was trapped in a PM2 restart loop, generating millions of restarts because runtime dependencies required by server.js were missing.

We debugged these independently instead of treating the VPS as a single black box.

2. Debugging Google OAuth

The original application error was generic:

Login Google non riuscito

That message was not enough to identify whether the problem was caused by:

the authorization code,
OAuth state,
callback URI,
client ID,
client secret,
Google configuration,
proxy forwarding,
or application logic.

We therefore instrumented the token exchange temporarily.

The relevant backend flow exchanges the authorization code against Google's token endpoint and then retrieves the verified Google user profile.

The diagnostic logging revealed the actual failure:

status: 401
error: invalid_client
description: The provided client secret is invalid.

This immediately changed the investigation.

The callback itself was reaching the backend correctly.

The OAuth state existed.

The authorization code existed.

The problem was the credentials used during the token exchange.

3. Validating the Runtime Environment

One of the important lessons from this debugging session was that validating .env alone is not sufficient.

We checked both the configuration file and the running PM2 process.

The Google client configuration was inspected without printing the complete secret.

We verified properties such as:

CLIENT ID
SECRET PRESENT
SECRET LENGTH
SECRET FORMAT
CALLBACK URL

The configured callback was:

https://sepolia.myzubster.com/api/auth/social/google/callback

We then discovered that the previously configured secret was not a valid secret for the OAuth client being used.

After replacing it with the correct Google OAuth client secret, the token exchange started succeeding.

4. Google OAuth Successfully Completed

After correcting the OAuth credentials, the backend began reporting successful callbacks.

The flow became:

Browser
   |
   v
Google Authorization
   |
   v
/api/auth/social/google/callback
   |
   v
Authorization Code Exchange
   |
   v
Google Access Token
   |
   v
Google UserInfo
   |
   v
MyZubster account verification
   |
   v
Social login ticket
   |
   v
Frontend

The backend successfully received callbacks containing both the authorization code and OAuth state.

Most importantly, the previous invalid_client token error disappeared.

The public provider endpoint also confirmed:

{
  "success": true,
  "data": {
    "providers": {
      "google": true,
      "github": false,
      "facebook": false
    }
  }
}

Google authentication is therefore enabled in the Sepolia environment.

5. Fixing the OAuth Redirect

Authentication succeeding at Google was only part of the problem.

The backend was still redirecting successful logins to an old Vercel deployment.

The runtime destination was effectively:

old Vercel deployment

instead of:

https://sepolia.myzubster.com

We corrected the runtime environment so the relevant public URLs resolve to the Sepolia deployment.

The running PM2 process was verified with:

FRONTEND_URL=https://sepolia.myzubster.com
GATEWAY_PUBLIC_URL=https://sepolia.myzubster.com
PUBLIC_APP_URL=https://sepolia.myzubster.com

This was an important distinction.

Updating .env is not necessarily equivalent to updating the environment of an already running process.

For production-style deployments, the effective runtime configuration must always be verified.

6. Testing the Complete OAuth Path

We then tested the complete flow again.

The callback reached the backend with:

hasCode: true
hasState: true
hasError: false

The Google token exchange completed successfully.

No new Google token error appeared.

The backend remained online.

The provider endpoint remained available.

The PM2 process PID remained stable.

This gave us evidence that the fix was not simply masking an application crash.

7. Removing Temporary Debug Instrumentation

Temporary instrumentation is useful during incident analysis, but it should not become permanent application noise.

After confirming the OAuth path, we removed the temporary HTTP callback logger.

We retained useful error-level logging around OAuth failures:

console.error('[OAUTH-CALLBACK-ERROR]', {
  provider,
  message: String(error?.message || error)
});

This provides enough information for future troubleshooting without logging authorization codes, OAuth state values, access tokens, or secrets.

That distinction matters.

Authentication debugging must provide observability without accidentally creating a credential-leak mechanism.

8. Stabilizing Urban Lab / I-ECO-01

At the same time, another service on the VPS had a completely different problem.

The urbanlab PM2 process had accumulated an extremely large restart count.

Inspection of the error log showed:

Error: Cannot find module 'helmet'

The server required:

require('dotenv').config();
const express = require('express');
const cors = require('cors');
const helmet = require('helmet');
const mongoose = require('mongoose');
const { createServer } = require('http');
const { Server } = require('socket.io');
const winston = require('winston');

But package.json originally contained only:

cors
dotenv
express
mongoose

The runtime therefore expected modules that the project did not declare/install.

We added the missing dependencies:

helmet
socket.io
winston

After installation, each required module could be resolved successfully.

9. Stopping a Multi-Million-Restart PM2 Loop

Once the missing dependencies were installed, Urban Lab started normally.

The server reported:

Server started on port 5002
MongoDB connected

Its health endpoint returned:

{
  "status": "ok",
  "service": "I-ECO-01",
  "mongodb": "connected"
}

We then compared the PM2 process before and after a waiting period.

The PID remained unchanged.

The restart counter remained unchanged.

That confirmed that the restart loop had actually stopped.

This is much more meaningful than simply seeing:

status: online

for a few seconds.

A process can repeatedly appear online while constantly crashing and restarting.

Process stability needs to be measured over time.

10. Current VPS Architecture

The resulting environment now looks approximately like this:

                    Internet
                       |
                       v
                Cloudflare Tunnel
                       |
                       v
            sepolia.myzubster.com
                       |
          +------------+------------+
          |                         |
          v                         v
     React Frontend            Express Backend
                                   |
                    +--------------+--------------+
                    |              |              |
                    v              v              v
                 MongoDB       Google OAuth   Sepolia EVM
                    |
                    v
              Application Data

Additional VPS services:

PM2
 |
 +-- MyZubster Sepolia Backend
 +-- MyZubster Sepolia Frontend
 +-- MyZubster Social
 +-- MyZubster Web
 +-- Urban Lab / I-ECO-01
 +-- Cloudflared

PM2 is configured under systemd, and the active process list has been saved using:

pm2 save

This means the current process configuration can be restored after a reboot through the existing PM2 startup integration.

11. Git Hygiene

Another important part of the work was separating debugging artifacts from actual source changes.

During debugging we created several temporary backups.

Before committing, those files were removed from the working tree.

We also explicitly verified that .env remained ignored.

The final source change was isolated and committed as:

fix(auth): improve OAuth callback error logging

The resulting commit was:

02f06f22

The working tree was clean after the commit.

The branch:

vps/sepolia-e2e-test

was then pushed to the remote repository.

This gives us a reproducible Git checkpoint corresponding to the working VPS state without committing the environment secrets.

12. Security Considerations

One of the most important lessons from this work concerns secrets.

OAuth client secrets, API keys, database credentials, private keys, GitHub tokens, JWT signing secrets, and similar credentials should never be committed to Git.

The repository correctly ignores .env.

But .gitignore is only one layer of protection.

The longer-term architecture should move production credentials toward dedicated secret management and controlled deployment-time injection.

We also deliberately avoided logging:

Google client secret
OAuth authorization code
OAuth state
access tokens
JWTs
database passwords
private keys

Observability and secret exposure must remain separate concerns.

13. What Is Working Now

At this stage we have achieved several concrete milestones.

Google OAuth can complete the server-side authorization-code flow.

The Sepolia backend exposes Google as an enabled authentication provider.

The callback is routed through the public Sepolia infrastructure.

The application runtime is using the intended Sepolia frontend/public URLs.

PM2 is running the MyZubster services.

PM2 configuration has been persisted.

Urban Lab / I-ECO-01 is no longer trapped in its missing-module restart loop.

Its MongoDB connection is operational.

The I-ECO-01 health endpoint responds successfully.

The OAuth debugging changes have been cleaned up.

The relevant source change has been committed and pushed to a dedicated branch.

14. What Still Needs to Be Done

A working OAuth callback does not mean the entire platform is production-ready.

Several areas still need engineering work.

Authentication hardening

The OAuth implementation should receive additional automated tests covering:

invalid state
expired state
missing code
provider errors
invalid client configuration
token endpoint failures
userinfo failures
unverified email
expired social-login tickets
replay attempts

GitHub and Facebook providers are currently disabled according to the provider endpoint and can be implemented or enabled separately when their configuration is ready.

Deployment reproducibility

The VPS currently represents an important working environment.

The next goal should be making that environment reproducible from source and configuration rather than depending on manual server history.

That means documenting and eventually automating:

dependency installation
environment injection
PM2 configuration
Cloudflare configuration
frontend startup
backend startup
health checks
rollback

Monitoring

A process being online is not sufficient operational monitoring.

We want visibility into:

restart rate
HTTP error rate
OAuth failure rate
MongoDB availability
response latency
memory consumption
CPU consumption
Sepolia RPC failures
WebSocket availability

Secret management

Production credentials should eventually be rotated and managed through a dedicated secret-management workflow.

CI/CD

The current branch and commit structure gives us a good foundation for moving toward automated deployment.

The desired flow is:

Developer
   |
   v
Git branch
   |
   v
Pull Request
   |
   v
Automated Tests
   |
   v
Security Checks
   |
   v
Merge
   |
   v
Sepolia Deployment
   |
   v
Health Checks
   |
   v
Production Promotion

15. Where We Want to Go

The objective is larger than simply making Google login work.

The Sepolia environment is becoming the integration layer where we can validate the interaction between identity, application state, Web3 payments, AI services, Urban Lab infrastructure, and MyZubster's broader ecosystem.

The architecture we are moving toward is approximately:

                         MyZubster
                             |
        +--------------------+--------------------+
        |                    |                    |
        v                    v                    v
     Identity             Web3 Layer          AI Layer
        |                    |                    |
   Google OAuth          Ethereum           AI Services
   GitHub OAuth          Sepolia            Zorgax / Pytho
   Social Identity       Wallets            Automation
        |                    |                    |
        +--------------------+--------------------+
                             |
                             v
                        Core Backend
                             |
             +---------------+---------------+
             |                               |
             v                               v
          MongoDB                       Urban Lab
                                            |
                                  +---------+---------+
                                  |                   |
                                  v                   v
                              IoT / GPS        Environmental Data

The Sepolia environment gives us a place to integrate these components without treating production as the testing environment.

16. Engineering Lessons

Several lessons from this debugging cycle are worth keeping.

Inspect the real provider response

Login failed tells you almost nothing.

401 invalid_client
The provided client secret is invalid

tells you exactly where to investigate.

`.env` is not necessarily the runtime environment

Always inspect what the actual process sees.

This becomes especially important with PM2, containers, systemd, CI/CD, and cloud deployment systems.

`online` does not necessarily mean healthy

Urban Lab demonstrated this clearly.

A service may be repeatedly restarted while PM2 continues reporting it as online between crashes.

PID stability, restart counters, health checks, and logs provide a much better picture.

Debug instrumentation should be temporary

Add enough instrumentation to understand the failure.

Verify the fix.

Then remove unnecessary diagnostic logging while retaining safe operational error reporting.

Infrastructure fixes should end in Git

A VPS fix that exists only in shell history is technical debt.

Once the behavior is verified, the corresponding source changes should be cleaned, reviewed, committed, and pushed.

Conclusion

What began as a Google login failure exposed several deeper infrastructure issues.

We traced the OAuth flow to the actual Google token exchange, identified an invalid client secret, corrected the runtime configuration, fixed the frontend redirect destination, validated the public callback path, cleaned the debugging instrumentation, stabilized a separate Urban Lab service that had entered a massive PM2 restart loop, verified MongoDB connectivity, persisted the PM2 process state, cleaned the Git working tree, committed the relevant OAuth changes, and pushed the working branch.

The important outcome is not simply:

“Google login works.”

The more important outcome is that the Sepolia environment is becoming observable, reproducible, testable, and suitable for real end-to-end integration work.

The next phase is to convert these manually validated infrastructure improvements into automated tests, deployment pipelines, monitoring, stronger secret management, and repeatable environment provisioning.

That is the path from a working development VPS to a maintainable engineering platform.

Debugging 6.5 Million PM2 Restarts: Stabilizing MyZubster Sepolia and I-ECO-01

Sat, 26 Sep 2026 14:17:31 +0000

Debugging 6.5 Million PM2 Restarts: Stabilizing MyZubster Sepolia and I-ECO-01

Today I completed a significant infrastructure debugging session across the MyZubster Sepolia E2E environment and the I-ECO-01 / UrbanLab backend.

What initially looked like a few unrelated configuration problems turned into a useful example of why production debugging needs to happen layer by layer.

The work involved Node.js, PM2, MongoDB, Google OAuth 2.0, Cloudflare, Socket.IO, and a Linux VPS.

Here is what happened, what was broken, and how it was fixed.

1. Google OAuth Was Reaching the Backend — But Authentication Still Failed

The first problem appeared in the Google social-login flow.

The authorization request itself looked correct.

Google was receiving the expected client ID and redirect URI:

https://sepolia.myzubster.com/api/auth/social/google/callback

The callback also reached the backend correctly.

Debug instrumentation confirmed:

hasCode: true

hasState: true

hasError: false

So the browser redirect, public route, OAuth state and callback routing were all working.

The failure happened one step later: during the authorization-code exchange.

The backend logged:

401 invalid_client

and Google reported that the provided client secret was invalid.

That was an important distinction.

The problem was not the redirect URI.

It was not Cloudflare.

It was not Express routing.

It was not the OAuth state.

It was not the frontend.

The OAuth client credential itself was wrong.

After correcting the configuration, the backend reached:

OAUTH-CALLBACK-ENTER

followed by:

OAUTH-CALLBACK-SUCCESS

The server-side Google OAuth flow was working.

2. The `.env` File Was Correct — The Running Process Was Not

The next issue was more subtle.

The .env file contained:

FRONTEND_URL=https://sepolia.myzubster.com

and:

GATEWAY_PUBLIC_URL=https://sepolia.myzubster.com

Yet after successful OAuth authentication, the application could still redirect toward an old Vercel deployment.

This highlighted an important operational detail:

Changing .env does not automatically mean a long-running process is using the new values.

I inspected the configuration at several levels:

.env
Node + dotenv
PM2 process configuration
actual process environment
OAuth redirect generation
backend logs

The active PM2 environment was then aligned with the Sepolia deployment:

FRONTEND_URL=https://sepolia.myzubster.com

PUBLIC_APP_URL=https://sepolia.myzubster.com

GATEWAY_PUBLIC_URL=https://sepolia.myzubster.com

After that, the active process environment was verified directly.

3. Then I Found a Process With More Than 6.5 Million Restarts

While checking the rest of the VPS, another issue appeared.

The UrbanLab / I-ECO-01 PM2 process was shown as online, but its restart counter was enormous:

6,548,384 restarts

That immediately suggested a crash/restart loop.

This is one of the reasons I do not treat:

PM2 status = online

as proof that an application is healthy.

A process can start, crash immediately, restart automatically and still repeatedly appear online when inspected at the wrong moment.

4. Finding the Root Cause

PM2 was executing:

/opt/I-ECO-01/server.js

I inspected the server entrypoint and found these dependencies:

dotenv

express

cors

helmet

mongoose

http

socket.io

winston

Then I compared those requirements against package.json.

The project only declared:

cors

dotenv

express

mongoose

The logs confirmed the immediate failure:

Error: Cannot find module 'helmet'

The server crashed at:

const helmet = require('helmet');

Because PM2 automatically restarted the process, the cycle repeated continuously.

5. Fixing the Dependency Graph

Instead of installing only the first missing package and waiting for the next crash, I inspected all external imports in the server entrypoint.

Three npm dependencies were missing:

helmet

socket.io

winston

http did not require installation because it is part of Node.js itself.

The missing dependencies were installed and persisted to package.json.

The resulting application dependencies now include:

Express
CORS
dotenv
Helmet
Mongoose
Socket.IO
Winston

Each new module was then tested independently with require() before restarting the service.

All modules loaded successfully.

6. Application Startup Was Not Enough — I Tested Stability

After restarting UrbanLab, the application successfully initialized its Escrow controller.

The server reported:

Server started on port 5002

MongoDB connected successfully.

The health endpoint:

http://127.0.0.1:5002/health

returned:

status: ok

service: I-ECO-01

mongodb: connected

But I still did not consider the problem solved.

The original issue was a restart loop, so the correct test was to verify process stability over time.

Before the stability test:

PID = 1026452

restarts = 6548384

After 15 seconds:

PID = 1026452

restarts = 6548384

Same PID.

Same restart counter.

The loop had stopped.

7. Escrow Infrastructure Is Running Again

Once the dependency problem was resolved, the I-ECO-01 Escrow controller initialized with:

Threshold: 2/3

Timeout: 24 hours

Fee: 0.5%

The service also initialized its HTTP API and WebSocket infrastructure.

MongoDB remained connected and the health endpoint continued responding successfully.

So the service moved from an uncontrolled crash loop to an operational backend.

8. Old Logs Can Be Misleading

One interesting detail appeared after the repair.

Running:

pm2 logs urbanlab

still displayed:

Cannot find module 'helmet'

At first glance, that could suggest that the problem still existed.

But those entries were historical.

The current process had already successfully loaded Helmet, initialized the application, connected to MongoDB and remained alive with the same PID.

This is why timestamps, process IDs, restart counters and health checks matter when reading operational logs.

A log file can contain a real error that is no longer a current error.

9. What I Verified

By the end of the session:

Google OAuth provider: operational

Google OAuth callback: operational

OAuth authorization-code exchange: successful

OAuth state handling: verified

Sepolia public URL configuration: corrected

PM2 runtime environment: verified

I-ECO-01 Node.js dependencies: repaired

UrbanLab restart loop: stopped

MongoDB: connected

Health endpoint: healthy

Escrow controller: initialized

2-of-3 multisig configuration: loaded

Socket.IO: initialized

PM2/systemd environment: verified

10. The Main Lessons

This debugging session reinforced several useful principles.

`online` does not mean healthy

A process manager tells you about process state.

An application health endpoint tells you about application state.

You often need both.

`.env` does not equal runtime environment

Always verify what the running process actually sees.

This is especially important when using PM2, containers, systemd or other long-running process managers.

Debug OAuth one stage at a time

OAuth involves multiple independent stages:

authorization request → provider consent → callback → state verification → authorization-code exchange → profile retrieval → application session → frontend redirect.

Logging each boundary makes failures much easier to isolate.

Fix dependency problems systematically

When one require() fails, inspect the entire entrypoint instead of repeatedly installing dependencies one crash at a time.

Verify the original failure condition

The problem was a restart loop.

Therefore, successful startup alone was not enough.

The actual success condition was:

same PID + unchanged restart counter + healthy endpoint over time.

Final Result

The MyZubster Sepolia environment is now significantly more stable.

Google OAuth can successfully complete its backend authentication flow.

The Sepolia runtime URLs are correctly propagated to the active backend process.

I-ECO-01 is connected to MongoDB.

The Escrow service is initialized.

And, most importantly, a Node.js service that had accumulated more than 6.5 million PM2 restarts is now staying alive.

Sometimes the most useful development work is not adding another feature.

It is understanding exactly why the existing system is failing — and proving that the fix actually removed the failure.

Stack: Node.js · Express · MongoDB · PM2 · Google OAuth 2.0 · Socket.IO · Cloudflare · Linux · Sepolia

NodeJS #DevOps #WebDevelopment #BackendDevelopment #Debugging #PM2 #MongoDB #OAuth2 #GoogleOAuth #SocketIO #Web3 #Sepolia #Cloudflare #Linux #MyZubster #CoderLegion

Building MetaMask Wallet Ownership Verification in MyZubster

Fri, 25 Sep 2026 04:23:07 +0000

Building MetaMask Wallet Ownership Verification in MyZubster

I’ve started implementing MetaMask / EVM wallet ownership verification inside MyZubster.

At first, this might sound like a simple “Connect Wallet” feature.

But I wanted to avoid one of the most common mistakes in Web3 UX:

treating “wallet connected” as if it automatically meant “identity verified”, “payment authorized”, or “transaction completed”.

For MyZubster, those states need to stay separate.

The architecture I’m building follows this rule:

CONNECTED
!=
VERIFIED
!=
SIGNED
!=
PAID
!=
SETTLED

That distinction is now becoming part of the product.

What we already had

MyZubster already had several EVM-related pieces in the codebase:

ethers
Ethereum Sepolia verification
Base Sepolia anchoring
Marketplace ETH support
NFT experiments
blockchain evidence flows

What was missing was the user-side ownership layer.

In other words:

How can a logged-in MyZubster user prove that they actually control a specific Ethereum wallet?

Without giving MyZubster the private key.

Without sharing a seed phrase.

Without sending ETH.

Without paying gas.

That became the first milestone.

The new flow

The flow now looks like this:

Authenticated MyZubster user

↓

Connect MetaMask

↓

eth_requestAccounts

↓

Read wallet address

↓

Read chain ID

↓

Request challenge from MyZubster

↓

Server creates short-lived nonce

↓

User signs message in MetaMask

↓

personal_sign

↓

Signature sent to backend

↓

Backend recovers signer

↓

Address matches?

↓

WALLET_VERIFIED

The important part is that verification happens server-side.

The frontend cannot simply tell MyZubster:

{
"walletVerified": true
}

and be trusted.

New API

I added four authenticated endpoints:

GET /api/wallet/me
POST /api/wallet/challenge
POST /api/wallet/verify
DELETE /api/wallet/disconnect

Their responsibility is intentionally narrow.

/challenge creates the ownership proof request.

/verify validates the signature.

/me returns the current wallet relationship.

/disconnect removes the verified association from the MyZubster account.

The challenge

The challenge includes context such as:

MyZubster account
wallet address
chain ID
domain
application URI
random nonce
issued time
expiration time
intended action

The intended action is currently:

LINK_WALLET

A simplified message looks like:

www.myzubster.com wants you to link your Ethereum account to MyZubster:

0x...

Sign this message to prove wallet control.
This is not a payment and does not spend ETH.

URI: https://www.myzubster.com
Version: 1
Chain ID: 1
Nonce: ...
Issued At: ...
Expiration Time: ...
Request ID: LINK_WALLET:user-id

That “not a payment” line is deliberate.

I don’t want a wallet signature to ever be confused with a transaction.

Why use a nonce?

Because the signature must not be reusable forever.

Each challenge is:

random
short-lived
account-bound
wallet-bound
action-bound

The backend checks:

challenge exists
↓
challenge is not expired
↓
wallet matches
↓
message matches
↓
signature is valid
↓
recovered signer matches wallet

Only then does the wallet become verified.

Signature verification

The backend uses ethers to recover the signer:

const recovered = verifyMessage(message, signature);

Then the recovered address is normalized and compared with the expected wallet.

If another wallet signs the challenge, verification fails.

What gets stored

MyZubster never stores:

private key
seed phrase
mnemonic
MetaMask password
wallet secret

Only wallet relationship metadata is persisted.

Conceptually:

evmWallet: {
walletType: 'EVM',
provider: 'metamask',
address: '0x...',
chainId: 1,
status: 'WALLET_VERIFIED',
verifiedAt: Date,
lastVerifiedAt: Date
}

Temporary challenge information is also stored server-side until verification or expiration.

Wallet state

I’m avoiding a single boolean like:

walletConnected = true

Instead, MyZubster has explicit states:

WALLET_NOT_CONNECTED
WALLET_CHALLENGE_PENDING
WALLET_VERIFIED
WALLET_DISCONNECTED

That becomes useful both for the UI and for Zorgax.

For example:

MetaMask connected

is not the same as:

wallet ownership verified

and neither means:

payment completed
Frontend

I extended the existing WalletHubPanel rather than creating a completely separate wallet system.

The user now gets a:

Connect MetaMask

action.

The frontend uses:

eth_requestAccounts

then:

eth_chainId

and finally:

personal_sign

The signature is then sent back to the backend for verification.

The UI explicitly explains that the signature:

does not transfer ETH
does not cost gas
does not authorize a payment
Preventing duplicate wallet ownership claims

Another issue is what happens if two MyZubster accounts try to link the same verified wallet.

The current behavior is:

Account A
→ verifies 0xABC

Account B
→ tries to verify 0xABC

Result:
WALLET_ALREADY_LINKED

This does not mean that MyZubster treats a wallet as a legal identity.

It simply prevents contradictory wallet-to-account relationships inside the application.

What changes in MyZubster

This creates a new trust layer.

MyZubster can already work with:

account identity
GitHub identity
community profile
Marketplace identity
Zorgax context

Now it can also say:

this account proved control
of this EVM wallet
at this time

That is a reusable building block.

The next milestone: signed Marketplace requests

This is where the wallet integration becomes much more useful.

The next step is to let a verified wallet sign a canonical Marketplace request.

Something like:

{
"schema": "myzubster.marketplace-request.v1",
"intent": "MARKETPLACE_REQUEST",
"listingId": "listing-123",
"quantity": 1,
"buyerAccountReference": "user-42",
"walletAddress": "0x...",
"nonce": "...",
"issuedAt": "...",
"expiresAt": "..."
}

The buyer signs it off-chain.

The server verifies it.

Then MyZubster can create:

MarketplaceOrder: REQUESTED

with cryptographic evidence attached.

No gas.

No blockchain transaction.

No payment.

Just signed intent.

Why not put every action on-chain?

Because not every interaction needs blockchain finality.

If every Marketplace request required a transaction, the UX would become unnecessarily heavy.

Instead, the architecture I’m aiming for is:

Marketplace request
↓
off-chain signature
↓
seller acceptance
↓
exchange lifecycle
↓
structured evidence
↓
optional blockchain anchor

Blockchain should be used where permanence adds value.

Not everywhere by default.

User wallet vs infrastructure wallet

This is also an important boundary.

MyZubster already has server-side blockchain infrastructure for evidence anchoring.

That wallet must stay completely separate from user wallets.

User MetaMask
!=
MyZubster infrastructure wallet

User-controlled wallet actions remain user-controlled.

Server-side evidence operations remain isolated.

How Zorgax can use this

Once the wallet state is explicit, Zorgax can explain it correctly.

For example:

User:
Is my wallet verified?

Zorgax:
Yes. Your MetaMask wallet ownership has been verified.

But if the user asks:

Did I pay?

Zorgax should answer:

No confirmed payment evidence exists.
Your wallet only signed the request.

That distinction is essential.

Security tests

I’ve added tests covering:

valid signer
wrong signer
expired challenge
non-payment message semantics
authenticated API wiring
MetaMask provider integration

The goal is not just to make the feature work.

The goal is to make it hard to misunderstand.

Current status

The first implementation is now open as:

PR #1385 — MetaMask wallet ownership linking

The PR adds:

wallet model
challenge service
signature verification
authenticated API
MetaMask UI
disconnect flow
security tests

This is not yet an ETH payment system.

It is the identity and ownership layer that payments or signed Marketplace actions can build on later.

Why this matters for MyZubster

The bigger change is architectural.

MyZubster is gradually moving toward a model where different kinds of evidence are handled by different layers:

MyZubster account

↓

wallet control

↓

signed intent

↓

Marketplace state

↓

payment evidence

↓

optional blockchain evidence

instead of:

connect wallet
↓
everything becomes blockchain

For me, that is the more useful direction.

MetaMask should not just be a button.

It should become a cryptographic tool that adds proof exactly where proof is useful.

Project: MyZubster
Stack: Node.js, Express, MongoDB, React, ethers, MetaMask
Current milestone: EVM wallet ownership verification
Next milestone: signed Marketplace requests without gas

Repository:

https://github.com/MyZubster-Ecosystem/myzubster

Implementation PR:

https://github.com/MyZubster-Ecosystem/myzubster/pull/1385

Building a Human-in-the-Loop GitHub Profile Automation with MyZubster and Zorgax

Thu, 24 Sep 2026 14:25:14 +0000

thorization with the user.

Result

The full GitHub profile automation loop is now working:

MyZubster
→ Zorgax
→ GitHub analysis
→ generated README
→ final preview
→ explicit human approval
→ authenticated GitHub write
→ public commit

It took more than generating Markdown.

The difficult parts were identity, OAuth, state handoff, parsing, permission boundaries and making the final action auditable.

And those are exactly the parts that make AI automation interesting to build.

Project: MyZubster
AI assistant: Zorgax
Main repository: MyZubster-Ecosystem/myzubster
Test profile: @myzubster

Ditching the Monolith: An Introduction to Multi-Agent Systems for Node.js Devs

Wed, 23 Sep 2026 08:50:33 +0000

In the world of modern software design, we are constantly searching for ways to make our systems more resilient, adaptable, and scalable. If you have spent your career building traditional monolithic applications or synchronous microservices, you are likely familiar with the headaches of centralized control. When one coordinator service fails, the entire application can grind to a halt. To solve this, forward-thinking engineers are turning to a design paradigm known as multi-agent systems.

What is a Multi-Agent System?

A multi-agent system is a network of independent, self-contained software programs (called "agents") that work together to solve complex problems. Instead of relying on a single, massive program to make every decision, these systems break tasks down among specialized digital assistants that communicate and coordinate with one another. Each agent has its own specific goal, operational rules, and local knowledge, allowing them to cooperate to achieve a larger objective that none of them could finish alone.

The Kitchen Analogy: Who Is Cooking Your Order?

To understand how this behaves, think of a busy professional restaurant kitchen. You do not have one single super-chef cooking every single dish, washing the plates, taking customer orders, and running food to tables. That would create an immediate bottleneck.

Instead, you have a head chef, a sous chef, a pastry chef, and a dishwasher. Each of these people is an "agent" with a highly specific role. They do not need to know the exact details of how the others perform their tasks; the pastry chef does not need to know how to clean a fish, and the dishwasher does not need to know how to bake a soufflé. However, they pass messages and resources back and forth (such as "order up!" or "need clean plates!") to run the entire kitchen smoothly. If the pastry chef temporarily falls behind, the rest of the kitchen can still prepare appetizers and main courses independently.

Why It Matters Daily in the Tech Industry

In the tech industry, engineers use multi-agent architectures to prevent system-wide bottlenecks and cascading failures. In standard APIs, if your main process gets overwhelmed with database operations, your user interface freezes. By distributing these responsibilities among autonomous agents, you decouple your system's critical paths.

For example, in a high-traffic e-commerce platform, instead of having one monolithic script process a payment, update inventory, and send confirmation emails sequentially, you delegate these tasks to dedicated agents. If the notification agent experiences an outage or third-party API rate limit, the payment and inventory agents keep operating without interruption. This separation allows software teams to scale individual components dynamically and deploy updates to one agent without risking the stability of the entire network.

Simulating Multi-Agent Coordination in Node.js

Below is a simple Node.js example demonstrating how multiple independent agents can coordinate tasks using an event-driven system. We will set up an OrderAgent, an InventoryAgent, and a NotificationAgent that communicate via an event emitter.

const EventEmitter = require('events');
const agentBus = new EventEmitter();

// Agent 1: Takes care of incoming customer requests
class OrderAgent {
  constructor() {
    agentBus.on('order_placed', (order) => this.processOrder(order));
  }

  processOrder(order) {
    console.log(`[OrderAgent] Received order #${order.id}. Passing to inventory...`);
    // Trigger the next autonomous agent
    agentBus.emit('check_inventory', order);
  }
}

// Agent 2: Manages physical product availability
class InventoryAgent {
  constructor() {
    agentBus.on('check_inventory', (order) => this.verifyStock(order));
  }

  verifyStock(order) {
    console.log(`[InventoryAgent] Checking stock for ${order.item}...`);
    const itemInStock = true; // Simulating a local database lookup
    
    if (itemInStock) {
      console.log(`[InventoryAgent] Stock confirmed for order #${order.id}.`);
      agentBus.emit('dispatch_notification', order);
    } else {
      console.log(`[InventoryAgent] Stock unavailable for order #${order.id}.`);
    }
  }
}

// Agent 3: Handles user communications
class NotificationAgent {
  constructor() {
    agentBus.on('dispatch_notification', (order) => this.sendReceipt(order));
  }

  sendReceipt(order) {
    console.log(`[NotificationAgent] Email sent: Order #${order.id} for a ${order.item} is confirmed!`);
  }
}

// Instantiate our autonomous agents to start listening
new OrderAgent();
new InventoryAgent();
new NotificationAgent();

// Simulate a customer purchase
console.log('--- Initiating Customer Order Flow ---');
agentBus.emit('order_placed', { id: 4509, item: 'Mechanical Keyboard' });

The Takeaway

Transitioning to a multi-agent paradigm is less about writing more code and more about changing your architectural mindset from strict, top-down instruction to distributed, cooperative behavior. By giving your software components the independence to make decisions and communicate fluidly, you build highly resilient, loosely coupled applications that can gracefully handle the messy, unpredictable demands of real-world production traffic.

Originally published on my blog. You can read the alternative breakdown here.

Building MyZubster as a Verifiable AI Development Ecosystem

Tue, 22 Sep 2026 15:42:36 +0000

uilding MyZubster as a Verifiable AI Development Ecosystem: Metaverse, GitHub Evidence, Zorgax Knowledge, and DevelopmentRequest
Today we had the opportunity to present and demonstrate part of the MyZubster ecosystem during an institutional hearing.
Instead of presenting only slides or a conceptual roadmap, we showed the system directly. We opened the MyZubster metaverse, explained a real pilot involving an actual user, discussed how GitHub activity can become technical evidence, and described the architecture we are building around Zorgax, verified knowledge, software development, wallets, and human review.
The most important outcome for us was not the meeting itself.
It was realizing that the different components we have been building are starting to form a coherent technical system.
The architecture is moving toward this loop:
Real Person

↓

Real Activity

↓

Data / Evidence

↓

MyZubster

↓

Verified Knowledge

↓

Zorgax AI

↓

DevelopmentRequest

↓

Developer

↓

Code / GitHub

↓

Tests

↓

Independent Verification

↓

New Knowledge

↓

Identity / Wallet

↓

Metaverse Interaction
The goal is not simply to build another AI assistant, another marketplace, or another metaverse.
The goal is to connect all of these components around one principle:
AI can assist the process, but evidence and verification should remain inspectable.

The Metaverse Is Only the Interface
One of the things we demonstrated was the MyZubster metaverse.
For us, the metaverse is not the entire product.
It is one possible user interface for a much larger system.
The long-term idea is that a user should eventually be able to enter the environment with a digital identity, connect a wallet, interact with people or services, and have access to a history of technical contributions and activities.
That means the visible environment may look like a metaverse, but behind it we want something closer to a verifiable contribution graph.
A person may have:
Identity
↓
Wallet
↓
GitHub Account
↓
Issues
↓
Pull Requests
↓
Commits
↓
Tests
↓
Reviews
↓
Verified Contributions
The important distinction is that a wallet alone does not prove contribution.
A GitHub account alone does not prove contribution either.
What matters is the history of actions and the evidence attached to them.
A commit may be evidence.
A passing test may be evidence.
A reviewed pull request may be evidence.
A reproducible result may be evidence.
But none of these should automatically be interpreted as proof of everything.
That distinction becomes very important once AI is involved.
A Real Pilot: Nicola
We also presented a real pilot involving Nicola, one of the people testing and contributing to the ecosystem.
The first experiment was intentionally simple.
A physical kefir handover was performed and digitally recorded through MyZubster.
The workflow represented something like:
HAND_DELIVERY

↓

Receiver confirms receipt

↓

Status updated

↓

RECORDED
There was no payment involved.
The specific transaction was also not recorded as an on-chain blockchain transaction.
That is important because we do not want to describe the experiment as something it was not.
The purpose was not to scientifically validate kefir.
The purpose was to answer a much more technical question:
Can a real-world event be represented digitally in a way that produces useful evidence?

The answer was yes.
From there, Nicola also tested other parts of MyZubster, reported problems, and worked on his own software.
That gave us a more interesting participation model.
Instead of thinking about users as static roles, we can model participation as a progression:
User
↓
Tester
↓
Contributor
↓
Developer
↓
Evidence Producer
↓
Knowledge Contributor
This is one of the ideas that could eventually connect the metaverse, GitHub, wallets, community reputation, and Zorgax.
Zorgax: Chat Is Not Knowledge
One of the most important architectural decisions we have made recently concerns Zorgax.
We do not want to treat an AI-generated answer as verified knowledge.
That sounds obvious, but many systems effectively do exactly that.
A language model generates a response.
The response is stored.
Later the system retrieves that response as if it were truth.
That is dangerous.
So our architecture starts from a different assumption:
A message is not knowledge.

A message may contain useful information.
It may become a candidate for knowledge.
But it must go through additional stages.
Our conceptual flow looks like this:
User Question

↓

AI Response

↓

Candidate Information

↓

Sources

↓

Evidence

↓

Human / System Review

↓

Verified Knowledge
Only after that should the information become part of a trusted knowledge layer.
The Zorgax Knowledge Layer
We have been building a dedicated Knowledge Layer for Zorgax.
The knowledge model is designed to preserve more than just text.
A useful knowledge object needs context.
It may include concepts such as:
Knowledge
├── contributionId
├── title
├── content
├── contentHash
├── version
├── status
├── visibility
├── provenance
├── source
├── evidence
└── lineage
The contentHash is particularly important.
It gives us a deterministic identity for the canonical content.
If the content changes, the hash changes.
This allows downstream systems to know exactly which version of knowledge they are using.
For example, a development task may reference:
{
"contributionId": "ZK-123",
"contentHash": "f30c...",
"version": 3,
"title": "Metaverse Identity Architecture"
}
Later, the knowledge may evolve to version 4.
The development task should still be able to show:
This implementation was based on version 3.

That creates much better provenance.
Preview Before Persistent Writes
Another principle we implemented is:
The AI should not silently perform consequential writes.

For operations such as creating a DevelopmentRequest, the system first produces a preview.
Conceptually:
User Intent

↓

Normalized Input

↓

Knowledge Snapshot

↓

Preview

↓

Digest

↓

Explicit Confirmation

↓

Persistent Write
A digest is generated from the preview.
Then the user must confirm using a value such as:
CONFERMA DEV a1b2c3d4
If the preview changes, the digest changes.
If the digest does not match, the operation fails.
This prevents a situation where the user approves one thing and the system writes something different.
DevelopmentRequest: Turning Knowledge Into Work
The next architecture problem was:
How do we transform verified knowledge into software development?

Our solution is a new neutral object called DevelopmentRequest.
A DevelopmentRequest is not a bounty.
It is not a GitHub issue.
It is not a payment request.
It represents the work that needs to be done.
The current lifecycle is:
DRAFT
↓
OPEN
↓
IN_PROGRESS
↓
SUBMITTED
↓
VERIFIED
or:
SUBMITTED
↓
REJECTED
The model also supports cancellation.
Conceptually, the object contains:
DevelopmentRequest {
requestId
requestHash
createdBy

title
description

knowledgeRefs[]

requirements[]
acceptanceTests[]
evidenceRequired[]

status
assigneeId

submission
review

bountyReference // optional
}
The important part is that the development request is anchored to verified knowledge.
Snapshotting Knowledge
When a DevelopmentRequest is created, Zorgax does not simply store a knowledge ID.
It stores a snapshot reference.
For example:
knowledgeRefs: [
{

contributionId: "ZK-123",
contentHash: "abc123...",
version: 2,
title: "Verified Architecture"

}
]
Before the request is actually committed, the system revalidates those knowledge references.
If the underlying content has changed between preview and commit, the operation fails.
Conceptually:
if (
expected.contentHash !== current.contentHash ||
expected.version !== current.version
) {
throw new Error(

"Knowledge changed after preview"

);
}
This gives us something extremely useful:
development provenance.
We can eventually say:
Feature X
was implemented because of
DevelopmentRequest Y
which was based on
Knowledge Z version 2
which came from
Evidence A + Evidence B
That is much stronger than simply linking a GitHub issue to a random conversation.
Submission Is Not Verification
This is another important rule.
A developer may submit something.
That does not mean the work is verified.
So the workflow separates:
SUBMITTED
from:
VERIFIED
The contributor submits evidence such as:
{
"evidenceRefs": [

"test:integration:green",
"demo:https://..."

],
"commitRefs": [

"commit:abc123"

]
}
But a separate reviewer is responsible for verification.
The reviewer cannot be the creator or the assigned developer.
Conceptually:
if (
reviewer === request.createdBy ||
reviewer === request.assigneeId
) {
throw new Error(

"Review must be independent"

);
}
This is one of the foundations for future contribution reputation.
Strengthening the Evidence Model
We are now tightening this further.
The first implementation verified that evidence existed.
But that is not enough.
Imagine this DevelopmentRequest:
Evidence required:

Unit tests pass
Security review completed
Metaverse integration demonstrated
And the developer submits only:
commit:abc123
Technically, evidence exists.
But the required evidence is not actually satisfied.
So the stronger model connects each requirement to a specific reference.
For example:
{
"evidence": [ { "requirement": "Unit tests pass", "reference": "ci:test-run:4821" }, { "requirement": "Security review completed", "reference": "review:security:913" }, { "requirement": "Metaverse integration demonstrated", "reference": "demo:metaverse:42" } ]
}
Then the verification rule becomes:
VERIFIED
only if
every evidenceRequired item
has corresponding evidence
Conceptually:
const missingEvidence =
requiredEvidence.filter(
requirement =>
!coveredRequirements.has(requirement)
);

if (missingEvidence.length) {
throw new Error(

`Missing required evidence: ${missingEvidence.join(", ")}`

);
}
This may look like a small implementation detail.
It is not.
It changes the meaning of VERIFIED.
Without this rule:
VERIFIED = someone attached something
With this rule:
VERIFIED = the declared evidence requirements were actually covered
That is a much stronger guarantee.
DevelopmentRequest Is Not a Bounty
Another architectural decision we intentionally made was separating development from economics.
A DevelopmentRequest can exist without any reward.
That means:
Knowledge
↓
DevelopmentRequest
↓
Implementation
↓
Verification
can operate independently.
Then, optionally:
DevelopmentRequest
↓
Bounty
↓
Reward
↓
Settlement
may be attached later.
This distinction matters.
A university research project may not use bounties.
An open-source volunteer may contribute without payment.
A company may use fixed contracts.
A decentralized community may use tokens.
A marketplace may use fiat.
The technical work object should not care.
That is why the model contains only:
bountyReference: null
as an optional link.
The economics remain a separate subsystem.
GitHub as Evidence, Not Truth
GitHub is also important in the architecture, but we do not want to make the mistake of treating GitHub itself as truth.
A commit proves that a commit exists.
It does not automatically prove:
the feature works
the requirement is satisfied
the code is secure
the architecture is correct
the scientific result is valid
So GitHub becomes one evidence source among others.
Future DevelopmentRequest verification could combine:
GitHub commit
+
CI test result
+
integration test
+
human review
+
external evidence
This is also why we are working toward a Contribution Graph.
The graph can connect:
Developer
↓
Issue
↓
DevelopmentRequest
↓
Commit
↓
Pull Request
↓
Test
↓
Review
↓
Verified Contribution
That provides a much richer representation of technical reputation.
Where the Metaverse Fits Back In
At first glance, the metaverse and DevelopmentRequest may seem unrelated.
But they are actually part of the same architecture.
Imagine a person entering MyZubster.
The environment could eventually know:
Identity
Wallet
GitHub History
Verified Contributions
Knowledge Contributions
Development Activity
Community Activity
The user does not need to see all of this as raw database records.
The metaverse can become an interface for it.
For example, someone could enter a virtual lab or development room and interact with a project.
Behind the scene:
Metaverse Project

↓

DevelopmentRequest

↓

Verified Knowledge

↓

GitHub Repository

↓

Contributors

↓

Evidence
The virtual environment therefore becomes a way to navigate a real technical graph.
What We Demonstrated Today
During the hearing, we were able to connect several of these concepts in one presentation.
We demonstrated the MyZubster metaverse directly.
We explained Nicola's real-world pilot.
We discussed his AI-related development work.
We explained how user activity can evolve into testing and software contribution.
We described the role of GitHub history.
We explained Zorgax.
We described the difference between AI conversation and verified knowledge.
We presented our university outreach and the idea that scientific pilots require scientific methodology.
And we explained the broader direction:
People perform real activities, software records evidence, AI assists the process, developers build solutions, and independent verification creates new trusted knowledge.

Universities and Scientific Validation
We also discussed the academic side of the project.
We have started conversations with several universities and research groups around areas including AI, food science, fermentation, circular economy, environmental measurement, LCA/LCSA, and water reuse.
We are intentionally keeping the wording precise.
These are exploratory discussions.
They are not formal partnerships unless and until they are formally established.
The reason universities matter is structural.
MyZubster can build:
software
traceability
data collection
knowledge infrastructure
AI assistance
development workflows
evidence systems
But scientific validation requires additional layers:
scientific protocol
experimental design
control groups
measurement methodology
ethical assessment
statistical analysis
domain expertise
An AI system should not pretend to replace those responsibilities.
LIFE 2027 as a Possible Pilot Framework
We are also exploring whether some of these ideas could eventually fit into a LIFE 2027 project.
Again, this is an exploratory direction.
There is no approved or funded MyZubster LIFE project at this stage.
The interesting part for us is the possibility of connecting environmental pilots with the same verifiable architecture.
For example:
Circular Water Pilot

↓

Sensors / Data

↓

Evidence

↓

University Protocol

↓

Verified Knowledge

↓

Zorgax

↓

DevelopmentRequest

↓

Software Improvement
The same architecture could theoretically apply to fermentation, environmental monitoring, circular economy, Living Labs, or other pilot domains.
That is one reason we are trying to keep the core infrastructure domain-neutral.
The Broader Architecture
If we compress the whole system, the current direction looks like this:

                ┌──────────────────┐
                │    Real World    │
                └────────┬─────────┘
                         │
                         ▼
                ┌──────────────────┐
                │ Data / Evidence  │
                └────────┬─────────┘
                         │
                         ▼
                ┌──────────────────┐
                │ MyZubster        │
                │ Knowledge Layer  │
                └────────┬─────────┘
                         │
                         ▼
                ┌──────────────────┐
                │      Zorgax      │
                │   AI Assistant   │
                └────────┬─────────┘
                         │
                         ▼
                ┌──────────────────┐
                │DevelopmentRequest│
                └────────┬─────────┘
                         │
                         ▼
                ┌──────────────────┐
                │    Developer     │
                └────────┬─────────┘
                         │
                         ▼
                ┌──────────────────┐
                │ GitHub / CI      │
                └────────┬─────────┘
                         │
                         ▼
                ┌──────────────────┐
                │ Evidence Review  │
                └────────┬─────────┘
                         │
                         ▼
                ┌──────────────────┐
                │ Verified Result  │
                └────────┬─────────┘
                         │
                         ▼
                ┌──────────────────┐
                │  New Knowledge   │
                └────────┬─────────┘
                         │
                         ▼
                ┌──────────────────┐
                │ Identity/Wallet  │
                │ Metaverse Layer  │
                └──────────────────┘

This creates a feedback loop.
The output of development can become new knowledge.
That knowledge can generate new development requests.
Those development requests create more evidence.
The evidence can improve the knowledge layer.
What We Have Implemented So Far
At the technical level, the recent work includes:

a Zorgax Knowledge Layer with provenance, content hashing, versioning, visibility and evidence metadata;
explicit distinction between conversation, candidate information and verified knowledge;
fail-closed behavior for sensitive knowledge operations;
preview + digest + explicit human confirmation before persistent DevelopmentRequest creation;
snapshotting of Knowledge references using contentHash and version;
idempotent DevelopmentRequest creation;
a lifecycle covering DRAFT, OPEN, IN_PROGRESS, SUBMITTED, VERIFIED, REJECTED and CANCELLED;
contributor assignment;
submission of commit and evidence references;
independent review rules;
strict separation between DevelopmentRequest and bounty/payment systems;
integration points for future GitHub contribution tracking;
an evidence model being strengthened so every required evidence item must be satisfied before VERIFIED.
This is still evolving.
There are important areas we still want to improve, including stronger revision models, more precise evidence objects, route-level tests, better contribution graph integration, and stronger lineage across Knowledge revisions.
The Principle Behind the Architecture
The most important idea is probably this:
We are not trying to automate trust.

We are trying to make trust more inspectable.
AI can help generate code.
AI can help search.
AI can help organize knowledge.
AI can help transform requirements into development tasks.
AI can help developers.
But an AI response should not automatically become truth.
A commit should not automatically become proof.
A wallet should not automatically become reputation.
A submission should not automatically become verification.
The architecture should preserve the difference.
That is what we are trying to encode in the system itself.
What Comes Next
The next engineering steps are straightforward.
We want to finish the stronger DevelopmentRequest evidence invariant, integrate the workflow into the operational branch, and then continue toward the GitHub and Contribution Graph bridge.
After that, we want Zorgax to be able to move through a much richer development loop:
Question
↓
Verified Knowledge
↓
Architecture
↓
DevelopmentRequest
↓
Implementation
↓
Tests
↓
Independent Verification
↓
Delivery
↓
Optional Reward
↓
New Knowledge
At the same time, we will keep exploring how this architecture can work with real users, academic research, environmental pilots, and the MyZubster metaverse.
The meeting today was useful because we did not have to describe this only as an abstract roadmap.
We were able to show pieces of it working.
There is still a lot to build.
But the architecture is becoming increasingly clear.
Real people create activity. Activity creates evidence. Evidence can become verified knowledge. Knowledge can generate development. Development creates new evidence. AI assists the entire loop without becoming the sole authority.

That is the direction we are building toward with MyZubster and Zorgax.
MyZubster
Open Source • Verified Knowledge • AI-Assisted Development • GitHub Evidence • Metaverse • Digital Identity • Community

AI #OpenSource #SoftwareArchitecture #SoftwareDevelopment #GitHub #Metaverse #Web3 #KnowledgeGraph #ArtificialIntelligence #NodeJS #Developers #MyZubster #Zorgax

Stop Killing Your Database: The Power of Connection Pooling

Tue, 22 Sep 2026 08:51:06 +0000

Connection pooling is a software development technique used to maintain a cache of database connections that can be reused for future requests. Instead of opening and closing a brand-new connection to the database every time a user wants to read or write data, your application keeps a "pool" of active connections ready to go. This drastically reduces the overhead of establishing new database handshakes over and over again.

The Pizza Delivery Analogy

Imagine you run a busy pizza shop. Every time a delivery order comes in, instead of sending a driver who is already waiting at the shop, you post a job listing online, interview a candidate, hire them, buy them a car, have them deliver the pizza, and then fire them and sell the car immediately after.

It sounds completely insane, right? Yet, that is exactly what your server does when it creates a brand-new database connection for every single HTTP request, only to destroy it a millisecond later.

Connection pooling is like hiring a dedicated team of five delivery drivers who hang out at the shop. When an order is ready, one driver grabs it, delivers it, and immediately returns to the shop to wait for the next order.

Why Connection Pooling Matters Daily

In production environments, creating a database connection is an expensive operation that involves cryptographic handshakes, network latency, and authentication checks. If your Node.js application receives 100 concurrent API requests and tries to spin up 100 separate database connections simultaneously, your MySQL database will quickly run out of memory or reject connections altogether, throwing the dreaded "Too many connections" error.

By implementing a connection pool, you cap the maximum number of active database connections, protect your database from crashing under sudden traffic spikes, and shave off up to 90% of your API response latency because the connection is already warmed up and ready to run queries.

Implementation in Node.js & MySQL

Here is how simple it is to set up a connection pool in an Express.js backend using the popular mysql2 driver:

const express = require('express');
const mysql = require('mysql2/promise');

const app = express();

// Create a connection pool instead of a single connection
const pool = mysql.createPool({
  host: 'localhost',
  user: 'root',
  database: 'store_db',
  waitForConnections: true,
  connectionLimit: 10, // Max 10 active connections kept in the pool
  queueLimit: 0
});

app.get('/products', async (req, res) => {
  try {
    // The pool automatically rents out a connection, runs the query,
    // and returns the connection back to the pool when finished.
    const [rows] = await pool.query('SELECT * FROM products LIMIT 10');
    res.json(rows);
  } catch (error) {
    console.error('Database query error:', error);
    res.status(500).send('Database error');
  }
});

app.listen(3000, () => {
  console.log('Server running on port 3000');
});

The Takeaway

Optimizing your backend performance isn't always about writing faster algorithms; often, it's about managing your external resources wisely. By treating database connections as a reusable, limited public utility rather than disposable single-use items, you build highly resilient APIs capable of weathering massive traffic spikes without sweating.

Originally published on my blog. You can read the alternative breakdown here.

Why Your Node.js App Crashes Under Traffic: The Magic of Connection Pooling

Sun, 20 Sep 2026 08:56:50 +0000

Imagine launching your shiny new web application. During development, everything runs smoothly. But the moment you launch on social media and a few hundred users hit your API at the exact same time, your server grinds to a halt and starts throwing nasty database connection timeout errors. This common disaster is almost always caused by a lack of database connection pooling.

What is Connection Pooling?

Database connection pooling is a performance optimization technique where a set of active database connections is kept open and shared among multiple requests. Instead of creating and destroying a new database connection every single time an application needs to run a query, the application borrows an existing connection from a pre-allocated pool, executes the query, and immediately returns the connection back to the pool. This prevents the costly overhead of starting new communication channels from scratch.

The Rental Car Analogy

To understand this concept, think of a rental car agency at an airport. Imagine if every time a traveler landed at the airport, the rental agency had to buy a brand-new car from the factory, register it with the DMV, customize it for the driver, and then completely crush and destroy the car at the end of the traveler's weekend trip. That would be an absurdly slow and expensive process.

Instead, the agency maintains a "pool" of 50 pre-registered, fully fueled, ready-to-drive cars parked in a lot. When a traveler arrives, they borrow a car from the lot, use it for their trip, and then return it to the lot. The car is washed and immediately made available for the next traveler. This is connection pooling: instead of building and destroying connections on the fly, you manage a recycled fleet of them.

Why It Matters Daily in the Tech Industry

Opening a database connection is an incredibly resource-intensive operation. To establish a new connection, your backend application and database server must complete a network handshake, negotiate security protocols (like TLS/SSL), authenticate user credentials, and allocate dedicated RAM and CPU threads to handle that specific session.

If you build an application where every user request opens a brand-new connection, a sudden spike in web traffic will force your Node.js server to attempt to open hundreds of database connections simultaneously. Your database server (like MySQL) has a built-in safety limit of maximum allowable concurrent connections. Once that limit is hit, the database will aggressively reject all incoming connection requests, throwing "Too many connections" errors, causing API endpoints to return 500 errors, and crashing your platform. Connection pooling solves this by capping the total number of open database connections and keeping extra queries safely queued up in memory until a connection becomes available.

Connection Pooling in Action (Node.js & MySQL)

Here is how you implement connection pooling using Express and the standard mysql2/promise library in a Node.js environment:

const express = require('express');
const mysql = require('mysql2/promise');
const app = express();

// 1. Create a global connection pool with configuration limits
const pool = mysql.createPool({
  host: 'localhost',
  user: 'root',
  password: 'secure_password',
  database: 'ecommerce_db',
  waitForConnections: true,    // Queue queries if no connections are free
  connectionLimit: 10,         // Maximum number of open connections
  queueLimit: 0                // No limit on the number of queued queries
});

app.get('/api/products', async (req, res) => {
  try {
    // 2. pool.query automatically borrows an idle connection, 
    // executes the query, and returns it to the pool.
    const [products] = await pool.query('SELECT id, name, price FROM products LIMIT 20');
    res.json(products);
  } catch (error) {
    console.error('Database failure:', error);
    res.status(500).json({ error: 'Internal Server Error' });
  }
});

app.listen(3000, () => {
  console.log('Server is running on port 3000');
});

The Takeaway

Software scalability is not just about writing clean logic or optimizing loops; it is about protecting and sharing finite hardware resources under heavy load. By implementing connection pooling, you protect your database from crashing, reduce API response latency, and build backend systems capable of handling traffic spikes gracefully without costing a fortune in extra server infrastructure.

Originally published on my blog. You can read the alternative breakdown here.

Supercharge Your Node.js App: Why You Need Database Connection Pooling

Sat, 19 Sep 2026 08:17:43 +0000

What is Connection Pooling?

Database connection pooling is a technique where a cache of database connections is kept open and shared among multiple queries, rather than opening and closing a brand-new connection for every single query. When your application needs to talk to the database, it borrows an existing connection from the pool, runs the query, and immediately returns it. This avoids the heavy performance penalty of creating and destroying connections constantly.

The Pizza Scooter Analogy

Imagine a busy local pizza shop. Every time a customer orders a pizza, if the shop had to buy a brand-new delivery scooter from the dealership, register it, insure it, deliver the pizza, and then scrap or sell the scooter immediately afterward, the business would go bankrupt in an hour.

Instead, the shop maintains a stable fleet of five delivery scooters parked in the back. When a delivery is ready, a driver grabs an available scooter from the fleet, completes the delivery, and parks it back in the lot for the next driver to use. Connection pooling is that stable fleet of scooters.

Why Connection Pooling Matters Daily

In backend development, establishing a connection to a database like MySQL is an incredibly heavy operation. It requires several steps:

A TCP network handshake.
SSL/TLS security negotiation.
User authentication and privilege verification.
Memory allocation on the database server.

If your Express.js server handles 1,000 requests per second and you open a new connection for each one, your database server's CPU will spike to 100% just managing handshakes. Eventually, the database will run out of resources and throw a dreaded "Too many connections" error, crashing your app. By using a connection pool, you bypass this startup overhead. Your queries run almost instantly because the connection is already active and authenticated.

Simple Code Example

Let's see the difference in a MERN stack application using Node.js, Express, and the mysql2 package.

The Bad Way (New Connection per Request)

const express = require('express');
const mysql = require('mysql2/promise');
const app = express();

const dbConfig = { host: 'localhost', user: 'root', database: 'store' };

app.get('/products', async (req, res) => {
  try {
    // Slow and resource-intensive: creating a connection on every request
    const connection = await mysql.createConnection(dbConfig);
    const [rows] = await connection.execute('SELECT * FROM products LIMIT 10');

    // Closing it manually
    await connection.end();
    res.json(rows);
  } catch (err) {
    res.status(500).json({ error: err.message });
  }
});

app.listen(3000);

The Good Way (Using a Connection Pool)

const express = require('express');
const mysql = require('mysql2/promise');
const app = express();

const dbConfig = { host: 'localhost', user: 'root', database: 'store' };

// Create a pool of up to 10 reusable connections
const pool = mysql.createPool({
  ...dbConfig,
  connectionLimit: 10,
  waitForConnections: true,
  queueLimit: 0
});

app.get('/products', async (req, res) => {
  try {
    // Automatically borrows an active connection and returns it when done
    const [rows] = await pool.execute('SELECT * FROM products LIMIT 10');
    res.json(rows);
  } catch (err) {
    res.status(500).json({ error: err.message });
  }
});

app.listen(3000);

Key Takeaway

Database connection pooling is not just a performance optimization; it is a fundamental survival mechanism for production databases. It turns chaotic database traffic into a predictable, queued stream, ensuring your server remains responsive and your database stays healthy under heavy load.

Originally published on my blog. You can read the alternative breakdown here.

We Turned a Real Knowledge Transfer Into Verifiable Evidence on Base Sepolia

Sat, 19 Sep 2026 02:18:27 +0000

We Turned a Real Knowledge Transfer Into Verifiable Evidence on Base Sepolia

Today we completed an experiment inside MyZubster that connects real human collaboration, GitHub history, cryptographic commitments, and blockchain verification.

The goal was not to “put knowledge on blockchain.”

The goal was more precise:

Can we take a real technical collaboration, freeze the resulting public work into a canonical record, hash it deterministically, anchor that commitment on-chain, and verify it independently later?

We used the ongoing Nicola / N4K48 MyZubster pilot as the real case.

The result is now public and reproducible.

The real source of evidence: GitHub

Nicola’s public MVP is here:

https://github.com/nicolaususnicola-lgtm/myzubster-mvp

Live pilot:

https://myzubster-mvp.onrender.com/

During the September 18 collaboration, the repository advanced across:

economic provenance;
persistent ledger storage;
revenue and asset APIs;
derived balances;
source-specific revenue breakdown;
UTC timestamps;
asset/revenue history;
MYZ ledger UI;
public documentation;
revenue simulation.

Instead of describing that work loosely, we identified an exact Git history window:

Base commit:
8da7435ddc1b9ae8b2f5c2eea5debd2df4bc5f47

Head commit:
b9ac4e8801d65b84ef882baf78fa09ce5b6b5059

That snapshot contains 30 commits.

This gives us a concrete technical object to reference.

We introduced a knowledge-transfer manifest

We created a new schema:

myzubster.knowledge-transfer.v1

with transfer ID:

KNOWLEDGE-N4K48-2026-09-18-001

The manifest records:

source identity;
recipient identity;
repository;
base commit;
head commit;
changed files;
knowledge scope;
blockchain target;
evidence boundaries.

The important design choice is that we separate:

source assertion
from
recipient attestation

At anchor time, Nicola’s recipient attestation was still:

PENDING

That is intentional.

A blockchain transaction should never be interpreted as automatic consent from another human.

Deterministic hashing

The manifest is canonicalized by:

recursively sorting object keys;
preserving array order;
serializing compact JSON;
UTF-8 encoding;
calculating SHA-256.

The resulting hash is:

39ab3a177734b5e3e254657cfe6015100644bcda2fb008cf2561d000669b9e14

The blockchain payload is:

MZ-KNOWLEDGE-V1:39ab3a177734b5e3e254657cfe6015100644bcda2fb008cf2561d000669b9e14

This is the core architecture:

large public evidence set

↓

canonical manifest

↓

SHA-256

↓

small on-chain commitment

We do not need to put source code, documentation, personal data, or full knowledge content directly on-chain.

Anchoring on Base Sepolia

For the pilot we used:

Network: Base Sepolia
Chain ID: 84532
Value: 0 ETH

The transaction was confirmed:

0xff3c108275625673ad22a886da2df7120ae81b8f0106ec833613513b03c7bc31

Block:

47000958

Confirmed at:

2026-09-18T22:23:24.000Z

Explorer:

https://sepolia.basescan.org/tx/0xff3c108275625673ad22a886da2df7120ae81b8f0106ec833613513b03c7bc31

A real integration bug: MetaMask rejected self-targeted calldata

Our first implementation tried to submit:

from: wallet
to: same wallet
value: 0
data: commitment

MetaMask rejected it with:

External transactions to internal accounts cannot include data

So we changed the design to use a fixed zero-value evidence sink:

0x000000000000000000000000000000000000dEaD

No ETH is transferred.

Only the commitment is carried in calldata.

Then smart accounts changed the transaction shape

After the transaction confirmed, the first version of our verifier reported:

MISMATCH

At first glance that looked wrong.

The receipt was successful.

The chain was correct.

The value was zero.

But the transaction destination and top-level calldata were not what our verifier expected.

The reason: MetaMask submitted the request through a delegated smart-account transaction.

The top-level transaction was type 4 and the anchor execution was wrapped inside:

redeemDelegations(...)

This forced us to improve the verifier.

Instead of only checking:

tx.to === evidenceSink
tx.data === expectedPayload

we added delegated execution decoding.

The verifier now inspects the nested execution and validates:

target
value
calldata
execution mode

Inside the delegated call we found:

Target:
0x000000000000000000000000000000000000dEaD

Value:
0

Payload:
MZ-KNOWLEDGE-V1:39ab3a177734b5e3e254657cfe6015100644bcda2fb008cf2561d000669b9e14

The result became:

MATCH

This was one of the most useful parts of the pilot.

A synthetic demo probably would not have exposed this distinction between a normal EOA transaction and a delegated smart-account execution.

Public verifier

We deployed a public verifier endpoint:

https://www.myzubster.com/api/knowledge-anchor/n4k48/0xff3c108275625673ad22a886da2df7120ae81b8f0106ec833613513b03c7bc31

It checks:

chainId ✅
receipt success ✅
top-level value = 0 ✅
delegated call recognized ✅
nested target match ✅
nested value = 0 ✅
exact calldata match ✅
execution mode match ✅

Current status:

{
"success": true,
"status": "MATCH"
}

Human-readable page:

https://www.myzubster.com/knowledge-anchor-n4k48

What exactly is proven?

This is where engineering language matters.

The blockchain anchor proves evidence about:

existence
integrity
timestamp

for the exact canonical manifest.

It does not prove:

authorship of every idea
learning
understanding
scientific validity
legal ownership
commercial value
truth of every human claim

This distinction is fundamental.

Blockchain is useful here as a tamper-evident timestamp and integrity layer, not as a replacement for human verification, peer review, contracts, or scientific validation.

The final human layer

The next step belongs to Nicola.

He now needs to add a separate attestation from his own GitHub account confirming that the record actually represents knowledge, guidance, and collaboration he received and used.

We intentionally do not modify the original anchored manifest, because that would change its hash.

Instead, Nicola can create something like:

knowledge/N4K48-ATTESTATION-KNOWLEDGE-2026-09-18.md

and commit a statement acknowledging:

KNOWLEDGE-N4K48-2026-09-18-001

and its anchored hash.

This creates a stronger chain:

source assertion

↓

public development evidence

↓

canonical hash

↓

blockchain anchor

↓

independent verifier

↓

recipient acknowledgement
Why we think this architecture matters

A lot of blockchain systems start from the wrong question:

“What can we put on-chain?”

A better question is:

“What evidence should stay off-chain, and what minimal commitment should be anchored on-chain?”

For MyZubster, this leads to a cleaner architecture:

GitHub → evidence history
SHA-256 → integrity
Base Sepolia → timestamp anchor
Human attestation → consent / acknowledgement
Research review → validation
MYZ ledger → internal accounting

Each layer does one job.

That separation avoids a lot of semantic confusion.

Knowledge is not the token

Another design principle we are keeping explicit:

knowledge ≠ blockchain asset
MYZ ≠ crypto token
GitHub commit ≠ legal ownership
blockchain timestamp ≠ scientific proof

MYZ remains an internal MyZubster accounting unit.

The knowledge anchor is a separate evidence mechanism.

The two may interact later through explicit business rules, but they should never be silently merged into one concept.

Public references

Nicola / N4K48 repository:

https://github.com/nicolaususnicola-lgtm/myzubster-mvp

Live pilot:

https://myzubster-mvp.onrender.com/

MyZubster verification page:

https://www.myzubster.com/knowledge-anchor-n4k48

Verifier endpoint:

https://www.myzubster.com/api/knowledge-anchor/n4k48/0xff3c108275625673ad22a886da2df7120ae81b8f0106ec833613513b03c7bc31

BaseScan:

https://sepolia.basescan.org/tx/0xff3c108275625673ad22a886da2df7120ae81b8f0106ec833613513b03c7bc31

MyZubster Core:

https://github.com/MyZubster-Ecosystem/myzubster

What we learned

The biggest lesson was not about hashes or blockchains.

It was about evidence boundaries.

Real systems need to distinguish:

recorded
from
verified

verified
from
validated

validated
from
owned

owned
from
understood

Once those boundaries are explicit, blockchain becomes much more useful.

Not as a magical source of truth.

But as one precise component in a larger evidence architecture.

Building Zorgax in Production: Hybrid AI Routing, Grounding, Fallbacks, and UX

Fri, 18 Sep 2026 15:05:04 +0000

Building Zorgax in Production: Hybrid AI Routing, Grounding, Fallbacks, and UX

Building an AI chatbot is easy.

Building an AI feature that survives production is a very different problem.

Over the last development cycle, we evolved Zorgax, the AI copilot inside MyZubster, from a simple conversational interface into a hybrid AI layer with model routing, cost controls, fallback logic, product grounding, runtime observability, and tighter integration with the rest of the platform.

The most interesting lesson was that the LLM API call ended up being one of the smallest parts of the system.

The goal

Zorgax is designed to act as the conversational entry point to MyZubster.

A user should be able to ask a normal question, explore the Marketplace, understand a LIFE Pilot, start a Seller journey, navigate the Metaverse, or receive guidance about the wider ecosystem.

But we did not want every request to go through the same AI model.

A simple message such as:

Ciao Zorgax

does not need the same infrastructure as:

Analyze the current LIFE workflow, evaluate KPI/MRV evidence,
and explain the deployment and repository architecture.

So we introduced a routing layer.

Hybrid model routing

The current architecture separates requests into broad standard and complex categories.

Research mode, long prompts, and signals related to coding, deployment, research, LIFE, KPI/MRV, automation, GitHub, or Vercel can move a request into the complex route.

Conceptually:

const tier =
useResearch ||
complexSignalCount >= 2 ||
text.length >= 2500

? "complex"
: "standard";

The architecture then becomes:

User
↓
Zorgax API
↓
Task classification
↓
AI Router
├── Standard → Ollama / general AI gateway
│
└── Complex / Research → OpenAI

                        ↓
                     fallback

This gives us a hybrid system instead of making the whole product dependent on one remote provider.

Making the model visible

One of the first improvements was exposing the actual route used by the backend.

The API returns metadata like:

{
"ai_provider": "openai",
"ai_model": "gpt-5.6-sol",
"ai_fallback_reason": null
}

The interface displays it directly below the response:

AI: OpenAI · gpt-5.6-sol

If something goes wrong:

AI: Ollama · qwen2.5:3b · fallback: openai_rate_limit

This looks like a small UI feature, but operationally it changes a lot.

We no longer have to ask:

Did OpenAI actually generate this answer?

The application tells us.

The first production surprise

Once OpenAI routing was enabled, the first production request did not succeed.

Zorgax returned a valid answer, but the logs showed:

aiProvider: ollama
aiModel: qwen2.5:3b
aiFallbackReason: openai_error

The OpenAI call had returned HTTP 429.

At first, that was not enough information.

A 429 can represent very different conditions: a temporary rate limit, exhausted API credits, a project spending limit, or another usage restriction.

So we improved the error handling rather than simply retrying blindly.

Precise OpenAI diagnostics

Zorgax now extracts safe diagnostic information from failed requests:

HTTP status
error code
error type
request id

without exposing credentials.

Fallback states can now be much more precise:

openai_credit_balance_exhausted
openai_rate_limit
openai_rate_limit_exceeded
openai_project_spend_limit_exceeded

That immediately paid off.

Our next production test returned:

openai_credit_balance_exhausted

The application was working correctly. The account simply had no API credit.

After adding credit, the next request succeeded with:

AI: OpenAI · gpt-5.6-sol

and production logs confirmed:

aiProvider: openai
aiModel: gpt-5.6-sol
aiFallbackReason: null

That was the first full end-to-end verification.

Retry without retrying everything forever

Transient failures still need resilience.

For rate-limit-style failures, Zorgax now has controlled retry behavior.

The primary route can retry once, respecting Retry-After when the wait is short enough. A secondary OpenAI model can then be attempted before the system falls back to the local/general route.

The flow is roughly:

gpt-5.6-sol

↓

controlled retry

↓

secondary OpenAI model

↓

Ollama fallback

Quota errors are different.

If the API explicitly says credits are exhausted, retrying repeatedly would only waste time, so the fallback happens immediately.

That distinction is important: resilience does not mean retrying every failure.

Cost controls belong in the architecture

Remote AI usage also needs application-level cost controls.

Before sending a complex request, Zorgax checks its monthly OpenAI budget and reserves enough capacity for the request.

The flow looks like this:

Classify request

↓

Check monthly spend

↓

Estimate request cost

↓

Reserve budget

↓

Call OpenAI

↓

Record actual usage

↓

Settle reservation

If the reservation cannot be made, the conversation can continue through the fallback path.

We also record the model that actually completed the request, so usage accounting still makes sense if a secondary OpenAI model is used.

Grounding became the harder problem

Once OpenAI was running correctly, another issue appeared.

The model became too conservative.

We asked Zorgax which MyZubster features were live, experimental, planned, or unverified.

The answer classified nearly everything as unverified.

That was technically cautious, but it was also wrong.

The reason was architectural: the OpenAI route and the Ollama route were not receiving exactly the same runtime product context.

The local route had product facts.

The OpenAI route mostly had the system persona and the user message.

So instead of weakening our anti-hallucination rules, we fixed the input architecture.

One canonical runtime context

Both model paths now receive the same canonical product context.

It includes facts such as:

myzubster.com is live

/marketplace is an implemented route

Seller starts from Marketplace

/social-login is the authentication path

/metaverse is an implemented route

/life-pilot is an implemented pilot path

MYZ is internal reward/accounting

It also contains an important rule:

A live route does not prove every possible feature associated with that route.

That distinction prevents two opposite failures.

The assistant should not invent features that do not exist, but it should also not deny the existence of routes that are actually implemented.

Product state as part of the prompt

Zorgax now reasons with four capability states:

LIVE / IMPLEMENTED
PILOT / EXPERIMENTAL
PROPOSED / PLANNED
UNKNOWN / UNVERIFIED

For example, /metaverse being live means the route exists.

It does not automatically mean Sellers can create virtual stores.

Likewise, /life-pilot being live does not prove that GPS field reporting, scientific validation, public map publication, or environmental measurements are all implemented.

After introducing the shared runtime context, the same test produced a much better result.

Marketplace, Seller flow, authentication, Metaverse, LIFE Pilot, open-source contribution, and MYZ were described according to their verified status, while advanced dashboards, IoT integrations, IPFS/IPNS workflows, and external settlement remained unverified.

And this time the answer was generated by:

AI: OpenAI · gpt-5.6-sol
The chat itself needed work too

Backend architecture was only half the problem.

The original Zorgax interface had a very small conversation area.

Long AI answers appeared with the scroll position close to the end of the response, forcing the user to scroll upward to begin reading.

We changed the page so that the conversation became the main content area.

New assistant replies are now positioned at the beginning of the new response rather than the end.

The scroll logic is roughly:

function scrollMessageToTop(element) {
const delta =

element.getBoundingClientRect().top -
messages.getBoundingClientRect().top;

messages.scrollTo({

top: messages.scrollTop + delta - 12,
behavior: "smooth"

});
}

We also prevent textarea focus from modifying the viewport:

input.focus({ preventScroll: true });

This made long-form AI responses dramatically easier to read.

Zorgax also needed an exit

There was one more product problem.

Once a user entered Zorgax, the assistant page felt too isolated from the rest of MyZubster.

A product copilot should help users move through the product, not trap them inside the AI interface.

So we added permanent navigation in the Zorgax header:

🏠 Home MyZubster
🛒 Marketplace

The page also contains guided actions for Seller, Marketplace, Metaverse, LIFE Pilot, University/research, Community, and other supported journeys.

This turned Zorgax from a standalone AI page into a real navigation layer for the wider product.

Observability closed the loop

Vercel runtime logging now gives us a compact record of every AI routing decision.

A successful production event can look like:

event: zorgax_message_sent
plan: pro
webResearch: true
sourceCount: 0
aiProvider: openai
aiModel: gpt-5.6-sol
aiFallbackReason: null

During debugging, this was one of the most valuable pieces of the system.

Without it, an answer that looked fine could still have silently come from the fallback model.

With it, we can trace the actual production path.

What the architecture looks like now

The current system can be summarized like this:

User
↓
Zorgax UI
↓
Access policy
↓
Optional research
↓
Task classification
↓
Canonical product context
↓
AI Router
├── OpenAI
│ ├── primary model
│ ├── controlled retry
│ └── secondary model
│
└── Ollama / general AI fallback
↓
Grounded answer
↓
Provider + model metadata
↓
Runtime logs
↓
User

That is much closer to an AI product architecture than a simple chatbot.

What we learned

The LLM call was the easy part.

The actual production system is:

model

routing
budget controls
retry policy
fallbacks
grounding
authorization
observability
UI
navigation

The more capable an AI model becomes, the more important it is for the application around it to define what is real.

The application has to know:

what exists
what is experimental
what is verified
what the AI may do
what actually happened

That is the direction we are taking with Zorgax and MyZubster.

AI can reason.

The product still owns truth, permissions, evidence, and execution.

We did not come to conquer. We came to build together.

Per CoderLegion userei come titolo:

Building Zorgax in Production: Hybrid AI Routing, Grounding, Fallbacks, and UX

Come sottotitolo:

How we evolved the MyZubster assistant from a chat interface into an observable, budget-aware, multi-model product copilot.

We Built Zorgax as a Hybrid AI Layer for MyZubster — Here’s How It Works

Fri, 18 Sep 2026 13:20:52 +0000

We Built Zorgax as a Hybrid AI Layer for MyZubster — Here’s How It Works

Over the last development cycle, we turned Zorgax into a real AI layer inside MyZubster.

The goal was not to add “another chatbot”.

We wanted Zorgax to become a product copilot capable of:

deciding which AI provider should handle a request,
using OpenAI only when it makes sense,
falling back to a local/general AI route,
controlling cloud AI costs,
showing which model actually answered,
distinguishing live product features from pilots and proposals,
and keeping persistent actions behind explicit user confirmation.

The result is now running in production.

A hybrid AI architecture

The current flow is simple in concept:

User
↓
MyZubster /zorgax
↓
POST /api/zorgax/assistant/chat
↓
Access / research policy
↓
Task classification
↓
AI Router
├── OpenAI
│ └── complex / research tasks
│
└── Ollama / general AI gateway

    └── standard tasks + fallback

↓
Zorgax system persona
↓
Grounded response
↓
UI + routing metadata

The important part is that we do not hide the routing decision.

A response can include:

{
"ai_provider": "openai",
"ai_model": "gpt-5.6-sol",
"ai_fallback_reason": null
}

and the UI shows:

AI: OpenAI · gpt-5.6-sol

If OpenAI cannot be used:

AI: Ollama · qwen2.5:3b · fallback: openai_error

That gives us transparency for both users and developers.

Not every prompt should use the expensive model

One of the first design decisions was to avoid sending every request to a cloud model.

A basic message like:

Ciao Zorgax

does not need the same infrastructure as:

Analyze this LIFE research workflow, evaluate KPI/MRV evidence,
compare deployment behavior, and explain the GitHub implementation.

So Zorgax classifies requests into broad tiers.

Signals such as:

GitHub
Vercel
deployment
debugging
code
research
LIFE
KPI
MRV
automation

can contribute to the request being classified as complex.

Research mode or very long prompts can also move a request into that tier.

The simplified logic is:

const tier =
useResearch ||
complexSignalCount >= 2 ||
text.length >= 2500

? "complex"
: "standard";

Complex requests can use OpenAI.

Standard requests can remain on the more lightweight route.

OpenAI as a runtime capability

During testing we found an interesting production bug.

The OpenAI API key was already present in Vercel, but Zorgax was still responding through the fallback model.

The runtime logs showed:

aiProvider: ollama
aiModel: qwen2.5:3b
aiFallbackReason: astra_disabled

That made the problem immediately clear: the integration existed, but the runtime gate was still disabling it.

We fixed the routing so that the OpenAI path becomes available when the server has a configured OPENAI_API_KEY.

There is still a dedicated emergency kill switch:

ZORGAX_ASTRA_KILL_SWITCH=true

so remote inference can be disabled deliberately without removing credentials.

The status API now exposes safe observability data:

{
"ai": {

"openai_configured": true,
"astra_enabled": true,
"astra_kill_switch": false,
"astra_model": "gpt-5.6-sol"

}
}

No API key is exposed.

Cost control is part of routing

We did not want “use the stronger model” to mean “ignore cost”.

Zorgax keeps an application-side monthly OpenAI budget.

Before a request is sent remotely, the system checks current usage and reserves an estimated amount.

The flow is roughly:

classify request

↓

check monthly spend

↓

estimate worst-case request cost

↓

reserve budget

↓

call OpenAI

↓

record real token usage

↓

settle reservation

If the budget cannot be reserved, Zorgax falls back instead of failing the conversation.

That makes cost control part of the architecture rather than an afterthought.

The OpenAI path

The server calls the OpenAI Responses API.

The key never reaches the browser.

Conceptually:

const response = await fetch(
"https://api.openai.com/v1/responses",
{

method: "POST",
headers: {
  Authorization: `Bearer ${process.env.OPENAI_API_KEY}`,
  "Content-Type": "application/json"
},
body: JSON.stringify({
  model,
  input,
  max_output_tokens: 4096
})

}
);

After the response returns, token usage is recorded.

If OpenAI fails, the reserved budget is released and Zorgax switches to the fallback route.

That means a remote AI failure does not necessarily make Zorgax unavailable.

The harder problem: product truth

The most important lesson was not about model selection.

It was about grounding.

Once Zorgax became capable of producing richer answers, it also became capable of confidently describing features that would make sense but were not necessarily implemented.

For a platform like MyZubster, that is dangerous.

The ecosystem contains:

live product features,
MVP components,
active pilots,
experimental concepts,
roadmap items.

Those cannot be treated as the same thing.

We therefore introduced explicit capability states:

LIVE / IMPLEMENTED

PILOT / EXPERIMENTAL

PROPOSED / PLANNED

UNKNOWN / UNVERIFIED

Zorgax is instructed not to silently promote one state into another.

For example, it must not invent:

Seller → advanced shipping dashboard

or:

LIFE Pilot → GPS environmental incident reporting

or:

Seller → virtual Metaverse showroom

unless the current product evidence actually supports those claims.

If something is hypothetical, Zorgax should say it is hypothetical.

Product-first answers

Another improvement was changing how Zorgax introduces MyZubster.

An AI assistant can easily over-explain architecture.

A newcomer asks:

What is MyZubster?

and gets blockchain terminology, repository details, validation states, evidence theory, and technical architecture.

Interesting for developers.

Bad for onboarding.

Our new system prompt uses a PRODUCT FIRST rule.

The assistant should first explain what the user can do:

Marketplace
Seller
Community
Metaverse
LIFE Pilot
GitHub
Zorgax

Only then should it go deeper into implementation details.

And when a user has a concrete goal, Zorgax should become a step-by-step copilot rather than a directory.

The preferred interaction becomes:

current step

↓

expected result

↓

next step

instead of dumping an entire workflow at once.

Web research as a separate evidence layer

Zorgax can also perform research through external providers.

The current architecture contains adapters for sources such as:

Brave Search
Tavily
Wikipedia

depending on which services are configured.

Results are normalized into a common structure:

{
"label": "W1",
"provider": "wikipedia",
"title": "...",
"url": "...",
"snippet": "..."
}

That information can be added to the AI context and displayed separately in the interface.

An important rule is that retrieved web content is treated as evidence, not as trusted instructions.

This matters for any assistant that performs web research because external pages can be wrong, outdated, or malicious.

AI is not allowed to silently write data

We also separated conversational reasoning from persistent actions.

If a user asks Zorgax to store data, Zorgax first creates a preview.

It then generates a digest and a confirmation token:

CONFERMA ab12cd34

Only after explicit confirmation can the authenticated write endpoint persist the record.

The flow is:

ANSWER
↓
UNDERSTAND
↓
COLLECT MISSING DATA
↓
VALIDATE
↓
CONFIRM
↓
SUBMIT

This is an architectural pattern we want to keep expanding.

The AI can prepare an action.

The application still controls authorization and persistence.

We also had to fix the chat itself

A good backend does not help much if the UI is uncomfortable.

The original Zorgax conversation area was too small.

Long replies also caused the chat to scroll to the bottom of the answer.

So the user would receive a long response and immediately have to scroll upward to find where it started.

We changed the layout so the conversation is the main area of the page.

We also changed the scrolling logic so a new assistant response moves to the top of that response, not the bottom.

Conceptually:

function scrollMessageToTop(element) {
const delta =

element.getBoundingClientRect().top -
messages.getBoundingClientRect().top;

messages.scrollTo({

top: messages.scrollTop + delta - 12,
behavior: "smooth"

});
}

And when focus returns to the input:

input.focus({ preventScroll: true });

This prevents browser focus behavior from undoing the intended scroll position.

Small detail.

Huge usability improvement for long AI answers.

Observability made the debugging possible

The runtime now logs routing metadata for successful Zorgax requests.

A simplified example:

event: zorgax_message_sent
plan: pro
webResearch: true
sourceCount: 0
aiProvider: openai
aiModel: gpt-5.6-sol
aiFallbackReason: null

That was how we discovered that our first production test was still using Ollama.

Without observability, the answer looked fine.

With observability, we knew the exact provider, model, fallback reason, and deployment involved.

That changed the debugging process from:

"I think OpenAI is running"

to:

"this exact production request used this exact route"
The production model today

The architecture now behaves approximately like this:

STANDARD REQUEST

↓

general/local AI route

COMPLEX / RESEARCH REQUEST

↓

OpenAI

↓

budget reservation

↓

usage tracking

↓

response

↓

fallback if required

The application also exposes enough metadata to verify the decision.

The latest production runtime reports:

OpenAI configured: yes
Astra/OpenAI route enabled: yes
kill switch: off
model: gpt-5.6-sol
The main lesson

The AI API call was the easy part.

The real system is:

AI model

router
grounding
budget control
fallbacks
authorization
observability
UX

A stronger model does not automatically produce a more reliable product.

In some ways, the opposite is true.

The more convincing the model becomes, the more important it is for the application to define:

what exists
what is experimental
what is verified
what the AI is allowed to do
what actually happened

That is the direction we are taking with Zorgax.

Not AI controlling the entire product.

AI reasoning inside a product whose state, evidence, permissions, and execution rules remain explicit.

What comes next

Zorgax is now becoming the conversational layer that can connect more parts of MyZubster.

The next areas include deeper integration with:

Marketplace
Seller workflows
profiles
research workflows
LIFE pilots
GitHub contributions
community interactions

The goal is to keep improving intelligence without losing transparency.

That balance is becoming one of the core architectural principles of MyZubster.

We did not come to conquer. We came to build together.

Per CoderLegion userei come titolo:

How We Built Zorgax: A Hybrid AI Layer with OpenAI, Ollama, Grounding and Runtime Routing

Stop Cascading Failures: Implementing the Circuit Breaker Pattern in Node.js

Fri, 18 Sep 2026 08:33:49 +0000

In modern web development, our systems rarely run in isolation. Your Node.js server likely connects to external APIs, databases, authentication providers, and third-party payment gateways. But what happens when one of those external systems slows down or crashes entirely? If your system continues to blindly send traffic to a broken dependency, you risk crashing your own backend. This is where The Circuit Breaker Pattern comes into play.

What is the Circuit Breaker Pattern?

The Circuit Breaker pattern is an architectural safety mechanism that monitors outbound requests to external dependencies and halts execution when failure rates cross a certain threshold. Instead of allowing your application to repeatedly attempt an operation that is almost guaranteed to fail, the circuit breaker trips. It quickly returns an error or a fallback response, saving system resources and preventing minor, isolated downtime from turning into a massive, system-wide outage.

A Relatable Real-Life Analogy

Imagine you are trying to call a local pizza shop to place an order, but their phone line is completely down due to a storm. If you sit there redialing their number every ten seconds, you are wasting your own time, draining your phone's battery, and growing increasingly frustrated.

A circuit breaker is like having a smart digital assistant. The first few times you try to call, the assistant lets the call go through. But after three failed attempts, the assistant intercepts your future attempts. Instead of dialing, it instantly tells you, "The shop is unreachable right now. I am not going to let you waste your time calling them. Let's try again in ten minutes, or order from another place instead." It protects your time and energy from being wasted on a guaranteed failure.

Why It Matters Daily in the Tech Industry

In a production environment, when an external API becomes sluggish or goes offline, incoming HTTP requests to your Express server will start to pile up. Each waiting request holds onto a system thread, a database connection, and a chunk of memory. Within seconds, your server runs out of available resources and crashes.

Engineers use circuit breakers to prevent these cascading failures. By failing fast, the circuit breaker immediately releases resources back to the server. Your application can then degrade gracefully—perhaps by serving cached data or displaying a polite "service temporarily offline" message—ensuring that the rest of your application remains fast and responsive for your users.

Code Example: Simulating a Circuit Breaker in Node.js

Here is a simple, plain Node.js implementation of a circuit breaker class protecting a route in an Express application:

const express = require('express');
const app = express();

class CircuitBreaker {
  constructor(requestFunction, failureThreshold = 3, cooldownPeriod = 10000) {
    this.requestFunction = requestFunction;
    this.failureThreshold = failureThreshold;
    this.cooldownPeriod = cooldownPeriod;
    this.state = 'CLOSED'; // States: CLOSED, OPEN, HALF-OPEN
    this.failureCount = 0;
    this.nextAttemptTime = Date.now();
  }

  async execute(...args) {
    if (this.state === 'OPEN') {
      if (Date.now() > this.nextAttemptTime) {
        this.state = 'HALF-OPEN';
        console.log('[Circuit Breaker] Transitioning to HALF-OPEN. Testing service...');
      } else {
        throw new Error('Circuit is OPEN. External service temporarily offline.');
      }
    }

    try {
      const result = await this.requestFunction(...args);
      this.reset();
      return result;
    } catch (error) {
      this.handleFailure();
      throw error;
    }
  }

  reset() {
    this.state = 'CLOSED';
    this.failureCount = 0;
    console.log('[Circuit Breaker] Success! Circuit reset to CLOSED.');
  }

  handleFailure() {
    this.failureCount++;
    console.warn(`[Circuit Breaker] Failure recorded. Count: ${this.failureCount}`);
    if (this.failureCount >= this.failureThreshold) {
      this.state = 'OPEN';
      this.nextAttemptTime = Date.now() + this.cooldownPeriod;
      console.error(`[Circuit Breaker] Threshold reached! Circuit is now OPEN for ${this.cooldownPeriod}ms.`);
    }
  }
}

// A mock external API function that always fails
const unstableExternalAPI = async () => {
  throw new Error('API Timeout');
};

const breaker = new CircuitBreaker(unstableExternalAPI);

app.get('/external-data', async (req, res) => {
  try {
    const data = await breaker.execute();
    res.json({ success: true, data });
  } catch (error) {
    res.status(503).json({
      success: false,
      message: error.message,
      fallbackData: { items: ['Cached Item 1', 'Cached Item 2'] } // Graceful fallback
    });
  }
});

app.listen(3000, () => console.log('Server running on port 3000'));

Key Takeaway

Designing resilient distributed systems means accepting that third-party dependencies will inevitably fail. Instead of hoping for 100% uptime from external APIs, developers must use the Circuit Breaker pattern to isolate those inevitable failures, shield their own resources, and keep their core application functional.

Originally published on my blog. You can read the alternative breakdown here.

From a Real-World Kefir Handover to an On-Chain Commitment with Node.js, SHA-256 and Base

Thu, 17 Sep 2026 09:09:36 +0000

From a Real-World Kefir Handover to an On-Chain Commitment with Node.js, SHA-256 and Base

Most blockchain demos begin with a smart contract.

This one began with a real person handing kefir culture to another person.

While building MyZubster, I wanted to test a practical question:

How can we connect a real-world event to public blockchain evidence without storing personal data on-chain — and without pretending that blockchain proves the physical event itself?

We now have a working end-to-end experiment.

The architecture

The flow is intentionally simple:

Physical handover

↓

MarketplaceHandover

↓

RECORDED

↓

Canonical payload

↓

SHA-256

↓

Blockchain commitment

↓

Base Sepolia

↓

Transaction verification

↓

Blockchain receipt stored in MyZubster

The important part is that the blockchain is not the database.

MongoDB maintains the application state and structured evidence. The public blockchain provides an independent timestamped commitment.

Creating the handover record

The physical kefir exchange progressed through explicit application states:

ACCEPTED
↓
HANDED_OVER
↓
RECEIVED
↓
RECORDED

Only after reaching RECORDED do we allow preparation of the blockchain commitment.

That distinction matters.

Creating a hash does not mean something has been anchored.

Preparing a transaction does not mean it has been confirmed.

And storing a transaction ID does not automatically mean the expected commitment actually exists on-chain.

Building the commitment

For the first implementation we defined:

myzubster.marketplace-handover.v1

The committed payload contains the identifiers, handover method, state and relevant event timestamps.

From its deterministic representation we calculate a SHA-256 digest.

For this real handover:

ba9973f08ce86d16a3611c3cccbb9cc2cc779b9ea1cb6fd87a2e5864e557b6b9

Instead of publishing the original record, the transaction contains:

MZ-HANDOVER-V1:
ba9973f08ce86d16a3611c3cccbb9cc2cc779b9ea1cb6fd87a2e5864e557b6b9

This means the public chain receives the cryptographic commitment, not the participant's personal information.

Anchoring it on Base Sepolia

We then performed the actual transaction.

Network: Base Sepolia
Chain ID: 84532
Value: 0 ETH
Block: 46933575
Gas used: 24160
Status: CONFIRMED
Confirmed: 2026-09-17T08:57:18.000Z

Transaction:

0x998a98b1733312e248f74a1387319dae30aab8123a6115c517e4ffe0ef9584bf

You don't have to trust this article.

You can inspect the transaction yourself:

https://sepolia.basescan.org/tx/0x998a98b1733312e248f74a1387319dae30aab8123a6115c517e4ffe0ef9584bf

Verifying before trusting

Before writing the blockchain receipt back into MyZubster, we verified the transaction against the chain.

The verification included the expected chain ID, transaction, block, calldata commitment and confirmation.

Only after those checks did MyZubster persist the blockchain metadata.

The resulting state is conceptually:

{
state: "RECORDED",

blockchainCommitment: {

schema: "myzubster.marketplace-handover.v1",
algorithm: "SHA-256",
hash: "ba9973f08ce86d16a3611c3cccbb9cc2cc779b9ea1cb6fd87a2e5864e557b6b9",
network: "base-sepolia",
txId: "0x998a98b1733312e248f74a1387319dae30aab8123a6115c517e4ffe0ef9584bf",
confirmedAt: "2026-09-17T08:57:18.000Z"

}
}

And the application derives:

const onchainRecorded = Boolean(
commitment.network &&
commitment.txId &&
commitment.confirmedAt
);

So we deliberately keep these concepts separate:

state = RECORDED

onchainRecorded = true

ONCHAIN_RECORDED does not replace the domain state.

It adds another evidence layer.

Moving the experiment into reusable code

A one-off transaction isn't enough.

We therefore added blockchain commitment metadata to the MarketplaceHandover model and implemented an authenticated endpoint:

POST /:handoverId/prepare-blockchain-commitment

The model can now represent:

blockchainCommitment: {
schema,
algorithm,
hash,
preparedAt,
network,
txId,
anchoredAt,
confirmedAt
}

The endpoint requires an eligible RECORDED handover and checks participant authorization before generating the commitment.

The implementation is public here:

https://github.com/MyZubster-Ecosystem/myzubster/pull/1230

Main MyZubster repository:

https://github.com/MyZubster-Ecosystem/myzubster

Testing it

After installing the repository's declared dependencies, we ran the relevant marketplace test suites:

PASS tests/marketplaceFlow.test.js
PASS tests/marketplaceEvidence.test.js

Test Suites: 2 passed, 2 total
Tests: 4 passed, 4 total
TEST_EXIT=0

The point wasn't simply to get a transaction onto a block explorer.

The blockchain representation also needed to fit the existing marketplace/evidence architecture without breaking the application flow.

What does the blockchain actually prove?

This is the most important part.

The transaction does not prove that the kefir was safe.

It does not provide microbiological validation.

It does not independently prove that the physical handover occurred exactly as described.

And it certainly doesn't turn an observation into scientific evidence.

What it gives us is much narrower:

A public, timestamped cryptographic commitment that can later be compared with the corresponding digital record.

That's useful because it allows us to ask a precise technical question:

Does the record I have now produce the same commitment that was publicly anchored at that point in time?

Why I like this architecture

Instead of asking blockchain to become the source of all truth, we can separate responsibilities:

Human action

↓

Application evidence

↓

Structured representation

↓

Cryptographic commitment

↓

Public timestamp / receipt

Each layer proves something different.

That's a much more useful model for provenance than simply putting a "verified on blockchain" badge on an application.

Next step: independent verification

The natural next step is a verifier that can reproduce the process independently:

record
→ canonicalize()
→ sha256()
→ expectedCommitment
→ fetchTransaction()
→ decodeCalldata()
→ compare()
→ MATCH | NO_MATCH

Eventually, I'd like verification to require as little trust in the MyZubster application itself as practical.

Project links

MyZubster:

https://www.myzubster.com

Main source code:

https://github.com/MyZubster-Ecosystem/myzubster

MyZubster GitHub organization:

https://github.com/MyZubster-Ecosystem

On-chain handover integration — PR #1230:

https://github.com/MyZubster-Ecosystem/myzubster/pull/1230

Kefir / fermentation knowledge repository:

https://github.com/DanielIoni-creator/Myzubster-fermentation-kefir

MyZubster Marketplace:

https://github.com/DanielIoni-creator/MyZubster-Marketplace

Base Sepolia transaction:

https://sepolia.basescan.org/tx/0x998a98b1733312e248f74a1387319dae30aab8123a6115c517e4ffe0ef9584bf

What started as one person sharing kefir culture with another became a useful test of how physical events, application state, cryptographic evidence and public blockchain infrastructure can coexist without confusing one for another.

There is still a lot to build, particularly around canonicalization, independent verification, batching and evidence schemas.

But now we have something much more valuable than a diagram:

a real event, a real application record, real code, a reproducible SHA-256 commitment and a real confirmed public transaction.

And every layer can be inspected.

MyZubster — Share → Try → Observe → Improve → Share again.

nodejs #blockchain #web3 #opensource #mongodb #sha256 #base #cryptography #softwarearchitecture #provenance

The Silent Database Killer: Understanding and Fixing the N+1 Query Problem

Thu, 17 Sep 2026 08:54:44 +0000

The N+1 query problem is a performance bottleneck that occurs when an application makes database queries inside a loop. Instead of fetching all the required data in a single, efficient query, the system executes one initial query to fetch a list of parent records, followed by "N" separate queries to fetch related child records for each item in that list. This creates an exponential number of database requests as your data grows.

The Grocery Store Analogy

Imagine you are baking a cake and need five different ingredients from your local grocery store. Instead of writing all five items on a single list and making one trip, you drive to the store, buy flour, and drive home. Then, you realize you need sugar, so you drive back to the store, buy sugar, and drive home. You repeat this entire round-trip for every single ingredient. This is highly inefficient, wasting time, fuel, and energy. In software, each trip to the database is like one of those painful, redundant car rides.

Why It Matters in the Tech Industry

On a daily basis, software engineers actively fight N+1 queries to prevent application servers and databases from crashing under heavy traffic. Each extra query introduces network latency and forces the database to parse, plan, and execute a new command. This wastes valuable connection pools, spikes CPU usage, and causes page load times to balloon. Fixing this issue is often the difference between a snappy, responsive web app and one that times out during peak business hours.

Spotting and Fixing the Problem in Node.js

Let's look at how this happens in a Node.js and Express application querying a MySQL database, and how to fix it.

// THE PROBLEM: The N+1 Query Way
app.get('/users-with-posts-bad', async (req, res) => {
  // 1. First query to get all users (The "1" in N+1)
  const [users] = await db.query('SELECT id, username FROM users');

  const usersWithPosts = [];

  // 2. Loop through each user and query their posts (The "N" queries)
  for (const user of users) {
    const [posts] = await db.query('SELECT id, title FROM posts WHERE user_id = ?', [user.id]);
    usersWithPosts.push({
      ...user,
      posts
    });
  }

  res.json(usersWithPosts);
});

// THE SOLUTION: The Efficient Way (Using a SQL JOIN)
app.get('/users-with-posts-good', async (req, res) => {
  // A single query fetches all users and their posts at once
  const query = `
    SELECT u.id AS userId, u.username, p.id AS postId, p.title
    FROM users u
    LEFT JOIN posts p ON u.id = p.user_id
  `;
  const [rows] = await db.query(query);

  // Format the flat SQL result into structured JSON
  const usersMap = {};
  for (const row of rows) {
    if (!usersMap[row.userId]) {
      usersMap[row.userId] = {
        id: row.userId,
        username: row.username,
        posts: []
      };
    }
    if (row.postId) {
      usersMap[row.userId].posts.push({
        id: row.postId,
        title: row.title
      });
    }
  }

  res.json(Object.values(usersMap));
});

The Takeaway

Database interactions are highly expensive operations. Always aim to retrieve data in bulk sets rather than looping over individual records. By shifting your mindset from iterative programming to relational database operations, you can transform a slow, unstable backend into a highly optimized, production-ready system that scales effortlessly.

Originally published on my blog. You can read the alternative breakdown here.

Stop Crashing Your Database: A Developer's Guide to Connection Pooling

Wed, 16 Sep 2026 08:51:01 +0000

Database connections are the lifeblood of any web application, but they are also surprisingly expensive to create and destroy. When your application struggles under sudden traffic spikes, the database is often the first thing to break. This is where connection pooling comes in.

Connection pooling is a software development technique that creates and maintains a cache of active database connections, keeping them ready for immediate reuse. Instead of initiating and closing a brand-new connection every time a user requests data, your application borrows an already-established connection from this pre-allocated "pool." Once the query is complete, the connection is returned to the pool for the next request to use, rather than being discarded.

The Taxi Stand Analogy

To understand this clearly, think of a busy taxi stand outside a major hotel. If every guest who wanted to travel into town had to call a car dealership, buy a brand-new car, register it with the city, drive to their destination, and then crush the vehicle at a scrap yard when they arrived, transportation would be impossibly slow, expensive, and wasteful.

Instead, the hotel relies on a taxi stand. A small fleet of vehicles sits idling in the driveway. When a guest steps outside, they instantly hop into an available taxi, take their trip, and the driver returns to the stand to wait for the next passenger. The taxi is never destroyed; it is constantly reused. In this analogy, the guests are your incoming API requests, the taxis are your database connections, and the stand is your connection pool.

Why Connection Pooling Matters Daily

In real-world software engineering, establishing a database connection is not a simple task. It requires a multi-step process: initiating a TCP handshake over the network, negotiating security protocols (TLS/SSL), authenticating the database user's credentials, allocating memory on the database server, and setting up session variables. Doing this on every single HTTP request adds massive latency to your API responses.

Furthermore, databases have strict physical limits. Every active connection consumes a slice of the database server's RAM and CPU. If your Express server experiences a sudden burst of 500 concurrent users and tries to open 500 individual connections simultaneously, your MySQL database will quickly run out of memory. This leads to the dreaded "Too many connections" error, bringing your entire application to a grinding halt. Connection pooling caps the maximum number of connections, forcing extra requests to wait in a fast queue rather than crashing your database server.

Putting It into Code

Here is how you implement connection pooling in a Node.js application using the popular mysql2 driver. Instead of using createConnection, we use createPool to manage our resources efficiently.

const mysql = require('mysql2/promise');

// Configure and create the connection pool
const pool = mysql.createPool({
  host: 'localhost',
  user: 'root',
  password: 'your_secure_password',
  database: 'production_db',
  waitForConnections: true,    // Queue requests when pool is full
  connectionLimit: 10,         // Max number of simultaneous active connections
  queueLimit: 0                // No limit on waiting requests (0 means infinite)
});

// Helper function to fetch a user profile
async function getUserProfile(userId) {
  try {
    // The pool automatically checks out a connection, runs the query, and returns it
    const [rows] = await pool.execute(
      'SELECT id, username, email FROM users WHERE id = ?',
      [userId]
    );
    return rows[0];
  } catch (error) {
    console.error('Database query error:', error);
    throw error;
  }
}

// Example execution
getUserProfile(42).then(user => console.log('User retrieved:', user));

The Takeaway

Connection pooling is a quiet champion of system architecture, transforming database communication from a slow bottleneck into a reliable, high-performance system utility. By setting up a robust pool inside your application, you prevent database crashes, lower response times, and ensure that your backend can handle high-concurrency traffic without breaking a sweat.

Originally published on my blog. You can read the alternative breakdown here.

Building MyZubster: A Circular Marketplace Connected to a Social Metaverse

Mon, 14 Sep 2026 12:43:38 +0000

Building MyZubster: A Circular Marketplace Connected to a Social Metaverse
MyZubster is an open-source ecosystem that connects a circular marketplace, local communities, educational support, underground culture, visual storytelling and an experimental social metaverse.
The project is based on a cyclical model:
Discover → Connect → Exchange → Use → Learn → Repair → Reuse → Recover → Contribute
Instead of building another isolated marketplace or virtual world, MyZubster connects these activities through shared navigation, account-linked characters and measurable user journeys.
The MyZubster Marketplace
The Marketplace is the operational discovery layer of the ecosystem. It organizes products, services, skills and community activities into accessible categories and subcategories.
The current demo explores areas including:

local food and independent producers;
kefir and fermented products;
underground music and alternative culture;
sound-system equipment;
event and creative services;
university and educational assistance;
programming and digital services;
repair and reuse;
recycling and material recovery;
sustainable everyday products.
Explore the Marketplace:
🛒 https://www.myzubster.com/marketplace?utm_source=coderlegion&utm_medium=article&utm_campaign=myzubster_circular_funnel&utm_content=marketplace_overview
Why Kefir Is Part of the Ecosystem
Kefir is a useful example of how a small local product can become part of a larger circular process.
A visitor can discover a producer, learn about the production process, contact the seller and understand what happens to the packaging after consumption.
Local production
↓
Marketplace discovery
↓
Informed purchase
↓
Consumption
↓
Container reuse
↓
Recycling or material recovery
MyZubster does not automatically certify food safety, production standards or health benefits. Those claims require evidence and compliance from the responsible producer.
The platform’s role is to make sellers, information and provenance easier to discover and inspect.
Open the local-products Marketplace funnel:
🌱 https://www.myzubster.com/marketplace?category=food&utm_source=coderlegion&utm_medium=article&utm_campaign=myzubster_circular_funnel&utm_content=kefir_local_producers
Supporting Underground and Independent Culture
Underground communities often depend on informal networks connecting musicians, artists, technicians, venues and equipment owners.
MyZubster can provide optional infrastructure for discovering:
DJs, musicians and independent performers;
sound-system equipment;
audio repair specialists;
rehearsal and event services;
visual artists and graphic designers;
alternative clothing and records;
independent venues and cultural initiatives;
reusable creative materials.
A sound system can move through a complete circular lifecycle:
Equipment discovery
↓
Purchase or rental
↓
Event use
↓
Maintenance
↓
Repair
↓
Resale and reuse
↓
Component recovery
The objective is not to control underground culture. It is to give independent communities tools they can use on their own terms.
Explore the underground and sound-system funnel:
🔊 https://www.myzubster.com/marketplace?category=sound-system&utm_source=coderlegion&utm_medium=article&utm_campaign=myzubster_circular_funnel&utm_content=underground_sound_system
University and Educational Assistance
Education is another important part of the MyZubster cycle.
Students can discover legitimate support such as:
tutoring;
programming guidance;
language practice;
research-method explanations;
document and presentation formatting;
portfolio support;
study groups;
guidance toward internships;
open-source contribution opportunities.
Educational support should help students learn. It must not be used to impersonate students, complete prohibited examinations or submit deceptive academic work.
The educational cycle is:
Ask for support
↓
Learn
↓
Practise
↓
Build a project
↓
Document the work
↓
Contribute
↓
Help another learner
Explore the educational Marketplace funnel:
🎓 https://www.myzubster.com/marketplace?category=education&utm_source=coderlegion&utm_medium=article&utm_campaign=myzubster_circular_funnel&utm_content=university_help
Entering the MyZubster Metaverse
The MyZubster Metaverse is the visual and social navigation layer of the ecosystem.
Its current environment, Neon Plaza, allows visitors to:
enter as guests or authenticated users;
use account-linked characters;
move around a shared environment;
see other active visitors;
communicate through chat and emotes;
discover nearby participants;
visit ecosystem portals;
complete an exploration mission;
preserve visited-zone progress;
discover experimental virtual rooms.
Enter Neon Plaza:
🪐 https://www.myzubster.com/metaverse?utm_source=coderlegion&utm_medium=article&utm_campaign=myzubster_circular_funnel&utm_content=metaverse_entry
Account-Linked Characters
A visitor can explore without creating a verified identity claim.
Authenticated users can use persistent account-linked characters. The server determines which character belongs to the authenticated account instead of trusting a name supplied by the browser.
The current system includes:
canonical server-side character profiles;
account-linked character ownership;
authenticated profile retrieval;
server-backed mission progress;
visited-zone persistence;
live player presence;
session recovery;
expired-JWT handling;
protected character-progress updates.
A display name is not considered legal identity evidence. The MYZ VERIFIED label indicates that the character is linked to a MyZubster account, not that a person’s legal identity, professional skills or qualifications have been certified.
Mission Progress
Visitors receive an initial mission: move through Neon Plaza and reach one of its portals.
For guests, progress remains local to the browser. For authenticated characters, visited zones can be stored on the account-linked server profile.
The server accepts only known landmark identifiers. It derives character ownership from the verified authentication token and does not trust a character name submitted by the client.
This creates a safer progression model:
Authenticated account
↓
Canonical character
↓
Visit a valid landmark
↓
Server validates the landmark
↓
Progress is attached to the account-owned character
Try the mission funnel:
🧭 https://www.myzubster.com/metaverse?utm_source=coderlegion&utm_medium=article&utm_campaign=myzubster_circular_funnel&utm_content=first_mission
Experimental Virtual Rooms
MyZubster is beginning to connect Neon Plaza with persistent virtual rooms.
The discovery system applies access rules on the server:
guests can discover public rooms;
authenticated users can also discover account-only rooms;
private rooms remain hidden;
drafts and archived rooms are excluded;
only published, scheduled or live spaces are discoverable.
These rooms are still experimental. A dedicated room experience will be introduced only after navigation, access control, moderation and lifecycle handling are ready.
Follow the virtual-room funnel:
🏛️ https://www.myzubster.com/metaverse?utm_source=coderlegion&utm_medium=article&utm_campaign=myzubster_circular_funnel&utm_content=virtual_rooms
MyZubster Visual
MyZubster Visual provides the storytelling layer.
Illustrations, interactive popups and visual narratives can explain connections between:
physical products;
circular lifecycles;
environmental projects;
characters;
community contributions;
Marketplace categories;
Metaverse destinations.
A popup can introduce a story, provide additional context and direct a visitor toward the next relevant part of the ecosystem.
The intended funnel is:
Visual story
↓
Interactive popup
↓
Context and explanation
↓
Marketplace category or project
↓
Metaverse exploration
↓
Community participation
Explore the visual-story funnel:
🎨 https://www.myzubster.com/fumetto?utm_source=coderlegion&utm_medium=article&utm_campaign=myzubster_circular_funnel&utm_content=visual_popups
How the Complete Circular Funnel Works
Consider a visitor interested in locally produced kefir:
1. They discover MyZubster through this article.
2. They open the Marketplace.
3. They explore local food and producer listings.
4. They learn about the product and its provenance.
5. They contact or purchase from an authorized seller.
6. They reuse or correctly recycle the packaging.
7. They enter the Metaverse.
8. They visit the Marketplace or LIFE Projects portal.
9. They document or share a useful contribution.
10. That contribution helps another visitor discover the circular pathway.
  The same model can apply to underground sound equipment:
11. Discover equipment.
12. Use it for an event.
13. Maintain and repair it.
14. Resell or reuse it.
15. Recover components at the end of its usable life.
16. Share repair knowledge with the community.
  It can also apply to educational support:
17. Request tutoring.
18. Learn a skill.
19. Build a project.
20. Publish the code.
21. Contribute to MyZubster.
22. Help another learner.
  Technical Architecture
  The current implementation combines:
React for the user interface;
Node.js and Express for backend services;
MongoDB for persistent characters, progress and room records;
authenticated API routes;
server-authoritative access rules;
short-lived presence and chat records;
GitHub for open-source development;
Vercel for deployment and web analytics.
The architecture maintains an important boundary:
The client presents the experience, but the server controls identity, ownership, persistence and access decisions.
Measuring the Funnel
Each link in this article contains UTM parameters designed to distinguish traffic from Coder Legion.
The campaign uses:
utm_source=coderlegion
utm_medium=article
utm_campaign=myzubster_circular_funnel
The utm_content value identifies the selected journey:
marketplace_overview
kefir_local_producers
underground_sound_system
university_help
metaverse_entry
first_mission
virtual_rooms
visual_popups
github_repository
This makes it possible to compare which concepts attract attention and which destinations visitors choose next.
Build With Us
MyZubster is being developed openly. Developers can inspect the implementation, review the architecture, propose improvements and contribute code.
💻 https://github.com/MyZubster-Ecosystem/myzubster?utm_source=coderlegion&utm_medium=article&utm_campaign=myzubster_circular_funnel&utm_content=github_repository
The project remains an evolving prototype. Experimental functionality, verified evidence and operational services are kept clearly distinguishable.
The objective is not simply to create another marketplace or virtual environment. It is to build a connected system in which software helps communities discover opportunities, learn, exchange value, extend product lifecycles and create new contributions.
Explore the Ecosystem
🛒 Marketplace Demo
https://www.myzubster.com/marketplace?utm_source=coderlegion&utm_medium=article&utm_campaign=myzubster_circular_funnel&utm_content=marketplace_overview
🪐 MyZubster Metaverse
https://www.myzubster.com/metaverse?utm_source=coderlegion&utm_medium=article&utm_campaign=myzubster_circular_funnel&utm_content=metaverse_entry
🎨 MyZubster Visual
https://www.myzubster.com/fumetto?utm_source=coderlegion&utm_medium=article&utm_campaign=myzubster_circular_funnel&utm_content=visual_popups
💻 Open-source Repository
https://github.com/MyZubster-Ecosystem/myzubster?utm_source=coderlegion&utm_medium=article&utm_campaign=myzubster_circular_funnel&utm_content=github_repository
MyZubster #OpenSource #WebDevelopment #Metaverse #Marketplace #CircularEconomy #ReactJS #NodeJS #MongoDB #BuildInPublic

Building MyZubster Marketplace Demo v2: A Marketplace You Can Explore Without Fake Transactions

Sun, 13 Sep 2026 16:25:17 +0000

Building MyZubster Marketplace Demo v2: A Marketplace You Can Explore Without Fake Transactions

I’ve been working on the next evolution of MyZubster Marketplace, and this update is less about adding another button and more about making the Marketplace something people can actually explore and understand safely.

The result is Marketplace Demo v2.

From 6/12 to 12/12 categories

The Marketplace already supported 12 categories, but the integrated demo experience originally covered only six.

We completed the missing categories, bringing demo coverage to 12/12:

🌱 Seeds & Plants
🥕 Produce
👕 Clothing
🎒 Accessories
🎪 Event Equipment
🛠️ Tools
💼 Services
🤝 Volunteering
🐶 Pet Adoption
🔎 Pet Lost & Found
🐾 Pet Services

We also added automated regression coverage: if we introduce a new Marketplace category in the future without providing demo content for it, the tests can catch it.

But 12 cards aren't a marketplace experience.

So we went further.

Marketplace Demo v2 is interactive

Demo listings can now be opened and explored in more detail.

Instead of a static showcase, users can understand things like the listing's description, location, availability, category, exchange/payment mode and category-specific information.

We also introduced filtering across the demo content:

Category · Location · Search · FREE · BARTER · EUR · MYZ

This makes the demo behave much more like the real discovery experience we're building.

A request flow — without creating fake business

One of the most important additions is the simulated request flow.

A visitor can open a demo listing and try the journey from listing discovery to reviewing and confirming a request.

But there is a strict boundary:

Demo requests never become real transactions.

No real order is created.

No payment is processed.

No real seller receives a message.

No personal request data needs to be persisted.

This gives us a useful product demo without filling the production database with fake economic activity.

A pet adoption isn't the same thing as selling a product

This became particularly important when expanding the categories.

Marketplaces often make the mistake of forcing everything into the same product → price → checkout model.

That doesn't work for MyZubster.

A volunteering opportunity isn't a product.

A lost pet alert isn't a product.

Pet adoption shouldn't behave like an e-commerce checkout.

So Demo v2 introduces specialized contextual information for volunteering, pet adoption, lost & found and pet services.

The category isn't just a label anymore — it can influence how an opportunity is presented.

One Marketplace demo, not two

We previously had a separate /marketplace-demos showcase alongside the actual React Marketplace.

That was useful early in development, but maintaining two different demo experiences creates duplication.

We've now consolidated them.

The old demo route points toward the main Marketplace demo experience.

One Marketplace. One demo model. One source of truth.

Payments are getting the same treatment

At the same time, we've been cleaning up MyZubster's payment architecture.

One interesting production case involved a €9.90/month Stripe subscription.

Zorgax showed the user as Free.

Initially, that looked like a billing bug.

It wasn't.

The subscription was for Marketplace Seller, not Zorgax Pro.

Same price.
Same account.
Different product.

Instead of artificially granting Zorgax Pro, we separated the two states properly:

Zorgax: Free

Marketplace: Seller active

That's an architectural principle we're carrying into the Marketplace:

A payment tells us that money moved. It doesn't tell us what the user owns.

The product, purchase and resulting entitlement need to remain explicit.

Where MyZubster Marketplace is now

We now have 12/12 demo category coverage, interactive listing details, search and filtering, multiple exchange modes, specialized community categories, a simulated request journey and automated regression coverage.

And most importantly:

Real transactions stay real.

Demo transactions stay simulated.

Seller subscriptions stay separate from Zorgax subscriptions.

There's still plenty to build, but the Marketplace is starting to behave less like a collection of screens and more like a coherent system.

That's the part of development I enjoy most: not just making features appear, but figuring out how they should interact without creating problems later.

Building MyZubster, one layer at a time.