Coder Legion - Recent posts tagged rust

Let AI Help Diagnose Incidents - Without Giving It the Keys

Wed, 28 Oct 2026 10:22:53 +0000

Practical lessons from building Esapay, a prototype exploring safer recovery for payment infrastructure.

Payment systems can fail in ways that look deceptively similar. Slow checkouts might point to an overloaded service, a growing backlog, or trouble somewhere further downstream. Responding to every spike in the same way can make an incident worse: a change that helps one bottleneck may increase pressure on another.

I started building Esapay to explore a question that matters beyond payments: how can AI help engineers make sense of an incident while keeping important system changes controlled, reviewable, and reversible?

Esapay is an experimental project, not a production payment processor. Its recovery demonstrations use simulated workloads, and its results should not be read as guarantees about live payment traffic.

The Principle: Recommendations and Authority Are Different Things

AI can be useful when an incident has several possible explanations, when signals are noisy, or when an engineer needs a concise summary of what may be happening. But a model's recommendation should not automatically become an infrastructure change.

A safer design keeps three responsibilities conceptually separate:

Interpretation: Use telemetry and contextual reasoning to form a hypothesis.
Decision control: Check a proposed response against current conditions, operational limits, and applicable policy.
Execution and verification: Make only an authorized change, then inspect whether the outcome matched expectations.

This separation makes it possible to benefit from AI without requiring the system to trust every output it generates. It also makes the overall system easier to reason about when a model is unavailable or returns an answer that cannot be used safely.

1. Keep Automated Actions Deliberately Narrow

A free-form instruction is difficult to validate. A bounded operation is easier to inspect before it is allowed to run.

One lesson from Esapay is to define the kinds of changes a system is permitted to consider, rather than allowing a reasoning component to invent arbitrary operational commands. The enforcement layer should independently check a proposed change and remain responsible for whether it is accepted.

This does not make automation safe by itself. The permitted operations still need clear limits, appropriate authorization, and tests for invalid inputs and unexpected conditions.

A useful design question is: Could the system explain why this change was allowed, and could a separate part of the application reject it? If the answer is no, the authority boundary may be too vague.

2. Plan for Missing, Slow, or Malformed AI Responses

A system that depends on an AI service should define what happens when that service is unavailable. Network problems, timeouts, model errors, and unexpected output are ordinary failure modes—not exceptional situations that can be ignored until deployment.

For critical workflows, the fallback should be intentionally limited. Depending on the system, that might mean using established deterministic checks, asking for human review, or declining to take an automated action. The correct choice depends on the risk; continuing automatically is not always the safest option.

The broader lesson is to design the failure path at the same time as the success path. If a workflow only behaves safely when the model responds perfectly, its reliability depends on an assumption that will eventually be tested.

3. Treat Decisions as Snapshots of a Changing World

A recommendation can be reasonable when it is created and unsafe a moment later. Other requests may have changed the resource, its limits, or the surrounding conditions while the system was preparing a response.

Before applying a change, a robust system should establish that the information behind the decision is still current. Where concurrent changes are possible, that check should be coordinated with the mutation itself; checking a version and changing state as two unrelated operations can leave a race between them.

This is a general backend engineering problem, not something unique to AI. AI can make it more visible because reasoning may take long enough for the world to change before the recommendation is ready.

Tests should deliberately create competing updates and verify that outdated decisions are rejected or reconsidered. Happy-path tests alone will not reveal these timing problems.

4. Verify Effects Instead of Assuming Success

A successful function call or accepted command does not necessarily mean an incident has been resolved. A change may be applied but fail to improve the symptom, improve one metric while worsening another, or fail to reach the external system it was intended to affect.

That is why recovery should include an outcome check. Before acting, define what improvement is expected. Afterward, compare the observed state with that expectation and retain enough context to understand what happened.

Reversibility also deserves care. A rollback that restores an application's local view does not automatically restore an external service, cluster, or payment provider. Systems that cross those boundaries need a way to verify the external result and handle partial failure.

The aim is not to claim that every action can always be undone. It is to make uncertainty explicit and avoid declaring success before there is evidence for it.

5. Make Decisions Explainable After the Incident

When an automated response succeeds, engineers need to know what changed and why. When it fails, they need to reconstruct the path to that outcome.

Useful operational records can include:

The observed symptoms.
The rationale for a recommendation.
The decision outcome.
The change that was attempted.
The result that followed.

Such records should avoid storing credentials or unnecessary personal and payment data. Access controls and retention rules matter just as much as the record format.

Integrity checks can help reveal whether recorded events have been altered, but they are not a substitute for durable storage, restricted access, backups, or a clear operational process. An audit trail is only as useful as its coverage and the trustworthiness of the environment that stores it.

These details are easy to underestimate when building a prototype. They become central when someone must explain an incident to a teammate, an operator, or an auditor later.

6. Evaluate the Trade-offs Honestly

AI-assisted recovery introduces costs as well as potential benefits. Additional reasoning can improve a recommendation in some cases, but it can also add latency, complexity, operational dependencies, and new failure modes. A simpler baseline may be preferable for predictable incidents.

For Esapay, I use a local simulation and benchmark harness to explore these trade-offs. Such experiments are valuable for comparing approaches under controlled conditions, reproducing scenarios, and identifying regressions. They do not establish how a system will behave under every real-world workload.

A useful evaluation should state what was simulated, what was measured, what baseline was used, and what remains untested. It should also consider more than one headline number:

Recovery quality.
Time spent in a degraded condition.
Unintended changes.
Resource overhead.
Behavior when the reasoning service fails.

The goal of a benchmark is not to make a project look fast. It is to help answer a specific engineering question and show the limits of the evidence.

Questions I Ask When Designing AI-Assisted Automation

Before trusting an automated recovery workflow, I want clear answers to questions like these:

What is the smallest set of actions the system genuinely needs?
Which independent checks can veto a recommendation?
What happens if the model is unavailable, uncertain, or returns unusable output?
How do we know the state has not changed since the recommendation was prepared?
What evidence tells us the action helped—and what happens if it did not?
Can we reconstruct the decision without exposing secrets or unnecessary user data?
Which results come from controlled simulation, and which have been validated in a real environment?

These questions are useful even when the final design does not use multiple agents or a language model. They force the system's authority, failure behavior, and evidence to be discussed explicitly.

Closing Thought

The most interesting part of applying AI to infrastructure is not giving a model more power. It is deciding where reasoning adds value, where deterministic controls are necessary, and how to tell whether the final outcome was actually good.

Esapay remains a prototype, and there is more work to do before its ideas could support production-critical workloads. Building it has reinforced one principle for me:

AI can help inform an operational decision, but the system should retain clear control over authorization, execution, and verification.

For engineers exploring similar ideas, I would start with narrow permissions, explicit failure paths, deliberate concurrency tests, careful outcome measurement, and honest reporting of limitations. Those foundations matter more than making an architecture sound autonomous.

Explore Esapay

Interested in exploring the project or discussing potential contributions? Visit the repository:

GitHub repository: https://github.com/sujithputta02/Esapay

If you have ideas for improving reliable AI-assisted systems, I'd be interested in hearing your thoughts and learning from the developer community.

The Future of Offline Payments: The DeFi Approach

Sun, 18 Oct 2026 22:34:57 +0000

Over the last two decades, digital payments have become an integral part of our everyday lives. Today, we can send money across borders, purchase goods using mobile wallets, and transfer digital assets without necessarily relying on traditional financial intermediaries. Despite these advances, one fundamental challenge remains: accessibility.

For people living in rural communities, traders operating in areas with unreliable mobile networks, and businesses experiencing network congestion, completing a digital payment can become unexpectedly difficult. In these situations, the problem is not necessarily access to a bank account or a digital wallet. Rather, it is the inability to connect to the infrastructure required to authorize, verify, or complete a transaction.

This challenge has not gone unnoticed. It has attracted interest across both traditional finance (TradFi) and decentralized finance (DeFi). Research into offline payment protocols and blockchain-based payment systems is exploring new ways to make digital payments more resilient when connectivity is unavailable.

Within the Web3 ecosystem, developers and researchers are exploring how digital transactions can be initiated, authorized, and exchanged without requiring a constant internet connection.

In this article, we'll explore emerging approaches to offline payments in DeFi. I'll also introduce PAT (Primitive Airborne Transaction), my contribution to this evolving field, which I had intended to submit to Colosseum's World Fair hackathon.

Innovations and Approaches to Offline Payments in Web3

1. Solana: Fast Payments, QR Codes, and Offline Signing

Solana is particularly interesting in this discussion because it combines a high-performance blockchain with a payment ecosystem designed for consumer applications.

From exploring its documentation, I learned that Solana supports offline and multi-party transaction signing, including the preparation of transaction messages, independent signatures, transaction validity, and eventual submission. For instance, a transaction can be prepared and signed independently of the device that eventually broadcasts it. This was one of the reasons I chose to build PAT on Solana.

However, Solana Pay's standard QR-based payment flow is not itself a complete offline payment solution. QR codes solve a communication and user-interface problem. They do not eliminate the need to validate transaction state or prevent a payer from attempting to spend the same funds more than once.

Bitcoin: Offline Transaction Authorization

In June 2026, researchers presented Offline Bitcoin: An Air-Gapped Protocol for Trustless Bitcoin Transactions at the IEEE International Conference on Blockchain and Cryptocurrency.

The research describes a protocol and application for making Bitcoin payments using devices without internet access. It leverages Bitcoin's Taproot capabilities and QR codes to exchange payment information between offline and online devices without requiring the offline device to connect to the internet. The intended receiver can claim the funds using a secret, while the sender can recover them after a configurable timeout if the receiver does not act.

The fundamental difficulty with offline blockchain payments is that disconnected devices cannot query the latest blockchain state. This creates challenges involving transaction validity, forgery, and double-spending.

This brings us to an important question:

How can a system preserve the accessibility of offline payments without compromising the integrity of digital money?

It is within this context that I began building PAT.

PAT (Primitive Airborne Transaction)

Although I did not get to submit PAT to Colosseum's World Fair hackathon, I built Primitive Airborne Transaction, a Solana-based project exploring how digital payment workflows can operate during periods of intermittent connectivity.

The underlying idea is simple:

Reserve while connected. Spend while disconnected. Settle when reconnected.

Step 1: Connect your wallet.

The sender connects a compatible Solana wallet to PAT while online.

Step 2: Reserve funds for offline spending.

While connected, the sender reserves funds for a future payment. This establishes the transaction's context before the sender goes offline.

Step 3: Authorize the payment offline.

When the sender wants to pay, PAT generates a payment authorization tied to the intended receiver's wallet address. This authorization is represented by a cryptographic signature, which can be shared through a QR code or copied for transfer.

The authorization is valid for a limited period. This helps define how long the payment request can be used.

Step 4: Submit the payment for settlement.

The merchant receives the authorization and, when connected to the internet, scans the QR code or enters the provided signature and initiates settlement on Solana.

The blockchain then checks the relevant transaction conditions before the payment can be completed.

This means the payment authorization can be exchanged without requiring the sender to have an active internet connection at that moment, while the merchant can submit the transaction when connectivity is available.

PAT is an amalgamation of multiple ideas:

Solana-based reservation state
Cryptographic authorization
Local persistence
QR-based exchange
Deferred settlement

The goal is to address the gap that emerges when the normal online payment workflow is interrupted. However, the current prototype still has shortcomings, particularly in user experience and operational reliability, that need to be addressed.

You can explore the application here:

PAT Web App

GitHub: Hilda-Enyioko/pat

While we share ideas and contribute to financial accessibility, a key question is ask is:

Can the concept behind PAT be replicated in TradFi systems?

Pyintel Mem

Sat, 17 Oct 2026 21:25:54 +0000

The Problem: The Agent Memory Bloatware Crisis

If you have built an autonomous AI agent recently (using frameworks like LangChain, Mem0, or Letta), you have probably noticed a glaring absurdity:

To give an agent long-term memory across sessions, you are often forced to run three separate database engines:

A vector database (Qdrant, Pinecone, Chroma) for dense semantic search.
A keyword/full-text engine (Elasticsearch, BM25) to catch exact tokens, port numbers, or UUIDs.
A graph database (Neo4j, FalkorDB) to track relationships and dependencies between entities.

The result? Docker container fatigue, runaway cloud bills, massive Python virtual environments, and query latencies exceeding 200 ms simply to recall what the user said 10 minutes ago.

We asked a straightforward question: Why can't agent memory live in a single, compact binary file that boots in 1 ms, runs completely offline, and queries in under 100 microseconds?

What SQLite did for relational data and Git did for version control, we set out to build for agent memory.

Meet Pyintel Mem (.pmem).

The Architecture: Single-File Binary Wire Layout (`0x50 0x4D`)

pyintel-mem compiles to a 536.5 KB standalone binary with 0 external runtime crate dependencies (no tokio, no serde, no rocksdb, no sqlite, no ndarray).

All data structures—vectors, HNSW graph layers, BM25 hash postings, and CSR temporal edges—live inside a single binary .pmem file with a 64-byte wire header:

Zero-Copy OS Memory Mapping (`mmap`)

When a process executes pmem recall "query", the file is not parsed or copied into heap memory. Instead, it utilizes native operating system memory mapping (MapViewOfFile on Windows, mmap on POSIX). Pages are faulted in directly by CPU cache lines on-demand.

The 4-Signal Hybrid Retrieval Engine

Instead of running separate queries across three distinct databases and reranking them with a slow LLM pass, pyintel-mem evaluates a unified 4-Signal Hybrid Search in a single memory-mapped pass:

\[\text{Score} = \Big(\alpha \cdot S_{\text{vector}} + \beta \cdot S_{\text{bm25}} + \gamma \cdot S_{\text{graph}}\Big) \cdot e^{-\lambda \cdot \Delta t} \cdot \text{Importance}\]

1. Vector Quantization: SQ8 + BQ1 Hardware POPCNT

Symmetric int8 (SQ8): Compresses 32-bit floats 4x with 16-way loop-unrolled SIMD dot-product accumulation.
1-bit Binary Signatures (BQ1): Compresses vectors 32x into u64 bitmasks. A single-cycle hardware POPCNT instruction calculates Hamming distance to instantly discard 90% of non-matching candidates before running dot products.

2. Zero-Allocation Hash-Keyed Okapi BM25+

Tokenizes text into u64 FNV-1a hash postings with CamelCase and digit boundary splitting (AutoVLA -> auto, vla).
Built-in Damerau-Levenshtein <= 1 typo recovery: queries like "what is the autoVLA ADE scre?" automatically recover "score".

3. CSR-Encoded 2-Hop Temporal Knowledge Graph

Nodes are linked by directed weighted edges: RELATES_TO, CAUSES, SUPERSEDES, PART_OF, DEPENDS_ON.
Traversals use Compressed Sparse Row (CSR) layout with spreading activation.
Superseding Logic: When fact B replaces fact A, linking them with SUPERSEDES suppresses the outdated fact during recall while preserving historical lineage.

4. Ebbinghaus Spaced Repetition & O(1) Tombstoning

Unaccessed memories decay exponentially based on elapsed time ($\Delta t$).
Recalling a memory automatically reinforces its access_count and refreshes its timestamp.
Critical rules can be --pinned to never decay.
Outdated facts can be soft-deleted in O(1) time via the TOMBSTONE flag.

Solving "Filtered Recall Collapse" with the Adaptive Router

A known limitation of standard HNSW vector search is that when you filter by metadata (e.g., namespace or tag where only 5% of nodes match), greedy graph traversals hit dead ends, causing recall to collapse to near 0%.

pyintel-mem includes an Adaptive Selectivity Router:

High Selectivity (>15% match): Routes through the logarithmic HNSW graph.
Low Selectivity (<15% match): Dynamically bypasses the graph and runs a pre-filtered SIMD SQ8 scan over candidate bitsets.
Result: 100% recall retention across all filter densities in under 350 microseconds.

Empirical Benchmark

Measured on an AMD Ryzen / NVMe setup (x86_64-pc-windows-gnu, release build with LTO + strip):

Metric	Result
Standalone Binary Size	536.5 KB (0 external crates)
Indexed Nodes & Edges	2,500 nodes (128-D) + 833 edges
.pmem File Size	1,088 KB (~445 bytes / node)
Single-Store Query	31.9 µs (sub-millisecond)
Federated Global Query (3 stores)	84.0 µs (1.00 ms total mmap)
Selective 5% Filtered Query	332.2 µs p50 (100% recall)

Quickstart

1. Install via Cargo

cargo install --git https://github.com/Pyintel/pyintel-mem.git

2. Shorthand CLI

# Store facts with namespace and tags
pmem store "Apex Studio UDP listener migrated to port 4141" --ns apex --tags network,udp --importance 2.0

# Sub-millisecond recall (typo tolerant)
pmem recall "what is the apex udp port?"

# Link causal / superseding relationships
pmem link --src 2 --dst 1 --kind SUPERSEDES --weight 1.0

# Delete / forget outdated memories by ID or query
pmem forget "port 3000"

# Federated cross-store search across EVERY .pmem file on disk in <100 µs
pmem global "what is the apex udp port?"

3. Connect as an MCP Server (Cursor / Claude Code / Antigravity)

Add this to your MCP settings (mcp_config.json):

{
  "mcpServers": {
    "pyintel-mem": {
      "command": "pmem",
      "args": ["mcp", "workspace.pmem"]
    }
  }
}

Exposes mem_store, mem_recall, mem_forget, mem_link, and mem_stats over stdio JSON-RPC 2.0.

One-Click Chat Memory Harvester

Have an ongoing Cursor, Claude, or ChatGPT conversation that you want to preserve into .pmem without dumping a giant wall of text?

Grab our autonomous Memory Harvester prompt:

PowerShell:

irm https://pyintel.cc/pmem-harvest.prompt | claude -p

Bash / macOS / Linux:

curl -fsSL https://pyintel.cc/pmem-harvest.prompt | claude -p

It instructs the agent to deconstruct the session into atomic 1–2 sentence facts (20–50 words), categorize them into namespaces, pin core rules, and link causal relationships automatically.

Summary & Open Source

Pyintel Mem is completely open-source under the MIT license. We provide Authenticode-signed pre-built binaries for Windows (x86_64 & ARM64), macOS Universal (Apple Silicon & Intel), and Linux (x86_64 & ARM64).

GitHub: https://github.com/Pyintel/pyintel-mem
Website & Documentation: https://pyintel.cc/products/pyintel-mem
Memory Harvester Prompt: [https://pyintel.cc/pmem-harvest.prompt

How to Catch Rogue LLM Agents on a Local Machine? Meet TopGent

Thu, 15 Oct 2026 23:26:52 +0000

As a security guy, I kept asking myself what an AI agent does on my machine apart from answering my questions.

Can it read my secrets? Which IPs and countries does it talk to? What files can it reach? What processes does it start? Can I stop it if it does something I don't like?

I could not find one tool that shows all of that in one place. So I decided to make my own, so meet Topgent.

What is Topgent

The name is top plus agent. top shows the running processes on Linux and macOS. Topgent monitors real behaviour of the AI agents on your local machine.

For each agent it shows the files and credentials it can reach, the networks and countries it connects to, the processes it starts, and the model it uses. Then it gives the agent a risk grade, and you can stop it.

It is old school on purpose. Detection and scoring are deterministic rules, and I chose not to have any LLMs involved. It runs locally and collects metadata only.

How does an OS tell a process from an agent?

It doesn't. To the OS, Claude Code is a process like any other, with a PID (Process Identifier number), an owner, a path and some network sockets. There is no "agent" flag for OS.

So topgent looks at the executable's identity and where it was installed.

Three questions

Most tools mix these up:

Declared: what the agent says it can do
Observed: what it has actually done
Reachable: what it could do right now

Example: your SSH key. If an agent never touched it, a tool that only watches events sees nothing and the agent looks clean. But the agent could still read that key any time it wants. Topgent shows that as reachable. It never opens the key to test it. It asks the OS whether the account is allowed to read that file.

Why Rust

Topgent reads OS data like socket tables, audit logs, config files. Memory safety matters when you use that data and parse it. The parsers have 11 fuzz targets(OSS-Fuzz), and trufflehog, gitleaks, osv-scanner and semgrep run on every push to ensure we catch some obvious security issues.

Topgent itself runs unprivileged, and there is an optional packet capture helper that needs a privilege, and it stays off until you turn it on.

Pitfalls so far

1. I didn't want another antivirus. Topgent reports first and acts only when you ask. When you stop a process, it rechecks the PID and start time right before signalling, so it never kills a reused PID by accident.

2. Three operating systems have three different ways to get security events. Linux has audit, macOS and Windows have their own, and none of them share access easily. Linux has been the least painful. On Windows I can't check file readability yet, so there are no reachability findings there, and a Windows score is not comparable with a Linux one.

3. How much detail to show. The operating system reports a huge amount of detail, and most of it is noise. Nobody wants to read thousands of lines to find one problem. So Topgent shows a simple risk grade first. If you want to know why an agent got that grade, one click shows the evidence behind it.

4.Short-lived processes. Topgent looks at the machine in snapshots, like taking a photo. In my tests it found every process that stays running, but it could not find the quick ones. If a process starts and exits between two snapshots, it never appears in either photo. An agent can run a quick command and be gone before Topgent looks. That is why I want eBPF on Linux, which watches events as they happen instead of taking photos. I am planning to release this soon.

eBPF, in simple words

eBPF lets you run small programs inside the Linux kernel after a safety check. You can watch process starts and connections as they happen. I am planning to add it in the next release to catch short-lived processes and look inside containers.

No proxy

Many tools inject themself between the agent and the user. I decide to go the other way and work with the OS. Topgent looks into the interaction between the agent and the OS. As mentioned Topgent can also capture network packets (headers only).

Where it is now

v0.5.1 works on macOS, Linux and Windows. Nineteen agent families are defined, including Claude Code, Codex CLI, Gemini CLI, Aider, OpenHands, Goose, OpenCode, Cursor, Ollama and Cline. Some are verified per platform and the README table shows which.

Releases are unsigned for now. I also need to get a macOS developer ID and a paid Apple account.

The tool is Apache-2.0. I also plan an enterprise version for teams that need one view of all agents.

Try it

I would appreciate your honest feedback. If you want to contribute or work on this together, I am happy to collaborate. Windows and macOS event visibility is where I need the most help.

How do you spot and stop a rogue AI agent on your machine?
https://github.com/farikonsec/topgent

Returning to Salesforce After 8 Years: Building a Cross-Platform Task Ecosystem in Tauri v2 & Rust

Wed, 14 Oct 2026 04:25:10 +0000

Hey CoderLegion! I recently returned to the Salesforce ecosystem after an 8-year hiatus, and the shift in integration architecture is night and day. It’s no longer about writing legacy point-to-point plugins—it's about orchestrating zero-trust multi-platform workflows across Salesforce, Jira, Asana, ServiceNow, and HubSpot.

To solve this without hitting our users with memory-hungry backend processes (avoiding heavy Chromium/Electron bundles that consume hundreds of megabytes of RAM), we engineered an ultra-lightweight desktop daemon using Tauri v2 and Rust.

🛠️ The Architecture Stack

• Web UI Center: A clean, highly visual 4-column Kanban command stream built with React 18, TypeScript, and Tailwind CSS.
• Native Desktop Client: An operational desktop system tray daemon built using a memory-safe Tauri v2 + Rust core running smoothly with a footprint of sub-15MB RAM and <0.05% CPU.
• Multi-OS compilation: Compiling native runtime binaries for Windows (MSI), macOS (DMG), and Linux (DEB) via unified DevOps pipelines.

🛡️ DevSecOps & Privacy Isolation

When deploying fleet tools inside corporate networks, strict compliance handles are non-negotiable.

We integrated an absolute "Privacy by Design" model targeting exclusive task state lifecycles (In Progress ➔ Blocker ➔ Complete). There is zero keystroke logging and zero screen capturing, creating a pipeline that is 100% compliant with EU GDPR and German Works Council (Betriebsrat) frameworks.

I wrote an architectural breakdown covering how this ties technical delivery frameworks (like Model Context Protocol / AI Agents) into front-end business databases, mapping true task completion speeds to accurate customer acquisition costs.

I’d love to hear how the community is managing cross-cloud event loops and native desktop triggers in Tauri v2 or Rust right now! Let me know your thoughts on the approach.
Read the full breakdown:
👉 https://www.seosiri.com/2026/10/multi-platform-task-automation-ecosystem-tauri-rust.html

Building aegis-filter: Why Standard Kalman Filters Fail in Adversarial Environments

Fri, 09 Oct 2026 05:45:50 +0000

When building telemetry filters for autonomous drones or robotics, the textbook recommendation is always the same: “Just slap a Kalman Filter on it.”

In theory, the linear Kalman filter is mathematically optimal under Gaussian white noise. In the real world, however, physical sensors do not live in textbook conditions:

Adversarial GPS/Baro Spoofing: A rogue transmitter or RF jammer injects sudden, massive delta jumps (e.g., +150 meters in a single millisecond).
Numerical Covariance Degeneracy: Floating-point round-off errors over millions of iterations cause the error covariance matrix $P$ to lose its positive-definiteness ($P \not\succ 0$), producing negative variances and exploding state estimates.
Severe Non-Linearities: Radar range calculations ($r = \sqrt{x^2 + y^2}$) and orientation quaternions break linear assumptions; Extended Kalman Filters (EKF) require calculating brittle analytical Jacobian matrices that drift rapidly.

To solve these mission-critical vulnerabilities, we built and open-sourced aegis-filter under Exploidus—a cyber-resilient, #![no_std], hard real-time estimation engine written from scratch in pure Rust.

Here is the engineering breakdown of how it works and what the simulation numbers prove.

1. The Core Architecture

aegis-filter is engineered with three core defenses:

A. Joseph-Stabilized Covariance Update

Standard Kalman filters update covariance using $P_k = (I - K_k Hk) P{k|k-1}$. In finite-precision IEEE 754 floating-point arithmetic, subtraction can yield asymmetric, non-positive-definite matrices.

aegis-filter enforces the Joseph Stabilized Form:
\[P_k = (I - K_k H_k) P_{k|k-1} (I - K_k H_k)^T + K_k R_k K_k^T\]

Because both terms are quadratic forms ($A P A^T + B R B^T$), positive-definiteness is mathematically guaranteed across infinite operational lifetimes.

B. Chi-Square / 3-Sigma Outlier Rejection Gating

Before any raw measurement is allowed to alter the state vector, the innovation $\tilde{y}_k = z_k - Hk \hat{x}{k|k-1}$ is evaluated against the innovation covariance $S_k = Hk P{k|k-1} H_k^T + R_k$:
\[\text{Reject if } |\tilde{y}_k| > \gamma \sqrt{S_k}\]

If an adversary transmits a spoofed altitude or an IMU glitches, the reading is gated at line-rate. The filter falls back safely to its physical state propagation without corrupting the state.

C. Unscented Kalman Filtering (UKF) & $H_\infty$ Robust Minimax

UKF (Merwe Scaled Sigma Points): Replaces brittle Jacobian derivations by propagating $2n + 1$ deterministic sigma points through non-linear observation closures.
$H_\infty$ Minimax: Optimizes state estimation against the worst-case disturbance rather than assuming well-behaved Gaussian noise.

2. Flight Telemetry & Spoofing Resilience Simulation

To prove real-world resilience, we simulated a quadcopter ascending at $1.5\text{ m/s}$ subjected to atmospheric turbulence, normal sensor noise, and deliberate adversarial spoofing:

Step 12: Instantaneous $+150\text{m}$ GPS spoofing attack.
Step 22: Severe $-80\text{m}$ sensor hardware dropout glitch.

Here is the telemetry log produced by running cargo run --example flight_telemetry:

===============================================================================
  AEGIS-FILTER :: FLIGHT TELEMETRY & ADVERSARIAL RESILIENCE SIMULATION
  Developed under Exploidus (https://github.com/Exploidus)
===============================================================================

Step   | True Alt (m) | Sensor Raw (m) | Aegis Est (m)  | Abs Error    | Status    
-------+--------------+----------------+----------------+--------------+-----------
1      | 1.50         | 1.79           | 1.75           | 0.254        | VALID     
2      | 3.00         | 3.22           | 3.24           | 0.238        | VALID     
3      | 4.50         | 3.46           | 4.31           | 0.187        | VALID     
4      | 6.00         | 6.74           | 6.06           | 0.057        | VALID     
5      | 7.50         | 7.67           | 7.58           | 0.081        | VALID     
6      | 9.00         | 8.90           | 9.04           | 0.045        | VALID     
7      | 10.50        | 9.68           | 10.39          | 0.113        | VALID     
8      | 12.00        | 12.98          | 12.08          | 0.075        | VALID     
9      | 13.50        | 13.41          | 13.55          | 0.047        | VALID     
10     | 15.00        | 14.69          | 14.99          | 0.009        | VALID     
11     | 16.50        | 16.04          | 16.42          | 0.080        | VALID     
12     | 18.00        | 168.00         | 17.92          | 0.080        | SPOOF REJECT
13     | 19.50        | 19.08          | 19.36          | 0.138        | VALID     
14     | 21.00        | 20.65          | 20.83          | 0.172        | VALID     
15     | 22.50        | 22.43          | 22.34          | 0.156        | VALID     
16     | 24.00        | 24.93          | 24.01          | 0.012        | VALID     
17     | 25.50        | 24.79          | 25.40          | 0.099        | VALID     
18     | 27.00        | 26.78          | 26.88          | 0.116        | VALID     
19     | 28.50        | 28.76          | 28.44          | 0.060        | VALID     
20     | 30.00        | 30.64          | 30.04          | 0.044        | VALID     
21     | 31.50        | 30.61          | 31.41          | 0.095        | VALID     
22     | 33.00        | -47.00         | 32.91          | 0.095        | SPOOF REJECT
23     | 34.50        | 34.96          | 34.50          | 0.003        | VALID     
24     | 36.00        | 36.25          | 36.04          | 0.036        | VALID     
25     | 37.50        | 36.60          | 37.39          | 0.110        | VALID     
26     | 39.00        | 39.34          | 38.96          | 0.041        | VALID     
27     | 40.50        | 40.99          | 40.54          | 0.039        | VALID     
28     | 42.00        | 41.84          | 42.01          | 0.009        | VALID     
29     | 43.50        | 42.78          | 43.40          | 0.100        | VALID     
30     | 45.00        | 45.60          | 45.00          | 0.003        | VALID     

-------------------------------------------------------------------------------
  SIMULATION SUMMARY:
  - Total Steps Processed      : 30
  - Valid Readings Fused       : 28
  - Adversarial Spikes Blocked : 2 (100% Gated)
  - Final Variance Confidence  : 0.1185
===============================================================================

Key Takeaways from the Data:

Millimeter Tracking Precision: Under normal noisy conditions (Step 10, Step 30), estimation error dropped to 0.003–0.009 meters (3–9 mm).
Flawless Spoof Rejection: When the raw altitude leaped to $168\text{m}$ at Step 12, the filter isolated the anomaly, kept the aircraft altitude at $17.92\text{m}$ (error: $0.080\text{m}$), and prevented a fatal motor cut.

3. Quick Start (Rust)

Add aegis-filter to your Cargo.toml:

[dependencies]
aegis-filter = "0.1.0"

For bare-metal Cortex-M/RISC-V embedded targets:

[dependencies]
aegis-filter = { version = "0.1.0", default-features = false }

1D Outlier-Gated Fusion Example:

use aegis_filter::{ScalarKalmanFilter, FilterError};

fn main() -> Result<(), FilterError> {
    // Initial state: 0.0m, Initial variance: 5.0, Process noise: 0.02, Measurement noise: 0.8
    let mut filter = ScalarKalmanFilter::new(0.0, 5.0, 0.02, 0.8)?;

    let sensor_stream = [1.5, 3.1, 4.4, 168.0 /* Spoofed spike */, 6.0];

    for &raw_reading in &sensor_stream {
        filter.predict(1.5)?; // Predict forward by 1.5m climb

        match filter.update(raw_reading, Some(3.0)) { // 3-sigma gate
            Ok(estimated_alt) => println!("Estimated Altitude: {:.2}m", estimated_alt),
            Err(FilterError::MeasurementGated) => {
                println!("Spoofed sensor reading ({:.1}m) rejected! Holding state.", raw_reading);
            }
            Err(e) => return Err(e),
        }
    }

    Ok(())
}

4. Open Source Links

The engine is open-source under dual MIT/Apache-2.0 licenses:

Crates.io: crates.io/crates/aegis-filter
GitHub Repository: Exploidus/aegis-filter
Documentation: docs.rs/aegis-filter

Feedback, issues, and PRs from control engineers and systems developers are warmly welcome!

Data Types Across Programming Languages: What You Actually Need to Know

Tue, 06 Oct 2026 14:07:00 +0000

Every introduction to programming eventually reaches data types.

An integer is a whole number. A string contains text. A Boolean is true or false. A floating-point number represents numbers with fractional components.

Then you learn another programming language and discover that apparently none of this was quite as simple as advertised.

An integer in Python is not the same thing as an integer in C. JavaScript's number covers values other languages separate. Rust would rather make absence explicit than let null wander through a program. R has several different ways for something to be missing.

The definitions are not wrong. They are incomplete. A data type does more than tell a computer how to store a value. It expresses what a language assumes you should be able to represent, which distinctions matter, and which mistakes should be caught before the program runs.

So instead of memorizing another list of types, let's give several languages the same small collection of facts and see what happens.

Meet Alex.

Alex is about to have a complicated afternoon, mostly because of one missing middle name.

The deceptively simple data model

Suppose our program needs to represent a customer:

name = "Alex"
age = 42
discount = 0.15
middle_name = no value
favorite_numbers = 3, 7, 11
status = ACTIVE

Nothing exotic. Text, an integer, a decimal value, an absent value, a collection, and a status chosen from a known set.

If data types were merely different spellings for the same concepts, moving this model between languages would be syntax conversion.

It isn't. And watch which field causes the most trouble.

Python: let the values tell us what they are

Python makes our first version pleasantly straightforward.

name = "Alex"
age = 42
discount = 0.15
middle_name = None
favorite_numbers = [3, 7, 11]
status = "ACTIVE"

At runtime those values have types such as str, int, float, NoneType, and list. We did not declare them before assigning because Python is dynamically typed.

That makes Alex easy to get into the system. It also means constraints exist only if we deliberately add them.

Nothing about status = "ACTIVE" prevents a later status = "ACTVE". Python does not know our domain recognizes ACTIVE, SUSPENDED, and DELETED and has never heard of ACTVE. We can model that with an Enum, type hints, validation, or classes, but the bare string does not carry the rule.

Python's integers hold another surprise for anyone arriving from C or Java. A normal Python int is not limited to a fixed 32-bit range.

age = 999999999999999999999999999999999999

An absurd customer age, and a perfectly valid Python integer.

What nothing looks like in Python

Spelling	What it means	Who makes you deal with it
`None`	The only nothing available	Nobody

Already, "integer" has stopped meaning one universal thing. And None is a single, undifferentiated way of saying nothing is here, which will turn out to be a choice rather than an inevitability.

JavaScript: one `number`, and two different nothings

const name = "Alex";
const age = 42;
const discount = 0.15;
const middleName = null;
const favoriteNumbers = [3, 7, 11];
const status = "ACTIVE";

Similar enough, until we ask JavaScript about the values.

typeof age;         // "number"
typeof discount;    // "number"
typeof middleName;  // "object"

Yes, typeof null returns "object", a famous historical artifact that cannot be fixed without breaking existing code.

More important for our comparison, 42 and 0.15 share the ordinary type number, based on IEEE 754 double-precision floating point. JavaScript also has BigInt for integers beyond the safe range of number.

Absence is where it gets interesting. An uninitialized variable is undefined. A variable deliberately set to nothing is null. The language provides two spellings for emptiness and then declines to tell you which one means what. In practice one tends to mean "nobody has assigned this yet" and the other "somebody decided this is empty," except when a codebase uses them the other way around, or interchangeably.

Loose equality shows what the language really thinks. Zero, the empty string, false, and an empty array all agree they are the same thing. The two values that actually mean nothing is here do not join in:

0 == "";             // true
"" == false;         // true

null == undefined;   // true
null === undefined;  // false
null == 0;           // false
null == "";          // false

null and undefined are loosely equal to each other and to nothing else in the language. They form their own small island.

So JavaScript does distinguish absence from emptiness, and the rule is arguably the right one: a missing thing is not a zero thing. The problem is where that rule lives. It sits inside an equality operator famous for being unpredictable, next to a typeof that reports null as "object". Alex Dorey's JavaScript Equality Table renders every comparison as a grid, if you want the full picture and a certain amount of despair.

What nothing looks like in JavaScript

Spelling	What it means	Who makes you deal with it
`undefined`	Never assigned	Nobody
`null`	Deliberately empty, by convention	Nobody

A type system is already more than a vocabulary list.

C: representation becomes difficult to ignore

#include <stdint.h>

typedef enum {
    ACTIVE,
    SUSPENDED,
    DELETED
} Status;

typedef struct {
    char *name;
    int32_t age;
    double discount;
    char *middle_name;
    int favorite_numbers[3];
    Status status;
} Customer;

Representation is visible almost everywhere. We chose an explicitly 32-bit signed integer for age, double for the discount, an array of exactly three integers, pointers to characters for names, and an enumeration for status.

A null pointer might represent an absent middle name:

customer.middle_name = NULL;

But C does not attach our intended meaning to that pointer. NULL here could mean Alex has no middle name, that we have not loaded it yet, that the allocation failed, or that somebody forgot to initialize the struct. The program establishes the convention and the program is responsible for respecting it. The compiler will not help.

What nothing looks like in C

Spelling	What it means	Who makes you deal with it
`NULL` pointer	Whatever this program decided it means	Convention, and only convention

C also forces us to care directly about ranges and representation. A fixed-width signed integer cannot grow forever. This is not C being gratuitously difficult. C was designed around a different relationship between programmer and machine, and its type system reflects that.

Java: primitives, references, and the null question

enum Status {
    ACTIVE,
    SUSPENDED,
    DELETED
}

class Customer {
    String name;
    int age;
    double discount;
    String middleName;
    int[] favoriteNumbers;
    Status status;
}

Java distinguishes primitive types such as int, double, and boolean from reference types such as String, Customer, and arrays.

An int cannot be null. A String can.

That single asymmetry does a lot of work. It means absence is available for some of Alex's fields and structurally impossible for others, and the choice was made by whoever picked the field types rather than by anyone thinking about the domain. Unless we add conventions or tooling, String middleName does not tell us whether null is an expected state or a bug waiting for the right execution path.

What nothing looks like in Java

Spelling	What it means	Who makes you deal with it
`null`	An empty reference	Nobody
(not available)	Primitives cannot be absent at all	The compiler, by refusing

Java also has wrapper classes such as Integer, which means two things that sound like "an integer" behave differently:

int age;
Integer optionalAge;

One of those can be absent. The other cannot. Learning the keyword is only the beginning; the useful question is what guarantees come with it.

Rust: make some invalid states harder to represent

enum Status {
    Active,
    Suspended,
    Deleted,
}

struct Customer {
    name: String,
    age: u32,
    discount: f64,
    middle_name: Option<String>,
    favorite_numbers: Vec<i32>,
    status: Status,
}

Look at middle_name: Option<String>.

Instead of allowing a String to secretly contain null, Rust represents the possibility of absence in the type itself. The value is either Some(...) or None, and code consuming it has to deal with that possibility before it can get at the string.

This is the same information every other language has been carrying around implicitly, finally written down somewhere the compiler can read it. The other languages let you forget. Rust makes forgetting a compile error.

The Status enum similarly makes the valid alternatives explicit. We cannot accidentally assign ACTVE, because no such variant exists.

What nothing looks like in Rust

Spelling	What it means	Who makes you deal with it
`None`, inside `Option<T>`	This value may legitimately be absent	The compiler

A type system can encode valid states, not merely storage categories. That does not make incorrect programs impossible. It moves some classes of error from runtime behaviour into representations the compiler can reject.

The type system is participating in the design.

Go: useful zero values and deliberate simplicity

type Status string

const (
    Active    Status = "ACTIVE"
    Suspended Status = "SUSPENDED"
    Deleted   Status = "DELETED"
)

type Customer struct {
    Name            string
    Age             int
    Discount        float64
    MiddleName      *string
    FavoriteNumbers []int
    Status          Status
}

One important Go idea appears before we populate Alex at all: types have zero values. The zero value of an int is 0, of a bool is false, of a string is "". Pointers, slices, maps, functions, channels, and interfaces can be nil.

That creates a question. If Age contains 0, is Alex zero years old, or did nobody supply an age?

The type cannot answer. If the application uses a zero value to stand for “not supplied,” that state becomes indistinguishable from a legitimately supplied zero. If the distinction matters, the model has to represent it deliberately, which is why MiddleName above is a *string rather than a string.
What nothing looks like in Go

Spelling	What it means	Who makes you deal with it
`0`, `""`, `false`	Zero/default value; cannot by itself tell us whether a domain value was supplied	The model must distinguish this if it matters
`nil`	Empty pointer, slice, map, channel, or interface	Nobody

Convenient defaults acquire meaning only when they enter a domain.

R: "missing" is not one simple state

name <- "Alex"
age <- 42L
discount <- 0.15
middle_name <- NA_character_
favorite_numbers <- c(3L, 7L, 11L)
status <- "ACTIVE"

R was designed around statistical computing, and its types and missing-value semantics reflect that.

NA represents a missing value, with typed forms such as NA_integer_ and NA_character_. R also has NULL, which is not another spelling of NA, and numeric computation can produce NaN.

These answer different questions. A missing observation in a dataset is not the same thing as the absence of an object. Neither is the same thing as a mathematically undefined result. R treats those as three distinct facts, because in statistics they are three distinct facts, and conflating them produces wrong answers rather than crashes.

Consider:

mean(c(10, 20, NA))
mean(c(10, 20, NA), na.rm = TRUE)

What nothing looks like in R

Spelling	What it means	Who makes you deal with it
`NA`	A missing observation	Functions ask, via `na.rm`
`NULL`	No object at all	Nobody
`NaN`	An undefined numeric result	Nobody

The second call makes an explicit methodological choice to exclude missing observations. The language made you say so.

The field that caused all the trouble

Alex's missing middle name has now been through seven general-purpose languages. Stack those seven tables on top of each other and the third column tells the whole story.

Python gave it None, one undifferentiated nothing. JavaScript offered two and left the meaning to you. C gave a null pointer whose meaning lives entirely in a convention the compiler cannot see. Java made absence available for reference types and impossible for primitives, as a side effect of a decision about memory. Rust put it in the type signature and refused to let you ignore it. Go made it indistinguishable from a valid value unless you reach for a pointer. R split it into three concepts because its users needed three.

Not one of those languages is wrong. They disagree about a prior question: how much does the programmer have to admit they do not know?

Representations of absence carry semantics whether or not the programmer acknowledges them. The difference between these languages is who is required to do the acknowledging, and when.

Which is why the missing middle name turned out to be more interesting than the integer.

The table gets weird quickly

Language	Typing	Ordinary integers	`42` and `0.15` same type?	How absence is spelled
Python	Dynamic	Arbitrary size	No	`None`
JavaScript	Dynamic	Fixed (`BigInt` for more)	Yes, both `number`	`null` and `undefined`
C	Static	Fixed width, chosen	Usually no	Null pointer, by convention
Java	Static	Fixed width	No	`null`, references only
Rust	Static	Fixed width	No	`Option<T>`, in the type
Go	Static	Fixed width	No	Zero values, `nil` for some types
R	Dynamic	Fixed width	Usually distinct	`NA`, `NULL`, `NaN`

Closed sets of alternatives vary just as much. Rust, Java, and C have real enums, Python offers Enum if you reach for it, Go builds them from a named type plus constants, JavaScript models them by hand, and R uses factors. Collections differ too: C, Java, Rust, and Go enforce an element type in the declaration, Python and JavaScript do not, and R's atomic vectors quietly coerce everything toward a common type.

There is no universal translation saying:

Python int = JavaScript number = C int = Java int = Rust i32 = Go int = R integer

Those types overlap. They do not carry identical guarantees, ranges, operations, failure modes, or assumptions.

The same name does not imply the same contract.

If you were designing a language, what would you choose?

This gets much more interesting when you reverse the problem.

Suppose you are designing a language.

You need an integer.

Easy.

Except now you have questions. Fixed widths, or one integer that grows until memory runs out? What happens on overflow? Should 3 become 3.0 on its own? Should "42" + 1 be legal, and if so, what should it mean?

Then absence, which by now you can see is the hard one. Do you have null? Can every reference be null, or do you require an explicit optional type? Do you distinguish a missing observation from a non-existent object? Whichever you choose, every program ever written in your language inherits that decision.

Then collections, where you decide whether elements must share a type and whether values can be moved or consumed. Then domain state, where you decide whether a programmer can say a value is only ever ACTIVE, SUSPENDED, or DELETED, and whether the compiler can prove every case was handled.

Finally, timing. When should a bad type relationship surface: while parsing, during compilation, at runtime, or only when the offending path finally executes?

There is no universally correct set of answers. There are consequences.

Every choice makes some programs easier to express, some mistakes harder to make, some implementations simpler, some runtimes more complicated, and some programmers wonder why on Earth the language designer did that.

Sometimes the answer is historical compatibility. Sometimes performance. Sometimes safety. Sometimes the problem domain.

And sometimes the language designer has a very good explanation and would appreciate it if everyone stopped bringing up typeof null.

One language where absence is not the question

Everything above is a general-purpose language, and every one of them had to decide what nothing means. That is not a universal preoccupation. It is what happens when a language expects you to model a world.

The e language, commonly associated with Cadence Specman, was designed for hardware verification. Verification code describes legal ranges of values, generates constrained test data, and explores systems whose state spaces are far too large to enumerate by hand. So its declarations carry constraints rather than storage categories:

struct customer {
    age : uint (bits: 8);
    keep age in [18..120];

    status : [ACTIVE, SUSPENDED, DELETED];
    keep soft status == ACTIVE;
};

There is no absence card for e in this article, and that is the point. In ordinary application programming we ask what type of value this is. A verification environment asks what values are legal here, and how valid combinations should be generated so we can explore what the system does.

If you look only at Python, JavaScript, Java, and C, it is easy to assume programming languages are all solving approximately the same problem with different punctuation.

They aren't.

Types are compressed language philosophy

At the beginner level, "a type tells us what kind of data a value contains" is perfectly serviceable. Eventually that runs out of road. A type can tell us what values exist, how they are stored, which operations are permitted, which conversions happen silently, what counts as absence, whether absence must be acknowledged, whether alternatives form a closed set, and in some languages how long a value lives and who owns it. "Integer, string, Boolean, float" is not the definition of a type system. It is the introductory vocabulary.

Which is why learning the types in a new language beats memorizing a conversion table.

Python's arbitrary-precision integers are a tradeoff. So is C asking you to care about width. Java's primitive and reference split quietly decides which of your fields are allowed to be empty. Go buys simplicity with zero values and pays for it with ambiguity at the domain boundary. R distinguishes forms of missingness because missing observations are a normal feature of statistical work, so its problem domain is visible in its types. A verification language making constrained generation first-class tells you what its designers expected programmers to spend their days doing.

A language's types are partly a record of what its designers thought mattered, and partly a record of what they were willing to let you avoid thinking about.

They are compressed language philosophy.

Alex survived

After all of this, Alex is still 42, still has no recorded middle name, still likes 3, 7, and 11, and remains active. Nothing about the underlying person changed. But every language forced different decisions about how those facts became computation, and the fact that changed most was the one that was not there.

Knowing that Python has int, Java has int, and Rust has i32 is useful when you are trying to get a program running. Understanding why they are not interchangeable is useful when you are trying to understand languages.

Once you start asking what a language lets you represent, what it makes you acknowledge, and what it refuses to let you say, data types stop being the boring chapter before loops. They become one of the quickest ways to see what a language believes programming should be.

What is the strangest, most useful, or most frustrating type decision you've encountered in a programming language? More importantly, what problem do you think the language designer was trying to solve with it? Or, if you were designing your own language, which of these decisions would you make differently?

Building a Line-Rate Linux Kernel Firewall in Pure Rust with Aya and eBPF/XDP

Sat, 03 Oct 2026 13:29:33 +0000

Volumetric Distributed Denial of Service (DDoS) and transport-layer flooding attacks pose an existential challenge to modern cloud infrastructure, edge gateways, and container workloads.

Conventional Linux packet filtering utilities such as Netfilter (iptables and nftables) evaluate incoming frames only after the kernel has allocated a socket buffer (sk_buff) descriptor and scheduled softirq processing through the core TCP/IP stack. Under multi-gigabit line-rate flood conditions (millions of packets per second), the operating system spends the vast majority of its CPU cycles simply allocating and freeing sk_buff memory metadata, triggering kernel starvation and packet drop bottlenecks even before firewall rules are evaluated.

To address this fundamental OS-level bottleneck, I built xdp-guard: a production-grade, memory-safe in-kernel defense engine and per-IP rate limiter implemented strictly in Pure Rust using the Aya eBPF framework.

Why eBPF / XDP at the Driver Layer?

The eXpress Data Path (XDP) provides a programmable, bare-metal packet processing execution hook directly inside the Network Interface Card (NIC) driver subsystem before any memory descriptor allocation occurs.

Conventional Path:
[NIC RX] -> [IRQ / SoftIRQ] -> [Allocate sk_buff] -> [Netfilter/iptables] -> [Socket / App]
                                      ^
                           Severe CPU Allocation Overhead

xdp-guard Path:
[NIC RX] -> [eBPF / XDP Driver Hook] ---> (Match Blocklist) ---> [ XDP_DROP ] (Near-Zero CPU)
                             |
                      (Match Allowlist)
                             |
                             v
                   [ Pass to OS Stack ]

When hostile or unapproved packets are identified, returning XDP_DROP causes the network driver to immediately recycle the packet ring buffer without allocating a single byte of kernel socket memory.

Architectural Highlights

1. Zero C Dependencies (Pure Rust)

Traditional eBPF tools rely heavily on C toolchains (clang, llvm, libbpf). xdp-guard is built entirely in Rust from the user-space management CLI down to the no_std kernel eBPF bytecode using Aya.

2. Strict 64-Bit Memory Boundary Alignments

To ensure safe zero-copy communication across the kernel-userspace boundary, shared structures use explicit padding fields to guarantee 8-byte alignment:

#[repr(C)]
#[derive(Clone, Copy, Debug)]
pub struct RuleValue {
    pub action: u32,       // 0 = Pass, 1 = Drop
    pub _pad: u32,         // Explicit 4-byte padding for 8-byte boundary
    pub created_at: u64,   // Monotonic timestamp
    pub ttl_secs: u64,     // Time-to-live expiration
}

The presence of _pad: u32 is critical: it prevents architecture-specific compiler padding discrepancies and satisfies the Linux Kernel BPF Verifier.

3. Dynamic BPF Virtual Filesystem (bpffs) Synchronization

Firewall rules (blocklist and allowlist) are dynamically synchronized using pinned BPF hash maps located in /sys/fs/bpf/xdp_guard. Rules can be inserted or deleted instantaneously from the CLI without tearing down network interfaces or interrupting traffic flows.

4. In-Kernel Token-Bucket Rate Limiter

In addition to static blocklists, xdp-guard embeds a stateful token-bucket rate limiter directly in kernel space. Monotonic arrival intervals ($\Delta t = t{now} - t{last}$) are computed using nanosecond bpf_ktime_get_ns() clocks, discarding over-limit burst floods at line-rate.

Empirical Verification: Real Linux Kernel Telemetry

To validate xdp-guard under realistic operating conditions, we set up an automated Linux network namespace testbed over virtual ethernet (veth) pairs (xdp_client at 10.10.0.2 and xdp_server at 10.10.0.1).

Telemetry streamed directly from the kernel STATS_MAP array demonstrated instantaneous 100.00% packet mitigation upon inserting an attacker IP into the kernel blocklist, followed by zero-loss recovery:

TIMESTAMP            PASSED (PKTS)   DROPPED (PKTS)  DROP RATIO     
-----------------------------------------------------------------
09:56:56 | Pass: 1        (30.2 KB) | Drop: 0        (0 B)   | Ratio: 0.00%
[!] Inserting IP 10.10.0.2 into BLOCKLIST_MAP... [SUCCESS]
09:56:57 | Pass: 0        (30.2 KB) | Drop: 1        (98 B)  | Ratio: 100.00%
09:56:58 | Pass: 0        (30.2 KB) | Drop: 1        (196 B) | Ratio: 100.00%
09:56:59 | Pass: 0        (30.2 KB) | Drop: 1        (294 B) | Ratio: 100.00%
...
09:57:55 | Pass: 0        (30.2 KB) | Drop: 1        (5.7 KB)| Ratio: 100.00%
[!] Removing IP 10.10.0.2 from BLOCKLIST_MAP... [SUCCESS]
09:57:56 | Pass: 1        (30.3 KB) | Drop: 0        (5.7 KB)| Ratio: 0.00%
09:57:57 | Pass: 1        (30.4 KB) | Drop: 0        (5.7 KB)| Ratio: 0.00%

Every single drop occurred directly within the NIC driver hook, requiring zero kernel TCP/IP stack overhead.

Source Code & Research Paper

The complete project, documentation, and formal research paper are fully open-source:

GitHub Repository: https://github.com/rahadbhuiya/xdp-guard
Citable Zenodo DOI: 10.5281/zenodo.23120995
Automated CI: Continuous testing across Rust Stable and Nightly LLVM targets via GitHub Actions.

If you are working on Linux kernel systems, eBPF, or Rust networking, I would love to hear your thoughts, feedback, or suggestions!

Mike Dabydeen on Rust, AI & Engineering Leadership

Fri, 02 Oct 2026 03:10:07 +0000

Developer Stories: Mike Dabydeen on Rust, AI & Engineering Leadership

Welcome to another episode of CoderLegion Developer Stories!

In this episode, Tom Smith sits down with Mike Dabydeen, an engineering leader and technology educator from Toronto, Canada.

Mike shares his experience across engineering leadership, enterprise APIs, Rust, systems programming, AI, DevOps, and SRE, as well as his perspective on building reliable software in a rapidly changing technology landscape.

🎥 Watch the Interview

▶️ Watch Mike Dabydeen's Developer Story on YouTube

💡 What We Discuss

🦀 Rust & Memory Safety — Exploring Rust, ownership, borrowing, and safe systems programming
🤖 AI & Reliable AI — Building and working with AI systems in real-world environments
⚙️ Enterprise APIs — Designing and working with APIs at scale
👨💻 Engineering Leadership — Lessons from leading engineering teams and projects
☁️ DevOps & SRE — Reliability, automation, cloud, and modern engineering practices
🎓 Teaching & Mentoring — Mike's experience sharing technology knowledge with the next generation of developers

👨💻 About Mike Dabydeen

Mike Dabydeen is an engineering leader who works across enterprise APIs, decentralized systems, systems programming, and AI.

He also teaches at Sheridan College and Conestoga College and is active in the developer community.

His technical interests and experience include Rust, AI, DevOps, SRE, Java, C, Python, C#/.NET, Go, Kubernetes, AWS, Azure, Docker, React, TypeScript, and more.

🌐 Visit Mike's Website

🎙️ Developer Stories

Developer Stories is a CoderLegion series featuring developers, engineers, technical leaders, and technology professionals from around the world.

We talk about their journeys, experiences, challenges, technologies, and lessons learned along the way.

👉 Join the CoderLegion community: https://coderlegion.com

Guest: Mike Dabydeen
Interviewer: Tom Smith
Series: CoderLegion Developer Stories

developer-stories #rust #ai #artificial-intelligence #software-engineering #engineering-leadership #systems-programming #devops #sre

Semantic Search in C++ without Python, libtorch or ONNX Runtime

Wed, 30 Sep 2026 21:39:52 +0000

title: Semantic Search in C++ without Python, libtorch or ONNX Runtime
published: true
description: Run a transformer model from C++ in four commands. No libtorch, no ONNX Runtime, no Python, no model conversion. Works from C++11 upward.
tags: cpp, machinelearning, rust, opensource

canonical_url: https://kjarni.ai/blog/cppinference/

Ask how to run a transformer model from C++ and you get two answers: link libtorch, or convert the model and link ONNX Runtime. Both work. Both are large, both want a toolchain of their own, and both put a second inference engine inside your process.

There is a third answer, and it takes four commands.

mkdir kjarni-quickstart && cd kjarni-quickstart
curl -sL https://github.com/olafurjohannsson/kjarni/releases/latest/download/kjarni-x86_64-linux.tar.gz | tar xz
curl -sO https://raw.githubusercontent.com/olafurjohannsson/kjarni/main/crates/kjarni-ffi/examples/cpp/hello.cpp
g++ -std=c++23 hello.cpp -I. -L. -lkjarni_ffi -Wl,-rpath,'$ORIGIN' -o hello && ./hello

related:   0.5510
unrelated: -0.0630

That is a transformer model, downloaded, loaded and run, from an empty directory. No package manager, no Python, no model conversion step. The archive holds the shared library, kjarni.h (the C ABI) and kjarni.hpp (a header-only C++23 wrapper). macOS and Windows
builds are on the same [releases page(https://github.com/olafurjohannsson/kjarni/releases).

Here is what hello.cpp contains:

#include "kjarni.hpp"
#include <print>

int main() {
    // Downloaded once and cached under ~/.cache/kjarni, then loaded from disk.
    auto embedder = kjarni::Embedder::create({.model = "minilm-l6-v2"});
    if (!embedder) {
        std::println("{}", embedder.error().message());
        return 1;
    }

    auto question = embedder->encode("How do I get my money back?");
    auto related = embedder->encode("What is your refund policy?");
    auto unrelated = embedder->encode("The weather in Reykjavik is unpredictable.");

    // No shared words with the question, but the same meaning.
    std::println("related:   {:.4f}", kjarni::cosine(*question, *related));
    std::println("unrelated: {:.4f}", kjarni::cosine(*question, *unrelated));
}

"How do I get my money back?" and "What is your refund policy?" share no words at all, and score 0.55. The sentence about the weather scores below zero. That gap is the entire idea behind semantic search.

How semantic search works

An embedding model reads text and returns a vector, an array of floats, 384 numbers for the model above. Text with similar meaning produces vectors that point in similar directions.

"refund policy"  -> [0.12, -0.34, 0.56, ...]  (384 numbers)
"get money back" -> [0.11, -0.33, 0.55, ...]  (384 numbers)  <- close
"weather today"  -> [-0.45, 0.23, -0.12, ...] (384 numbers)  <- far

You compare two vectors with cosine similarity, which measures the angle between them and ignores their length. It runs from 1 for identical direction to -1 for opposite.

What you actually link against

This is the part that decides whether the approach is worth anything, so it is worth checking rather than believing. Ask the linker:

$ ldd hello
    linux-vdso.so.1
    libkjarni_ffi.so => /home/you/kjarni-quickstart/libkjarni_ffi.so
    libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6
    /lib64/ld-linux-x86-64.so.2
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2

Kjarni, the C++ runtime, and the parts of glibc every program already uses: libc, libm, libgcc, libpthread and libdl. That is the whole list. No libtorch, no onnxruntime, no Python, no CUDA runtime. The binary is 283 KB and the library is 19.4 MB, which includes the tokenizer, the model loaders and every kernel.

The -Wl,-rpath,'$ORIGIN' in the build line is what makes that first entry resolve to the library sitting next to your binary rather than something in /usr/local/lib. Keep it, and
the directory you built in is a directory you can copy somewhere else and run.

A dependency you cannot see in ldd is a dependency that cannot break you on a machine that is not yours.

Errors are values

Every fallible call returns std::expected<T, kjarni::Error>. Nothing in the header throws except std::bad_alloc.

auto embedder = kjarni::Embedder::create({.model = "minilm-l6-v2"});
if (!embedder) {
    std::println(stderr, "could not load model: {}", embedder.error().message());
    return 1;
}

Whether a missing model file is exceptional depends on the program. A batch job should die; a desktop application should show a message and carry on. Returning the failure lets the caller decide, and puts it in the signature where it cannot be missed.

The options are a designated-initialiser aggregate, so a call names only what it changes:

Embedder::create({.model = "mpnet-base-v2", .gpu = true}).

Searching a corpus

Encode the documents once, encode the query at search time, sort by similarity.

#include "kjarni.hpp"

#include <algorithm>
#include <print>
#include <ranges>
#include <string_view>
#include <vector>

int main() {
    auto embedder = kjarni::Embedder::create({.model = "minilm-l6-v2"});
    if (!embedder) {
        std::println(stderr, "could not load model: {}", embedder.error().message());
        return 1;
    }

    constexpr std::array docs = {
        std::string_view{"How do I reset my password?"},
        std::string_view{"What is your refund policy?"},
        std::string_view{"Do you ship internationally?"},
        std::string_view{"How do I update my billing address?"},
        std::string_view{"Where can I track my order?"},
    };

    std::vector<std::vector<float>> corpus;
    corpus.reserve(docs.size());
    for (std::string_view d : docs) {
        auto v = embedder->encode(d);
        if (!v) {
            std::println(stderr, "encode failed: {}", v.error().message());
            return 1;
        }
        corpus.push_back(v->to_vector());
    }

    constexpr std::string_view query = "I need to change my login credentials";
    auto q = embedder->encode(query);
    if (!q) {
        std::println(stderr, "encode failed: {}", q.error().message());
        return 1;
    }

    std::vector<std::pair<float, std::string_view>> scored;
    for (auto [i, doc] : std::views::enumerate(docs))
        scored.emplace_back(kjarni::cosine(q->values(), corpus[i]), doc);

    std::ranges::sort(scored, std::ranges::greater{},
                      &std::pair<float, std::string_view>::first);

    std::println("query: \"{}\"", query);
    for (auto [score, doc] : scored)
        std::println("  {:.4f}  {}", score, doc);
}

query: "I need to change my login credentials"
  0.5981  How do I reset my password?
  0.4067  How do I update my billing address?
  0.0767  Where can I track my order?
  -0.0027  What is your refund policy?
  -0.0451  Do you ship internationally?

"Change my login credentials" matches "reset my password" at 0.60 while sharing no words with it, and "update my billing address" comes second because changing account details is a
related idea. That is what you cannot get from keyword matching.

encode returns a kjarni::Embedding, which owns the array the C API returned and frees it in its destructor. It hands out a std::span<const float> through values(), so it drops
straight into ranges, and to_vector() copies when the data has to outlive the object. There is no raw pointer to forget.

Classification and reranking

The other models follow the same shape.

auto clf = kjarni::Classifier::create({.model = "roberta-sentiment"});
if (!clf) { std::println(stderr, "{}", clf.error().message()); return 1; }

for (std::string_view t : {"I love this product!",
                           "Terrible quality, broke after one day."}) {
    auto top = clf->top(t);
    if (!top) { std::println(stderr, "{}", top.error().message()); return 1; }
    if (*top) std::println("  {:<9} {:5.1f}%  \"{}\"", (*top)->name, (*top)->score * 100, t);
}

  positive   98.5%  "I love this product!"
  negative   94.1%  "Terrible quality, broke after one day."

top() returns Result<std::optional<Label>>, which puts two separate questions in the type: did the call work, and did it produce a label. A model with nothing above threshold is
not a failure.

Reranking uses a cross-encoder, which reads the query and each document together instead of comparing two independently produced vectors. It is slower and much more precise, so it runs
as a second pass over whatever the embeddings retrieved:

auto rr = kjarni::Reranker::create();
if (!rr) { std::println(stderr, "{}", rr.error().message()); return 1; }

const std::vector<std::string> docs = {
    "Machine learning is a subset of artificial intelligence.",
    "Deep learning uses neural networks with many layers.",
    "The weather today is sunny.",
};

auto ranked = rr->rerank("What is machine learning?", docs);
if (!ranked) { std::println(stderr, "{}", ranked.error().message()); return 1; }

for (const auto& r : *ranked)
    std::println("  {:>9.4f}  {}", r.score, docs[r.index]);

    10.5139  Machine learning is a subset of artificial intelligence.
    -5.5301  Deep learning uses neural networks with many layers.
   -11.1001  The weather today is sunny.

The scores are logits, not probabilities. What matters is the ordering and the size of the gap between one document and the next. Ranked gives back an index into your input rather than a copy of the text, so whatever IDs, URLs and permissions came with your documents stay attached to them.

One signature detail: rerank takes std::span<const std::string> rather than string_view, because the C call underneath needs an array of null terminated pointers and a string_view does not promise one.

The same numbers in every language

Those figures are not specific to C++. The reranker scores 10.5139, -5.5301 and -11.1001 are
the same values the C# document search post prints, and the
similarity scores are the ones in Semantic Search in C#.

That is the reason a C ABI is the right shape here. There is one engine and one set of
kernels, and the C++ header is a wrapper over the same entry points the C#, Go and Python
packages call. There is no second implementation to drift from the first.

Choosing a model

Model	Dimensions	Input limit	Notes
`minilm-l6-v2`	384	256 tokens	Default. Fast, good quality per byte
`mpnet-base-v2`	768	384 tokens	Higher quality, slower
`nomic-embed-text`	768	8192 tokens	Long documents, though trained at 2048
`bge-m3`	1024	8192 tokens	Large, multilingual

Mind the input limit column. minilm-l6-v2 reads 256 tokens, roughly 900 characters, and silently drops the rest: no error, no warning, just a vector computed from the part it saw.
The cross-encoder has its own limit, reading query and document as one sequence capped at 512 tokens. If your documents are longer than that, chunk them. There is a measurement of
what the truncation costs in Your MiniLM Embeddings Are Probably Truncating at 256 Tokens.

Practical notes

Threading. The handles are not individually thread safe. Give each thread its own, or serialise calls. The engine already parallelises across cores inside a single call, so one embedder will use the machine.

C++23 is only needed for std::expected. Everything else in kjarni.hpp is C++20, and GCC 13 or Clang 17 and newer will build it.

There is no package manager integration yet. No Conan recipe, no vcpkg port. The four commands above are the install story on Linux, and the equivalent archives for macOS and
Windows are on the releases page.

If you are on C++11 or C++17

Plenty of codebases are, and a header that demands C++23 is not much use to them. It is worth being clear about what the requirement actually covers: kjarni.hpp needs C++23 for
std::expected, and nothing else does. kjarni.h is plain C, it is the interface every other language binding is built on, and it works from C++11 upward.

The whole of the C++23 convenience is one RAII wrapper and a copy:

#include "kjarni.h"

#include <cstdio>
#include <memory>
#include <string>
#include <vector>

namespace {

struct EmbedderDeleter {
    void operator()(KjarniEmbedder* p) const { kjarni_embedder_free(p); }
};
using EmbedderPtr = std::unique_ptr<KjarniEmbedder, EmbedderDeleter>;

std::vector<float> encode(KjarniEmbedder* emb, const char* text) {
    KjarniFloatArray arr{};
    if (kjarni_embedder_encode(emb, text, &arr) != KJARNI_ERROR_CODE_OK) {
        std::fprintf(stderr, "encode failed: %s\n", kjarni_last_error_message());
        return {};
    }
    std::vector<float> out(arr.data, arr.data + arr.len);
    kjarni_float_array_free(arr);   // copied out, so release the engine's buffer
    return out;
}

} // namespace

int main() {
    KjarniEmbedderConfig cfg = kjarni_embedder_config_default();
    cfg.model_name = "minilm-l6-v2";
    cfg.quiet = 1;

    KjarniEmbedder* raw = nullptr;
    if (kjarni_embedder_new(&cfg, &raw) != KJARNI_ERROR_CODE_OK) {
        std::fprintf(stderr, "could not load model: %s\n", kjarni_last_error_message());
        return 1;
    }
    EmbedderPtr embedder(raw);

    const std::vector<float> question  = encode(embedder.get(), "How do I get my money back?");
    const std::vector<float> related   = encode(embedder.get(), "What is your refund policy?");
    const std::vector<float> unrelated = encode(embedder.get(), "The weather in Reykjavik is unpredictable.");
    if (question.empty() || related.empty() || unrelated.empty()) return 1;

    std::printf("related:   %.4f\n",
                kjarni_cosine_similarity(question.data(), related.data(), question.size()));
    std::printf("unrelated: %.4f\n",
                kjarni_cosine_similarity(question.data(), unrelated.data(), question.size()));
}

g++ -std=c++17 hello17.cpp -I. -L. -lkjarni_ffi -Wl,-rpath,'$ORIGIN' -o hello17

related:   0.5510
unrelated: -0.0630

The same numbers as the C++23 version, because it is the same engine underneath. That compiles unchanged under -std=c++11 as well.

Two rules cover the manual memory. Any KjarniFloatArray you receive is freed with kjarni_float_array_free once you have copied what you need out of it, and any handle is freed with its matching _free function. Wrapping the handle in a unique_ptr with a
custom deleter, as above, means the second rule takes care of itself on every return path.

Error text comes from kjarni_last_error_message(), and it reports the most recent failure process wide, so read it immediately after the call that failed rather than saving it up.

Compared to the alternatives

	libtorch	ONNX Runtime	Kjarni
Install	download SDK, match ABI	package plus model conversion	one archive
Model format	TorchScript	.onnx, converted	HuggingFace safetensors and GGUF directly
Extra entries in `ldd`	many	its own stack	none beyond libc
Tokenizer	bring your own	bring your own	included
GPU	CUDA toolkit	CUDA or DirectML	WebGPU, no toolkit

The trade is scope. libtorch runs anything expressible in TorchScript. Kjarni runs the model families it implements: BERT-style encoders, cross-encoders, Llama-family decoders, T5, BART and Whisper. For an arbitrary research model, convert it and use ONNX Runtime. For embeddings, classification, reranking or chat inside a C++ program that has to ship somewhere, one archive with no toolchain is a smaller problem than either.

Getting it

kjarni.ai - documentation and the rest of these posts
Releases - the archive used above, for Linux, macOS and Windows
GitHub - source, including kjarni.hpp and the C++ examples
NuGet - the same engine from C#
npm - the same engine in the browser, via WebAssembly
Go module - the same engine from Go
crates.io - the Rust engine itself

Other Resources

Semantic Search in C# - The same engine and the same vectors, from .NET
Build a Document Search Engine in C# - Keyword and semantic retrieval combined, with reranking
Why I Built a Native ML Inference Engine in Rust - What is underneath all of this
ML from the Command Line - The same models as a UNIX tool
Your MiniLM Embeddings Are Probably Truncating at 256 Tokens - Measured, with the cost

Uncle Bob's uml-viewer on Rust: 35 dependency cycles down to 3

Tue, 22 Sep 2026 07:13:36 +0000

TL;DR. Uncle Bob released uml-viewer, a clickable architecture diagram that draws Dependency Rule violations in red and is meant to be driven together with a coding agent. It only parses Clojure, so I wrote a 200-line script that turns my Rust crate into its input file. The first picture showed 167 red arrows and 35 dependency cycles. Six commits and 315 files later: 109 red arrows, 3 cycles, and app.rs down from 1,087 lines to 642. No behavior changed. The tool matters less than the loop it forces on you: look, point at one arrow, let the agent move code, look again. The original post has the same text with a collapsible FAQ and the source list at the end.

Uncle Bob's argument for the tool is short: agents still need supervision, and reading every line they write is the bottleneck. So supervise the structure instead of the text: draw the system, find the shape that is wrong, tell the agent to fix that shape, and check the new drawing.

I wanted to know if that works on a real codebase, so I pointed it at Uptimepage, about 146,000 lines of Rust in one crate. It does, with one condition: the picture is only as honest as the input file you feed it.

Five of the 167 red arrows and two of the 35 cycles, and where the code went. Every arrow that pointed up now points down.

What uml-viewer is

uml-viewer is an open-source desktop tool by Uncle Bob (Robert Martin) that draws a codebase as a UML-like diagram you can click. It is written in Clojure and needs the Clojure CLI and Java 21 or newer.

Namespaces are components. The modules inside a namespace are the component's elements. Nesting can go as deep as your source tree does. You double-click a component to open the next level, double-click a module to see its class card, and click a function to open the source file at that line.

Color comes from CRAP and mutation scores, so a red box is code with high complexity and weak tests, or code with no metrics at all, which the README counts as the worst grade. Red arrows come from the Dependency Rule: you tell the tool which namespaces sit at which architectural level, and every dependency that points from an inner level to an outer level is drawn red.

The tool is built to run next to an agent. By default it opens a tmux window with Grok in the examined project and the two talk through a small mailbox directory. You can also drive it by hand from any agent session: regenerate the input file, press R in the viewer, and the diagram reloads.

Getting a Rust crate into it

The only parser it ships reads Clojure. That sounded like the end of the experiment, until I read what the viewer actually consumes. It never sees source code. It reads one EDN file: a list of classes with a namespace and a level, a list of edges with a from, a to and a kind, and a list of levels. That is a format any script can emit.

So I asked the agent to write a generator for Rust. It came out at about 200 lines of Python and does four things:

Every .rs file is one class. mod.rs stands for its directory and lib.rs is skipped.
Every use crate::…, super::… and self::… path, plus every inline crate::a::b path in a function body, becomes a dependency edge to the nearest enclosing module that exists as a file.
Comments are stripped first, so a path mentioned in a doc comment does not count as a dependency.
A hand-written LEVELS list gives each top-level module a rank.

The last item is the only opinion in the script:

# inner (high level) first, matching the Dependency Rule ranks
LEVELS = [
    ["domain", "error", "text", "metric_names"],
    ["storage", "security", "net", "http_client", "config",
     "quotas", "observability", "pagination"],
    ["pipeline", "escalation", "notifier", "email", "telegram",
     "whatsapp", "billing", "auth", "analytics",
     "jobs", "http_outbound", "targets"],
    ["api", "web", "mcp", "marketing", "agent", "worker", "scheduler",
     "request", "public_status", "templates", "oauth",
     "ad_hoc_dispatch", "channels"],
    ["app", "router", "bootstrap", "main", "bin"],
]

Rank 0 is vocabulary that everyone may use: domain types, error codes, text helpers. Rank 1 is infrastructure: storage, security primitives, config. Rank 2 is the services that do the work: probing, escalation, notification, billing. Rank 3 is every way into the system: HTTP handlers, HTML views, the MCP server, the marketing site. Rank 4 is assembly: the app state, the router, main.

The rule is then mechanical. An edge is a violation when the module it comes from has a smaller rank than the module it points to. Same rank is allowed. Foreign crates like axum and sqlx are ovals outside the diagram and are never compared.

The picture is only as honest as the levels list

The generator does not decide the architecture. I do, in that list. If you put api in the same group as domain, the diagram turns green and you have learned nothing. Write the list you want to be true, then let the red arrows show how far the code is from it.

What the first picture showed

The first diagram had 167 red arrows, 35 pairs of modules that imported each other, 17 modules importing api, and an app.rs of 1,087 lines.

Behind the numbers were shapes I half knew about and had never seen drawn:

storage reached up into api for error codes and dashboard read models, and into worker for heartbeat state.
api and web imported each other. The JSON side took session and token extractors, cookies and client IP from the HTML side, and the HTML side took the heartbeat read model from the JSON handlers.
quotas read accounts and organizations from storage, while storage embedded quota SQL fragments and read plan types from quotas. A cycle in both directions.
marketing and oauth imported web for three template filters.
config imported auth and billing for two enums.

None of this was visible in the review of any single pull request. Every one of those imports was reasonable on the day it was written. A diff shows one import at a time, so the sum of them never appeared in any review.

The loop

Every round had the same six steps:

Regenerate the EDN file and reload the viewer.
Pick one red arrow. Hover it to see which module pairs it bundles.
Tell the agent one thing: what must not import what, and where the shared piece should live.
The agent moves the code and runs the compiler and the tests.
Regenerate. Check that the arrow is gone and count the new ones.
Run the full suite, review the diff, commit.

The instruction in step 3 is the part that makes this work. It looks like this:

storage must not import api. The error codes and the dashboard read models
it takes from there are crate-wide vocabulary. Move the codes to error::codes
and the read models to domain::metrics, and point every reader at the new
path. No behavior changes.

That is one rule, one violation, and one place to put the result. The agent does not have to guess what "cleaner" means. It has a named arrow to remove, and the next diagram says whether it did.

I ran this with Claude Code, but nothing in the loop depends on it. Codex, Cursor, Grok, or any agent that can edit files and run a test suite gets the same instruction and the same picture afterwards.

Six rounds took the crate from 167 red arrows to 109:

Round	Red arrows I pointed at	Where the code went
1	`storage` into `api` and `worker`, `config` into `auth`	error codes to `error::codes`, 12 read models to `domain::metrics`, provider enums to `domain::credential`
2	`storage` into `auth` and `api`	token hashing, HMAC and SHA-256 helpers to `security`, redaction scrubbers to `security::redaction`
3	`api` and `web` into each other	extractors, cookies, client IP and host resolution to a new `request` module that imports neither
4	`storage` and `quotas` into each other	SQL fragments to `storage::count_sql`, plan types to `domain::quota`
5	`marketing`, `oauth` and `api` into `web`	template filters and formatters to `templates`, `app.rs` split into `config::boot`, `observability::readiness`, `targets::status` and `net`
6	`public_status` and `web` into `api`, `security` into `http_client`	page envelopes to `pagination`, `Cipher::from_config` into `security`

And the totals:

Six commits, 315 files, same test suite before and after. Violations down 35 percent, cycles down 91 percent.

	Before	After
Dependency Rule violations	167	109
Module pairs importing each other	35	3
Modules importing `api`	17	4
Lines in `app.rs`	1,087	642
Files touched		315
Behavior changes		0

I regenerated the graph at every one of the six commits to see what each round removed:

Cycles and api importers per round. The first two rounds did most of the work on the red count; rounds three to six were about the cycles.

The shape surprised me. The red arrow count stopped moving after round two. Rounds three to six barely touched it, but they took the cycles from 23 down to 3, because a cycle between two modules at the same level is not a Dependency Rule violation at all. The rule catches arrows that point up and says nothing about two modules on the same level that import each other, so you need both counts.

Not every move survived. In a later round the agent moved the health-check paths into observability, and a coupling test on the marketing module said no, because that module is only allowed to reach a short list of leaf modules. Another move put the strict JSON parser under request, then had to come back because that parser reads the OpenAPI document. Both reverts took minutes, because the diagram and the test said so before the commit did.

Why I stopped at 91

After the six commits, 109 red arrows were left. Most pointed into app, but ten did not: a sampler and a silence job that lived under observability but reached into the scheduler and the notifier, two background jobs reading public_status, the HTTP metrics layer, and the rate-limit middleware in quotas reading the app state. That middleware was also the third of the three cycles. One more round moved each of those into the module that owns it and left 91 red arrows and 2 cycles.

Every one of the 91 is the same shape. Handlers read the app state, and the app state is built from the modules those handlers live in. app imports api and web to mount them; api and web import app to get the state.

Fixing it is possible. Split the state into per-handler sub-states and hand each handler only its slice. I counted what that would touch: about 90 files of plumbing that make the code harder to read, not easier. I stopped because I could name every remaining arrow, and at that point the diagram had told me everything it was going to.

Stop when every arrow has a name

Drive the count down until every arrow that is left has a name and a reason, then stop. The 91 arrows left in my crate are all "this handler reads the app state", and a diagram that shows them is more honest than one that hides them behind 90 files of indirection.

Keeping it fixed

Two things stop the graph from drifting back.

The marketing module has a test that is an allow-list: the exact set of leaf modules it may import. A new reach into the app fails the build. It used to be a deny-list, which only catches the mistakes you already thought of.

The other is a habit, not a test. Regressions arrive with feature commits, not with refactors. After every push I regenerate the graph and read the list of red arrows that do not point into app. If the list is empty, the feature stayed in its layer. If it is not, the offending edge is one instruction away from gone, before the next feature builds on it.

Why this beats "refactor the codebase"

Two months ago I wrote about mapping this codebase for humans and AI agents. The lesson then was that a model is good at shape and bad at numbers, so I had to check every count it produced by hand.

This is the same lesson, applied to refactoring. Here the agent never counts anything; the generator does. The agent gets one rule with one violation and a picture that says pass or fail after every change. It is the same reason a failing test is a better instruction than a paragraph of requirements: the acceptance criterion exists before the work starts, and it is not the agent that judges it.

What I did not get is the color. CRAP and mutation scores come from Uncle Bob's Clojure tooling, and there is no Rust equivalent wired in yet. The README is clear that a box with no metrics is painted as the worst grade, so the fill on my boxes means nothing until coverage and mutation numbers exist for Rust. The arrows alone were worth the setup.

Do it yourself

Install the Clojure CLI and Java 21 or newer, then clone uml-viewer.
Write a generator for your language. One class per module, one dependency edge per import, output as EDN with :hierarchical true. The generated examples/uml-viewer.edn in the repo is a complete example of the format. In Rust, imports are use paths; in TypeScript they are import statements; in Python, import and from. A regex over each file is enough to start.
Write the :levels list by hand, inner layer first. Describe the architecture you want; the red arrows will show where the code differs.
Run the viewer against your file. On a fresh start it waits for its companion agent; the --restart flag restores the last view without spawning one, which is what you want when you drive it from your own agent session.
Pick one red arrow. Write one instruction that names the two modules and the new home. Regenerate. Repeat.

Key takeaways

uml-viewer reads a plain EDN file, not source code. A 200-line script gets any language in.

The levels list is the only opinion in the input, so write the architecture you want and let the red show the distance.

Give the agent one rule, one violation and one destination per instruction. The next diagram is the acceptance test.

Rust will not tell you two modules import each other. Build the graph and count the mutual pairs yourself.

Stop when every remaining red arrow has a name. Mine are all "handler reads app state", and that is fine.

Turn the result into an allow-list test, and regenerate the graph after every feature push, because that is when regressions arrive.

The six commits are public in the Uptimepage repository if you want to read what moved and why.

Sources

Robert C. Martin, uml-viewer on GitHub. The README documents the EDN format, the :levels rule and the companion agent mailbox.
Robert C. Martin, The Clean Architecture, The Clean Code Blog, August 2012. The Dependency Rule.
Uptimepage, source repository, AGPL. Commits bda59342 through cee12c44 are the six rounds described here.

I build Uptimepage because I wanted uptime checks that say why something failed instead of "transport error": HTTP with per-phase timing, TLS and domain expiry, ping and TCP, heartbeats for background jobs, and browser flows, from several regions. It is AGPL-3.0 open source, which is why the six commits above are public and you can check every number in this post against them.

Now the question for the comments: what draws the module graph for your language, and does it show the cycles or only the layer violations? My rounds three to six only moved the cycle count, and I would have missed that with a tool that colors arrows but never counts pairs.

Real-Time AI Interview Help: Keeping Technical Answers Relevant as Questions Change

Tue, 15 Sep 2026 08:22:45 +0000

A technical interview rarely follows a straight line. You suggest a solution, the interviewer changes a constraint, and suddenly the most important part of your original answer no longer applies. The challenge is keeping your explanation aligned with the problem as it develops.

That is the angle I find most interesting about real-time AI interview help. Generating a plausible answer is only one part of the task. An assistant also needs to recognize what changed, preserve the details that still matter, and avoid pulling the conversation back toward an outdated solution.

I’m Peter, founder of Linkjob AI. In this article, I’ll use a hypothetical backend interview to explain how I think about useful assistance during technical conversations. These are design principles, not performance claims about a particular model.

Start with a concrete problem

Imagine an interviewer asks you to design a service that accepts uploaded files and processes them in the background. You propose an API, object storage, a job queue, and workers.

That is a reasonable starting point, but listing components does not explain the decisions behind them. A clearer answer connects each component to a requirement:

Object storage holds the uploaded files.
The queue separates upload requests from processing work.
Workers process jobs independently of the request lifecycle.
A status endpoint lets users check progress.

An assistant should help the candidate articulate those relationships. Producing a longer list of technologies would add little unless the interviewer has introduced requirements that justify them.

Treat a follow-up as a change to the problem

Now the interviewer asks: “What happens if a worker finishes processing a file but crashes before acknowledging the message?”

The conversation has moved from architecture to failure handling. Repeating the original system diagram would miss the question.

A useful response should focus on the sequence: processing completed, acknowledgment did not, and the queue may deliver the job again. The candidate can then discuss how to make repeated processing safe.

A short prompt might be enough:

“Explain redelivery, identify the operation that must be idempotent, and describe how completion is recorded.”

That leaves room for the candidate to reason through the actual design. It also avoids claiming that adding a queue automatically solves reliability.

If the interviewer then asks about two workers receiving the same job concurrently, the assistant should update its guidance again. Checking whether a job is complete is not sufficient if both workers can pass that check before either records completion.

Keep requirements separate from assumptions

Technical answers become confusing when assumptions silently turn into facts. In the file-processing example, the interviewer might never have specified file size, processing duration, traffic volume, or whether jobs must finish in order.

An assistant could fill those gaps with invented numbers and produce an impressive-looking architecture. The candidate would then be defending decisions built on requirements that nobody agreed to.

A better approach is to identify which unknown changes the design. For example:

Large files may affect upload strategy and storage access.
Long-running jobs may require attention to visibility timeouts or lease renewal.
Strict ordering may constrain how work is distributed.
External side effects may make retries harder to handle safely.

The candidate does not need to ask every possible question. They need to surface the assumptions that materially affect their answer.

Suggest the next reasoning step

During a live conversation, a full essay can compete with the interviewer for attention. I would rather see an assistant offer a compact sequence the candidate can develop in their own words.

For the worker-crash question, that sequence could be:

State that the job might be delivered again.
Explain what duplicate execution would do.
Describe a mechanism for preventing or safely handling duplicate effects.
Acknowledge the remaining failure case.

The fourth step matters. Suppose the worker calls an external billing service and then crashes before recording success locally. A local “completed” flag alone cannot establish whether the charge happened. The answer needs to consider the external service’s behavior, such as whether it supports idempotency keys.

This is the kind of distinction that makes technical assistance useful: helping someone identify the next issue they should reason about.

Do not manufacture the candidate’s experience

There is also a difference between explaining a design and claiming to have implemented it. An assistant may be able to describe a distributed job-processing system without any evidence that the candidate has built one.

If the interviewer asks, “Have you handled this in production?”, the response should reflect the candidate’s actual background. It should not invent a migration, an incident, or a performance improvement.

A candidate can explain how they would approach an unfamiliar problem while being clear about their experience. Assistance should preserve that distinction, particularly when using a résumé or project notes as context.

My interest in real-time AI interview help centers on making relevant information easier to use in the moment. That includes recognizing when the available information does not support a claim.

Evaluate how well the assistant follows the conversation

Testing an interview assistant with isolated questions misses an important part of the experience. A response can be correct for the opening question and irrelevant by the third follow-up.

For the file-processing example, I would test a sequence that introduces duplicate delivery, concurrent workers, external side effects, and then a simpler traffic requirement that removes the need for some earlier complexity.

The questions I would ask during evaluation are practical:

Does the assistant notice each changed constraint?
Does it abandon advice that no longer applies?
Does it distinguish a proposed design from a proven guarantee?
Can the candidate understand the suggestion while continuing to listen?
Does the output encourage clarification when necessary?

These checks examine whether the assistant supports an evolving explanation.

Keep the candidate in control

A real-time AI interview assistant should leave room to ignore a suggestion, pause assistance, or request clarification. Interviewers may also have specific rules about external tools, so candidates should establish what assistance is permitted before using it in an assessed conversation.

The design goal I care about is straightforward: help someone explain their reasoning clearly as the question changes. In a technical interview, that means tracking constraints, discussing failure cases, and knowing which assumptions still need to be checked.

A useful suggestion gives the candidate something they can understand, evaluate, and explain. That is a more meaningful standard than how much text an assistant can generate.

Disclosure: I’m the founder of Linkjob AI. The product links in this article lead to my company’s website.

Booting ARM64 Linux in my Rust KVM Hypervisor: Debugging a Memory Issue

Sun, 06 Sep 2026 18:39:35 +0000

Motivation

The time has come: the big test. I got some simple binaries running on my Rust
KVM hypervisor (at this point I am thinking about giving it a proper name). The
next thing has to be Linux. For this series I will show how to actually boot
Linux on my Rust KVM hypervisor on ARM64 CPU (I am using Raspberry Pi 4 which
supports hardware accelerated virtualization).

Cross-compiling Linux kernel

In one of my previous articles I have described how to
set up the cross-compilation environment with a simple Dockerfile. We can
reuse it to cross-compile the Linux kernel for ARM64 (after adding some Linux
build dependencies like bc, flex, python3, libssl-dev and bison):

docker run --network host -it --rm \
       -e CROSS_COMPILE=aarch64-linux-gnu- -e ARCH=arm64 \
       -v "$(pwd)":"/home/docker/kvm-rust" -w "/home/docker/kvm-rust/linux-7.1.3" \
       arm-rust-build /bin/bash -c 'make defconfig && make'

The resulting ARM64 Linux kernel image (Image) should be located in
arch/arm64/boot/ directory.

ARM64 boot protocol

The full ARM64 Linux boot protocol is quite straightforward and can be found in
the Linux kernel documentation:

https://www.kernel.org/doc/html/v7.1/arch/arm64/booting.html

We simply need to parse the 64-byte header and verify the magic field (it should
equal 0x644d5241). I will just highlight a few important points. We should
read image_size and text_offset fields and note that the image should be
loaded at an offset of text_offset bytes at a 2 MiB boundary. For our kernel,
as I checked, text_offset is zero and image_size is the amount of memory the
decompressed kernel image occupies in RAM, which is not necessarily the same as
the file size as reported by stat. Finally, the registers x1, x2 and x3
must be set to zero, while x0 must point to the device tree blob (dtb) in
the RAM. So, for this I chose to load the dtb at address 0x40000000 and
Image itself on the first 2 MiB boundary:

const LOAD_ADDR: u64 = 0x40_000_000;
const KERN_OFFS: u64 = 0x200_000;

Obtaining the device tree blob

So, for ARM64 we do need a device tree blob that will tell the Linux kernel on
what kind of hardware it is running on. And this step is quite easy, we will
can simply reuse our command for running QEMU, with one small addition (i.e.
adding dumpdtb option to -M virt argument):

#!/bin/sh

KERNEL=/home/pi/Image
qemu-system-aarch64                                             \
        -M virt,dumpdtb=virt.dtb                                \
        -smp 1                                                  \
        -enable-kvm                                             \
        -cpu host                                               \
        -kernel ${KERNEL}                                       \
        -append "console=ttyAMA0"                               \
        -serial stdio                                           \
        -nographic                                              \
        -nodefaults

This gives us a consistent snapshot of the hardware we were running on (in this
case it's emulated hardware for QEMU virt machine). We could (and we should)
edit and prune it for our own purposes, but let's leave that for another time.

Implementing boot in Rust

So, loading the kernel in Rust is quite straightforward:

impl VM {
    pub fn arch_load_linux(&mut self, vcpu: &mut VCPU, args: &Args) -> io::Result<()> {
        let header = get_linux_header(&args.binary)?;
        let linux_mem_idx = self.add_mem_region((KERN_OFFS + header.image_size) as usize, LOAD_ADDR)?;

        match &args.dtb {
            Some(dtb_path) => { self.load_file_to_memory(linux_mem_idx, &dtb_path, 0x0)?; },
            None => { return Err(io::Error::other("Device tree blob not specified.")); }
        }

        self.load_file_to_memory(linux_mem_idx, &args.binary, KERN_OFFS)?;

        vcpu.set_one_reg("x0", LOAD_ADDR)?;
        vcpu.set_one_reg("x1", 0x0)?;
        vcpu.set_one_reg("x2", 0x0)?;
        vcpu.set_one_reg("x3", 0x0)?;

        vcpu.set_one_reg("pc", LOAD_ADDR + KERN_OFFS)?;
    }
}

Note that the get_linux_header() function was omitted here as it only parses
the aforementioned ARM64 Linux image headers in a boilerplate manner.

Function not implemented

As soon as I tried to run my hypervisor with Linux Image, I ran into an issue:

pi@raspberrypi:~ $ ./rust -d virt Image 
x0 = 0x0
pc = 0x1000
magic 0x644d5241, image_size: 38731776, file_size: 38040064, text_offset: 0x0
Loading "virt" to 0x0...
Loading "Image" to 0x200000...
x0 = 0x40000000
pc = 0x40200000
x0 = 0xfffffbfffda35000
pc = 0xffffd902f0f8a42c
Error: Os { code: 38, kind: Unsupported, message: "Function not implemented" }

Trying strace

First thing I turned to was strace; it's easy to use and gets some nice
information really fast:

ioctl(5, KVM_ARM_SET_DEVICE_ADDR or KVM_GET_ONE_REG, 0x7fcee98768) = 0
x0 = 0x40000000
ioctl(5, KVM_ARM_SET_DEVICE_ADDR or KVM_GET_ONE_REG, 0x7fcee98768) = 0
pc = 0x40200000
ioctl(5, KVM_RUN, 0)                    = -1 ENOSYS (Function not implemented)
ioctl(5, KVM_ARM_SET_DEVICE_ADDR or KVM_GET_ONE_REG, 0x7fcee988a8) = 0
x0 = 0xfffffbfffda35000
ioctl(5, KVM_ARM_SET_DEVICE_ADDR or KVM_GET_ONE_REG, 0x7fcee988a8) = 0
pc = 0xffffd902f0f8a42c
Error: Os { code: 38, kind: Unsupported, message: "Function not implemented" }
+++ exited with 1 +++

I didn't learn much other than the fact that KVM_RUN failed completely with
ENOSYS error code. My first attempt to find out what was going on was
searching the ARM64 KVM source code for ENOSYS in Linux repository. It didn't
take long, as it appeared only in one place which was a promising lead:

https://elixir.bootlin.com/linux/v7.1.3/source/arch/arm64/kvm/mmio.c#L188

As we can see, the function in question is io_mem_abort() in the Linux kernel.
Now, strace got us that far, but this is a Linux kernel function and not much
we can find out more about it with strace (which traces only system calls like
ioctl). I haven't even considered GDB because this was the host kernel,
and this would require a really complex time-consuming setup. Luckily, there was
a much easier way.

Analyzing IO memory abort with eBPF

The first thing I did was to list kprobes in bpftrace and see if I can hook
into the io_mem_abort at all via eBPF:

pi@raspberrypi:~ $ sudo bpftrace -l 'kprobe:io_mem_abort'
kprobe:io_mem_abort

I was in luck, so I could at least confirm that this function was the one
causing the issue, so I attached a kretprobe to it, ran bpftrace alongside
my hypervisor and got the confirmation I needed:

pi@raspberrypi:~ $ sudo bpftrace -e '
>    kretprobe:io_mem_abort {
>        printf("ret=%d\n", retval);
>    }'
Attaching 1 probe...
ret=-38

Finding out what really happened

Now that I have confirmed io_mem_abort was the culprit, it was time to
analyze it more deeply. I attached an additional kprobe so I could print both
the return value and the fault_ipa argument:

int io_mem_abort(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa)

The updated bpftrace script printed the fault address.

pi@raspberrypi:~ $ sudo bpftrace -e '
>    kretprobe:io_mem_abort {
>        printf("ret=%d\n", retval);
>    }
>    kprobe:io_mem_abort {
>        printf("0x%llx\n", arg1);
>    }'
Attaching 2 probes...
0x47fff000
ret=-38

So the result surprised me as I initially suspected it was trying to access some
device that I haven't created; however, I realized that this was actually just
beyond the memory I had allocated for the Linux kernel to use. So it clicked: I probably hadn't allocated enough guest memory. Instead of allocating just
header.image_size + KERN_OFFS I increased it to 1 GiB:

let linux_mem_idx = self.add_mem_region(1024 * 1024 * 1024, LOAD_ADDR)?;

Conclusion

With that adjustment, the ENOSYS error disappeared. The guest was clearly
executing code, although I still wasn't seeing any output on the serial console.
That meant the memory issue was solved, but there was at least one more thing
left before we would actually see Linux boot.

Can Rust Make Unsafe AI Agent Actions Unrepresentable?

Sun, 06 Sep 2026 14:56:00 +0000

I went looking for a better runtime check and found a different way to think about the problem.

I have spent a lot of time recently thinking about what an AI agent should be allowed to do.

Reading data is one thing. Writing durable state is another. Sending an email, approving a refund, changing a production configuration, or deleting a record moves farther along the same spectrum.

The usual answer is to put a guardrail in front of the dangerous operation.

That makes sense, and I have built examples that work exactly this way.

Then I started looking at how I might implement the same kind of boundary in Rust.

Rust asked a more interesting question:

Why does the dangerous function accept an unchecked action in the first place?

Runtime Checks Are Still Checks

The complete runnable example is on GitHub if you want to make the compiler angry yourself.

Consider a simplified AI agent that can write information into durable memory.

The agent proposes a write:

struct ProposedWrite {
    key: String,
    value: String,
    authority: String,
}

Somewhere else in the application we have a function that persists it:

fn persist(write: ProposedWrite) {
    // write to durable storage
}

Obviously we should validate the write first.

fn validate(write: &ProposedWrite) -> bool {
    // validate provenance
    // evaluate authority
    // apply policy
    true
}

Then our application does this:

if validate(&write) {
    persist(write);
}

Perfectly reasonable.

It also means persist() will happily accept a ProposedWrite that has never been validated. We are relying on every caller to remember the protocol.

That is not necessarily a problem in a small example. In a large agentic system with multiple tools, services, developers, and execution paths, it becomes a much more interesting assumption.

What happens when somebody adds this six months later?

persist(write);

The compiler sees nothing wrong. The type is correct. The application is wrong.

Make the State Part of the Type

Rust gives us another option. Instead of treating "proposed" and "approved" as metadata attached to the same object, we can make them different types.

struct ProposedWrite {
    key: String,
    value: String,
    authority: String,
}

struct AdmittedWrite {
    key: String,
    value: String,
    authority: Authority,
}

Now persistence accepts only the second one:

fn persist(write: AdmittedWrite) {
    // write to durable storage
}

This seemingly small change alters the boundary. An agent can produce a ProposedWrite. It cannot produce an AdmittedWrite directly, provided we control how that type is constructed. Something trusted has to perform the transition.

The persistence layer no longer asks:

Has somebody remembered to validate this?

Its API says:

Give me something that has already crossed the admission boundary.

That is a much stronger contract.

This has a name in Rust circles: the typestate pattern, encoding a value's state in its type so that only valid transitions type-check. It is the type-level cousin of a principle Alexis King named "parse, don't validate". Instead of checking a value and handing the same type onward, hoping every later caller re-checks, you transform it into a new type whose very existence proves the check already happened. The check is not something you remember to run. It is something the type system will not let you skip.

The Compiler Becomes Part of the Boundary

The phrase "provided we control how that type is constructed" is doing all the work in the previous section, so let us actually deliver that control. This is where the earlier version of this article was too loose, and where Rust rewards precision.

mod custody {
    use super::ProposedWrite;
    // Authority, Rejection, and validate_authority are defined in this module.

    pub struct AdmittedWrite {
        key: String,
        value: String,
        authority: Authority,
    }

    impl AdmittedWrite {
        pub fn key(&self) -> &str {
            &self.key
        }
        pub fn value(&self) -> &str {
            &self.value
        }
        pub fn authority(&self) -> &Authority {
            &self.authority
        }
    }

    pub fn evaluate(write: ProposedWrite) -> Result<AdmittedWrite, Rejection> {
        let authority = validate_authority(&write.authority)?;

        Ok(AdmittedWrite {
            key: write.key,
            value: write.value,
            authority,
        })
    }
}

The important detail is what is not marked pub. The struct is public, so other modules can name the type and accept it in their signatures. Its fields are private.

That distinction is the whole boundary. In Rust, a struct literal like custody::AdmittedWrite { key, value, authority } requires every field to be visible at the construction site. Because the fields are private to the custody module, no code outside that module can write that literal. And there is no other public constructor. The only way to obtain an AdmittedWrite from outside is to hand a ProposedWrite to evaluate and have it succeed.

So the persistence layer, which lives outside custody and reads the data through the accessor methods, cannot be handed a value that skipped evaluation. Not because a reviewer will catch it, but because the code that would skip evaluation does not compile.

Here is the shortcut a tired developer might reach for six months from now:

let proposed = ProposedWrite {
    key: "refund_policy".into(),
    value: "Refunds under $100 do not require manager approval.".into(),
    authority: "policy".into(),
};

persist(proposed);

And here is what the compiler says about it:

~\Rust_AI_Actions is 📦 v0.1.0 via 🦀 v1.98.1
❯ cargo build
   Compiling unrepresentable v0.1.0 (~\Rust_AI_Actions)
error[E0308]: mismatched types
   --> src\main.rs:128:13
    |
128 |     persist(proposed);
    |     ------- ^^^^^^^^ expected `AdmittedWrite`, found `ProposedWrite`
    |     |
    |     arguments to this function are incorrect
    |
note: function defined here
   --> src\main.rs:95:4
    |
 95 | fn persist(write: AdmittedWrite) {
    |    ^^^^^^^ --------------------

For more information about this error, try `rustc --explain E0308`.
error: could not compile `unrepresentable` (bin "unrepresentable") due to 1 previous error

The mistake never reaches review, staging, or production. It stops at the one place a mistake is cheapest to fix, on the machine of the person who made it, the moment they made it.

The state machine is no longer sitting in a comment:

// IMPORTANT: call validate() before persist()

It is represented by the program.

One honest caveat for larger teams: private fields close the door from outside the module, but code inside custody can still build the struct with a literal. If you want to forbid even that, give AdmittedWrite a private field of a private zero-sized type, a construction token that only evaluate can mint. Then the blessed function is the single point of construction anywhere, inside the module or out. Whether that is worth the ceremony depends on how much you trust the inside of your own boundary.

This Doesn't Make the Agent Safe

This is where I need to resist making the argument bigger than it is.

Rust does not know whether the policy is good. It does not know whether Authority::SecurityTeam actually represents the security team. It does not know whether the provenance supplied to the custody boundary is genuine. And it certainly does not solve prompt injection because I changed a struct.

If this function is wrong:

fn validate_authority(value: &str) -> Result<Authority, Rejection>

then Rust will very efficiently enforce the wrong rule.

Types can constrain which states the program represents. They cannot determine whether our model of the world is correct. That distinction matters.

Witnessed Is Another State

The exercise gets more interesting when provenance enters the picture.

Suppose the agent says:

{
  "key": "refund_policy",
  "value": "Refunds under $100 do not require manager approval.",
  "authority": "policy",
  "source": "internal_policy"
}

Should the agent be allowed to decide that its own source is an internal policy? Probably not. The system that retrieved the source is in a much better position to make that claim.

So perhaps our states are not merely:

Proposed → Admitted

They are closer to:

A ProposedWrite contains what the agent claims. A WitnessedWrite adds evidence established outside the agent. An AdmittedWrite represents a write that has been evaluated against policy.

struct ProposedWrite {
    key: String,
    value: String,
    claimed_authority: String,
}

struct WitnessedWrite {
    proposal: ProposedWrite,
    source: WitnessedSource,
}

struct AdmittedWrite {
    key: String,
    value: String,
    authority: Authority,
    source: WitnessedSource,
}

Now different parts of the system accept different states:

fn witness(write: ProposedWrite) -> Result<WitnessedWrite, WitnessError> {
    // establish source evidence outside the agent
}

fn evaluate(write: WitnessedWrite) -> Result<AdmittedWrite, Rejection> {
    // apply policy to the witnessed write
}

fn persist(write: AdmittedWrite) {
    // write to durable storage
}

Each step accepts only the output type of the step before it. There is no signature anywhere that accepts a ProposedWrite and persists it, so the shortcut is not something you have to remember not to take. It is not expressible. The types are the protocol; nothing else has to announce it.

What Rust Changed for Me

I started this experiment thinking about how to implement an AI safety boundary in another language. The more interesting lesson was that Rust made me reconsider where the boundary should live.

In many systems, we encode state like this:

{
  "status": "approved"
}

Then every downstream consumer has to inspect status and behave correctly.

Rust encourages another question:

If these states permit fundamentally different operations, why are they represented by the same type?

That question matters for AI agents because agentic systems cross consequential boundaries constantly.

A model proposes a tool call. A runtime authorizes it. A tool executes it. A result becomes memory. Memory later becomes context. Context influences another action.

At each transition, we can either carry another flag saying what happened, or change what the next component is capable of accepting. Those are not equivalent designs.

Invalid States Versus Invalid Reality

There is an important limit here.

Suppose a write was legitimately admitted yesterday because Alice had authority to approve it. Alice leaves the company today. The AdmittedWrite type does not magically expire. Likewise, a policy can be superseded, a credential revoked, or evidence later discovered to be wrong.

The compiler can enforce:

This value crossed the required transition.

It cannot establish:

Everything that justified that transition remains true forever.

That still requires runtime governance, lifecycle management, revocation, and revalidation.

So I would not claim Rust makes unsafe AI actions impossible. What it can do is make certain classes of architecturally invalid transitions harder to express accidentally.

That is narrower. It is also much more believable.

Could I Do This in Another Language?

Of course.

You can model state transitions in Go, Java, TypeScript, Python, C#, or plenty of other languages. You can build wrapper types, sealed classes, discriminated unions, private constructors, capability objects, and carefully designed APIs. Rust does not own the idea, and the typestate pattern predates it.

What I found useful is that Rust keeps pushing the design conversation in this direction. Ownership asks who controls a value. Visibility asks who can construct it. The type system asks what operations are valid for it. Result makes failure part of the function signature.

None of those concepts exists specifically for AI safety. Together, though, they provide an unusually direct vocabulary for designing agent boundaries, and the defaults nudge you toward making the boundary structural instead of remembered.

The Bigger Lesson

A lot of AI safety architecture is necessarily dynamic. Policies change. Users have different authority. Tools expose different capabilities. Context changes what an action means. We are never going to compile all of that uncertainty away.

But not every invariant is dynamic.

If an unwitnessed write must never be persisted, perhaps persist() should not accept unwitnessed writes. If an unevaluated tool call must never execute, perhaps execute() should not accept unevaluated tool calls. If a rejected action must never cross a boundary, perhaps rejection should produce a state for which crossing that boundary is not an available operation.

That is what Rust changed in how I think about this problem.

I started by asking:

How do I check that the agent is allowed to do this?

Rust made me ask:

Why does this function accept something the agent is not allowed to do?

That is a much better question.

How to Create and Distribute Rust Packages with Cargo

Tue, 01 Sep 2026 18:21:40 +0000

Hey everyone! I put together a quick guide on building and releasing Rust crates using Cargo.

Highlights covered:

Distinguishing between binary crates (src/main.rs) and library crates (src/lib.rs).

Manifest management in Cargo.toml.

Pre-release checklist: testing, docs, and authentication on crates.io.

Read the complete article on SunnyWriteUps. Feel free to share your thoughts or tips on your Rust package workflow!

https://sunnyblog.apexmansunny.workers.dev/posts/how-to-create-your-rust-package/

When WebAssembly Is Worth The Build Step In A Web Game

Mon, 31 Aug 2026 11:53:33 +0000

Most of the wasm material floating around is about the toolchain: install Emscripten, pass these flags, ship the .wasm. The decision that comes before it gets far less attention, and for this crowd it is the more interesting one, so here is the version I would have wanted to read first.

What A Module Actually Is

WebAssembly is a binary instruction format, not a language you write. You compile C, C++ or Rust into a .wasm file, the browser validates it and turns it into real machine code, and that compile step is where the speed comes from. There is no warm-up period while a JIT works out which functions are hot, so a wasm function runs at full speed from its first call.

What surprises people is how little a module can do on its own. It has no DOM, no canvas, no network, no audio. It gets a block of linear memory, a set of exported functions JavaScript can call, and a set of imported functions JavaScript hands to it. Everything it does to the outside world crosses that boundary. Sandbox with a bridge is the mental model, and nearly every wasm performance surprise traces back to it.

The Boundary Is The Whole Design

Calling into a wasm function is fast. It is not free. Cross the boundary a few thousand times a frame with tiny calls and you can spend the entire advantage you compiled for.

The pattern that works is a coarse boundary. Pass data in batches, do as much work as possible inside the module before returning, and share state through linear memory instead of copying values back and forth. Rather than asking the module for each entity's position one call at a time, let it write every position into a shared buffer and read the whole buffer once on the JavaScript side.

This is also why a wasm rewrite of ordinary game logic often benchmarks slower than the JavaScript it replaced. The logic got faster and the plumbing got more expensive, and the plumbing won.

Two Toolchains Cover Almost Everything

For C and C++ it is Emscripten, which does much more than emit a binary. It ships a C library, maps OpenGL onto WebGL, simulates a filesystem, and generates the glue that loads the module and wires it to the page. Point it at an existing SDL or OpenGL game and with the right flags you get something playable without rewriting the source.

For Rust the support lives in the language ecosystem. wasm32-unknown-unknown is a first-class target, wasm-bindgen generates bindings with real types instead of raw numbers, and macroquad or Bevy sit on top depending on how ambitious the project is. Rust is what most people reach for when they are starting fresh rather than porting.

C#, Go and AssemblyScript all target wasm too, but for games those two paths have the deepest ecosystems and the most shipped titles behind them.

Your Engine Already Made This Choice

If you export from Unity or Godot, you are already shipping wasm. Both compile their C++ runtime to a module and drive a WebGL canvas from it, which explains the complaint everyone has: a trivial Unity web build is many megabytes because the engine runtime is the download, not your game. The load time you see is the browser compiling that binary, and streaming compilation is what hides most of it behind the download.

We put the longer version together on our own site, WebAssembly for web games, which goes through the toolchains, how each engine handles its web export, and the things wasm still cannot do without JavaScript holding its hand.

Reach for wasm when you have a hot loop running thousands of times per frame, a large existing native codebase, or a workload like physics or a software rasterizer where fixed types and explicit memory genuinely pay. Skip it when the honest answer is that your game logic runs fine in JavaScript. A build step you did not need is a real cost, and the speedup you imagined is often a rounding error.

Why GitWand is Rust, not Electron: what native performance actually buys a Git client

Fri, 21 Aug 2026 15:52:00 +0000

It's been a while since the last post here (v3.5, July 10). Since then, three releases have shipped: v3.6.0 (a handful of workflow polish items), v3.6.6 (a pure dev-loop and CI performance pass, no product surface), and v3.7.0 (Commit Review, a local AI pass over your staged changes). A straight changelog recap would cover all three, but it would miss the thing that actually connects them.

Every one of those releases is a direct consequence of a decision made on day one: build on Tauri 2 and Rust instead of Electron, and keep the actual conflict-resolution logic in a deliberately portable, zero-dependency TypeScript package instead of folding it into the Rust backend. That split sounds like an implementation detail. It isn't. It's the reason a 5-repo Launchpad refresh takes 30ms instead of most of a second, the reason a 30-second timeout bug fixed a few days ago (#166) had a one-line fix instead of a rewrite, and the reason the new Commit Review feature never has to touch a GitWand server. This post is about that decision, with real numbers pulled straight from the codebase instead of the usual "native is faster, trust us."

The bet, and who else made it

GitWand ships as roughly an 8 MB installed binary. GitKraken, built on Electron, ships in the ~150 MB class: a full bundled Chromium plus a full bundled Node.js runtime, running alongside each other, for an app that mostly renders text and spawns git. GitHub Desktop is the same story. The gap shows up at startup (GitWand is sub-1 second) and it shows up on large repos, where an Electron app's single JS heap has to hold and garbage-collect everything the UI touches.

To be fair to the field: GitWand isn't the only one making this bet anymore. GitButler is Tauri + Rust too. GitComet skips the webview entirely and goes pure Rust with GPUI. Strand is Tauri + React. Being native, in 2026, is closer to table stakes for a new Git client than a differentiator; RelaGit is the interesting counter-example, choosing Electron even with a SolidJS frontend for its fine-grained reactivity. The real question isn't "who avoided Electron," it's what each of these tools actually does with the native layer once it has it. That's the part worth walking through.

What Rust buys: fearless concurrency, with a number attached

Rust's ownership model turns a whole class of concurrency bugs into compile errors instead of production incidents. That's a nice sentence to put in a pitch deck. Here's what it looks like in a file that ships in the app today.

GitWand's multi-repo dashboard (Launchpad) needs the status of every repo in your workspace: branch, ahead/behind, modified-file count. The naive version spawns four git subprocesses per repo, one after another. GitWand's Rust backend does two things instead: each repo's status comes from a single in-process libgit2 call rather than four subprocess spawns, and every repo in the workspace gets processed in parallel via rayon's into_par_iter(), one core per repo instead of one core doing all of them in sequence. The comment sitting next to that code in the source states the measured result plainly: a 5-repo workspace listing went from about 750ms wall-clock to about 30ms in the typical case. That's not a marketing number, it's a comment next to a #[tauri::command] that ships today.

The same pattern shows up all over the backend, because once the tool exists, using it costs nothing extra: PR +/- stat enrichment across GitHub, GitLab, Azure DevOps, and Bitbucket all fan out per-PR diff computation across cores instead of looping; submodule status checks across a repo's submodules run in parallel; per-branch top-author lookups for the branch picker do too. A few days ago, a regression brought back a 30-second gh_list_prs timeout: the per-PR stats loop had gone back to running one git diff --numstat at a time. The fix wasn't a rewrite or a new thread pool, it was swapping a for loop for .par_iter_mut(). None of these needed hand-rolled synchronization or a bespoke worker pool. The parallel version and the serial version are the same number of lines, and if two of those parallel closures ever raced on the same mutable data, that's a compile error, not a bug report from a user with a corrupted status badge.

What Rust doesn't need to buy: the conflict engine stays TypeScript, on purpose

Here's the part a "Rust is fast" pitch usually skips: the actual pattern-classification engine, the code that decides same_change versus one_side_change versus complex and merges JSON, YAML, and lockfiles key by key, is not written in Rust. It lives in @gitwand/core, and that package is deliberately zero-Node, browser-safe TypeScript.

That's not a compromise, it's a second, equally deliberate architectural line. Classifying a few hundred lines of diff text against nine pattern rules finishes in single-digit milliseconds no matter what language it's written in; Rust's raw throughput wouldn't move that number in any way a user could feel. What Rust would have cost is portability: because the engine has no Node dependency, the exact same code runs inside the desktop app, in the CLI (npx @gitwand/cli resolve), inside the MCP server that AI agents call, and could run directly in a browser tomorrow with no changes. The rule GitWand actually follows isn't "everything is Rust." It's narrower and, we think, more useful: Rust owns the boundary with the operating system (spawning git, walking the filesystem, talking to forge APIs, the whole app shell), and the deterministic algorithmic core stays a small, dependency-light, portable package that isn't tied to any one runtime.

The unglamorous release: v3.6.6

v3.6.6 shipped no new product surface at all, just a dev-loop and CI performance pass, and it's the release that best shows the bet compounding on itself. Because the entire backend is one Rust crate, the same profiling discipline used to make the app fast also applies to making the build fast: a [profile.dev] tuned for lighter debug info cut a post-edit cargo build from about 21 seconds to about 6.5. On the CI side, a desktop job that produced three fully-signed release bundles on every single push to main, bundles nobody downloaded, was replaced with a fast rust-check job (fmt, clippy, check, test) that finally runs the app's 217 Rust unit tests in CI for the first time, saving roughly 230 billed CI minutes per push. On the TypeScript side, Vitest's global jsdom environment (needed by maybe a fifth of the test files, paid by all of them) was scoped down per-file, cutting environment setup time from about 96 seconds to about 17 across the suite.

Worth mentioning: before touching any of it, every claim in the plan for this work got re-measured against the actual codebase, not assumed. Several turned out to be wrong. "Any edit recompiles and relinks everything" wasn't true, cargo check after a real edit came back in under 2 seconds; the real cost was in the codegen-and-link step of a full cargo build. "The old jsdom setup costs 100 to 300ms per file" was an underestimate by roughly 10x once actually profiled. That's the same discipline the conflict engine's confidence scores are built on: don't ship a number you haven't checked.

Commit Review, and why local-first is the real differentiator

v3.7.0's headline feature, Commit Review, runs an AI pass over your staged changes right in the Changes panel before you commit, with inline findings and a "Fix with agent" action that hands them to a terminal AI session (Claude Code, opencode, or Codex) already running against your repo. What matters about it architecturally is what it doesn't do: it never calls a GitWand-run server. The review runs against whichever AI CLI you already have installed, on your machine, over your own already-open agent session.

That's the same "zero mandatory cloud" value the conflict engine already embodies, applied to review instead of merge. It's also the sharper contrast with where a chunk of the market is headed: tools positioning code-intelligence features as a metered cloud API (Greptile's own "Genius API" is priced at $0.45 per request) are betting that developers will pay per call for something GitWand ships as a local, free, opt-in feature built on tools you already run. Being local-first for conflict resolution and local-first for review isn't two separate decisions, it's the same one, applied twice.

The small stuff: v3.6.0

Between the two, v3.6.0 shipped things that don't need an architecture section to justify: a one-click "Update branch" prompt when you check out something behind its upstream, and a non-blocking handoff when an interactive rebase hits a conflict (issue #128), so the conflict editor no longer traps you behind a modal. These are the kind of small interactions that only feel good because there's no Electron process and no garbage-collector pause standing between the click and the result. The architecture doesn't just enable the big numbers, it's also why the boring stuff feels instant.

Where the bet goes next

The roadmap's next stop, v3.9.0 (Live Repo), applies the same philosophy one layer up: a Rust notify crate watcher replaces the current 2-second status poll with real filesystem events, more read paths move to in-process libgit2, and separately, the portable TypeScript core's diff/parse hot path gets moved off the main thread into a Web Worker. Same rule, two languages: Rust handles OS-level events natively and cheaply, and the portable core gets pushed off the render thread the way any JavaScript does it. One architecture, one non-negotiable: the UI thread never waits on either of them.

The market read

The Git GUI space is splitting into two camps: Electron incumbents charging a subscription for cloud AI features (GitKraken at $8/mo, with its advanced and AI features gated behind an account), and a newer wave of native, mostly free, often open-source tools built on Rust or a comparable native stack (GitButler, GitComet, Strand, and GitWand, with RelaGit staying on Electron by choice). Being native is fast becoming the price of entry for that second group, not what sets any one of them apart. What actually differentiates GitWand inside it is what the native layer gets used for: a conflict-resolution engine that classifies instead of guesses, multi-repo workflows that get genuinely (and measurably) faster because of how the backend is built, and an AI layer that never assumes your code should leave your machine.

Download GitWand for macOS, Linux, or Windows from GitHub Releases, read the full changelog, or see how the numbers stack up against specific tools on the compare pages.

Bun Rust Rewrite: 64 Claude Agents, 535k Lines of Zig

Sat, 01 Aug 2026 14:39:27 +0000

A couple of weeks ago, in the post about pnpm rewriting its install engine in Rust, I called Bun "the odd one out: it's written in Zig, not Rust." That line is wrong, and the awkward part is that it was wrong the week I wrote it. Jarred Sumner announced Bun's Zig-to-Rust rewrite on July 8; the pnpm post went up July 15, so the claim was stale at publication. I've since corrected that post so it acknowledges the rewrite and links here, which means if you click through you won't find the quoted line anymore. Bun is rewriting itself from Zig to Rust, and the rewrite is in canary for Bun 1.4, the first release where the runtime, bundler, and test runner are all Rust. Jarred Sumner didn't do it by hand. He ran about 50 Claude Code workflows over 11 days, a few at a time, and let them loose on 535,496 lines of Zig.

This is the part of the JavaScript toolchain story I want to get right, because it's the one most likely to be flattened into a press release. The headline is real: Bun 1.4 fixes 128 bugs, ships a roughly 20% smaller binary, and runs a bit faster. The headline is also doing some work the underlying numbers don't fully support, and there is a genuine, public argument between Sumner and Andrew Kelley, the creator of Zig, over what actually happened. Both of them are partly right.

TL;DR

Bun ported its entire runtime from Zig to Rust in 11 days with 64 Claude agents, and the rewrite is in canary for Bun 1.4, not stable.
The numbers that hold up are the leak fixes. An in-process build loop that ballooned to 6.7 GB in 1.3.14 levels off at 609 MB in 1.4, and the use-after-free and double-free bugs that dogged the Zig runtime are the class of failure safe Rust's Drop and borrow checker turn into compile errors.
What the announcement overclaims is the framing. The 2 to 5% speed gain and the roughly 20% smaller binary come from cross-language LTO, ICU trimming, and linker work that could have shipped in Zig, not from Rust as a language.
Don't move to 1.4 in production yet. Stable is still v1.3.14; run bun upgrade --canary on a throwaway branch, hammer the 2,000-build leak scenario against whatever you ship, and watch memory hold flat before you trust it.

What the Bun Rust rewrite actually is

Bun 1.4 is a mechanical port of the whole runtime from Zig to Rust, done one .zig file at a time, holding behavior as close to 1.3.14 as the port allows. The last Zig release is Bun v1.3.14, tagged May 13, 2026. The first Rust release is v1.4.0, currently in canary: run bun upgrade --canary to pull it.

"Mechanical port" is the load-bearing phrase. This is not a greenfield rewrite where Claude gets to redesign Bun. The Zig and Rust shown side by side in Sumner's announcement are deliberately near-identical: same function names, same scoping, same comments. The plan is to refactor toward idiomatic Rust after 1.4 ships, not during the port. That choice is what makes an 11-day timeline plausible, and it's what keeps the existing TypeScript test suite meaningful as the source of truth. The tests were never tied to the implementation language, so a faithful port should pass the same assertions.

The scope is everything Bun is: the JavaScript, TypeScript, and CSS transpiler, minifier, and bundler; the npm-compatible package manager; the Jest-like test runner; the Node.js API surface (fs, net, tls, http, http2, node:zlib); and the HTTP server. About 780,000 lines of Rust now sit where 535,496 lines of Zig used to. The "million lines" Kelley keeps citing is the diff: the announcement counts the port as adding a million-plus lines of new Rust. The 780,000 is the resulting codebase, the denominator for the unsafe-percentage figure. Both numbers are right; one is what went in, the other is what's there.

Why rewrite Bun from Zig to Rust?

The honest case isn't speed. It's stability. Sumner opens the announcement with a list of bugs fixed in 1.3.14, bugs the Zig runtime shipped until that release, and it's grim reading for a runtime people run in production:

heap-use-after-free in node:zlib when .reset() fires during an async .write() still in flight on the threadpool
use-after-free in node:http2 when a reentrant JS callback triggers a hashmap rehash that invalidates internal stream pointers
use-after-free in UDPSocket.send() where a valueOf() callback detaches the ArrayBuffer between payload capture and the actual send
a tlsSocket.setSession() leak of about 6.5 KB per call, every call, from a missing SSL_SESSION_free
fs.watch() watchers never collected after .close(), pinned as GC roots by a reference-count underflow

These are the canonical memory-safety failures: use-after-free, double-free, leaks on error paths. In safe Rust most of them become compile errors, and the rest get Drop running cleanup exactly once when a value goes out of scope. The argument is that compiler errors are a better feedback loop than a style guide enforced by review, and for this class of bug, lifetimes of garbage-collected values mixed with manually managed ones, it's hard to argue with. Zig's defer/errdefer model puts cleanup at every call site and trusts the author to get it right every time. Bun, at half a million lines, did not get it right every time, and nobody reasonable expected it to.

This is the part I'd already half-gotten wrong. When I lumped Bun into the "everything's Rust now" trend in the pnpm v12 post and flagged it as the Zig holdout, I treated language choice as a settled, one-way decision. Sumner says explicitly that it used to be one, and that AI-assisted porting is what made it reversible at this scale. Take that claim seriously even if you're skeptical of the rest.

How do you port 535,000 lines in 11 days?

You don't ask one agent to "rewrite Bun." Sumner's post is most useful as an engineering breakdown of the loops, and the loops are ordinary:

// Pseudocode, not real code:
let task;
while ((task = todoList.pop())) {
  const result = task();
  const feedback = await Promise.all([review(result), review(result)]);
  await apply(feedback, result);
}

Each workflow is one implementer plus two or more adversarial reviewers plus a fixer, with the reviewers running in separate context windows so the Claude that wrote the code isn't the Claude grading it. At peak, four workflows ran at once, each with 16 Claudes, for about 64 concurrent agents across four git worktrees. They started on May 3 and merged May 14, across 6,778 commits and 1,448 .zig files moved to .rs.

The part worth copying is the prep work. Before any porting, Sumner spent about three hours with Claude producing a PORTING.md that mapped Zig patterns to Rust patterns, then ran a workflow that traced every struct field's lifetime across the codebase and wrote the results to a LIFETIMES.tsv. Three files were ported first as a trial, reviewed against the guides, then the full sweep ran. The false starts are instructive: two Claudes ran git stash and git reset --hard over each other's work, so the workflow rule became "no git except committing one specific file, no cargo, no slow commands." Claude also tried to "fix" compile errors by stubbing functions out, so a rule was added: if a workaround needs a paragraph-long comment, the code is wrong, fix the code.

The token bill for all of this is roughly 5.9 billion uncached input tokens, 690 million output tokens, and 72 billion cached reads, about $165,000 at API pricing, on what was then a pre-release build of Claude Fable 5, the Mythos-class model Anthropic shipped a month later. That's the part the search summaries will quote and the part that most obscures what happened. The money bought an engineered loop with human supervision, not a prompt and a prayer.

Do the numbers hold up?

Here's the skeptic's reading. The announcement lists 128 bugs fixed versus 1.3.14, 19 known regressions (all fixed), zero tests skipped or deleted, and 1,386,826 expect() calls on Debian. The leak numbers are the most convincing: every in-process Bun.build() leaked about 3 MB in 1.3.14, so 2,000 builds ran the process to 6,745 MB. In 1.4 it levels off at 609 MB.

Builds	Bun 1.3.14	Bun 1.4 (canary)
500	1,914 MB	526 MB
1,000	3,506 MB	586 MB
1,500	5,097 MB	608 MB
2,000	6,745 MB	609 MB

If you run a dev server that bundles on every request, that's the difference between an OOM restart at 3 AM and a stable process. A previous Zig attempt at this fix never merged because, Sumner says, the lack of Drop made it too risky to trust. That's the strongest single argument for the rewrite.

The speed and size numbers want more scrutiny. Bun 1.4 is 2 to 5% faster on HTTP and app workloads: Bun.serve goes 169.6k to 177.7k req/s (up 4.8%), and next build goes 13.62s to 13.03s. The binary shrinks roughly 20%, from 94 to 76 MB on Windows and 88 to 70 MB on Linux. But the announcement itself attributes the throughput to cross-language LTO between C/C++ and Rust, and attributes the binary shrink partly to ICU trimming and Identical Code Folding. None of that is Rust-only. The initial Rust-only binary reduction came from dropping Zig's heavy comptime use; the rest came from linker work that could have shipped in Zig.

That's where Andrew Kelley's pushback lands hardest. The short version: a meaningful slice of the "Bun is better in Rust" win was engineering work Bun could have shipped without changing languages, and the post doesn't separate the two clearly.

The Sumner vs. Kelley fight, both sides

Andrew Kelley's response is not a calm post, and it's not framed as a technical critique. It opens on Sumner's "beginner energy" and builds to Kelley calling him "a stinky manager. Poor communication, unrealistic expectations, low empathy, no experience." But buried in the personal framing are four technical objections that hold up on their own:

The test-suite contradiction. The argument for merging a million lines of largely unreviewed code is that the test suite catches everything. But the same post opens with a long list of bugs in the Zig version the test suite didn't catch. Kelley's question: if it wasn't sufficient to catch Zig bugs, why is it sufficient to declare one million lines of unreviewed AI-authored Rust clean?
The style-guide sleight of hand. The post frames the choice as "style guide vs. language feature," which skips the main way projects actually eliminate bugs, which is putting engineers on it. Kelley points at TigerBeetle, another Zig project, as the team that did the work Bun didn't.
The fuzzing claim. The announcement implies diligent Zig-side fuzzing; Kelley says Bun told the Zig Software Foundation directly that they weren't fuzzing anything. The 24/7 coverage-guided fuzzing the post now brags about is new and Rust-side.
The omitted build speed. The Zig compiler, roughly 600,000 lines, builds clean in 16s and recompiles in 90ms with incremental enabled, Kelley writes. Bun's post doesn't give post-rewrite build numbers, and for a mechanical port of a codebase this size that's a conspicuous gap.

Each of those is fair. The first is the sharpest. Sumner's "0 tests deleted, 1.4M expect() calls, adversarial review, human in the loop" defense addresses process, not the logical gap. A suite that missed a setSession leak bleeding 6.5 KB a call for years is not a suite you can then cite as proof a million lines are correct. The suite is necessary and it caught a lot, the 128 fixes prove that, but "the test suite caught regressions" and "the test suite proves correctness" are different claims, and the announcement slides between them.

Sumner's side holds up too, and it's the one that matters more for users. The Zig bug list isn't marketing; those are real crashes and real leaks in a runtime with 22 million monthly CLI downloads that now backs Claude Code. The Drop argument is the real argument, and Kelley doesn't really answer it. He shifts to "you should have put in engineering hours instead," which is true and also not a refutation of the fact that Zig's manual-lifetime model was bleeding. You can fault how Bun got to Rust and still accept that being in Rust, with Drop and a borrow checker and Miri, materially lowers the chance of the next setSession leak.

Where I land: the rewrite is net good for Bun's users, the AI-authorship process is more disciplined than its critics assume, and the announcement overclaims by bundling genuinely Rust-driven gains (leaks, use-after-free) with engineering work that didn't need a rewrite (LTO, ICU, comptime cleanup). Read the two posts as correcting each other and you get closer to the truth than either one alone.

What about the `unsafe` code and undefined behavior?

The thing people will actually worry about with AI-authored Rust is unsafe. The announcement gives the number: about 4% of the Rust sits in unsafe blocks, roughly 13,000 unsafe keyword uses across the ~780,000-line codebase, and 78% of those blocks are a single line, a pointer from JavaScriptCore or one call into a C library. That's lower than I expected, and it's the floor, not a ceiling. Bun embeds JSC, uWebSockets, BoringSSL, and SQLite, so unsafe will never hit zero. The plan is to push it down as the mechanical port gets refactored toward idiomatic Rust.

The worry has a concrete footprint. Issue #30719, filed May 14 by AwesomeQubic, is titled "PathString::slice dangling reference UB - add Miri to CI": a core::slice::from_raw_parts call constructing a dangling &[u8]. Miri was not in Bun's CI when the bug was filed; the reporter ran it locally to find it. The issue is closed, fixed by #30876 ("Add cargo-miri support and fix HiveArray aliasing UB", merged May 17), with #30728 an earlier attempt still open on GitHub. The reporter's parting line, "Please consider not vibe coding rust as AIs are not good at writing Rust and also hire a real rust dev," is the tweet-sized version of the whole debate. The bug is real, the kind of UB that turns up in any large unsafe-touching codebase, and Miri is exactly the tool meant to catch it. What's worth being precise about is who supplied the mechanism: the reporter ran it on a build Bun hadn't instrumented yet, not Bun's CI. The announcement now says Miri runs on a growing chunk of code in CI, which is the state after #30876 landed. So it's still the system working, not failing, but it's the community catching the bug and Bun turning the catch into infrastructure. And it's the proof that "AI wrote it" doesn't free you from reviewing unsafe.

Where this leaves the JavaScript toolchain

The pattern is established enough that I'd stop calling it a trend and start calling it the default. TypeScript went to Go for build speed. pnpm is going to Rust for install speed. Astro 7 rebuilt its compiler in Rust. Deno is Rust from the ground up. Bun just went to Rust for stability. The reasons differ, but the move is the same: the JavaScript world's foundation is being rewritten in languages with a real memory model, and the engines running the JS itself, V8 and JavaScriptCore with their tiered JIT pipelines, are the one layer staying put.

What's new in the Bun story is who's doing it and who owns it. Anthropic acquired Bun in December 2025. Claude Code ships as a Bun executable to millions of users, and the rewrite ran on Claude Fable 5 a month before Anthropic shipped it. Prisma launched a Prisma Compute beta on the Rust rewrite; Alexey Orlenko is quoted in the announcement on the memory leaks and the connection pool "that couldn't recover after a VM was paused and resumed," handled "perfectly" by the Rust build. That's the financial and structural context the announcement's lead leaves out: this is Anthropic making its own tooling's runtime safer, with Anthropic's model. The independence question is real, even if the code stays MIT.

The subtler shift is what "an engineer can do in a year" means. Sumner's closing line, that one engineer can do a lot more today than a year ago, is the part that's broader than Bun. If a faithful port of half a million lines, supervised, costs $165k and 11 days, the bottleneck for a class of large rewrites stops being the typing and starts being taste: knowing which port to do, how to scope it, and how to review the output. Bun's loop is a template people will copy, and the teams that internalize it first get to attempt things that used to require freezing development for a year.

Should you run Bun 1.4 today?

If you're on Bun in production on 1.3.x, the answer is "not yet, but soon, and watch the bug list." As I write this, the latest stable tag in Bun's release feed is still v1.3.14 from May 13. The only newer entries are consolidation-step-N-green canary tags from Bun's automation, not a stable v1.4.0.

To try it on a throwaway branch:

bun upgrade --canary

The honest test isn't the benchmark. Run the leaky-build scenario from the announcement, 2,000 Bun.build() calls in one process, against whatever you ship, and watch memory. If it holds flat the way the announcement's table shows, the rewrite did the thing you actually care about. If it doesn't, file the bug, because the whole premise is that this class of issue is now catchable and fixable instead of permanent.

I wrote that Bun was the Zig holdout two and a half weeks ago, and the line was stale the week it went up: Sumner had already announced the rewrite. The less comfortable lesson, for anyone writing about this stack, is that "X is written in Y" is now a claim with a short half-life, short enough that being a week behind the news is enough to be wrong. Bun's Rust rewrite is the cleanest example I've seen of AI-assisted porting at a scale that used to take a year, and the open questions about it, how much of the win was the language versus the engineering hours, whether machine-authored unsafe can be trusted, who owns the runtime your agent runs on, are the ones the rest of the JavaScript toolchain is going to answer next, one rewrite at a time.

Porting my Rust KVM hypervisor to ARM64: Running the binary

Wed, 22 Jul 2026 18:02:25 +0000

Recap

In the previous article we ported the register handling
of my Rust KVM hypervisor from x86 to ARM64. By tracing QEMU with strace and
inspecting the Linux headers, we discovered that ARM64 accesses CPU state
through the KVM_GET_ONE_REG and KVM_SET_ONE_REG interface and reconstructed
the register id encoding in Rust.

At that point the hypervisor compiled successfully, but trying to access even
the program counter resulted in an ENOEXEC error. In this article we'll
continue the reverse-engineering process, initialize the vCPU correctly, and
finally execute our first ARM64 guest.

Second roadblock: ENOEXEC when getting / setting PC

I was rather optimistic this would work out of the box, but even after trying
to do something as simple as reading the PC register I hit a snag:

ioctl(5<anon_inode:kvm-vcpu:0>, KVM_ARM_SET_DEVICE_ADDR or KVM_GET_ONE_REG, 0x7fdf2c81e8) = -1 ENOEXEC (Exec format error)
Error: Os { code: 8, kind: Uncategorized, message: "Exec format error" }

Luckily, I had a concrete lead now: error code and KVM command. So I searched
for the KVM_GET_ONE_REG in the Linux kernel code on Bootlin Elixir and found
the following code in arch/arm64/kvm/arm.c:

case KVM_SET_ONE_REG:
case KVM_GET_ONE_REG: {
struct kvm_one_reg reg;

r = -ENOEXEC;
if (unlikely(!kvm_vcpu_initialized(vcpu)))
break;

This is exactly the error we were getting and we immediately found what was
missing: vCPU initialization. Actually, if we take a closer look at the code we
will see the comment kernel developers have left us:

int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
{
int err;

/* Force users to call KVM_ARM_VCPU_INIT */
vcpu->arch.target = -1;

Initializing the vCPU

Following the lead I looked for KVM_ARM_VCPU_INIT in the original strace log
and found:

137152 ioctl(12<anon_inode:kvm-vcpu:0>, 0x4020aeae /* KVM_ARM_VCPU_INIT */, 0x7fff57e028) = 0
137156 ioctl(10<anon_inode:kvm-vcpu:0>, 0x4020aeae /* KVM_ARM_VCPU_INIT */, 0x7f82f70e68) = 0
137152 ioctl(10<anon_inode:kvm-vcpu:0>, 0x4020aeae /* KVM_ARM_VCPU_INIT */, 0x7fff57dd78) = 0
137152 ioctl(10<anon_inode:kvm-vcpu:0>, 0x4020aeae /* KVM_ARM_VCPU_INIT */, 0x7fff57e048) = 0

This wasn't really helpful: we again only see addresses, but we see no hints of
underlying structure. So let's first take a look at how KVM_ARM_VCPU_INIT is
defined in the Linux headers:

$ grep -Rn KVM_ARM_VCPU_INIT /usr/include/
/usr/include/linux/kvm.h:1622:#define KVM_ARM_VCPU_INIT   _IOW(KVMIO,  0xae, struct kvm_vcpu_init)

A quick search for struct kvm_vcpu_init revealed:

$ grep -Rn 'struct kvm_vcpu_init' /usr/include/ -A3
/usr/include/aarch64-linux-gnu/asm/kvm.h:112:struct kvm_vcpu_init {
/usr/include/aarch64-linux-gnu/asm/kvm.h-113-   __u32 target;
/usr/include/aarch64-linux-gnu/asm/kvm.h-114-   __u32 features[7];
/usr/include/aarch64-linux-gnu/asm/kvm.h-115-};

Getting struct data with Python GDB

I didn't want to spend much time understanding every target and feature flag. My
goal was simply to reproduce QEMU's behavior as quickly as possible (and I
didn't even have to understand it completely). Luckily, this structure was
really simple (basically an array of eight 32-bit integers), so I decided to
tackle this with Python GDB automation. I created a breakpoint to stop exactly
on ioctl call when we are in KVM_ARM_VCPU_INIT:

import gdb
import struct

def log(line):
    print(line)
    with open("gdb-debug.out", "a") as f:
        f.write(line + "\n")

class IoctlBreakpoint(gdb.Breakpoint):

    def stop(self):
        if x1 == 0x4020aeae: # KVM_ARM_VCPU_INIT
            ptr = int(gdb.parse_and_eval("$x2"))
            vcpu_init_mem = gdb.selected_inferior().read_memory(ptr, 32) # u32 + 7 * u32
            target_features = struct.unpack("<8I", bytes(vcpu_init_mem))
            log(f"KVM_ARM_VCPU_INIT: target={target_features[0]} features={target_features[1:]}")        
            return False

if __name__ == "__main__":
    tracer = IoctlBreakpoint("ioctl")
    gdb.execute("handle SIGUSR1 SIGUSR2 nostop noprint pass")
    gdb.execute("run")

I ran my QEMU command with GDB:

gdb -ex 'source ./trace-arm.py' ./start-qemu.sh

This has automatically yielded great results:

KVM_ARM_VCPU_INIT: target=5 features=(0, 0, 0, 0, 0, 0, 0)
KVM_ARM_VCPU_INIT: target=5 features=(12, 0, 0, 0, 0, 0, 0)
KVM_ARM_VCPU_INIT: target=5 features=(12, 0, 0, 0, 0, 0, 0)
KVM_ARM_VCPU_INIT: target=5 features=(12, 0, 0, 0, 0, 0, 0)

For now I want to try just setting target, but leave features zeroed out. I
could immediately reconstruct this in Rust:

impl VCPU {
    pub fn arm_vcpu_init(&self) -> io::Result<()> {
        let mut vcpu_init : kvm_vcpu_init = unsafe { std::mem::zeroed() };
        // KVM_ARM_VCPU_INIT: target=5 features=(0, 0, 0, 0, 0, 0, 0)
        vcpu_init.target = 5;
        let ret = unsafe { libc::ioctl(self.fd, KVM_ARM_VCPU_INIT, &mut vcpu_init) };
        if ret < 0 {
            return Err(io::Error::last_os_error());
        }
        Ok(())
    }
}

Third roadblock: No KVM exit

Soon after this I was able to get and set registers properly. The first ARM test
binary I compiled was based on working UART code from my hobby operating system
I started on Raspberry Pi. I expected at least a fault exit, if not an MMIO exit
when UART code was called. However, KVM_RUN never returned.

This is where I was stuck for a while. I considered creating a separate thread
that repeatedly queried KVM_GET_ONE_REG for PC, but from my own working
experience I knew that calling KVM ioctl calls asynchronously was a really bad
idea. I also considered adding hardware breakpoint capabilities to at least
trigger KVM_EXIT_DEBUG, but this would require more infrastructure for just
checking simple binary.

Writing a simple binary

I decided to try adding a signal handler instead. If I interrupted KVM_RUN
call with SIGINT maybe I could get registers then synchronously. I lowered
my expectations for initial binary and just created something really simple:

.section .text

.macro curr_el_to reg
mrs\reg, CurrentEL
lsr\reg, \reg, #2
and\reg, \reg, #0xf
.endm

.globl _start
_start:
mrsx0, mpidr_el1
andx0, x0, #0xFF
cbzx0, control
b .

control:
curr_el_to x0
b .

It looks like much, but this code will just load current CPU privilege level to
x0 register (being that level EL=1 is operating-system level, x0 should be
set to 0x1). The mpidr_el1 code is for checking if we are on control CPU
(this should be unnecessary here, but I added this out of paranoia caused by
real-life issues I had on ARM).

Installing signal handler

Finally, I added a signal handler:

extern "C" fn handler(_sig: libc::c_int) { }

unsafe fn install_interrupt_signal() {
    let mut sa: libc::sigaction = std::mem::zeroed();
    sa.sa_sigaction = handler as *const() as usize;
    libc::sigemptyset(&mut sa.sa_mask);
    sa.sa_flags = 0;
    libc::sigaction(libc::SIGINT, &sa, std::ptr::null_mut());
}

The handler itself does nothing. I only needed the signal to interrupt the blocking KVM_RUN call. So my final KVM loop looked like:

    unsafe { install_interrupt_signal() };

    let run = vcpu.kvm_run_mem as *mut kvm_run;

    loop {
        let ret = unsafe { libc::ioctl(vcpu.fd, KVM_RUN, 0usize) };
        if ret < 0 {
            if io::Error::last_os_error().raw_os_error() == Some(libc::EINTR) {
                vcpu.print_regs()?;
            }
            return Err(io::Error::last_os_error());
        }

        let exit_reason = unsafe { (*run).exit_reason };

Running the binary

Finally, it was time to run the gues:

pi@raspberrypi:~ $ ./rust hello-world-arm.img
x0 = 0x0
pc = 0x1000
Loading "hello-world-arm.img" to 0x1000...
^Cx0 = 0x1
pc = 0x101c
Error: Os { code: 4, kind: Interrupted, message: "Interrupted system call" }

We can see that PC did change and that x0 really is set to current CPU
privilege level: 0x1. Why it didn't work for my UART code is probably that the
guest itself requires more initialization than I though (x86 went more smoothly
and felt more plug-and-play).

Conclusion

Although x86 and ARM64 share the same KVM interface at a high level, their
userspace APIs differ significantly. Porting my hypervisor turned out to involve
much more than renaming registers: ARM64 uses per-register ids instead of a
single register structure, requires explicit vCPU initialization through
KVM_ARM_VCPU_INIT and demanded a fair amount of reverse engineering before I
could successfully execute guest code. Fortunately, the same workflow that
worked on x86 like combining strace, GDB and the Linux kernel source proved just
as effective on ARM64 (although required a bit more effort).

The code from this article can be found on my GitHub page:

https://github.com/StjepanPoljak/kvm-rust/tree/kvm-arm-part2-code

Astro 7: What's New, What's Faster, and What Breaks

Tue, 21 Jul 2026 20:52:08 +0000

Astro 7, released June 22, 2026, is a speed release. The .astro compiler got rewritten in Rust, the Markdown pipeline got rebuilt in Rust, and the bundler switched to Vite 8 with Rolldown. It is the same framework on faster infrastructure. Builds on real production Astro sites are 15 to 61 percent faster, with the biggest wins on sites where Markdown and .astro compilation dominate the build.

What Astro 7 actually is

Astro 7 is an infrastructure release. The component format, the island architecture, the routing model: all the same. What changed is what sits underneath. The official announcement calls it the speed release, and that is the honest framing. Four systems got rebuilt or swapped, five experimental features went stable, and the result is faster builds without a new mental model.

The closest parallel is TypeScript 7 rewriting its compiler in Go: keep the surface still, rebuild the engine. Astro 7 does the same, with Rust, across more layers.

Vite 8 and Rolldown

Astro 7 upgrades to Vite 8, which ships Rolldown as its bundler. Rolldown is a Rust bundler that replaces both esbuild and Rollup in the Vite pipeline, and runs 10 to 30 times faster than Rollup in benchmarks. If you had esbuild options or rollupOptions in your config, Vite 8 auto-converts them. The plugin API is the same, so existing Vite plugins keep working.

Vite is Astro's build core: dev server, dependency optimization, production bundling. A faster bundler there is a faster everything.

Astro 7's new Rust compiler

The .astro compiler was rewritten in Rust. It was Go before. The new compiler is built on oxc and Lightning CSS, and Astro reports about a 6 percent improvement in isolation. That sounds modest next to the Vite 8 jump, and it is. The compiler was already fast. But 6 percent compounds on a large build that runs thousands of .astro files.

Two behavior changes, both making the compiler stricter. It no longer auto-corrects HTML: markup is treated as-is. Unclosed tags are now errors, not warnings. If you had sloppy markup the Go compiler quietly fixed, the Rust compiler will tell you. It ships as native binaries per platform, with a WASM fallback.

Sätteri: Markdown and MDX in Rust

Sätteri is the new default Markdown and MDX pipeline. It is Rust-powered, built by Erika on pulldown-cmark and oxc, and it replaces the unified/remark/rehype stack Astro used before.

The performance is real. Astro's docs build and Cloudflare's docs build each shed over a minute. On a content-heavy site like this one, that is the win you feel.

Sätteri ships built-in support for GFM, smartypants, heading IDs, directives, math, frontmatter, and wikilinks. The plugin API works differently from unified: plugins declare the node types they care about, and Sätteri routes only those. If you need the old stack, unified() is still available via @astrojs/markdown-remark. But for most sites, Sätteri is the default and the faster path.

Queued rendering is now the default

Queued rendering was experimental in Astro 6. In Astro 7 it is stable and default. The old renderer walked the component tree recursively. Queued rendering uses a queue-based approach instead, and Astro reports roughly 2.4x faster rendering on expression-dense pages. You do not configure this. It is just on. The queuedRendering experimental flag is gone.

Advanced routing with src/fetch.ts

Advanced routing gives you a src/fetch.ts entrypoint with full control over the request pipeline. It uses the standard fetch handler pattern, the same Request in, Response out shape you see in Cloudflare Workers and WinterCG runtimes. It is Hono-compatible, so Hono middleware drops in directly.

The interesting part is composability. Individual Astro features can be composed as middleware: auth before your actions, a logging layer before everything, all in one file that owns the request lifecycle. src/fetch.ts is a reserved filename, so rename any existing file with that name.

Route caching and CDN providers

Route caching is now stable. You configure it with routeRules specifying maxAge and swr per route. Astro also ships experimental CDN cache providers for Netlify, Vercel, and Cloudflare, so cached responses live at the edge instead of in your origin's memory.

For a docs site or a blog, this is straightforward. Set a maxAge on your content routes, a shorter one with swr for pages that update occasionally, and the framework handles invalidation. The CDN providers mean the cache survives deploys on those platforms, which is the part that was missing before.

What breaks when upgrading to Astro 7

The migration guide lists the breaking changes. Here is what to expect:

Rust compiler is stricter. Unclosed tags error. No more HTML auto-correction. Fix your markup.
Sätteri is the default Markdown processor. If you depend on a specific remark or rehype plugin, install @astrojs/markdown-remark to keep the unified pipeline.
compressHTML defaults to jsx instead of true. Update your config if you relied on the old default.
Experimental flags removed: rustCompiler, queuedRendering, advancedRouting, cache, and logger. Their behaviors are now standard or gone.
@astrojs/db is removed. If you used the experimental database integration, you need a replacement.
src/fetch.ts is a reserved filename. Rename any existing file with that name.
Container renderer imports changed to the /container-renderer entrypoint.

There are also two AI-oriented additions: a background dev server mode for agents, and JSON logging output. Both are aimed at tooling that drives Astro programmatically.

Should you upgrade to Astro 7?

Yes, and soon if your site is content-heavy. The build speedup alone is worth it. The migration is mostly mechanical: remove the dead experimental flags, check your Markdown plugins, fix any markup the new compiler rejects.

Do the upgrade in an isolated git worktree so your main checkout keeps building while you shake out the Sätteri differences and the stricter compiler.

The broader story is the one I keep writing about. The JavaScript ecosystem is rewriting its foundations in native code. TypeScript went to Go, pnpm went to Rust, Vite swapped its bundler for Rolldown. And now Astro has rewritten its compiler, its Markdown pipeline, and its rendering engine in Rust, all in one release. The tools you use every day are getting faster underneath you, and the surface you write against is staying still. That is the best kind of progress.

Why I chose Rust over Go for an uptime monitor

Mon, 20 Jul 2026 14:44:36 +0000

I build Uptimepage, an open-source uptime monitor and status page written in Rust. People ask why Rust and not Go, since Go is the usual pick for this kind of network service. Here is the honest answer. It is not that Rust wins everywhere. It is that one part of this job made the choice for me.

The product is one promise: tell you fast and honestly when your site is slow or down. The numbers I show you, like your p99 response time, have to be clean. If my own code adds random delay, I blur the exact signal you pay for. So the runtime under the prober matters more here than it would for a normal app.

Garbage collection shows up in the tail

Go has a garbage collector. It is fast, and most apps never feel it. But it still has to do work to free memory, and that work can land inside the millisecond timings I report. Run tens of thousands of checks at once and a pause at the wrong moment lifts a p99 number. At that point I am measuring my own runtime, not your server.

Rust has no garbage collector. Memory is freed at a point I can see in the code. There is no background pause I did not write. For a tool that sells timing, that control is worth the extra work Rust asks for.

What that looks like in memory

Here are some real numbers, with a warning attached. They come from a load test on a developer laptop, not a production server. I use them to catch a slowdown between two versions of my code, not to plan capacity. A real server does better, a small box does worse. Treat them as a floor.

In one run, a single machine held 50,000 checks in flight and peaked at 933 MiB of memory. That is under one gigabyte for fifty thousand live checks. My running server uses about 42 MiB while watching its monitors, and it sits quiet when there is nothing to do. That kind of density is normal for a service with no garbage collector, and it means one small box covers a lot of monitors. I go deeper on the prober and the throughput numbers in the build story.

The bug that never compiles

Speed is only half of it. The other half is a class of bug that Rust refuses to build.

Picture many workers writing to one shared map of results at the same time. In Go this compiles and runs. Sometimes it is fine. Sometimes two goroutines write at once, you get corrupt data or a crash, and it only happens under load, which is the worst time to find out:

// compiles fine, races at runtime
results := map[string]int{}
for _, check := range checks {
    go func(c Check) {
        results[c.ID] = c.Latency // data race
    }(check)
}

Go gives you good tools for this. There is a race detector, a sync.Mutex, and channels. But remembering to reach for them is on you.

In Rust the same concurrent write does not compile. The compiler stops you until the shared map is wrapped in a lock:

// will not compile unless the shared map is a Mutex
let results = Mutex::new(HashMap::new());
std::thread::scope(|s| {
    for check in checks {
        let results = &results;
        s.spawn(move || {
            // each worker locks only for its own insert
            results.lock().unwrap().insert(check.id, check.latency);
        });
    }
});

Each worker holds the lock only for its own insert, so the writes stay safe without blocking the others for long. For code that runs day and night across many machines, "the compiler will not let you ship the race" removes a whole set of late-night bugs before they exist.

Where Go is the better pick

None of this makes Go a bad choice. Often it is the better one. Go is faster to learn. It builds in seconds. Its standard library for network services is excellent, and a new engineer can be useful in days. If I were building a normal web service, or something I had to ship this week, Go would be on the table and might win.

Rust asks more from you first. The compiler argues with you. The build is slower. You spend time on things Go would just handle. I take that trade because this job is narrow and it rewards tight control over memory and timing.

The compiler as a safety net

Heavy code review is how large teams catch data races and memory bugs. Rust gives you a lot of that for free. The compiler turns those mistakes into build errors, so they never reach production and never wake anyone up. Every change gets a strong, automatic check before it ships, which is a big part of why I trust the service to run unattended.

It is also why Uptimepage is open source and self-hostable. You can read the code, run it on your own hardware, and export your data whenever you want, so you are never locked into a single vendor. You can start from GitHub.

The honest version

The honest version is not "Rust beats Go." It is "for a tool that lives or dies by timing and runs at high concurrency, Rust fit better." Pick the language for the job in front of you. Mine happened to be a job that Rust is very good at.

If you have shipped a heavily concurrent network service, did you reach for Go or Rust, and did the GC ever actually show up in your tail latencies? Curious where people land when the timing is the product.

pnpm v12 Is Being Rewritten in Rust

Sat, 18 Jul 2026 22:37:54 +0000

Every package manager has a moment where you just wait. You add one dependency, or you pull a branch and run install, and there's a beat where the terminal sits there doing file I/O while you tab away. pnpm already made that beat shorter than most, staying ahead of npm and Yarn on most install scenarios thanks to its content-addressable store and hard-linked node_modules. But it was still a Node.js program, paying the Node startup cost and pushing tens of thousands of file operations through a single JavaScript runtime.

That is what changes in pnpm v12: it rewrites the install engine, the part that fetches and links packages, in Rust, and leaves the CLI, lockfile, and node_modules layout untouched. It's the same move TypeScript just made with Go, aimed at a different bottleneck. And like that one, the interesting detail is how little is supposed to change for you.

What pnpm v12 actually is

pnpm v12 is the same pnpm you already use, ported to Rust. pnpm has always been TypeScript running on Node.js, and the Rust work lives under the codename pacquet, whose repo describes it in one line: "The official pnpm rewrite in Rust." The key word there is rewrite, not reimagining. It's a port. The flags, defaults, error codes, lockfile format, and node_modules layout are all meant to match pnpm exactly.

Zoltan Kochan, pnpm's creator, put it plainly when the v12 plan came together: "v12 should behave the same as v11, the biggest change will be the full rewrite to Rust." That's the frame to hold onto. This is a performance release wearing a major version number, not a redesign of how pnpm works.

pacquet started as a separate experimental repo and moved into the main pnpm monorepo in May 2026, so the TypeScript and Rust versions now evolve side by side. The eventual goal is for the Rust engine to be what you run.

Why did pnpm choose Rust?

The motivation isn't abstract. Two things were capping how fast a Node-based pnpm could go.

The first is startup cost. Every invocation booted a Node.js runtime before it did any work. pnpm v12 ships as native per-platform binaries with no Node.js launcher, so a command pays no runtime bootstrap cost at all. On a fast, cached install where the actual work is milliseconds, that bootstrap was a real fraction of the wall-clock time.

The second is the file I/O itself. A cold install on a large monorepo is really a lot of filesystem operations: fetching tarballs, unpacking them into the store, and hard-linking thousands of files into place. Running all of that through a single JavaScript thread leaves throughput on the table. Rust handles that fan-out natively, without the runtime sitting in the middle of every syscall.

There's a softer reason too, and it's the same one that's driving the broader trend: the tooling to do this kind of rewrite has gotten dramatically better, and AI-assisted porting made a 1:1 translation of a large codebase far more tractable than it would have been a few years ago.

How much faster is pnpm v12?

pnpm publishes a benchmark suite comparing v11 against the Rust engine directly. As of this writing (July 2026), here's what it shows:

Scenario	pnpm v11	pnpm (Rust)
Everything warm (repeat install)	381 ms	12 ms
node_modules present, no cache	460 ms	40 ms
Clean install (nothing cached)	6.5 s	2.2 s

The warm case is everything already in place, so the install just re-verifies the tree, which is what happens on every CI job. That drops from 381ms to 12ms, roughly a 30x difference, and it's the number you'll feel most, because it runs constantly. The colder cases improve less dramatically but still meaningfully: a fully clean install with nothing cached goes from 6.5s to 2.2s, about 3x.

Treat these as pnpm's own numbers on pnpm's own hardware, and expect your mileage to vary with disk speed and project size. But the direction is not subtle, and the team's claim is that the Rust engine is faster than v11 in every scenario they measured.

It's already rolling out, incrementally

Here's the part that's easy to get wrong: pnpm v12 isn't a big-bang release. The Rust engine has been landing piece by piece through the v11 line, opt-in first.

pnpm 11.2 let you opt into pacquet as the install backend for materialization only. Rust did the fetch and link, while pnpm still resolved dependencies and wrote the lockfile.
pnpm 11.7 went further: a standard install could be delegated to pacquet end-to-end, with resolution, lockfile write, and linking all in a single Rust pass.
pnpm 11.10 added pnpm self-update next-12, so you can pull the v12 line and try the Rust binary directly.

In v12, the Rust engine for fetching and linking becomes the default rather than an opt-in. The official roadmap then works outward from there: first the headless frozen-lockfile installer, then full dependency resolution (add, update, remove, catalogs, overrides, peers), and finally the remaining commands like run, store, and publish. So even after v12 ships, parts of pnpm will still run through the TypeScript implementation for a while. The lockfile is what lets the two halves cooperate.

What changes for you?

The honest answer is: not much, on purpose. Same CLI, same lockfile, same node_modules. The whole point of a 1:1 port is that your muscle memory and your CI scripts keep working.

The breaking changes slated for v12 are deliberately small: removing some old migration code for git-hosted tarballs, rejecting unscoped auth settings, and deprecating the $ syntax in overrides in favor of catalogs. If none of those describe your setup, a v12 upgrade should be close to a no-op.

Will the upgrade break your CI?

Probably not, but the one thing worth checking is feature parity during the transition. Because the Rust engine is being ported incrementally, some newer or niche capabilities may lag the TypeScript version at any given moment. If your install depends on something specific (a particular linker mode, lockfile-verification settings, or pnpm deploy), test the bump in an isolated git worktree before you flip it on in CI, the same sandbox I used while exploring SolidJS 2.0's async data model. This is moving quickly, so the right list of gaps is whatever the roadmap says on the day you upgrade.

The bigger picture

pnpm is not doing this alone. The last few years have quietly rebuilt the JavaScript toolchain's foundations in native languages: Biome for linting and formatting, Oxc for parsing and transforming, Rolldown and Turbopack for bundling. TypeScript's compiler went to Go. Deno is Rust from the ground up. (Bun is the odd one out: it's written in Zig, not Rust, so don't lump it into the "everything's Rust now" story.)

There's a competitive angle underneath it, too. Bun and Deno both ship as single fast binaries with package management built in, and a chunk of their appeal is exactly that no-runtime-bootstrap speed. A native-binary pnpm with no Node launcher closes that gap while keeping the thing that makes pnpm pnpm: the strict, content-addressable store and the non-flat node_modules that catches phantom dependencies.

Should you upgrade to pnpm v12 yet?

If you're waiting for a stable release: keep waiting, but not for long. As of mid-July 2026, v12 is in alpha. The latest tag is v12.0.0-alpha.12, and the stable line is still v11.13.0. This is not something to point production CI at today.

But if you want to feel the difference now, you already can. Run pnpm self-update next-12 on a branch, or opt into the pacquet backend on your current v11, and watch a warm install that used to take a beat return before you've finished reading the command. The type of release this is, where the language underneath gets an order of magnitude faster while everything on top stays exactly where you left it, is a rare and good one. pnpm was already the fast package manager. It's about to stop being one you ever wait on.

I built a programming language because C++ almost killed my drone

Fri, 17 Jul 2026 14:10:22 +0000

I'm 17. Last year I was writing drone firmware in C++ and Python. One afternoon my ultrasonic sensor returned null during a landing sequence. The code didn't check for it. The drone tried to divide by zero, the motor controller panicked, and my DJI Tello-sized quadcopter dropped from 3 meters onto concrete.

That was a $400 lesson in why runtime errors in robotics aren't just bugs. They're physics problems.

So I built a programming language.

The problem nobody talks about

When you're writing a web app and something crashes, you reload the page. When you're writing drone code and something crashes, the drone crashes. Into whatever's below it. Usually something expensive or fragile or alive.

C++ and Python dominate robotics. Both assume you'll handle edge cases yourself. Sensor returns null? That's your problem. Timing deadline missed? Good luck. No fallback path for when the lidar fails over water? Figure it out.

ROS (Robot Operating System) helps with some of this, but it's an entire infrastructure layer. What I wanted was something simpler: a language where the compiler catches the stuff that kills robots before you ever compile it.

Here's what I mean. In C++, you write:

float distance = sensor.read();
float speed = target_distance / distance;
motor.set(speed);

If distance is 0, you get a division by zero. If sensor.read() returns NaN, you get NaN propagation through your motor controller. The compiler doesn't care. It'll happily compile this and let your drone fall out of the sky.

In Fabric (the language I built), you'd write:

sensor distance: Sensor<f32, ±0.02>
actuator motor: Motor

loop {
    let speed = target_distance / distance
    motor.write(speed)
}

If you try to use distance without checking that the sensor returned a valid reading, the compiler throws an error. If distance could be zero and you didn't add a fallback path, the compiler throws an error. If your sensor has an uncertainty bound of ±0.02 and you're doing math that could amplify that uncertainty past a safety threshold, the compiler throws an error.

The compiler knows about physics. Not perfectly, but enough to catch the stuff that actually kills robots.

How it works under the hood

I wrote the whole thing in Rust. About 4,500 lines across 8 crates. Here's the pipeline:

.fab source → Lexer → Parser → AST → Type Checker → Fallback Graph → IPET Timing → CodeGen → .py / .c

Each step catches a different category of mistakes.

The type system tracks sensor uncertainty as a first-class concept. When you declare Sensor<f32, ±0.02>, that ±0.02 isn't decoration. It propagates through every math operation. Add two sensors with ±0.02 uncertainty and the result has ±0.04. Multiply by a constant and the uncertainty scales accordingly. If the final result's uncertainty exceeds what's safe for your actuator, the compiler catches it.

The fallback graph checks that every sensor has a plan B. Not just declared, but actually reachable. If your fallback function A calls fallback function B, and B calls A, that's a cycle. The compiler detects it. If sensor X has no fallback at all, the compiler refuses to build.

IPET timing analysis uses an Integer Linear Program to prove worst-case execution time at compile time. It builds a control flow graph, assigns cycle costs to each instruction (ARM Cortex-M4 model: ALU = 1 cycle, float division = 14 cycles, sensor read = 2 cycles), and solves for the maximum possible execution time across all code paths including loops.

I didn't come up with this technique. It's from Li & Malik's 1994 paper on implicit path enumeration. But I implemented it in Rust using a pure-Rust ILP solver called good_lp with the microlp backend. No external solver dependency. No C FFI. Just Rust solving linear programs.

The solver maximizes sum(cost_i * x_i) subject to flow conservation constraints. If the solver fails (which happens on really complex control flow), the system falls back to a conservative estimate instead of crashing.

The two-target code generation

Fabric compiles to two backends: Python for Webots simulation, and C for ARM Cortex-M hardware.

Python output generates a Webots controller class with sensor initialization, motor setup, fallback state tracking, and all the logic. You can test your drone code in simulation before touching hardware.

C output generates code that includes a hal.h hardware abstraction layer. Static sensor handles, #define TIMEOUT_MS for fallback deadlines, proper C types (float, int), and deadline enforcement using hal_get_time_ms().

Same source file, two completely different targets. No #ifdef in your code. No conditional compilation. The compiler handles the target-specific stuff.

What I'd do differently

The parser is hand-rolled. About 700 lines of Pratt parser with recursive descent for statements. I started with chumsky (a Rust parser combinator library) but their API changed between versions and the documentation was sparse when I needed it most. So I wrote my own.

It's fine. But it means every time I want to add a new syntax construct, I'm writing parser code by hand. A proper parser generator would've saved time in the long run, even with the initial investment.

The loop bound estimation is a heuristic. If your loop body has 1-3 statements, the compiler assumes it runs 10 times. 4-6 statements? 5 times. 7+? 3 times. This is obviously wrong for real programs. Proper static bound analysis would make the IPET results much more accurate. It's on my list.

The C output references a clamp() function that it never defines. If you actually try to compile the generated C code on hardware, you'll get a linker error. I know about it. I just haven't fixed it yet.

Why I'm publishing this

I put all 8 crates on crates.io. The lexer, parser, type checker, everything. The repo is at github.com/subhansh-dev/fabric.

Partly because I want other people to try it. If you're working on robotics or drones and you've been bitten by runtime errors, Fabric might save you some pain.

Mostly because I think the idea matters more than my implementation. The safety-critical systems world has had formal methods and static analysis for decades. But it's all locked behind expensive tools and enterprise licenses. I wanted to build something that a 17-year-old with a Raspberry Pi could use.

Is my implementation perfect? No. The IPET loop bounds are guesses. The C backend doesn't handle drone swarms. The match expression codegen references some variables it doesn't always generate. I know about all of these. They're bugs, not features, and I'll fix them.

But the core idea is sound: compile-time safety for robotics shouldn't require a $50,000 license for Simulink. Sometimes the right answer to "how do we prevent this bug" isn't "write better tests" or "add more runtime checks." Sometimes it's "make the compiler say no."

What's next

I'm working on adding hardware-in-the-loop testing support. The idea is that Fabric could instrument your actual hardware and compare runtime sensor readings against the compile-time uncertainty bounds. If reality diverges from what the compiler predicted, you get a warning before your next flight.

Also working on proper static loop bound analysis. The current heuristic is fine for demos but useless for real deployments. I'm looking at abstract interpretation techniques to actually prove bounds instead of guessing.

If any of this sounds interesting, check out the repo. Open an issue. Try it on your own robot. Break it and tell me how.

That's how this gets better.

Fabric is MIT licensed. The crates are published on crates.io. The docs are in the README because I'm 17 and haven't written proper docs yet.

gh repo link :- https://github.com/subhansh-dev/fabric

pnpm v12 Is Being Rewritten in Rust

Wed, 15 Jul 2026 15:13:28 +0000

What pnpm v12 actually is

Why did pnpm choose Rust?

The motivation isn't abstract. Two things were capping how fast a Node-based pnpm could go.

How much faster is pnpm v12?

pnpm publishes a benchmark suite comparing v11 against the Rust engine directly. As of this writing (July 2026), here's what it shows:

Scenario	pnpm v11	pnpm (Rust)
Everything warm (repeat install)	381 ms	12 ms
node_modules present, no cache	460 ms	40 ms
Clean install (nothing cached)	6.5 s	2.2 s

It's already rolling out, incrementally

Here's the part that's easy to get wrong: pnpm v12 isn't a big-bang release. The Rust engine has been landing piece by piece through the v11 line, opt-in first.

pnpm 11.2 let you opt into pacquet as the install backend for materialization only. Rust did the fetch and link, while pnpm still resolved dependencies and wrote the lockfile.
pnpm 11.7 went further: a standard install could be delegated to pacquet end-to-end, with resolution, lockfile write, and linking all in a single Rust pass.
pnpm 11.10 added pnpm self-update next-12, so you can pull the v12 line and try the Rust binary directly.

What changes for you?

The honest answer is: not much, on purpose. Same CLI, same lockfile, same node_modules. The whole point of a 1:1 port is that your muscle memory and your CI scripts keep working.

Will the upgrade break your CI?

Probably not, but the one thing worth checking is feature parity during the transition. Because the Rust engine is being ported incrementally, some newer or niche capabilities may lag the TypeScript version at any given moment. If your install depends on something specific (a particular linker mode, lockfile-verification settings, or pnpm deploy), test the bump in an isolated git worktree before you flip it on in CI. This is moving quickly, so the right list of gaps is whatever the roadmap says on the day you upgrade.

The bigger picture

Should you upgrade to pnpm v12 yet?

Postgres vs ClickHouse? I use both. 4 tricks from the split.

Thu, 09 Jul 2026 05:57:37 +0000

TL;DR. My uptime monitor keeps config and incidents in Postgres and check results in ClickHouse. The split is one rule: does a row ever change? Config gets edited, so it wants Postgres transactions and constraints. A check result is written once and never touched again, so it goes to ClickHouse, where the right codec, a careful sort key, and a per-row TTL make billions of rows cheap. Four tricks and a bonus below, useful even if you only ever run one database.

Every monitor I run writes a row every 20 seconds, from every region, and never stops. One monitor is about 4,300 rows a day, per region. Multiply by every monitor on the platform and the rows only ever go up.

That stream would slowly crush a normal Postgres table, and watching it is the whole product. So the check results do not live in Postgres. They live in ClickHouse. Everything else, the monitors and incidents and teams, lives in Postgres. The interesting part is the line between them.

People ask why not one database. "Postgres vs ClickHouse" is the wrong question, because the two are not fighting over the same job. Here is the line I draw, and the one rule under all of it: split your data by how it is written, not by which engine is faster.

1. Does the row change? That picks the database

Forget benchmarks for a second. One question sorts a table into one store or the other: after you write a row, will you ever change it?

one monitor, two stores:

  you edit a monitor  ->  Postgres   (targets, incidents, team)
                          the row changes, has constraints, lives in a transaction

  a probe checks it   ->  ClickHouse (check_results)
                          one row, written once, never updated

A monitor is a row that changes. You toggle it off, edit the URL, change the interval. So it lives in Postgres:

CREATE TABLE targets (
    id            UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    name          TEXT NOT NULL,
    check_spec    JSONB NOT NULL,
    interval_secs INTEGER NOT NULL CHECK (interval_secs >= 10),
    enabled       BOOLEAN NOT NULL DEFAULT true,
    updated_at    TIMESTAMPTZ NOT NULL DEFAULT now()
);

That CHECK means a 3-second interval can never reach the table, no matter which part of the app tried to write it. This is Postgres doing the thing it is best at: a small set of rows that must stay correct while many callers change them at once.

A check result is the opposite. It is written once when a probe finishes, and then it never changes. Nothing ever updates it, so it does not need any of that:

CREATE TABLE check_results (
    org_id      UUID,
    target_id   UUID,
    region      LowCardinality(String),
    timestamp   DateTime('UTC') CODEC(DoubleDelta, ZSTD(1)),
    status      Enum8('up' = 1, 'down' = 2, 'degraded' = 3, 'error' = 4),
    duration_ms UInt32 CODEC(T64, ZSTD(1))
) ENGINE = MergeTree

No updated_at, no foreign key, no transaction. Just an append-only stream that grows forever. That is the shape ClickHouse is built for and the shape that slowly hurts Postgres.

Notice the types too. status is an Enum8, one byte on disk, not the string "up". region is LowCardinality, stored once in a dictionary and referenced by a small id instead of repeating the text on every row. On a table that only ever grows, a byte saved per row is a byte saved times billions.

2. The right codec turns a day of timestamps into almost nothing

Once the results are in a column store, the type on each column is most of the compression, and the default setting wastes a lot of space.

Look at that timestamp again. A monitor checks every 20 seconds, so for one monitor a day of timestamps is a run of 20, 20, 20, 20, over and over. DoubleDelta stores the change in the change. For a steady interval the first delta is a constant 20 and the second delta is zero, so each row after the first packs down to about a bit. The codec is not a ClickHouse invention: it comes from Facebook's Gorilla time-series paper, built for exactly this, measurements taken at a steady rate.

timestamp   DateTime('UTC')  CODEC(DoubleDelta, ZSTD(1)),
duration_ms UInt32           CODEC(T64, ZSTD(1)),
dns_ms      Nullable(UInt16) CODEC(T64, ZSTD(1)),

I also store the timestamp at whole seconds on purpose. The smallest interval is 20 seconds, so no two checks for one monitor ever land in the same second, and the sub-second detail lives in duration_ms where it belongs. Whole seconds that step by a fixed amount compress far harder than milliseconds would.

The latency columns get T64 instead, because they are small integers that stay in a small range, a different shape from a steady clock. T64 crops the unused high bits off a block of values, so a UInt32 that never climbs past a few thousand milliseconds stops paying for all 32 bits. That is the trick worth copying, and it is not the specific codec names. It is that a timestamp ticking by a fixed step and a latency staying in a small range are two different shapes, and telling the database which is which does the work.

3. Put the tenant in the sort key, not the partition

I got this one wrong first, so learn it from my mistake instead of your own.

This is a multi-tenant app, so every row carries an org_id and almost every query filters by it. My first schema partitioned by org, one slot per customer, because it felt tidy. ClickHouse turned that into a flood of tiny parts, the merges could not keep up, and startup got slower the more customers I had. I ripped it out. A partition is not a folder for tidiness. It is a physical unit ClickHouse merges and expires, and you want few large ones, not many small ones.

Here is what it should be:

) ENGINE = MergeTree
PARTITION BY toYYYYMMDD(timestamp)
ORDER BY (org_id, target_id, region, timestamp)

org_id leads the ORDER BY, so a per-org query walks the sorted primary index and reads only that org's slice. But it stays out of PARTITION BY. The rule I follow now: partition by something low-cardinality that you also delete by, here the day, and put the high-cardinality tenant key in the sort order. Sort key answers "find this org fast". Partition answers "drop old data cheaply", which is the next trick.

4. Make retention a number in a row, not a schema change

Different plans keep history for different lengths of time. The clumsy way is a migration or a cleanup job per plan. The clean way is to make the retention window a column:

ttl_days UInt16 DEFAULT 30 CODEC(ZSTD(1))
) ENGINE = MergeTree
PARTITION BY toYYYYMMDD(timestamp)
ORDER BY (org_id, target_id, region, timestamp)
TTL timestamp + toIntervalDay(ttl_days)

Each row carries its own ttl_days, stamped from the org's plan at the moment it is written. A free plan keeps 30 days, a paid plan keeps more, and changing that needs no ALTER, no migration, no backfill. The next write just stores a different number. The daily partitions line up with the TTL, so ClickHouse expires old data by dropping whole parts, close to free, and a whole column of the same 30 compresses away to nothing. The flexibility costs no space.

One more piece makes reading that history cheap. A materialized view rolls raw checks into per-minute and per-hour summaries as they land:

CREATE MATERIALIZED VIEW check_results_1m
ENGINE = AggregatingMergeTree
ORDER BY (org_id, target_id, region, minute)
AS SELECT
    org_id, target_id, region,
    toStartOfMinute(timestamp)  AS minute,
    countState()                AS total_checks,
    countIfState(status = 'up') AS up_checks,
    avgState(duration_ms)       AS avg_duration_ms
FROM check_results
GROUP BY org_id, target_id, region, minute;

So a dashboard showing 30 days across a thousand monitors reads a few thousand minute-buckets, not millions of raw rows. Recent views read the minute rollup, long history reads an hour rollup, and the raw rows underneath expire on their own TTL. The firehose is there when you need to drill into one bad minute, and left alone the rest of the time.

One more, because everyone hits it: make inserts idempotent

Here is the trick I wish someone had told me first. My agents batch check results and send them to ClickHouse. A network blip after the server commits but before my side gets the ack means the batch retries and sends the exact same block again. Without protection, that double-counts every row in it, and your uptime numbers quietly drift.

ClickHouse has a fix built in, but for a plain (non-Replicated) MergeTree it is off until you turn it on:

SETTINGS index_granularity = 8192,
         non_replicated_deduplication_window = 1000;

The server hashes each inserted block and remembers the last 1,000 hashes. A retry sends an identical block, the hash matches, and the server drops it instead of appending it again. The retry becomes safe to do blindly, so the client stays simple: send, and if unsure, send again. Make the window bigger than the most blocks a single retry could resend, and you stop having to worry about duplicate writes.

So, one database or two?

For almost every app, one. If your data fits in Postgres and stays fast, a second database is a cost you should not pay: two schemas, two clients, two things to back up and think about.

Reach for the second store only when one table stops looking like the rest of your tables. For me that table was check_results. It is written once and never edited, it grows without end, and every question I ask it is a summary over a time range. That is a different shape from my monitors and my incidents, so it wanted a different database.

The rule I would give my past self: do not split by "which database is faster". Split by how the data is written. Rows that change and must stay correct want Postgres. An append-only stream you only ever summarize wants a column store like ClickHouse. Most of the tricks above are that one idea pushed down into the schema.

Uptimepage is open source, AGPL-3.0, and both schemas are in the repo: github.com/uptimepage/uptimepage. The probe that writes those rows is its own post, and the wider build story, one binary and two databases, is here. Or start free on the hosted tier and point a check at something.

4 things writing an HTTP prober in Rust taught me

Tue, 07 Jul 2026 14:10:44 +0000

TL;DR. An uptime prober is a strange HTTP client, so I swapped reqwest for raw hyper. It needs the opposite of what a normal client gives you: no connection pool, so the cold path gets measured instead of hidden; every phase timed, even when connect fails; a hard SSRF block on the URL the user typed; and exact failure reasons instead of one flat error. Four tricks below, each worth copying even if you never build a monitor.

I built an uptime checker in Rust. Like most people, I used reqwest first. Then I dropped it and moved down to raw hyper.

Not because reqwest is slow. It is great, and for app code you should use it. But a prober is a strange kind of HTTP client. It makes one request, and its whole job is to measure that request and report exactly what happened. A normal client is built to do the opposite: make requests fast and hide the messy details. Every trick below is a messy detail I needed to keep.

Here are four things building it taught me. Each one is a habit worth copying, even if you never write a monitor.

1. A connection pool lies to a monitor

The first thing reqwest gives you is a connection pool, and it is the first thing I had to remove.

A pool keeps TCP connections open and reuses them. The second request to a host skips DNS, skips the TCP handshake, skips TLS, and comes back much faster. For app code that is a free speed boost. For a monitor it is a lie. If I reuse a warm connection, the "connect time" I report is near zero, because I did not connect, I borrowed. A user watching a slow TLS handshake slowly rise in one region would see nothing, because the pool hid the handshake.

So the connector makes exactly one request per connection, then drops it:

/// No pooling: the caller drives exactly one request over the returned
/// stream, then drops it.
pub(crate) async fn timed_connect(
    p: &ConnectParams<'_>,
    host: &str,
    port: u16,
    is_https: bool,
) -> Result<TimedConnection, ConnectError> {

Every check pays the full cost of a fresh connection, because the full cost is the thing I am trying to measure. The lesson is general: if you are timing a request, a warm pool is measuring the wrong thing. Reuse is a feature that erases exactly the numbers a monitor exists to show.

2. Put the timings inside the error

Splitting a request into DNS, TCP, TLS, and time to first byte is easy on the happy path. The interesting question is what happens when connect fails.

The naive design loses everything on failure. You get an Err and no numbers, so a hung TLS handshake looks the same as a dead DNS server. Both are just "failed." That is the worst moment to have no data.

The fix is small, and I now use it everywhere: make the error carry the phases that finished before the one that broke.

pub(crate) enum ConnectError {
    Dns(anyhow::Error),
    NoAddrs { dns_ms: u16 },
    Connect { err: io::Error, dns_ms: u16 },
    Tls { err: io::Error, dns_ms: u16, connect_ms: u16 },
}

A TLS failure still reports how long DNS and TCP took. So when a cert handshake hangs, the chart still shows DNS was 12ms, connect was 45ms, and then TLS took the rest. The shape of the error tells you what broke. You are never left guessing which layer stalled, even on the path where everything went wrong. reqwest gives you one flat Error at the very end. Here the error type holds the answer.

one probe, one fresh connection, each phase on its own clock:

  DNS         TCP connect      TLS handshake     request + TTFB
  +-----------+----------------+-----------------+---------------->
   12ms        45ms             (hung)            never reached

  TLS failed. The error still carries dns_ms=12, connect_ms=45,
  so you know exactly which layer stalled, not just "it failed".

3. The SSRF check goes after DNS, not before

This is the one most people building a fetch-any-URL feature get wrong, and it is the scary one.

A checker fetches whatever URL the user typed. That is the feature. It is also a classic SSRF engine. Someone signs up, points a check at http://169.254.169.254/latest/meta-data/, and now your prober is reading cloud credentials from inside your own network and showing them the reply. Point it at http://10.0.0.5:6379 and it is a port scanner for your private subnet.

The trap is thinking you can block this by checking the hostname. You cannot. The classic bypass is DNS rebinding: the name looks public and passes your check, but it resolves to 127.0.0.1. The block has to happen after resolution, on the real IP you are about to dial:

let addrs: Vec<SocketAddr> = p
    .resolver
    .resolve_addrs(host)
    .await
    .map_err(ConnectError::Dns)?
    .into_iter()
    .filter(|ip| p.ssrf_guard.allow(*ip))   // block loopback, private, link-local, metadata
    .map(|ip| SocketAddr::new(ip, port))
    .collect();

Resolve first, filter every resolved IP, then connect only to what survives. And because each redirect hop reconnects through this same guarded path, a Location: header pointing at 169.254.169.254 gets rejected exactly like a directly typed one. The safety sits in the single place all connections pass through, so there is nothing to forget on the redirect path. If you ever fetch a user-supplied URL server-side, this is the check that matters, and the hostname version is not it.

4. Stop grepping your error strings

When a check goes red the user gets one line, and that line has to be right. "Something went wrong" trains people to ignore alerts. "Certificate expired" tells them what to fix in five seconds.

The tempting way to produce that line is to match on error message text. It is a trap: those strings change between library versions and read differently on every OS. The message is not an API. So instead the connector downcasts to the real error and reads the reason straight out of it:

fn tls_reason(io: &io::Error) -> &'static str {
    use rustls::CertificateError;
    let Some(rustls::Error::InvalidCertificate(cert)) =
        io.get_ref().and_then(|e| e.downcast_ref::<rustls::Error>())
    else {
        return "tls";
    };
    match cert {
        CertificateError::Expired => "certificate expired",
        CertificateError::UnknownIssuer => "certificate not trusted",
        CertificateError::NotValidForName => "certificate hostname mismatch",
        CertificateError::Revoked => "certificate revoked",
        _ => "certificate invalid",
    }
}

An expired cert, an untrusted issuer, and a hostname mismatch are three different problems with three different fixes, and the user should never have to guess which one they have. The io::ErrorKind and the rustls error already carry the truth. The trick is to refuse to flatten it into a string and then re-parse the string you just made.

So, reqwest or not?

For your program? Almost certainly reqwest. Going down to hyper means you own redirects, decompression with a size cap, timeout budgeting per hop, and the request-target rules for HTTP/1 versus HTTP/2. That is real code with real edge cases, and every one is a place reqwest would have been correct for free. If I needed to fetch a config file at startup, writing this connector would be a mistake.

It is worth it here for one reason: measuring and guarding the request is the product, not a detail of it. When the request itself is the thing you sell, you want to own every millisecond and every IP it touches.

Uptimepage is open source, AGPL-3.0, and the whole probe path is in the repo: github.com/uptimepage/uptimepage. If you want the wider build story, one binary and two databases, that is a separate post. Or just start free on the hosted tier and point a check at something.

This Smart-Home Agent Treats Its Own 1B Model as Untrusted Input

Tue, 07 Jul 2026 12:19:43 +0000

Reliability in an AI agent is a harness property, not a model property.

The cleanest proof arrived at the bottom of the model-size ladder: a 688 MB model controlling a smart home, showcased by the model's own maker. The part worth studying is the 25 MB of Rust wrapped around it.

EdgeHome Harness pairs MiniCPM5-1B, 688 MB on disk and about 1 GB at runtime, with a Rust harness between 25 and 64 MB, targeting devices with 2 GB of RAM. A community developer built it; OpenBMB put it on stage. The line everyone will quote is that a full agent stack fits on ultra-constrained edge hardware. The design decision that matters is different: the model is not allowed to do anything.

Everything After the JSON Is Deterministic

EdgeHome Harness is a Rust framework that makes a 1B model act as a reliable smart-home controller. The model only generates a candidate JSON command. Deterministic Rust code validates the schema, resolves devices against a registry, checks capabilities, applies fail-closed policy gates, plans a dry run, and records a replayable trace.

The repo states the thesis outright: the model sits at the top of the pipeline, and everything after candidate generation belongs to the harness. There is a trust table in the docs. The model's output type is labeled untrusted. Not partially trusted. Untrusted, the way you treat user input in a web form.

The harness ships payload adapters for Home Assistant, MQTT, and MIoT or Matter bridges. The model itself talks to nothing but Ollama, through a structured output request.

The Model Is the First Thing Sacrificed

A 1B model repeats itself, rambles past the closing brace, and emits JSON that almost parses. EdgeHome budgets for all three: an output governor with dead-loop detection, a retry policy, and fallback classification. Short-term memory stays out of the context window too; Rust tracks the last target and confirmed aliases, because context is the scarcest resource on a 2 GB board.

The sharpest detail is the pressure policy. When memory runs low, the harness shrinks the model's context, then its output budget, and under critical pressure drops the model entirely and falls back to rules. In this architecture, the language model is the most disposable component in the stack. Execution backends ship disabled by default, every plan is a dry run first, and a 108-case eval gate decides releases.

The Same Boundary Holds at Frontier Scale

I run the identical pattern at the opposite end of the size ladder. My coding agent works inside a sandbox where the harness owns the filesystem and network boundaries, whatever the model believes. My wallet experiment puts a policy gate with spend caps and an allowlist between an agent and the chain, because an approval step the agent can call is an approval step the agent can forge. Enforcement lives outside the model or it does not exist.

I have made this argument for coding agents and for cheap-model pipelines. EdgeHome is the same argument compiled to 25 MB and pointed at a light switch.

This project settles a different question than the one in the announcement. Not whether a 1B model can run your smart home. How few decisions a model of any size should be making. Trace tonight's agent failures back to their source: how many were the model being too small, and how many were your harness letting the model decide something a validator should own? EdgeHome's answer is a trust table. Most agent stacks never wrote one.

I write field notes from real builds — AI integration, cron-driven automation, and the parts that break in production. New posts every two weeks; if this one was useful, notes on human-in-the-loop approval boundaries for agents is the companion download.

The Security Liability of Memory Allocation in TEEs: A Design Decision Log (no_std Rust)

Fri, 03 Jul 2026 00:54:14 +0000

Memory allocation is not a feature — it is a security liability.

In high-assurance Trusted Execution Environments (TEEs), you cannot afford the jitter or the fragmentation of a probabilistic global heap. When building the sakshi-core attestation loop for the Sovereign Spine architecture, the requirement was absolute: determinism.

Following my previous teardown on binding LLM context to a TPM root of trust, this post shifts the focus to the memory substrate itself — specifically, why GlobalAlloc creates an unacceptable attack surface within an enclave.

The Problem: Why GlobalAlloc Fails the TEE Test

In a standard Rust environment, we lean on the global allocator. In a TEE, however, the global allocator is a massive attack surface.

Jitter: Allocation time varies based on heap state, leaking metadata through timing side-channels.

Fragmentation: Heap fragmentation can lead to unpredictable exhaustion, a vector for Denial of Service (DoS) within the enclave.

TCB Bloat: The allocator logic itself adds thousands of lines of code to your audit surface, increasing the Trusted Computing Base (TCB) size beyond what is strictly necessary.

The Solution: Session-Scoped Bump Buffer

To enforce architectural certainty, I stripped away the dependency on standard heap allocation in the enclave. Instead, I implemented a session-scoped bump buffer.

This is a contract-based memory model:

Constant-time execution: Allocation is a pointer increment operation, taking 1-2 CPU cycles.

Zero-fragmentation: Memory is allocated linearly and cleared atomically at the session boundary.

Simplified TCB: By removing GlobalAlloc, the enclave memory logic is reduced to a handful of lines of verifiable code.

Implementation Concept

The core logic relies on a pre-allocated static region. We do not ask the system for memory; we own a dedicated slab of silicon-backed memory and manage it strictly within the request lifecycle.

// Conceptual implementation of the session-scoped buffer
pub struct BumpBuffer {
    buffer: &'static mut [u8],
    offset: usize,
}

impl BumpBuffer {
    pub fn alloc(&mut self, size: usize) -> Option<&mut [u8]> {
        if self.offset + size <= self.buffer.len() {
            let start = self.offset;
            self.offset += size;
            Some(&mut self.buffer[start..self.offset])
        } else {
            None // Enforce strict limit, no OOM panic
        }
    }
}

The Architectural Takeaway

We must stop hoping for performance and start architecting for it. By moving the decision boundary from the runtime heap to a statically-defined buffer, we remove entire classes of vulnerabilities.

In the Sovereign Spine, the memory model is not just an implementation detail—it is a security guarantee.

Explore the Implementation

The architectural patterns discussed here—including the session-scoped bump buffer — are part of the sakshi-core module within the Citadel Protocol Repository.

View the code: citadel-protocol/sakshi-core

Building an uptime monitor in Rust: one binary, two databases, 130K checks/sec per core

Tue, 30 Jun 2026 17:20:57 +0000

I spent the last few months building Uptimepage, an open-source uptime monitor and status page written in Rust. This post is the build story: the decisions that shaped it, the parts I rewrote, and the numbers that came out the other side.

The whole thing ships as one self-contained binary of about 23 MB, plus Postgres and ClickHouse. You can docker compose up and self-host it, or use the hosted tier. Source is AGPL-3.0 on GitHub.

Why one binary

A status page is a small surface with a lot of moving parts behind it: a scheduler, probe workers, an HTTP client, a time-series writer, an incident detector, an alerting fan-out, a web UI, a JSON API. The usual answer is a handful of services and a message bus between them.

I went the other way. One process, one binary, the same image whether it runs the control plane or a remote probe. No queue to operate, no version skew between services, no "which container is wedged" at 3am. The cost is that you have to be careful about what shares a thread and what can stall what. Most of the engineering below is about keeping those boundaries clean inside a single process.

Stack: Rust 1.95 (edition 2024), Tokio, Axum 0.8, Askama for compile-time HTML templates, HTMX 2 for partial swaps so there is no SPA framework to ship. The API stays the single source of truth because every UI mutation hits the same /api/v1/* endpoint a script would.

Two databases, on purpose

Monitors are low-cardinality relational data that gets mutated by API calls: targets, regions, channels, incidents, plans. That is Postgres. Check results are append-only, high-cardinality, and almost always queried by time range. That is ClickHouse.

Trying to force one of those into the other is where uptime monitors usually fall over. Putting billions of check results in Postgres turns every dashboard query into a sequential scan. Putting your relational config in ClickHouse means fighting its update model forever. So: Postgres 18 for the world, ClickHouse 26.3 for the firehose. Both run their migrations at process startup, so there is no separate migrator to forget.

The HTTP client I did not want to write

The first version used a popular high-level HTTP client. It worked, but a monitor is a weird HTTP workload: you connect once per target per interval, you never reuse the connection, and you care about the timing of each phase more than the body.

So I dropped down to hyper and hyper-util with rustls and built a connector that times DNS resolution, TCP connect, and the TLS handshake as separate numbers, then runs the request over hyper::client::conn and aborts the connection task the moment the body is read. Each result carries dns_ms, connect_ms, tls_ms, and ttfb_ms as distinct columns, which is what makes "it got slow but it is the DNS, not your server" a thing the dashboard can actually say.

The rewrite paid for itself. On a single core the client sustains around 130K checks/sec at saturation, roughly 7.7 microseconds per check. That was a 44 to 56 percent throughput gain over the old path. A chunk of it was a url::parse call hiding in the redirect policy that cost 7.5 percent on its own and just vanished. Two cores get to about 153K. Scaling goes sub-linear past four cores because of shared HTTP/2 connection state and the pool mutex, which is a fine problem to have for this workload.

These numbers come from a laptop and a loadtest binary, so I treat them as regression detection, not capacity planning. The headline in-process run pushed 252K requests/sec sustained at p99 393ms. The design goal is around 50K concurrent in-flight checks per node with under 50ms p99 of per-check overhead on top of the network.

One heap, not a timer per target

The naive scheduler spawns a timer task per monitor. That falls apart at fleet size: thousands of tasks, thousands of wakeups, memory that grows with the number of targets.

Instead there is a single driver task that owns one BinaryHeap<Reverse<Due>>, a min-heap keyed by the next due Instant for the whole fleet. Memory stays flat in fleet size. Each target gets a deterministic jitter offset hashed from its UUID so a thousand monitors on a 60s interval do not all fire on the same tick. Generation and sequence counters mean a re-scheduled target supersedes its stale heap entry instead of double-firing. The registry refresh that pulls config from Postgres runs on its own task with exponential backoff, so a Postgres hiccup never stalls dispatch: the scheduler keeps running on what it last knew.

Failure isolation inside one process

Because everything is one process, one bad target cannot be allowed to take down the rest. Three patterns do most of that work. Per-host circuit breakers trip when a host keeps failing, so it fails fast with circuit_open instead of tying up a worker on a timeout, then probes half-open after a cooldown. A per-tenant host throttle bulkhead caps how many checks can be in flight against one host at once; over the cap, a check is recorded as throttled and degraded rather than piling on, and it never pages. And singleflight on RDAP collapses domain-expiry checks for the same domain across many tenants into one upstream probe, with sticky last-good state so a flaky registrar does not flip the monitor red.

The worker pool itself is a task-per-dispatch gated by a semaphore sized to a max-concurrency setting, with an SSRF guard filtering resolved IPs before any connect.

Modeling time series in ClickHouse

The check_results table is a MergeTree ordered by (org_id, target_id, region, timestamp). The org_id leads the sort key so each tenant gets a sparse-index slice, but it is deliberately kept out of the partition key (partition is by day) so we do not end up with millions of partitions.

The columns are where the storage savings live:

Timestamps are DateTime('UTC') with CODEC(DoubleDelta, ZSTD(1)). Check intervals are near-constant, so DoubleDelta crushes the gaps to almost nothing.
Numeric phase columns (duration_ms, dns_ms, connect_ms, tls_ms, ttfb_ms, response code and size) use CODEC(T64, ZSTD(1)).
region and agent_id are LowCardinality(String), status is an Enum8.

Retention is per row. There is a ttl_days column with TTL timestamp + toIntervalDay(ttl_days), stamped from the org's plan at write time. A plan that buys longer history needs zero schema change: new rows just carry a bigger number.

On top of the raw table sit two AggregatingMergeTree materialized views: a per-minute rollup kept 30 days and an hourly rollup kept 13 months, both holding quantilesState for p50/p95/p99 and per-status counts. Reads route by range: anything inside 30 days hits the minute rollup, older ranges hit the hour rollup, and raw reads are capped at a 90-day span. A dashboard asking for "last 24h p99 latency" never touches a raw row.

Regional probes without a second brain

You can run probes in multiple regions, but I did not want each region to be its own little system. So an agent is the same binary in agent mode, running as a stateless probe with no database, no web, no alerting. Adding a region adds execution capacity, never a second control plane.

An agent pulls its region's config from the control plane with ETag/304 handling, serves its last-known config if the control plane blips, and pauses if its token is revoked. It ships results back in batches that reuse one UUID across retries so a lost ack cannot double-count. Region and agent identity are derived server-side from the bearer token, never sent in the payload, so a probe cannot claim to be somewhere it is not. A separate long-poll loop handles interactive "check now" so a button press in the UI runs on a real remote probe within milliseconds.

Region is the partition dimension end to end, so every read can slice by region: per-region latency series, per-region incident scope, "down in Singapore, up in Helsinki."

Incidents as a follower, not a gatekeeper

The incident detector is a background task that follows the check_results stream and writes into the Postgres incidents table. The rule I held to: it never touches the hot write path, never gates check execution, and never produces alerts directly.

The detection itself is boring on purpose: two or more consecutive unhealthy results with no open incident opens one, two or more consecutive healthy results closes it. A small flap threshold absorbs single-result blips. The cross-tenant walk is keyset-paginated so memory stays bounded as tenants grow, and a unique index on open incidents resolves the race when two ticks try to open the same one: only the winner pages.

Opening or resolving fires a non-blocking signal to the escalation engine, which does repeat-until-acknowledged paging on a per-monitor cadence across about fourteen transports (Slack, generic signed webhooks, Telegram, PagerDuty, ntfy, Pushover, Discord, email, and more), with sharded per-incident locks so a reconcile and an inbound signal can never double-page the same episode. Channel secrets are sealed at rest with AES-GCM and never echoed back.

Automation as a first-class surface

Because the API is the single source of truth, the rest came almost for free:

A self-describing OpenAPI 3.1 spec with Swagger UI.
An official Terraform provider so you can manage monitors and notification channels as code.
An MCP server so an LLM client can query your monitors and incidents over OAuth 2.1, scope-gated and audited, with per-action confirmation on the few write tools.

Where it is

Uptimepage is live, free to start with no card, and AGPL-3.0 open source. The core is not paywalled: checks, status pages, subscribers, the API, and every alert channel are in the free tier. It monitors itself and serves its own status badges from the running binary, which is the most honest dogfood I could think of.

Site and hosted tier: https://uptimepage.dev
Source: https://github.com/uptimepage/uptimepage
More build notes: https://uptimepage.dev/blog

Happy to answer anything about the Rust internals in the comments.

uptimepage

Tue, 30 Jun 2026 17:20:37 +0000

Uptimepage is an open-source uptime monitor and status page, built as one self-contained Rust binary backed by Postgres and ClickHouse. Run it with docker compose up, or use the free hosted tier.

I built it because I wanted a monitor I could run as a single binary and drive entirely as code, without operating a fleet of services to do it.

What it does

Checks: HTTP, TCP, DNS, TLS-certificate expiry, and domain expiry, from one region or many.
Public status page: HTML, JSON, and RSS, per-component opt-in, incidents, maintenance windows, email and webhook subscribers.
Alerting across about fourteen channels: Slack, generic signed webhooks, Telegram, WhatsApp, SMS, PagerDuty, ntfy, Pushover, Discord, Google Chat, MS Teams, email. Fire once, recover, and repeat until acknowledged.
Incidents with a separate internal state and public phase, so you can work an incident before you publish anything.
Multi-region probes: each agent is the same binary in a stateless probe mode. Run your own anywhere with an org-scoped token.
Automation: REST API with an OpenAPI spec, a Terraform provider, and an MCP server an LLM client can query over OAuth.

A few engineering bits

A custom hyper and rustls client instead of a high-level one, so each result carries separate DNS, connect, TLS, and TTFB timings. About 130K checks a second on one core in the load test.
A single min-heap scheduler for the whole fleet rather than a timer task per monitor, so memory stays flat in fleet size.
ClickHouse with per-row TTL and minute and hour rollups, so dashboards never scan raw rows.

Full write-up of how it is built: https://uptimepage.dev/blog/building-an-uptime-monitor-in-rust

Free and open

The core is not paywalled. Checks, status pages, subscribers, the API, and every alert channel are in the free tier, with no card to start. It is AGPL-3.0, and it monitors itself.

Site and hosted tier: https://uptimepage.dev
Source: https://github.com/uptimepage/uptimepage

Happy to answer questions about the internals or the self-host setup in the comments.

Building a KVM Virtual Machine in Rust: Running a binary

Sun, 28 Jun 2026 05:24:29 +0000

Recap

This is a continuation of Part 2 of KVM in Rust series,
where we successfully set up a memory region for our guest VM. Now we will see
how to proceed further and run a very simple binary in our hypervisor.

Loading the binary

Now, we have only set up a memory region, but we still haven't loaded any binary
or code to actually run. For this, I have written a small "Hello world" x86
assembly file:

https://github.com/StjepanPoljak/kvm-rust/blob/kvm-part3-code/samples/hello-world.asm

Notice that the org 0x1000 header is quite arbitrary and it's actually a good
lead of what we also need to do (set the instruction pointer). Let's compile the
assembly file with:

nasm ./samples/hello-world.asm

Then, we want to load our binary into the memory region at address 0x1000:

let mut file = File::open("./samples/hello-world")?;
let mut code = Vec::new();
file.read_to_end(&mut code)?;

unsafe {
std::ptr::copy_nonoverlapping(code.as_ptr(),
  (mem_ptr as *mut u8).add(0x1000),
  code.len()) };

Here we are using the more optimal copy_nonoverlapping because we can be quite
certain that the memory where code is stored is not overlapping with our
memory region.

Creating a vCPU

As we saw in Part 1, we have also discovered that QEMU also
calls KVM_SET_VCPU to obtain a vCPU file descriptor on which it will call
KVM_RUN. So, all that we have to do to set up a vCPU is to take a look at the
ioctl and recreate it in Rust:

// 140904 ioctl(9<anon_inode:kvm-vm>, 0xae41 /* KVM_CREATE_VCPU */, 0) = 10<anon_inode:kvm-vcpu:0>
let vcpu_fd = unsafe { libc::ioctl(self.fd, KVM_CREATE_VCPU, 0usize) };
if vcpu_fd < 0 {
return Err(io::Error::last_os_error());
}

Investigating vCPU operations

It would be tempting to simply call KVM_RUN on this file descriptor, however,
we don't really know how vCPU is set up, where the instruction pointer is and
even if the registers are in the valid state. If we look at the ioctl calls
referring to kvm-vcpu:0 in our log (and filtering out the spamming KVM_RUN
and KVM_SMI), we can see a lot of lines like these being repeated:

$ grep 'kvm-vcpu:0' kvm.log | grep -v 'RUN\|SMI'

                ### omitted a lot of repetitive strace output ###

140904 ioctl(10<anon_inode:kvm-vcpu:0>, 0xc080aebe /* KVM_GET_NESTED_STATE */, 0x7768f8002010) = 0
140904 ioctl(10<anon_inode:kvm-vcpu:0>, 0xc028ae92 /* KVM_TPR_ACCESS_REPORTING */, 0x77690198f090) = 0
140904 ioctl(10<anon_inode:kvm-vcpu:0>, 0x4140aecd /* KVM_SET_SREGS2 */, 0x77690198f2f0) = 0
140904 ioctl(10<anon_inode:kvm-vcpu:0>, 0x4090ae82 /* KVM_SET_REGS */, {rax=0x20, ..., rsp=0x6d88, rbp=0, ..., rip=0x18, rflags=0x246}) = 0
140904 ioctl(10<anon_inode:kvm-vcpu:0>, 0x5000aea5 /* KVM_SET_XSAVE */, 0x7768f8001000) = 0
140904 ioctl(10<anon_inode:kvm-vcpu:0>, 0x4188aea7 /* KVM_SET_XCRS */, 0x77690198f2c0) = 0
140904 ioctl(10<anon_inode:kvm-vcpu:0>, 0x4008ae89 /* KVM_SET_MSRS */, 0x7768f80040a0) = 59
140904 ioctl(10<anon_inode:kvm-vcpu:0>, 0x4040aea0 /* KVM_SET_VCPU_EVENTS */, 0x77690198f500) = 0
140904 ioctl(10<anon_inode:kvm-vcpu:0>, 0x4008ae89 /* KVM_SET_MSRS */, 0x7768f80040a0) = 1
140904 ioctl(10<anon_inode:kvm-vcpu:0>, 0x4080aea2 /* KVM_SET_DEBUGREGS */, 0x77690198f500) = 0
140904 ioctl(10<anon_inode:kvm-vcpu:0>, 0xc008ae88 /* KVM_GET_MSRS */, 0x77690198f5a0) = 1

So we see that QEMU is setting vCPU registers via KVM_SET_REGS and in this log
we can see their exact values. It's also calling KVM_SET_SREGS2 which is
setting system registers; unfortunately, we cannot see the values used here.
Also, we do not need to follow the whole logic here, we just want to see what
QEMU is doing with registers before calling KVM_RUN:

$ awk '/KVM_RUN/ {print; exit} {print}' kvm.log | grep '[GS]ET_[S]\?REGS\|KVM_RUN'
140904 ioctl(10<anon_inode:kvm-vcpu:0>, 0x4140aecd /* KVM_SET_SREGS2 */, 0x77690198f310) = 0
140904 ioctl(10<anon_inode:kvm-vcpu:0>, 0x4090ae82 /* KVM_SET_REGS */, {rax=0, ..., rsp=0, rbp=0, ..., rip=0xfff0, rflags=0x2}) = 0
140904 ioctl(10<anon_inode:kvm-vcpu:0>, 0x8090ae81 /* KVM_GET_REGS */, {rax=0, ..., rsp=0, rbp=0, ..., rip=0xfff0, rflags=0x2}) = 0
140904 ioctl(10<anon_inode:kvm-vcpu:0>, 0x8140aecc /* KVM_GET_SREGS2 */, 0x77690198f310) = 0
140904 ioctl(10<anon_inode:kvm-vcpu:0>, 0x4140aecd /* KVM_SET_SREGS2 */, 0x77690198f310) = 0
140904 ioctl(10<anon_inode:kvm-vcpu:0>, 0x4090ae82 /* KVM_SET_REGS */, {rax=0, ..., rsp=0, rbp=0, ..., rip=0xfff0, rflags=0x2}) = 0
140904 ioctl(10<anon_inode:kvm-vcpu:0>, 0xae80 /* KVM_RUN */, 0) = -1 EINTR (Interrupted system call)

Since QEMU first calls SET_SREGS2 and SET_REGS we should actually
investigate what the default (if any) values are in our own implementation.

Implementing register get/set operations in Rust

The procedure for this is as with all other KVM ioctl calls we implemented.
We find out how to construct their numbers (consult the previous article here)
and reimplement these constants in Rust:

const KVM_GET_SREGS2 : u64 = _IOR::<kvm_sregs2>(KVMIO, 0xcc);
const KVM_SET_SREGS2 : u64 = _IOW::<kvm_sregs2>(KVMIO, 0xcd);
const KVM_GET_REGS : u64 = _IOR::<kvm_regs>(KVMIO, 0x81);
const KVM_SET_REGS : u64 = _IOW::<kvm_regs>(KVMIO, 0x82);

We also need to add kvm_sregs2 and kvm_regs in our build.rs file:

    bindgen::Builder::default()
        .header("/usr/include/linux/kvm.h")
        .allowlist_type("kvm_sregs2")
        .allowlist_type("kvm_userspace_memory_region")
        .allowlist_type("kvm_regs")
        .generate_comments(false)
        .generate()?
        .write_to_file(out_path.join("kvm-bindings.rs"))?;

Then, ioctl calls are as follows (I will only give example for get_sregs2
and get_regs):

let mut sregs2 = unsafe { std::mem::zeroed() };
let ret = unsafe { libc::ioctl(self.fd, KVM_GET_SREGS2, &mut sregs2) };

let mut regs = unsafe { std::mem::zeroed() };
let ret = unsafe { libc::ioctl(self.fd, KVM_GET_REGS, &mut regs) };

Peeking at registers

After obtaining sregs2 and regs we can print them out and see what they are
set to by default (to accomplish this we consult the struct definition in our
kvm-bindings.rs (or in the Linux headers) and print out all the values of the
fields we find). For regs we see they are pre-set to:

RAX=0x0 RBX=0x0 RCX=0x0 RDX=0x600
RSI=0x0 RDI=0x0 RSP=0x0 RBP=0x0
R8=0x0  R9=0x0  R10=0x0 R11=0x0
R12=0x0 R13=0x0 R14=0x0 R15=0x0
RIP=0xfff0      RFLAGS=0x2

Taking a look at our trace from kvm.log we can see that the ones QEMU sets are
exactly the same (except for RDX but we can get away with this one).

Peeking at system registers

Now, for sregs2 it gets a bit more complicated. Our defaults are as follows:

CS      base=0xffff0000 selector=0xf000 limit=0xffff    type=0xb        present=0x1
        dpl=0x0         db=0x0          s=0x1   l=0x0   g=0x0           avl=0x0

DS      base=0x0        selector=0x0    limit=0xffff    type=0x3        present=0x1
        dpl=0x0         db=0x0          s=0x1   l=0x0   g=0x0           avl=0x0

ES      base=0x0        selector=0x0    limit=0xffff    type=0x3        present=0x1
        dpl=0x0         db=0x0          s=0x1   l=0x0   g=0x0           avl=0x0

FS      base=0x0        selector=0x0    limit=0xffff    type=0x3        present=0x1
        dpl=0x0         db=0x0          s=0x1   l=0x0   g=0x0           avl=0x0

GS      base=0x0        selector=0x0    limit=0xffff    type=0x3        present=0x1
        dpl=0x0         db=0x0          s=0x1   l=0x0   g=0x0           avl=0x0

SS      base=0x0        selector=0x0    limit=0xffff    type=0x3        present=0x1
        dpl=0x0         db=0x0          s=0x1   l=0x0   g=0x0           avl=0x0

TR      base=0x0        selector=0x0    limit=0xffff    type=0xb        present=0x1
        dpl=0x0         db=0x0          s=0x0   l=0x0   g=0x0           avl=0x0

LDT     base=0x0        selector=0x0    limit=0xffff    type=0x2        present=0x1
        dpl=0x0         db=0x0          s=0x0   l=0x0   g=0x0           avl=0x0

GDT     base=0x0        limit=0xffff

IDT     base=0x0        limit=0xffff

CR0=0x60000010          CR2=0x0         CR3=0x0         CR4=0x0         CR8=0x0

EFER=0x0                APIC_BASE=0xfee00900            FLAGS=0x0

PDPTRS[0]=0x0           PDPTRS[1]=0x0           PDPTRS[2]=0x0           PDPTRS[3]=0x0

Problem is that we don't see any values in our strace log, only the address of
the variable. So we need to be a bit creative here if we want to find out what
QEMU sets these values to. We could use ptrace (in fact strace is built upon
ptrace API), but it may be a bit too much. Same for uprobes and eBPF. We
do have GDB, though, and it's just perfect as a one-off thing here. All we have
to do is run QEMU under GDB and then execute:

(gdb) break ioctl if $rsi == 0x4140aecd
(gdb) run

Note that 0x4140aecd is the exact value that we extracted from SET_SREGS2
ioctl call (in x86 ABI, RSI is holding the value of second argument):

140904 ioctl(10<anon_inode:kvm-vcpu:0>, 0x4140aecd /* KVM_SET_SREGS2 */, 0x77690198f310) = 0

Once GDB breaks, we can dump memory. However, we don't know the exact size of
struct kvm_sregs2; we could guess or manually inspect, but quickest way to get
it is to actually have a small C program:

#include <linux/kvm.h>
#include <stdlib.h>
#include <stdio.h>

int main() {
  printf("%lu", sizeof(struct kvm_sregs2));
  return 0;
}

The program will return value of 320 (at least on my system) and we can then
use this in GDB:

(gdb) dump memory dump.bin $rdx $rdx+320

Note that RDX is holding the third argument, i.e. the address we actually saw
in our strace log. Now, we can reuse our function for printing sregs2 in
Rust. We simply hack our main function to load the file and reinterpret its data
as sregs2 and then print it. We got:

CSbase=0xffff0000selector=0xf000limit=0xfffftype=0xbpresent=0x1
dpl=0x0db=0x0s=0x1l=0x0g=0x0avl=0x0

DSbase=0x0selector=0x0limit=0xfffftype=0x3present=0x1
dpl=0x0db=0x0s=0x1l=0x0g=0x0avl=0x0

ESbase=0x0selector=0x0limit=0xfffftype=0x3present=0x1
dpl=0x0db=0x0s=0x1l=0x0g=0x0avl=0x0

FSbase=0x0selector=0x0limit=0xfffftype=0x3present=0x1
dpl=0x0db=0x0s=0x1l=0x0g=0x0avl=0x0

GSbase=0x0selector=0x0limit=0xfffftype=0x3present=0x1
dpl=0x0db=0x0s=0x1l=0x0g=0x0avl=0x0

SSbase=0x0selector=0x0limit=0xfffftype=0x3present=0x1
dpl=0x0db=0x0s=0x1l=0x0g=0x0avl=0x0

TRbase=0x0selector=0x0limit=0xfffftype=0xbpresent=0x1
dpl=0x0db=0x0s=0x0l=0x0g=0x0avl=0x0

LDTbase=0x0selector=0x0limit=0xfffftype=0x2present=0x1
dpl=0x0db=0x0s=0x0l=0x0g=0x0avl=0x0

GDTbase=0x0limit=0xffff

IDTbase=0x0limit=0xffff

CR0=0x60000010CR2=0x0CR3=0x0CR4=0x0CR8=0x0

EFER=0x0APIC_BASE=0xfee00900FLAGS=0x0

PDPTRS[0]=0x0PDPTRS[1]=0x4b275f5fce32f200PDPTRS[2]=0x0PDPTRS[3]=0x3

We can see that this more or less corresponds to the defaults we observed in our
KVM implementation. If we want to change our registers (which we will), then we
will always first get the ones that are current in the vCPU, change the ones we
need and then set them using KVM_SET_REGS (or KVM_SET_SREGS2). So one of
first things to do is to set CS to flat mapping:

let mut sregs2 = vcpu.get_sregs2()?;
sregs2.cs.base = 0;
sregs2.cs.selector = 0;
vcpu.set_sregs2(sregs2)?;

Recall that our assembly file was assembled with org 0x1000, so setting RIP to
0x1000 causes execution to begin at the start of the loaded binary:

let mut regs = vcpu.get_regs()?;
regs.rip = 0x1000;
vcpu.set_regs(regs)?;

Furthermore, we now have a very good process of finding out what QEMU does with
registers.

Running the binary

Now this is where strace stopped being useful and my understanding of the KVM
run loop became more useful. Another positive thing is that this part of QEMU
source code is quite readable and can be found in function kvm_vcpu_thread_fn.

The more x86-specific code with exit reasons can be found in the kvm_arch_handle_exit function.

Setting kvm_run shared memory region

To wrap things up, first we need to mmap the shared kvm_run structure that KVM
uses to communicate VM-exit information and other runtime state between the
kernel and userspace:

// 140904 ioctl(3</dev/kvm<char 10:232>>, 0xae04 /* KVM_GET_VCPU_MMAP_SIZE */, 0) = 12288
let kvm_run_size = unsafe {
libc::ioctl(kvm_fd, KVM_GET_VCPU_MMAP_SIZE, 0usize) };
let kvm_run_mem = unsafe {
libc::mmap(ptr::null_mut(), kvm_run_size as usize,
   libc::PROT_READ|libc::PROT_WRITE, libc::MAP_SHARED,
   vcpu_fd, 0) };

Then, when we call KVM_RUN we simply match the exit reasons and act
accordingly:

let run = vcpu.kvm_run_mem as *mut kvm_run;

loop {
    let ret = unsafe { libc::ioctl(vcpu.fd, KVM_RUN, 0usize) };
    if ret < 0 {
        return Err(io::Error::last_os_error());
    }

    let exit_reason = unsafe { (*run).exit_reason };

    match exit_reason {
        KVM_EXIT_MMIO => { /* omitted */ }
        KVM_EXIT_HLT => {
            println!("Guest halted.");
            break; }
        KVM_EXIT_SHUTDOWN => {
            println!("Guest shutdown.");
            break; }
        KVM_EXIT_INTERNAL_ERROR => {
            return Err(io::Error::other("KVM internal error.")); }
        _ => {
            println!("EXIT REASON = {}", exit_reason);
        }
    }
}

Our first output

Being that our assembly file uses VGA MMIO to output the "Hello world!" string,
I have only implemented KVM_EXIT_MMIO. Then we can finally see the output:

$ cargo run ./samples/hello-world
     ## ommitted sregs2 output and warnings ##
RAX=0x0 RBX=0x0 RCX=0x0 RDX=0x600
RSI=0x0 RDI=0x0 RSP=0x0 RBP=0x0
R8=0x0  R9=0x0  R10=0x0 R11=0x0
R12=0x0 R13=0x0 R14=0x0 R15=0x0
RIP=0x1000      RFLAGS=0x2
Hello from KVM!
Guest halted.
RAX=0xb800      RBX=0x0 RCX=0x0 RDX=0x600
RSI=0x102d      RDI=0x22        RSP=0x0 RBP=0x0
R8=0x0  R9=0x0  R10=0x0 R11=0x0
R12=0x0 R13=0x0 R14=0x0 R15=0x0
RIP=0x101b      RFLAGS=0x46

Conclusion

This concludes the three-part series on figuring out KVM via strace and
reimplementing it in Rust. A lot of next steps actually come down to x86
architecture and bootloader specifics, so we may leave it here for now. The full
code capable of running simple binaries can be found on GitHub:

https://github.com/StjepanPoljak/kvm-rust/tree/kvm-part3-code

KairoDB-Human-Readable Databases

Tue, 23 Jun 2026 17:28:42 +0000

I’ve been working on a small experimental project called KairoDB.

It’s not trying to replace anything big or compete with existing databases.
It’s more like a playground where I test ideas around data storage, structure, and performance.

What is KairoDB?

KairoDB is a lightweight database system built for experimentation and learning.

The goal is simple:

Understand how databases actually think internally
Experiment with storage and retrieval logic
Build something minimal but functional

Core Idea

Instead of overengineering from day one, I focused on a simple pipeline:


Input → Parser → Storage Layer → Query Engine → Output

Nothing fancy. Just clarity first.

Why I built it

Most people learn databases from theory.

I wanted to go one step deeper and actually build parts of it myself.

Not because it's "better", but because it makes concepts stick.

Current Features


* Basic data insertion
* Simple query system
* File-based persistence
* Lightweight architecture


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/tpn0lzmz7wno8390v4wt.png)

Things I learned

How messy real-world data handling can be
Why query optimization matters more than people think
That simplicity is harder than complexity

Project Links

GitHub:
https://github.com/lnfiniware/Kairo

Live / Project:
https://kairo.infiniware.bid

Final Note

This project is still evolving.
I’m not trying to make it perfect — just better every time I touch it.

Feedback is welcome.
Even the painful kind.

Building a KVM Virtual Machine in Rust: Memory Setup

Mon, 15 Jun 2026 05:20:13 +0000

Recap

This is a continuation of my previous article which dealt
with reverse-engineering QEMU with strace to learn how KVM works. Now it's
time to try and follow the steps we got from the strace logs to build our own
KVM-based virtual machine in Rust.

KVM Headers

I haven't actually used existing KVM libraries written specifically for Rust but
opted to use the libc crate which provides the required ioctl bindings and
helper macros. The main reason is that I want a complete understanding of what
is happening under the hood. Now for each of these KVM ioctl calls we can use
the Linux headers for reference. For example, to find out how to construct
ioctl number for KVM_CREATE_VM we simply can do:

$ grep -Rn 'KVM_CREATE_VM' /usr/include/linux/
/usr/include/linux/kvm.h:855:/* machine type bits, to be used as argument to KVM_CREATE_VM */
/usr/include/linux/kvm.h:882:#define KVM_CREATE_VM             _IO(KVMIO,   0x01) /* returns a VM fd */

Luckily in libc Rust crate we have macros for _IO (and the like), but we
still need KVMIO macro:

$ grep -Rn 'define KVMIO' /usr/include/linux/
/usr/include/linux/kvm.h:853:#define KVMIO 0xAE

We can now construct the ioctl number for KVM_CREATE_VM:

use libc::{_IOW, _IO, _IOR};

const KVMIO : u32 = 0xae;
const KVM_CREATE_VM : u64 = _IO(KVMIO, 0x01);

Note that exact integer type depends on the platform and libc definitions.
Then, in the main we can do:

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let file = OpenOptions::new()
        .read(true)
        .write(true)
        .open("/dev/kvm")
        .expect("failed to open /dev/kvm");

    let fd = file.as_raw_fd();
    let vm_fd = unsafe { libc::ioctl(fd, KVM_CREATE_VM, 0usize) };
Ok(())
}

This is the procedure we will follow for each relevant ioctl call. In fact,
we can "reverse-engineer" our own program and then compare it with the original,
to make sure we are doing the right thing:

$ strace cargo run

     ### omitted irrelevant strace output ###

openat(AT_FDCWD, "/dev/kvm", O_RDWR|O_CLOEXEC) = 3
ioctl(3, KVM_CREATE_VM, 0)              = 4

Important note

In production code we would immediately check for a negative return value and
convert errno into a Rust error. To keep the example focused, I am omitting
proper error handling in this article.

Setting memory region

Now, to recall, the next step is setting up the memory region used by both the
KVM guest and the host. This region will be the memory of our virtual machine, a
place where we will load our binary:

140900 mmap(NULL, 1075838976, 0 /* PROT_NONE */, 0x22 /* MAP_PRIVATE|MAP_ANONYMOUS */, -1, 0) = 0x7768b3e00000
140900 mmap(0x7768b3e00000, 1073741824, 0x3 /* PROT_READ|PROT_WRITE */, 0x32 /* MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS */, -1, 0) = 0x7768b3e00000
140900 ioctl(9<anon_inode:kvm-vm>, 0x4020ae46 /* KVM_SET_USER_MEMORY_REGION */, {slot=0, flags=0, guest_phys_addr=0, memory_size=1073741824, userspace_addr=0x7768b3e00000}) = 0

Recreating mmap call

So, first thing we need to do is follow the same logic for mmap which is also
available in the libc Rust crate. After creating the virtual machine, we could
simply recreate our own mmap calls based on the strace output. However,
notice that QEMU first reserves a larger address range with PROT_NONE and then
maps only the portion it actually intends to use. For our prototype we do not
actually need to mimic this exact reservation pattern.

let mem_size: u64 = 256 * 1024;
let mem = unsafe {
libc::mmap(ptr::null_mut(),
   mem_size as usize,
   libc::PROT_READ|libc::PROT_WRITE,
   libc::MAP_PRIVATE|libc::MAP_ANONYMOUS,
   -1,
   0)
};

For this experiment we are only allocating 256 kilobytes because we are not yet
booting a full operating system and therefore need very little guest memory.
Also note that, as with ioctl, production code should check whether mmap()
returned MAP_FAILED.

Recreating ioctl call

First we need to see the definition of the KVM_SET_USER_MEMORY_REGION:

$ grep -Rn 'define KVM_SET_USER' /usr/include/linux/ -A1
/usr/include/linux/kvm.h:1433:#define KVM_SET_USER_MEMORY_REGION _IOW(KVMIO, 0x46, \
/usr/include/linux/kvm.h-1434-                                  struct kvm_userspace_memory_region)

We see that for this one, we need struct kvm_userspace_memory_region. The
values we need are visible in the strace output, but copying struct definition
to our Rust program is really not advisable. Luckily, Rust has bindgen which
we can use to get this KVM struct (and others) from Linux headers. For this
purpose we have a separate build.rs file which will contain:

use bindgen;
use std::path::PathBuf;
use std::env;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let out_path = PathBuf::from(env::var("OUT_DIR")?);

    bindgen::Builder::default()
.header("/usr/include/linux/kvm.h")
.allowlist_type("kvm_userspace_memory_region")
.generate_comments(false)
.generate()?
.write_to_file(out_path.join("kvm-bindings.rs"))?;

    Ok(())
}

Now in the main.rs we can import these bindings with:

include!(concat!(env!("OUT_DIR"), "/kvm-bindings.rs"));

Then, after mmap() calls, we set the memory region, also imitating what
QEMU is doing in the strace output:

let region = kvm_userspace_memory_region {
slot : 0,
flags : 0,
guest_phys_addr : 0x0,
memory_size : mem_size,
userspace_addr : mem as u64
};

let _ret = unsafe { libc::ioctl(vm_fd, KVM_SET_USER_MEMORY_REGION, &region) };

We register this region starting at guest physical address 0x0, meaning the
first byte of our allocated host memory will appear as physical address 0 inside
the guest. Also note that the KVM_SET_USER_MEMORY_REGION call does not copy
memory, but rather tells KVM that guest physical address will be backed by a
specific userspace memory region.

Running the code

Next thing to do is to run it. We still haven't added any checks after mmap
and ioctl calls, but for this prototype we can again simply use strace our
own code:

$ strace -yy -X verbose -e trace=ioctl,mmap,openat,read,write cargo run

                ### omitted irrelevant strace output ###

openat(-100 /* AT_FDCWD */</home/stjepan/Develop/KVM/rust>, "/dev/kvm", 0x80002 /* O_RDWR|O_CLOEXEC */) = 3</dev/kvm<char 10:232>>
ioctl(3</dev/kvm<char 10:232>>, 0xae01 /* KVM_CREATE_VM */, 0) = 4<anon_inode:kvm-vm>
mmap(NULL, 262144, 0x3 /* PROT_READ|PROT_WRITE */, 0x22 /* MAP_PRIVATE|MAP_ANONYMOUS */, -1, 0) = 0x7d63fdfa2000
ioctl(4<anon_inode:kvm-vm>, 0x4020ae46 /* KVM_SET_USER_MEMORY_REGION */, {slot=0, flags=0, guest_phys_addr=0, memory_size=262144, userspace_addr=0x7d63fdfa2000}) = 0

We can see our output is fine and no errors were reported. Note that a full
working example with proper checking and Rust idiomatic approaches can be found
on my GitHub page:

https://github.com/StjepanPoljak/kvm-rust/tree/kvm-part2-code

Next steps

At this point we have a VM object and guest memory, but nothing is actually executing yet. In the next part we will create a vCPU, initialize its state, load a small binary into guest memory and enter the first KVM_RUN loop.

Learning KVM by Reverse-Engineering QEMU with strace

Thu, 11 Jun 2026 15:56:48 +0000

Motivation

I work with virtual machines in QEMU/KVM environment (a lot). In order to debug,
optimize and customize the VMs requires an in-depth knowledge of both QEMU and
KVM, the Linux kernel virtualization subsystem that exposes hardware
virtualization features such as Intel VT-x and AMD-V to userspace applications
like QEMU. Not only that, but I work on a lot of hobby projects requiring quick
bare-metal boot-ups and debugging workflows, and, to be honest, a lot of times
QEMU is an overkill for these sorts of tasks.

KVM vs TCG

Also when it comes to QEMU, it's worth noting that we always have an option of
using TCG, which has a completely different purpose than KVM. TCG is short for
Tiny Code Generator, which works by translating guest instructions into host
instructions at runtime. This is quite slow compared to running code without
translation overhead. So, if we want to test our bare-metal code, we may want to
test it on our own real CPU at native speed. This is where KVM comes in. Unlike
TCG, KVM does not emulate the CPU itself. Instead, it allows guest code to
execute directly on the host processor while the Linux kernel manages
transitions between guest and host execution.

How KVM works in Linux

So, I do already know some basics. KVM driver exposes a driver interface in
Linux root filesystem, /dev/kvm. Communicating with the driver is done via
ioctl() system call on a file descriptor. What we need to find out is how QEMU
communicates with Linux kernel and try and follow the QEMU logic without
reading QEMU source code and KVM API (both can be a bit more intimidating than
just seeing how it works under the hood).

Reverse-engineering KVM

Now we can take some lightweight Debian Linux image and load it into the QEMU,
with KVM enabled:

QEMU_IMAGE=./debian-12-nocloud-amd64.qcow2

qemu-system-x86_64\
-m 1024\
-drive file="${QEMU_IMAGE}",if=virtio,cache=none\
-serial stdio\
-enable-kvm\
-cpu host\
-nodefaults\
-nographic

This is a pretty straightforward way to run QEMU with minimal setup. The most
relevant options for us are -enable-kvm and -cpu host, which will enable
KVM and use host CPU instead of emulating some specific CPU.

Tracing QEMU/KVM with strace

Now, we want to see what QEMU is really doing by utilizing strace. We can put this command in a start-qemu.sh script and call it with:

strace -yy -f -X verbose\
   -e trace=ioctl,openat,read,write,mmap\
   -o kvm.log\
   ./start-qemu.sh

This command will trace all ioctl, openat, read, write and mmap system
calls. Although I mentioned only ioctl calls so far, I always like to include
some other common system calls that could be used. As far as we know /dev/kvm
is the interface to KVM driver and QEMU will probably use openat on it.
Similarly, we also want to see what QEMU is doing with memory and what it's
reading and writing in general.

Note: Information on strace arguments as above can be found in strace --help
or man strace, but essentially, the -yy tells strace to print all available
information when decoding file descriptors, -f follows forks (we need this one
as we're wrapping it in scripts and QEMU might also do similar stuff). The
-X verbose will print names of constants and flags (very important when
analyzing ioctl calls);

Interpreting the logs

Now, we start the above command and, as soon as system boots, we can kill it
with CTRL+C. This will be quite sufficient to see how QEMU/KVM works without
spamming our logs with redundant information. When we read the kvm.log file,
we will see a lot of traces that are not really interesting. However, we already
have some knowledge: we know QEMU should be opening /dev/kvm so a quick
search for kvm reveals exactly what we need:

140900 openat(-100 /* AT_FDCWD */</home/stjepan/Develop/KVM>, "/dev/kvm", 0x80002 /* O_RDWR|O_CLOEXEC */) = 3</dev/kvm<char 10:232>>
140900 ioctl(3</dev/kvm<char 10:232>>, 0xae00 /* KVM_GET_API_VERSION */, 0) = 12
140900 ioctl(3</dev/kvm<char 10:232>>, 0xae03 /* KVM_CHECK_EXTENSION */, 0x88 /* KVM_CAP_IMMEDIATE_EXIT */) = 1
140900 ioctl(3</dev/kvm<char 10:232>>, 0xae03 /* KVM_CHECK_EXTENSION */, 0xa /* KVM_CAP_NR_MEMSLOTS */) = 32764
140900 ioctl(3</dev/kvm<char 10:232>>, 0xae03 /* KVM_CHECK_EXTENSION */, 0x76 /* KVM_CAP_MULTI_ADDRESS_SPACE */) = 2
140900 ioctl(3</dev/kvm<char 10:232>>, 0xae01 /* KVM_CREATE_VM */, 0) = 9<anon_inode:kvm-vm>
140900 ioctl(9<anon_inode:kvm-vm>, 0xae03 /* KVM_CHECK_EXTENSION */, 0x9 /* KVM_CAP_NR_VCPUS */) = 4
140900 ioctl(3</dev/kvm<char 10:232>>, 0xae03 /* KVM_CHECK_EXTENSION */, 0x42 /* KVM_CAP_MAX_VCPUS */) = 4096

We can see that QEMU is opening /dev/kvm and that it's checking API version
and various extensions. We may skip these checks and focus on the calls that
look most important; one of these here is KVM_CREATE_VM which also returns a
file descriptor 9<anon_inode:kvm-vm> which we can use as a further reference.

Setting up memory regions

We know QEMU must eventually load firmware and guest memory into the VM. Looking
for file operations after KVM_CREATE_VM, we quickly encounter SeaBIOS being
loaded:

140900 openat(-100 /* AT_FDCWD */</home/stjepan/Develop/KVM>, "/usr/share/seabios/bios-256k.bin", 0 /* O_RDONLY */) = 12</usr/share/seabios/bios-256k.bin>
140900 mmap(NULL, 2359296, 0 /* PROT_NONE */, 0x22 /* MAP_PRIVATE|MAP_ANONYMOUS */, -1, 0) = 0x776900dc1000
140900 mmap(0x776900e00000, 262144, 0x3 /* PROT_READ|PROT_WRITE */, 0x32 /* MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS */, -1, 0) = 0x776900e00000
140900 openat(-100 /* AT_FDCWD */</home/stjepan/Develop/KVM>, "/usr/share/seabios/bios-256k.bin", 0 /* O_RDONLY */) = 12</usr/share/seabios/bios-256k.bin>
140900 mmap(NULL, 266240, 0x3 /* PROT_READ|PROT_WRITE */, 0x22 /* MAP_PRIVATE|MAP_ANONYMOUS */, -1, 0) = 0x776900fc0000
140900 read(12</usr/share/seabios/bios-256k.bin>, "\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 262144) = 262144

We can see QEMU opening and reading SeaBIOS binary and reserving memory along
with it. We can use the return of the read system call (the size of the
SeaBIOS binary) and see what KVM is doing with it. Searching through the log
for this size gives us further information on what KVM is doing with SeaBIOS:

140900 ioctl(9<anon_inode:kvm-vm>, 0x4020ae46 /* KVM_SET_USER_MEMORY_REGION */, {slot=3, flags=0x2 /* KVM_MEM_READONLY */, guest_phys_addr=0xfffc0000, memory_size=262144, userspace_addr=0x776900e00000}) = 0

So we see that it's now setting this as the memory region for KVM at guest
physical address 0xfffc0000 from userspace address that was actually
obtained by mmap in one of the traces above. In other words, KVM does not
allocate guest RAM itself; userspace applications such as QEMU remain
responsible for managing the backing memory.

Creating vCPU and running

Now, it gets very busy in the logs, but most of the stuff we see is still just
checking for extensions and capabilities. However, if we take a look at the
tail of the log, we will see a lot of these ioctl calls:

140904 ioctl(10<anon_inode:kvm-vcpu:0>, 0xae80 /* KVM_RUN */, 0) = 0
140904 ioctl(10<anon_inode:kvm-vcpu:0>, 0xaeb7 /* KVM_SMI */, 0) = 0
140904 ioctl(10<anon_inode:kvm-vcpu:0>, 0xae80 /* KVM_RUN */, 0) = 0
140904 ioctl(10<anon_inode:kvm-vcpu:0>, 0xae80 /* KVM_RUN */, 0) = 0
140904 ioctl(10<anon_inode:kvm-vcpu:0>, 0xae80 /* KVM_RUN */, 0) = 0
140904 ioctl(10<anon_inode:kvm-vcpu:0>, 0xae80 /* KVM_RUN */, 0) = 0
140904 ioctl(10<anon_inode:kvm-vcpu:0>, 0xaeb7 /* KVM_SMI */, 0) = 0

Now both KVM_RUN and KVM_SMI are operating on a kvm-vcpu file descriptor,
something we haven't yet seen. So if we search the logs for it, we can actually
see where it's created:

140904 ioctl(9<anon_inode:kvm-vm>, 0xae41 /* KVM_CREATE_VCPU */, 0) = 10<anon_inode:kvm-vcpu:0>

Conclusion

Now we have a more complete picture of how QEMU is setting up KVM. First,
/dev/kvm is opened to obtain a file descriptor representing the KVM subsystem.
From it we create a new virtual machine and get kvm-vm file descriptor. On
this file descriptor we are setting up memory regions and later use it to create
a vCPU, on which we can call KVM_RUN. The following diagram explains it
better:

/dev/kvm   kvm fd
    |
    +--> KVM_CREATE_VM   kvm-vm fd
            |
            +--> KVM_SET_USER_MEMORY_REGION
            |
            +--> KVM_CREATE_VCPU   kvm-vcpu fd
                            |
                            +--> KVM_RUN (loop)

As we can see from the logs, KVM_RUN appears repeatedly during guest
execution, while occasional KVM_SMI calls inject System Management Interrupts
into the guest. This repeated interaction between userspace and KVM is what
ultimately drives virtual CPU execution.

Next time we will recreate this exact behavior in Rust and also see about just
a few missing pieces to get our first virtual machine running in KVM.

Why I Still Choose Go After All These Years

Wed, 10 Jun 2026 13:38:44 +0000

Hello everyone.

I'd like to discuss a topic that has sparked endless debates among developers for years:

What programming language should you choose for a serious project?

Throughout my career, I've worked with C, Assembly, Go, and spent a considerable amount of time exploring modern languages such as Rust and C++. The more experience I gain, the more convinced I become that there is no perfect programming language. Every language has its own strengths and weaknesses.

Despite that, I deliberately chose Go as the foundation of my primary project: ForgeZero.

ForgeZero is a cross-platform build orchestration tool written in Go, with carefully optimized performance-critical components implemented in Plan 9 Assembly to minimize runtime overhead and eliminate unnecessary allocations.

At first glance, it might seem logical that after working with C and Assembly, the next step should be Rust or modern C++, since they are widely considered the future of systems programming.

Rust

Rust solves an impressive number of memory safety problems.

Its ownership model and borrow checker force developers to think about lifetimes and resource management before the code even compiles.

However, safety comes at a cost.

There are moments when it feels like the language dictates your architecture rather than the other way around. In large projects, satisfying the borrow checker can sometimes consume more time than solving the original problem itself.

Even with Rust's strict guarantees, bugs still exist. No language can eliminate every possible programming mistake.

Personally, I also find large Rust codebases relatively difficult to read. The amount of syntax and abstractions often makes understanding unfamiliar code slower than it should be.

C++

C++ is an incredibly powerful language.

Unfortunately, that power comes with enormous complexity.

Templates, metaprogramming, concepts, multiple paradigms, countless coding styles, and dozens of ways to solve the same problem can make maintaining large codebases challenging.

Sometimes reading someone else's C++ project feels less like software engineering and more like an archaeological expedition.

C

C remains one of the greatest programming languages ever created.

It provides maximum control over the system, minimal abstractions, and extremely predictable performance, making it an outstanding choice for low-level development.

The downside, however, is responsibility.

Every pointer, every allocation, and every memory access becomes your responsibility. Bugs such as use-after-free, buffer overflows, or segmentation faults can be devastating.

And perhaps even more importantly, building large software systems in C requires an enormous investment of engineering time.

Had ForgeZero been written entirely in C, both development and long-term maintenance would have been significantly more expensive.

Why Go?

Go occupies a very unique position.

It is simple enough that new contributors can understand a codebase within days, while still being powerful enough to build serious production-grade software.

Goroutines, channels, built-in concurrency, and a minimalistic syntax allow developers to focus on solving problems instead of fighting the language.

Of course, Go has its own limitations.

The runtime, garbage collector, and automatic memory management make it a poor choice for operating system kernels, drivers, or extremely low-level software.

Yet the common belief that Go cannot deliver high performance is simply incorrect.

With careful architecture, proper escape analysis, object pooling, and manual optimization of performance-critical paths, it is entirely possible to achieve nearly zero allocations where they actually matter.

That is exactly the direction ForgeZero takes.

For me, Go represents an excellent balance between development speed, maintainability, readability, and performance.

It is not perfect.

But it allows me to build complex tooling faster than most alternatives without sacrificing quality.

That is why, after all these years, I still choose Go.

Am I Betraying C?

Absolutely not.

C shaped me as a software engineer, and I still consider it one of the greatest programming languages ever designed.

In fact, one of ForgeZero's goals is to make C development easier, improving build workflows and project management for both newcomers and experienced engineers.

Go may be my primary language today, but that does not mean abandoning C.

A huge portion of modern software infrastructure still depends on it.

Operating system kernels, drivers, networking libraries, embedded software, and countless system components have been built and refined in C for decades.

Many developers now discuss Rust's introduction into the Linux kernel and the future of systems programming.

But realistically, no major company or open-source project is going to rewrite millions of lines of stable legacy C code overnight simply because a newer language exists.

Large migrations take years—sometimes decades—and happen gradually.

For that reason alone, C will remain relevant for a very long time.

And personally, I still use it whenever it is the right tool for the job.

Recently, while training a small neural network on my home machine, I encountered an interesting problem.

The hardware is far from modern: an eight-core CPU from 2014 paired with a GTX 1060 6GB (Pascal architecture). Although training was performed on the GPU through PyTorch, the CPU was still under significant thermal stress.

Instead of building a monitoring solution in Python or deploying heavyweight background services, I wrote a small daemon in C that continuously monitored CPU temperature and automatically terminated non-essential processes once critical thresholds were reached.

Could I have written the same utility in Go?

Absolutely.

But for this particular low-level task, C was simply the better tool.

No runtime, minimal dependencies, tiny resource usage, and complete control over the operating system.

That is exactly why I never claim that Go should replace C.

Quite the opposite.

I believe that Go + C + Assembly is an incredibly powerful combination.

Go excels at infrastructure, CLI applications, automation, backend services, and distributed systems.
C remains unmatched for systems programming and direct interaction with operating system internals.
Assembly is invaluable when every CPU cycle matters and maximum optimization is required.

A good engineer should choose the right tool for the problem—not the trendiest language on social media.

My Advice for Beginners

If I had to start my programming journey again, I would begin with C.

Not because it is the "best" language, but because it forces you to understand how computers actually work.

How memory is organized.

How the stack and the heap operate.

How pointers behave.

How a compiler transforms source code into assembly instructions and eventually into machine code executed by the processor.

Once those fundamentals become clear, learning Go, Rust, C++, Java, or almost any other language becomes dramatically easier.

You stop memorizing syntax and start understanding software engineering.

So my advice is simple:

Don't be afraid of low-level programming.

Even if your future career revolves around web services or cloud infrastructure, understanding what happens beneath the abstractions will make you a far stronger engineer.

Programming languages come and go.

Fundamental computer science knowledge stays with you forever.

Rust Concepts: dyn Trait, Custom Iterators, Deref/Drop & Axum REST API (Part 5)

Wed, 27 May 2026 07:37:07 +0000

title: Rust Concepts: dyn Trait, Custom Iterators, Deref/Drop & Axum REST API (Part 5)
published: true
tags: rust, webdev, intermediate, systems
cover_image: https://images.unsplash.com/photo-1504639725590-34d0984388bd?w=1000

series: Core Rust Concepts

This is Part 5 of the Core Rust Concepts series.

Part 1 — Ownership, Borrowing, Lifetimes, Traits, Result/Option, Pattern Matching

Part 2 — Closures, Iterators, Generics, Enums, Smart Pointers, Async/Await

Part 3 — Macros, Modules, Testing, Unsafe Rust, FFI

Part 4 — Threads, Channels, Send/Sync, Mutex, Atomics, clap

Trait Objects and Dynamic Dispatch
The Deref Trait
The Drop Trait
Custom Iterators
Building a REST API with Axum

25. Trait Objects and Dynamic Dispatch

In Part 2 we used impl Trait — that's static dispatch: the compiler knows the exact type at compile time and monomorphizes it. Sometimes you don't know the type until runtime. That's where trait objects (dyn Trait) come in.

Static vs Dynamic dispatch

trait Greet {
    fn hello(&self) -> String;
}

struct English;
struct Spanish;

impl Greet for English {
    fn hello(&self) -> String { String::from("Hello!") }
}

impl Greet for Spanish {
    fn hello(&self) -> String { String::from("¡Hola!") }
}

// Static dispatch — type known at compile time, zero overhead
fn greet_static(g: &impl Greet) {
    println!("{}", g.hello());
}

// Dynamic dispatch — type known at runtime via vtable
fn greet_dynamic(g: &dyn Greet) {
    println!("{}", g.hello());
}

fn main() {
    let e = English;
    let s = Spanish;

    greet_static(&e);
    greet_dynamic(&s);

    // Only dyn Trait allows mixing types in a collection
    let greeters: Vec<Box<dyn Greet>> = vec![
        Box::new(English),
        Box::new(Spanish),
        Box::new(English),
    ];

    for g in &greeters {
        println!("{}", g.hello());
    }
}

When to use `dyn Trait` vs `impl Trait`

	`impl Trait`	`dyn Trait`
Dispatch	Static (compile time)	Dynamic (runtime vtable)
Performance	Zero overhead	Small indirection cost
Heterogeneous collections	❌	✅
Return from function	✅ (one concrete type)	✅ (any type at runtime)
Sized	✅	❌ (must be behind `&` or `Box`)

Object-safe traits

Not every trait can be made into a trait object. A trait is object-safe if:

It has no methods that return Self
It has no generic methods

// ✅ Object-safe
trait Drawable {
    fn draw(&self);
    fn area(&self) -> f64;
}

// ❌ NOT object-safe — returns Self
trait Clone2 {
    fn clone2(&self) -> Self; // can't use as dyn Clone2
}

Real-world pattern — plugin system

trait Plugin {
    fn name(&self) -> &str;
    fn run(&self, input: &str) -> String;
}

struct UppercasePlugin;
struct ReversePlugin;

impl Plugin for UppercasePlugin {
    fn name(&self) -> &str { "uppercase" }
    fn run(&self, input: &str) -> String { input.to_uppercase() }
}

impl Plugin for ReversePlugin {
    fn name(&self) -> &str { "reverse" }
    fn run(&self, input: &str) -> String { input.chars().rev().collect() }
}

struct Pipeline {
    plugins: Vec<Box<dyn Plugin>>,
}

impl Pipeline {
    fn new() -> Self { Self { plugins: vec![] } }

    fn add(mut self, p: Box<dyn Plugin>) -> Self {
        self.plugins.push(p);
        self
    }

    fn run(&self, input: &str) -> String {
        self.plugins.iter().fold(input.to_string(), |acc, p| p.run(&acc))
    }
}

fn main() {
    let pipeline = Pipeline::new()
        .add(Box::new(UppercasePlugin))
        .add(Box::new(ReversePlugin));

    println!("{}", pipeline.run("hello")); // "OLLEH"
}

26. The Deref Trait

Deref controls what happens when you use the * operator on a type. It's also what enables deref coercions — Rust automatically calls deref() to convert types so you don't have to do it manually.

use std::ops::Deref;

// A simple smart pointer
struct MyBox<T>(T);

impl<T> MyBox<T> {
    fn new(x: T) -> MyBox<T> { MyBox(x) }
}

impl<T> Deref for MyBox<T> {
    type Target = T;

    fn deref(&self) -> &Self::Target {
        &self.0 // return a reference to the inner value
    }
}

fn hello(name: &str) {
    println!("Hello, {name}!");
}

fn main() {
    let x = MyBox::new(String::from("Rustacean"));

    // These are all equivalent due to deref coercions:
    hello(&x);           // MyBox<String> → &String → &str  (auto!)
    hello(&(*x)[..]);    // manual — what Rust does behind the scenes
    hello(x.deref());    // explicit deref
}

Deref coercion chain

Rust automatically chains Deref calls as many times as needed:

MyBox<String>  →  String  →  str
    &MyBox<String>  →  &String  →  &str

fn main() {
    let boxed = Box::new(String::from("hello"));

    // Box<String> derefs to String, then to str
    println!("{}", boxed.to_uppercase()); // HELLO
    println!("{}", boxed.len());          // 5

    let s: &str = &boxed; // Box<String> → &String → &str — all automatic
    println!("{s}");
}

DerefMut

DerefMut does the same for mutable contexts:

use std::ops::{Deref, DerefMut};

struct Wrapper(Vec<i32>);

impl Deref for Wrapper {
    type Target = Vec<i32>;
    fn deref(&self) -> &Vec<i32> { &self.0 }
}

impl DerefMut for Wrapper {
    fn deref_mut(&mut self) -> &mut Vec<i32> { &mut self.0 }
}

fn main() {
    let mut w = Wrapper(vec![1, 2, 3]);
    w.push(4);                     // DerefMut lets you call Vec methods
    println!("{:?}", w.as_slice()); // Deref: [1, 2, 3, 4]
}

27. The Drop Trait

Drop lets you run custom cleanup code when a value goes out of scope — like a destructor in C++. Rust calls drop() automatically; you never call it yourself (use std::mem::drop() to drop early).

struct Resource {
    name: String,
}

impl Resource {
    fn new(name: &str) -> Self {
        println!("acquiring: {name}");
        Resource { name: name.to_string() }
    }
}

impl Drop for Resource {
    fn drop(&mut self) {
        println!("releasing: {}", self.name);
    }
}

fn main() {
    let _a = Resource::new("database connection");
    let _b = Resource::new("file handle");
    println!("resources in use...");
    // Drop order is LIFO: _b dropped first, then _a
}

Output:

acquiring: database connection
acquiring: file handle
resources in use...
releasing: file handle
releasing: database connection

Dropping early

fn main() {
    let conn = Resource::new("db");
    println!("doing work...");

    drop(conn); // explicit early drop — calls Drop::drop()
    println!("conn already released");

    // conn.name; ← compile error: value was dropped
}

Drop in practice — RAII pattern

use std::sync::{Arc, Mutex};

struct DbPool {
    connections: Arc<Mutex<Vec<String>>>,
}

struct DbConnection {
    id: String,
    pool: Arc<Mutex<Vec<String>>>,
}

impl Drop for DbConnection {
    fn drop(&mut self) {
        // Return the connection back to the pool automatically
        self.pool.lock().unwrap().push(self.id.clone());
        println!("connection '{}' returned to pool", self.id);
    }
}

impl DbPool {
    fn new() -> Self {
        let conns = vec!["conn-1".into(), "conn-2".into(), "conn-3".into()];
        Self { connections: Arc::new(Mutex::new(conns)) }
    }

    fn get(&self) -> Option<DbConnection> {
        let id = self.connections.lock().unwrap().pop()?;
        println!("checked out: {id}");
        Some(DbConnection { id, pool: Arc::clone(&self.connections) })
    }
}

fn main() {
    let pool = DbPool::new();
    {
        let _c1 = pool.get(); // "conn-3"
        let _c2 = pool.get(); // "conn-2"
    } // both returned to pool here automatically
    println!("pool size: {}", pool.connections.lock().unwrap().len()); // 3
}

This is the RAII (Resource Acquisition Is Initialization) pattern — the same one Rust's MutexGuard, file handles, and socket wrappers use internally.

28. Custom Iterators

You can make any type iterable by implementing the Iterator trait, which requires only one method: next().

struct Counter {
    count: u32,
    max: u32,
}

impl Counter {
    fn new(max: u32) -> Self { Self { count: 0, max } }
}

impl Iterator for Counter {
    type Item = u32;

    fn next(&mut self) -> Option<Self::Item> {
        if self.count < self.max {
            self.count += 1;
            Some(self.count)
        } else {
            None // signals the iterator is exhausted
        }
    }
}

fn main() {
    let counter = Counter::new(5);

    // You get ALL iterator methods for free!
    let sum: u32 = counter.sum();
    println!("sum: {sum}"); // 15

    // Chain with other iterators
    let result: Vec<u32> = Counter::new(5)
        .zip(Counter::new(5).skip(1))   // pair consecutive values
        .map(|(a, b)| a * b)            // multiply each pair
        .filter(|x| x % 3 == 0)        // keep multiples of 3
        .collect();

    println!("{:?}", result); // [6, 12]
}

A real-world custom iterator — paginated results

struct Paginator {
    current_page: u32,
    total_pages: u32,
    page_size: u32,
}

impl Paginator {
    fn new(total_items: u32, page_size: u32) -> Self {
        Self {
            current_page: 0,
            total_pages: (total_items + page_size - 1) / page_size,
            page_size,
        }
    }
}

#[derive(Debug)]
struct Page {
    number: u32,
    offset: u32,
    limit: u32,
}

impl Iterator for Paginator {
    type Item = Page;

    fn next(&mut self) -> Option<Page> {
        if self.current_page < self.total_pages {
            self.current_page += 1;
            Some(Page {
                number: self.current_page,
                offset: (self.current_page - 1) * self.page_size,
                limit: self.page_size,
            })
        } else {
            None
        }
    }
}

fn main() {
    let pages = Paginator::new(95, 20);

    for page in pages {
        println!("Page {}: OFFSET {} LIMIT {}", page.number, page.offset, page.limit);
    }
}

Output:

Page 1: OFFSET 0 LIMIT 20
Page 2: OFFSET 20 LIMIT 20
Page 3: OFFSET 40 LIMIT 20
Page 4: OFFSET 60 LIMIT 20
Page 5: OFFSET 80 LIMIT 20

IntoIterator — making structs work in for loops

struct Stack<T> {
    items: Vec<T>,
}

impl<T> Stack<T> {
    fn new() -> Self { Self { items: vec![] } }
    fn push(&mut self, item: T) { self.items.push(item); }
}

impl<T> IntoIterator for Stack<T> {
    type Item = T;
    type IntoIter = std::vec::IntoIter<T>;

    fn into_iter(self) -> Self::IntoIter {
        self.items.into_iter()
    }
}

fn main() {
    let mut stack = Stack::new();
    stack.push(1);
    stack.push(2);
    stack.push(3);

    for item in stack { // works because of IntoIterator
        println!("{item}");
    }
}

29. Building a REST API with Axum

axum is the most popular async web framework in Rust, built on top of tokio and hyper. It uses Rust's type system to make routing and request extraction ergonomic and safe.

Setup

# Cargo.toml
[dependencies]
axum      = "0.7"
tokio     = { version = "1", features = ["full"] }
serde     = { version = "1", features = ["derive"] }
serde_json = "1"
uuid      = { version = "1", features = ["v4"] }

Full CRUD API — Todo List

use axum::{
    extract::{Path, State},
    http::StatusCode,
    response::Json,
    routing::{delete, get, post, put},
    Router,
};
use serde::{Deserialize, Serialize};
use std::sync::{Arc, Mutex};
use uuid::Uuid;

// --- Models ---

#[derive(Debug, Clone, Serialize, Deserialize)]
struct Todo {
    id: String,
    title: String,
    completed: bool,
}

#[derive(Deserialize)]
struct CreateTodo {
    title: String,
}

#[derive(Deserialize)]
struct UpdateTodo {
    title: Option<String>,
    completed: Option<bool>,
}

// --- Shared state ---

type Db = Arc<Mutex<Vec<Todo>>>;

// --- Handlers ---

async fn list_todos(State(db): State<Db>) -> Json<Vec<Todo>> {
    let todos = db.lock().unwrap();
    Json(todos.clone())
}

async fn create_todo(
    State(db): State<Db>,
    Json(payload): Json<CreateTodo>,
) -> (StatusCode, Json<Todo>) {
    let todo = Todo {
        id: Uuid::new_v4().to_string(),
        title: payload.title,
        completed: false,
    };
    db.lock().unwrap().push(todo.clone());
    (StatusCode::CREATED, Json(todo))
}

async fn get_todo(
    State(db): State<Db>,
    Path(id): Path<String>,
) -> Result<Json<Todo>, StatusCode> {
    let todos = db.lock().unwrap();
    todos
        .iter()
        .find(|t| t.id == id)
        .cloned()
        .map(Json)
        .ok_or(StatusCode::NOT_FOUND)
}

async fn update_todo(
    State(db): State<Db>,
    Path(id): Path<String>,
    Json(payload): Json<UpdateTodo>,
) -> Result<Json<Todo>, StatusCode> {
    let mut todos = db.lock().unwrap();
    let todo = todos.iter_mut().find(|t| t.id == id).ok_or(StatusCode::NOT_FOUND)?;

    if let Some(title) = payload.title { todo.title = title; }
    if let Some(completed) = payload.completed { todo.completed = completed; }

    Ok(Json(todo.clone()))
}

async fn delete_todo(
    State(db): State<Db>,
    Path(id): Path<String>,
) -> StatusCode {
    let mut todos = db.lock().unwrap();
    let before = todos.len();
    todos.retain(|t| t.id != id);

    if todos.len() < before { StatusCode::NO_CONTENT }
    else { StatusCode::NOT_FOUND }
}

// --- Router ---

fn app(db: Db) -> Router {
    Router::new()
        .route("/todos",      get(list_todos).post(create_todo))
        .route("/todos/:id",  get(get_todo).put(update_todo).delete(delete_todo))
        .with_state(db)
}

// --- Main ---

#[tokio::main]
async fn main() {
    let db: Db = Arc::new(Mutex::new(vec![]));

    let listener = tokio::net::TcpListener::bind("0.0.0.0:3000").await.unwrap();
    println!("listening on http://localhost:3000");

    axum::serve(listener, app(db)).await.unwrap();
}

Test it with curl

# Create todos
curl -s -X POST http://localhost:3000/todos \
  -H "Content-Type: application/json" \
  -d '{"title": "Learn Rust"}' | jq

curl -s -X POST http://localhost:3000/todos \
  -H "Content-Type: application/json" \
  -d '{"title": "Build an API"}' | jq

# List all
curl -s http://localhost:3000/todos | jq

# Get one (replace ID)
curl -s http://localhost:3000/todos/<id> | jq

# Update
curl -s -X PUT http://localhost:3000/todos/<id> \
  -H "Content-Type: application/json" \
  -d '{"completed": true}' | jq

# Delete
curl -s -X DELETE http://localhost:3000/todos/<id>

Adding middleware — request logging

use axum::middleware::{self, Next};
use axum::extract::Request;
use axum::response::Response;

async fn log_request(req: Request, next: Next) -> Response {
    let method = req.method().clone();
    let uri    = req.uri().clone();
    let res    = next.run(req).await;
    println!("{} {} → {}", method, uri, res.status());
    res
}

fn app(db: Db) -> Router {
    Router::new()
        .route("/todos",     get(list_todos).post(create_todo))
        .route("/todos/:id", get(get_todo).put(update_todo).delete(delete_todo))
        .layer(middleware::from_fn(log_request)) // add logging to all routes
        .with_state(db)
}

Wrapping Up

Concept	What it gives you
dyn Trait	Runtime polymorphism, heterogeneous collections
Deref	Transparent smart pointer ergonomics, auto-coercions
Drop	RAII — deterministic cleanup, no GC needed
Custom iterators	Any type can plug into Rust's iterator ecosystem
Axum	Type-safe, async REST APIs on top of tokio

The full series

Part	Topics
Part 1	Ownership, Borrowing, Lifetimes, Traits, Result/Option, Pattern Matching
Part 2	Closures, Iterators, Generics, Enums, Smart Pointers, Async/Await
Part 3	Macros, Modules, Cargo, Testing, Unsafe, FFI
Part 4	Threads, Channels, Send/Sync, Mutex, Atomics, clap
Part 5	dyn Trait, Deref, Drop, Custom Iterators, Axum REST API

What's in Part 6?

Workspaces and multi-crate projects
serde — serialization deep dive
Benchmarking with criterion
Error handling patterns with thiserror and anyhow
Writing a real-world CLI + library crate

Found this useful? Drop a ❤️ and follow for Part 6!

Rust Concurrency: Threads, Channels, Mutex & Sync (Part 4)

Wed, 27 May 2026 07:36:28 +0000

title: Rust Concurrency: Threads, Channels, Mutex & Sync (Part 4)
published: true
tags: rust, concurrency, intermediate, systems
cover_image: https://images.unsplash.com/photo-1558494949-ef010cbdcc31?w=1000

series: Core Rust Concepts

This is Part 4 of the Core Rust Concepts series.

Part 1 — Ownership, Borrowing, Lifetimes, Traits, Result/Option, Pattern Matching

Part 2 — Closures, Iterators, Generics, Enums, Smart Pointers, Async/Await

Part 3 — Macros, Modules, Testing, Unsafe Rust, FFI

Threads
Message Passing with Channels
Send and Sync Traits
Mutex and RwLock
Atomics
Building a CLI Tool with clap

19. Threads

Rust's standard library provides OS threads via std::thread. The ownership and type system prevents data races at compile time — if your code compiles, it's free of data races. That's a guarantee no other systems language makes.

Spawning a thread

use std::thread;
use std::time::Duration;

fn main() {
    let handle = thread::spawn(|| {
        for i in 1..=5 {
            println!("thread: {i}");
            thread::sleep(Duration::from_millis(50));
        }
    });

    for i in 1..=3 {
        println!("main:   {i}");
        thread::sleep(Duration::from_millis(80));
    }

    handle.join().unwrap(); // wait for the thread to finish
}

Moving data into a thread

Use move to transfer ownership of data into the thread closure:

use std::thread;

fn main() {
    let data = vec![1, 2, 3, 4, 5];

    let handle = thread::spawn(move || {
        // data is moved into this thread
        let sum: i32 = data.iter().sum();
        println!("sum: {sum}");
    });

    // println!("{:?}", data); ← compile error: data was moved

    handle.join().unwrap();
}

Spawning many threads

use std::thread;

fn main() {
    let handles: Vec<_> = (0..8)
        .map(|i| {
            thread::spawn(move || {
                println!("worker {i} done");
                i * i
            })
        })
        .collect();

    let results: Vec<_> = handles
        .into_iter()
        .map(|h| h.join().unwrap())
        .collect();

    println!("squares: {:?}", results);
}

thread::spawn returns a JoinHandle<T> where T is the return type of the closure. Always .join() your handles — a panicking thread silently fails otherwise.

20. Message Passing with Channels

Rust's standard library includes multiple-producer, single-consumer (mpsc) channels. The philosophy: "do not communicate by sharing memory; share memory by communicating."

Basic channel

use std::sync::mpsc;
use std::thread;

fn main() {
    let (tx, rx) = mpsc::channel();

    thread::spawn(move || {
        let msgs = vec!["hello", "from", "the", "thread"];
        for msg in msgs {
            tx.send(msg).unwrap();
        }
    });

    for received in rx {
        println!("got: {received}");
    }
}

Multiple producers

use std::sync::mpsc;
use std::thread;

fn main() {
    let (tx, rx) = mpsc::channel::<String>();

    for id in 0..4 {
        let tx_clone = tx.clone(); // each thread gets its own sender
        thread::spawn(move || {
            tx_clone.send(format!("msg from worker {id}")).unwrap();
        });
    }

    drop(tx); // drop the original so rx knows when all senders are gone

    for msg in rx {
        println!("{msg}");
    }
}

Synchronous channel (bounded)

use std::sync::mpsc;
use std::thread;

fn main() {
    // Buffer of 2 — sender blocks when full
    let (tx, rx) = mpsc::sync_channel::<i32>(2);

    thread::spawn(move || {
        for i in 0..5 {
            println!("sending {i}");
            tx.send(i).unwrap(); // blocks at i=2 until receiver reads
        }
    });

    thread::sleep(std::time::Duration::from_millis(200));

    for val in rx {
        println!("received {val}");
    }
}

For more advanced channel patterns (broadcast, watch, oneshot), use the tokio::sync or crossbeam-channel crates.

21. Send and Sync Traits

These two marker traits are the foundation of Rust's fearless concurrency. You rarely implement them manually — the compiler derives them automatically.

Trait	Meaning
`Send`	Safe to transfer ownership to another thread
`Sync`	Safe to share a reference across threads (`&T` is `Send`)

use std::thread;
use std::rc::Rc;
use std::sync::Arc;

fn main() {
    // Rc<T> is NOT Send — can't move across threads
    // let rc = Rc::new(42);
    // thread::spawn(move || println!("{}", rc)); ← compile error!

    // Arc<T> IS Send — atomic reference counting
    let arc = Arc::new(42);
    let arc_clone = Arc::clone(&arc);

    thread::spawn(move || {
        println!("thread sees: {}", arc_clone);
    }).join().unwrap();

    println!("main sees: {}", arc);
}

Types and their Send/Sync status:

Type	Send	Sync
`i32`, `f64`, `bool`	✅	✅
`String`, `Vec<T>`	✅	✅
`Rc<T>`	❌	❌
`Arc<T>`	✅	✅
`RefCell<T>`	✅	❌
`Mutex<T>`	✅	✅
`*mut T` (raw pointer)	❌	❌

If you try to share a non-Sync type across threads, the compiler refuses to compile. No runtime surprises.

22. Mutex and RwLock

When multiple threads need to mutate shared data, use a Mutex (mutual exclusion lock). Wrap it in Arc to share ownership across threads.

Mutex

use std::sync::{Arc, Mutex};
use std::thread;

fn main() {
    let counter = Arc::new(Mutex::new(0));
    let mut handles = vec![];

    for _ in 0..10 {
        let counter = Arc::clone(&counter);
        let handle = thread::spawn(move || {
            let mut num = counter.lock().unwrap(); // blocks until lock acquired
            *num += 1;
        }); // lock is released here when `num` drops
        handles.push(handle);
    }

    for h in handles { h.join().unwrap(); }

    println!("result: {}", *counter.lock().unwrap()); // 10
}

RwLock — many readers, one writer

use std::sync::{Arc, RwLock};
use std::thread;

fn main() {
    let data = Arc::new(RwLock::new(vec![1, 2, 3]));

    // Spawn 4 reader threads
    let readers: Vec<_> = (0..4).map(|i| {
        let data = Arc::clone(&data);
        thread::spawn(move || {
            let r = data.read().unwrap(); // multiple readers allowed
            println!("reader {i}: {:?}", *r);
        })
    }).collect();

    // One writer thread
    let data_w = Arc::clone(&data);
    let writer = thread::spawn(move || {
        let mut w = data_w.write().unwrap(); // exclusive access
        w.push(4);
        println!("writer pushed 4");
    });

    for r in readers { r.join().unwrap(); }
    writer.join().unwrap();
}

Mutex vs RwLock:

	`Mutex<T>`	`RwLock<T>`
Readers	One at a time	Many simultaneously
Writers	One at a time	One at a time (exclusive)
Best for	Write-heavy workloads	Read-heavy workloads
Overhead	Lower	Higher

Avoiding deadlocks

use std::sync::{Arc, Mutex};

fn main() {
    let a = Arc::new(Mutex::new(1));
    let b = Arc::new(Mutex::new(2));

    // ✅ Always acquire locks in the same order
    {
        let _lock_a = a.lock().unwrap();
        let _lock_b = b.lock().unwrap();
        println!("safe");
    }

    // ❌ Acquiring in different orders across threads = deadlock risk
    // thread 1: lock a → lock b
    // thread 2: lock b → lock a  ← deadlock!
}

23. Atomics

For simple counters and flags shared between threads, std::sync::atomic types are faster than a Mutex — they use hardware atomic instructions with no locking.

use std::sync::Arc;
use std::sync::atomic::{AtomicUsize, Ordering};
use std::thread;

fn main() {
    let counter = Arc::new(AtomicUsize::new(0));
    let mut handles = vec![];

    for _ in 0..8 {
        let counter = Arc::clone(&counter);
        handles.push(thread::spawn(move || {
            for _ in 0..1000 {
                counter.fetch_add(1, Ordering::Relaxed);
            }
        }));
    }

    for h in handles { h.join().unwrap(); }
    println!("total: {}", counter.load(Ordering::SeqCst)); // 8000
}

Memory ordering cheat sheet

Ordering	Use case
`Relaxed`	Counters — no ordering guarantees needed
`Acquire`	Load that synchronizes with a `Release` store
`Release`	Store that publishes data to other threads
`AcqRel`	Read-modify-write (e.g. `fetch_add` with sync)
`SeqCst`	Strongest — total order across all threads

use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::Arc;
use std::thread;

fn main() {
    let ready = Arc::new(AtomicBool::new(false));
    let ready_clone = Arc::clone(&ready);

    let worker = thread::spawn(move || {
        thread::sleep(std::time::Duration::from_millis(100));
        ready_clone.store(true, Ordering::Release);
        println!("worker: done");
    });

    // Spin-wait (not ideal in prod — use a Condvar or channel instead)
    while !ready.load(Ordering::Acquire) {
        thread::yield_now();
    }
    println!("main: worker finished");
    worker.join().unwrap();
}

Available atomic types: AtomicBool, AtomicI8/16/32/64, AtomicU8/16/32/64, AtomicUsize, AtomicIsize, AtomicPtr<T>.

24. Building a CLI Tool with clap

clap is the standard Rust library for building command-line interfaces. It gives you argument parsing, help text, and subcommands with minimal boilerplate.

# Cargo.toml
[dependencies]
clap = { version = "4", features = ["derive"] }

Basic CLI

use clap::Parser;

/// A simple file word counter
#[derive(Parser, Debug)]
#[command(name = "wordcount")]
#[command(about = "Count words in a string or file", long_about = None)]
struct Cli {
    /// The text to count words in
    #[arg(short, long)]
    text: Option<String>,

    /// Show character count too
    #[arg(short, long, default_value_t = false)]
    chars: bool,

    /// Verbosity level
    #[arg(short, long, action = clap::ArgAction::Count)]
    verbose: u8,
}

fn main() {
    let cli = Cli::parse();

    let input = cli.text.unwrap_or_else(|| String::from("hello world"));
    let words = input.split_whitespace().count();

    if cli.verbose > 0 {
        println!("Input: {:?}", input);
    }

    println!("Words: {words}");

    if cli.chars {
        println!("Chars: {}", input.chars().count());
    }
}

Running it:

$ cargo run -- --text "the quick brown fox" --chars -v
Input: "the quick brown fox"
Words: 4
Chars: 19

$ cargo run -- --help
Count words in a string or file

Usage: wordcount [OPTIONS]

Options:
  -t, --text <TEXT>   The text to count words in
  -c, --chars         Show character count too
  -v, --verbose...    Verbosity level
  -h, --help          Print help

Subcommands

use clap::{Parser, Subcommand};

#[derive(Parser)]
#[command(name = "mytool", about = "A multi-command CLI")]
struct Cli {
    #[command(subcommand)]
    command: Commands,
}

#[derive(Subcommand)]
enum Commands {
    /// Add two numbers
    Add { a: f64, b: f64 },
    /// Multiply two numbers
    Mul { a: f64, b: f64 },
    /// Greet someone
    Greet {
        name: String,
        #[arg(short, long, default_value = "Hello")]
        greeting: String,
    },
}

fn main() {
    let cli = Cli::parse();

    match cli.command {
        Commands::Add { a, b } => println!("{} + {} = {}", a, b, a + b),
        Commands::Mul { a, b } => println!("{} × {} = {}", a, b, a * b),
        Commands::Greet { name, greeting } => println!("{greeting}, {name}!"),
    }
}

Running subcommands:

$ cargo run -- add 3.5 2.5
3.5 + 2.5 = 6

$ cargo run -- greet Alice --greeting "Hey"
Hey, Alice!

$ cargo run -- mul 6 7
6 × 7 = 42

For a full production CLI, combine clap with anyhow for error handling, indicatif for progress bars, and colored or owo-colors for terminal color output.

Wrapping Up

Concept	What it gives you
Threads	OS-level parallelism, compile-time data race prevention
Channels	Safe message passing between threads (mpsc)
Send & Sync	Compiler-enforced thread-safety contracts
Mutex & RwLock	Shared mutable state without data races
Atomics	Lock-free counters and flags
clap	Production-grade CLI argument parsing

The full series so far

Part	Topics
Part 1	Ownership, Borrowing, Lifetimes, Traits, Result/Option, Pattern Matching
Part 2	Closures, Iterators, Generics, Enums, Smart Pointers, Async/Await
Part 3	Macros, Modules, Cargo, Testing, Unsafe, FFI
Part 4	Threads, Channels, Send/Sync, Mutex, Atomics, clap

What's in Part 5?

Trait objects and dynamic dispatch (dyn Trait)
The Deref and Drop traits
Custom iterators
Building a real REST API with axum

Found this helpful? Drop a ❤️ and follow for Part 5!

Sorting Wars: Benchmarking Selection, Insertion, Gnome e Bubble Sort em 5 Linguagens

Fri, 22 May 2026 14:58:16 +0000

Algoritmos quadráticos são lentos — isso todo mundo sabe. Mas o quão lentos? E a linguagem escolhida importa mais do que o algoritmo em si? Foi exatamente isso que nossa turma de Algoritmos e Estruturas de Dados I do CEFET-MG decidiu testar empiricamente.

Implementamos Selection Sort, Insertion Sort, Gnome Sort e Bubble Sort em C, C++, Rust, JavaScript e Python, rodamos benchmarks com até 1 milhão de elementos e coletamos os dados. Esse post resume o que encontramos — incluindo alguns resultados que nos surpreenderam bastante.

Por que estudar algoritmos O(n²) em 2025?

Antes de mergulhar nos números, vale responder a pergunta óbvia. Três razões principais:

Didática — a lógica é direta, ideal para aprender análise de complexidade e invariantes de laço.
Entradas pequenas — para N ≤ ~50, o overhead reduzido pode torná-los mais rápidos que algoritmos sofisticados.
Adaptatividade — Insertion Sort e Bubble Sort com parada antecipada chegam a O(n) em entradas quase ordenadas. É por isso que o TimSort (usado em Python e Java) usa Insertion Sort internamente para subconjuntos pequenos.

Os Algoritmos

Selection Sort — O inflexível

Seleciona o mínimo do subarray não ordenado e o move para a posição correta com uma única troca.

PARA i DE 0 ATE N - 2 FACA
  MIN_IDX <- i
  PARA j DE i + 1 ATE N - 1 FACA
    SE VETOR[j] < VETOR[MIN_IDX] ENTAO
      MIN_IDX <- j
  SE MIN_IDX != i ENTAO
    TROCA VETOR[i] COM VETOR[MIN_IDX]

Característica marcante: sempre realiza exatamente n(n-1)/2 comparações, independente da entrada. Não existe melhor caso em comparações — é Θ(n²) em todos os cenários.

Caso	Comparações	Complexidade
Melhor (ordenado)	n(n−1)/2	O(n²)
Médio (aleatório)	n(n−1)/2	O(n²)
Pior (invertido)	n(n−1)/2	O(n²)

✅ In-place · ❌ Não estável · ❌ Não adaptativo

Insertion Sort — O adaptativo

Mantém um subarray ordenado à esquerda e insere cada novo elemento na posição correta via shifting.

PARA i DE 1 ATE N - 1 FACA
  CHAVE <- VETOR[i]
  j <- i - 1
  ENQUANTO j >= 0 E VETOR[j] > CHAVE FACA
    VETOR[j + 1] <- VETOR[j]
    j <- j - 1
  VETOR[j + 1] <- CHAVE

Em entradas já ordenadas, o laço interno nunca executa: complexidade O(n) no melhor caso. Isso ficou claro nos nossos dados — até 10⁶ elementos, o cenário crescente terminou em microssegundos.

Caso	Comparações	Complexidade
Melhor (ordenado)	n − 1	O(n)
Médio (aleatório)	n(n−1)/4	O(n²)
Pior (invertido)	n(n−1)/2	O(n²)

✅ In-place · ✅ Estável · ✅ Adaptativo

Gnome Sort — O excêntrico

Opera como um "gnomo organizando vasos": avança comparando adjacentes, e ao encontrar desordem, troca e recua uma posição.

INDICE <- 0
ENQUANTO INDICE < N FACA
  SE INDICE = 0 OU VETOR[INDICE] >= VETOR[INDICE - 1] ENTAO
    INDICE <- INDICE + 1
  SENAO
    TROCA VETOR[INDICE] COM VETOR[INDICE - 1]
    INDICE <- INDICE - 1

Funcionalmente equivalente a um Insertion Sort por trocas — mas com movimento local em vez de shifting. Também adaptativo: O(n) no melhor caso.

Caso	Comparações	Complexidade
Melhor (ordenado)	n − 1	O(n)
Médio (aleatório)	(A DEPENDER)	O(n²)
Pior (invertido)	n(n−1)/2	O(n²)

✅ In-place · ✅ Estável · ✅ Adaptativo

Bubble Sort — O clássico (com parada antecipada)

Realiza passagens comparando pares adjacentes. A variável swapped permite encerramento antecipado quando o vetor já está ordenado.

PARA i DE 0 ATE N - 1 FACA
  SWAPPED <- FALSO
  PARA j DE 0 ATE N - i - 2 FACA
    SE VETOR[j] > VETOR[j + 1] ENTAO
      TROCA VETOR[j] COM VETOR[j+1]
      SWAPPED <- VERDADEIRO
  SE SWAPPED = FALSO ENTAO
    RETORNE  // Encerramento antecipado

Caso	Comparações	Complexidade
Melhor (ordenado)	n − 1	O(n)
Médio (aleatório)	n(n−1)/4	O(n²)
Pior (invertido)	n(n−1)/2	O(n²)

✅ In-place · ✅ Estável · ✅ Adaptativo (com parada antecipada)

Metodologia

Tamanhos de entrada: N ∈ { 10², 10³, 10⁴, 10⁵, 10⁶ }
Cenários: crescente (melhor caso), aleatório (caso médio), decrescente (pior caso)
Repetições: 5 execuções; média das 3 centrais (descartando menor e maior)
Otimizações testadas: -O0 e -O3 para C/C++/Rust
Medição: apenas tempo de ordenação (I/O descartado)

Cada linguagem usou sua estrutura de dados nativa:

Linguagem	Estrutura	Tipo
C	`double[]` com `malloc`	`double` 64-bit
C++	`std::vector<double>`	`double` 64-bit
Rust	`Vec<f64>`	`f64` 64-bit
JavaScript	`Array` (V8 float64)	`Number`
Python	`list` (refs a PyObject)	`float`

Resultados — Selection Sort

O Selection Sort foi o mais previsível: tempos quase idênticos nos três cenários, confirmando a não-adaptatividade.

N = 10⁵ — tempo em segundos (-O3 / otimização máxima)

Linguagem	Crescente (s)	Aleatório (s)	Decrescente (s)
C (-O3)	2.0500	3.5188	1.9618
C++ (-O3)	2.1247	3.5188	1.9089
Rust (-O3)	2.3265	4.2560	2.1320
JavaScript (V8)	3.5835	6.6564	3.5851
Python	93.164	164.890	98.378

Python para N = 10⁶ foi descartado — tempo estimado superior a 10 horas.

Hierarquia: C -O3 ≈ C++ -O3 < Rust < JavaScript ≪ Python

Resultados — Insertion Sort

Aqui a adaptatividade brilhou. No cenário crescente com N = 10⁶, todas as linguagens terminaram em menos de 12 ms. Já no cenário decrescente, o pior caso quadrático foi brutal:

N = 10⁵ — tempo em segundos (-O3 / otimização máxima)

Linguagem	Crescente (s)	Aleatório (s)	Decrescente (s)
C (-O3)	0.000055	0.615683	1.188966
C++ (-O3)	0.0000634	0.587483	1.183000
Rust (-O3)	0.000079	0.897895	1.774292
JavaScript (V8)	0.000809	1.525250	3.033301
Python	0.005321	136.295	226.560

Para N = 10⁶ no pior caso sem otimização, C++ chegou a 3670 s (~1 hora). Com -O3, caiu para 119 s — speedup de ~30×.

Resultados — Gnome Sort

O Gnome Sort trouxe a maior surpresa da série: Rust superou C e C++ no caso médio.

N = 10⁵ — tempo em segundos (-O3 / otimização máxima)

Linguagem	Crescente (s)	Aleatório (s)	Decrescente (s)
C (-O3)	0.000179	15.19068	28.90668
C++ (-O3)	0.000159	13.96807	28.25787
Rust (-O3)	0.000049	3.94810	8.27646
JavaScript (V8)	0.001584	13.90908	27.45163
Python	0.019030	21.28324	257.00624

Por quê? O padrão de acesso do Gnome Sort — muitos acessos locais e trocas adjacentes — se beneficia especialmente das otimizações do compilador LLVM (backend do Rust), que gera código de swap em registrador sem acesso intermediário à memória.

Resultados — Bubble Sort

O Bubble Sort entregou o resultado mais inesperado de todo o projeto:

N = 10⁵ — tempo em segundos (-O3 / otimização máxima)

Linguagem	Crescente (s)	Aleatório (s)	Decrescente (s)
C (-O3)	0.000042	20.489569	23.568626
C++ (-O3)	0.000045	21.165759	23.919268
Rust (-O3)	0.000051	13.513529	9.231813
JavaScript (V8)	0.001753	15.012168	6.905683
Python	0.003504	312.509064	403.412438

JavaScript bateu C e C++ no pior caso. O motor V8 identificou o padrão repetitivo de trocas adjacentes e gerou código de máquina altamente especializado via JIT. É um caso raro onde a compilação dinâmica supera a compilação estática.

Principais Aprendizados

1. Otimização de compilador importa muito mais do que parece

O impacto de -O3 vs -O0 para Rust no Bubble Sort chegou a 23× para N = 10⁴. Em modo debug, o Rust mantém verificações de bounds checking em todo acesso ao array — correto para desenvolvimento, devastador para performance.

2. Adaptatividade é real e mensurável

Insertion Sort com N = 10⁶ no cenário crescente: < 1 ms. No cenário decrescente: até 3670 s (sem otimização). Isso é uma diferença de mais de 10 ordens de magnitude no número de operações.

3. Python tem overhead estrutural, não só de interpretador

A list do Python armazena referências a objetos PyObject — cada acesso e comparação envolve uma indireção de ponteiro. Isso multiplica o custo por operação mesmo para tipos simples como int e float.

4. JIT pode superar código estático em padrões repetitivos

O V8 observa o comportamento em runtime e especializa o código gerado. Para algoritmos com padrões de acesso altamente previsíveis (como Bubble Sort), isso pode ser mais eficiente que otimizações estáticas conservadoras de um compilador AOT.

5. A hierarquia não é fixa

A ordem geral foi C ≈ C++ < Rust < JS < Python, mas exceções importantes existem — Rust venceu todos no Gnome Sort, JS venceu todos no Bubble Sort em pior caso. O melhor resultado depende do algoritmo, tamanho de entrada e padrão de acesso.

Conclusão

Algoritmos simples ensinam mais do que complexidade assintótica. Ensinam sobre a relação entre padrões de acesso à memória e otimizações de compilador, sobre como compilação JIT funciona na prática, e sobre por que constantes importam tanto quanto expoentes para tamanhos reais de entrada.

Se você quiser explorar o código completo, estrutura de Makefiles, geradores de dataset e scripts de benchmark, o repositório está público no GitHub.

{% embed https://github.com/Bernardo-Lebron/simple_sort_algorithms %}

Trabalho prático da disciplina de Algoritmos e Estruturas de Dados I — CEFET-MG Campus V, Divinópolis. Orientação: Prof. Michel Pires.

I'm Building a Multi-Target Compiler Backend from Scratch — No LLVM, No Crutches

Thu, 21 May 2026 20:34:21 +0000

Hi there, i'm Gideon. Roughly 1.5 years of writing C++ from the ground up — ray tracers, video codecs, and now a compiler. No frameworks. No LLVM. Just me, the hardware manuals, and a lot of wrong turns.

This post starts a series where I document the build in real time. I'm currently in the parser stage. By the end, I want a compiler that emits x86-64 and SPIR-V from a C++-like language, with SIMD vectorization and security-hardened codegen baked in.

What I'm Actually Building

Not a programming language. A compiler backend toolkit — the part that turns intermediate representation into fast machine code across multiple targets.

The pipeline:

Source → Parser → AST → SSMOL (HIR) → MREL (LIR) → x86-64 / SPIR-V / ARM64 / RISC-V / WASM

MREL is my target-agnostic low-level IR. It knows about virtual registers, stack slots, and machine operations — but not physical register names. The backend handles that per-target.

Why Not Just Use LLVM?

LLVM is 4 million lines of code. It solves everyone's problem and no one's perfectly. I need:

Fine-grained control over SIMD width selection per target
Constant-time crypto primitive emission with secret register annotations
Security obfuscation passes (control flow flattening, opaque predicates)
A codebase I fully understand and can license

Building from scratch is slower. But I own every decision.

Where I Am Right Now

Parser stage. Hand-written recursive descent. C++-like syntax with:

Functions, structs, basic types
Ownership semantics (borrowed from my Rust phase, simplified)
Explicit SIMD types (v128, v256, v512)

The parser emits an AST that gets lowered to SSMOL — my high-level IR that knows about types, ownership, and semantics.

What's Next

SSMOL → MREL lowering (types to sizes, structs to offsets, control flow to basic blocks)
MREL → x86-64 backend (register allocation, instruction selection, ELF emission)
One working program: compile, link, run main() that returns 42

Then SPIR-V compute kernels. Then the rest.

What I'll Write About

Each stage, when I hit it. The problems that took me three days to solve. The specs I wrote to keep myself honest. The wrong assumptions that cost me a week.

Not polished tutorials. Build logs from someone actually building.

Follow This Series If

You work in systems, compilers, or graphics
You're curious what "building from scratch" actually looks like
You want to see if I crash or ship

My Specs (For the Curious)

I write technical specifications to keep the design coherent across months of work. The MREL backend spec covers x86-64, ARM64, RISC-V, SPIR-V, and WebAssembly with calling conventions, opcode tables, and security passes. Link in bio.

Substack: (https://ayndlr.substack.com)

Closing

This is post 1 of however many it takes. Next post: parsing expressions with operator precedence and why I gave up on Pratt parsing.

Follow for the crash or the ship. Either way, it's real.

Data Sanitization is a Design Flaw: How Penta-V Prevents Data Pollution at the Hardware-Software

Mon, 18 May 2026 03:10:10 +0000

Go to GitHub, search for "data cleaning library," and you will find thousands of repositories. From Pandas to Pydantic, the entire software engineering industry has been conditioned to treat data sanitization as a reactive, post-facto chore. You ingest data, you realize it’s full of missing values, NaNs, or volatile floating-point noise (Inf), and then you write heavy, resource-expensive loops to "clean" it.

Here is the cold, hard truth that legacy architectures refuse to admit: If your system is spending CPU cycles cleaning data inside its execution core, your architecture has already failed. In high-frequency pipelines and autonomous AI environments, traditional cleaning introduces massive latency spikes, destroys CPU cache locality, and leads to Logic Drift—where the state of your system silently decays.

When we engineered the Penta-V Kernel (v0.4.0), we didn't build another data-cleaning library. We built src/processing/cleaner.rs (PentaCleaner) to prove a different architectural paradigm: We don't clean data; we structurally prevent data pollution from existing.

The Legacy Trap: Why Runtime Cleaning Kills High-Frequency Systems

In standard hybrid pipelines (Python bridging into Rust via PyO3), data passed from autonomous AI agents is inherently probabilistic and volatile.

When a naive system encounters corrupted data at runtime, it usually triggers one of two reactive paths:

The Allocation Nightmare: It instantiates dynamic buffers on the heap to reconstruct tables (e.g., df.fillna()), causing memory fragmentation and triggering garbage collection jitter.
The Unsafe Race: It forces raw parallel mutation via unsafe pointers, risking data races and undefined behavior (UB) under sudden systemic stress.

Penta-V eliminates this by transforming data purification from an operational step into an Adaptive Geometric Constraint.

1. Zero-Allocation Parallel Extraction (Eliminating Memory Debt)

Instead of allocating new memory frames to clean data, the PentaCleaner leverages a highly optimized, lock-free parallel extraction pipeline driven by safe Rust abstractions (Polars and Rayon).

Look at how the core execution loop handles a contaminated DataFrame:

// src/processing/cleaner.rs

pub fn geometric_sanitize(df: &mut DataFrame, state: &ProcessingState) -> PolarsResult<()> {
    let pressure = state.data_pressure;

    // SAFE & LOCK-FREE PARALLELISM:
    // We leverage low-level pointer duplication (shallow clones) to parallelize 
    // column purification across all available CPU cores without single-line heap allocations.
    let updated_columns: Vec<Series> = df
        .get_columns()
        .par_iter()
        .map(|series| {
            let mut clned = series.clone(); // Cheap shallow clone (O(1) pointer copy)
            Self::purify_column(&mut clned, pressure);
            clned
        })
        .collect();

    // Reconstruct the DataFrame in a single CPU cycle by consuming the safe vector
    *df = DataFrame::new(updated_columns)?;

    Ok(())
}

Why this is prevention, not cleaning:

By enforcing compile-time data aliasing rules, the Rust compiler guarantees that no two threads can cause a data race. The allocation cost is exactly zero because series.clone() in Polars is merely a shallow reference duplication. The system doesn't wait for data to become a problem; it enforces a strict parallel layout where invalid structures cannot propagate.

2. Pressure-Aware Structural Adaptation (The Anti-Fragile Gate)

A static data cleaner is blind. It executes the exact same parsing script whether your server is idling at 5% or burning at 99% capacity. Under extreme load, this blindness causes memory queues to back up, leading to catastrophic system throttles.

Penta-V introduces Pressure-Aware Sanitization. The kernel dynamically mutates its purification strategy based on the thermodynamic state of the infrastructure (data_pressure):

fn purify_column(series: &mut Series, pressure: f64) {
    // Dynamic Strategy Selection based on real-time hardware telemetry
    let strategy = if pressure > 0.8 {
        // High-Intensity: Forward fill maintains the continuous geometric sequence 
        // under extreme load without interrupting the ALU pipeline.
        FillNullStrategy::Forward(None)
    } else {
        // Standard Baseline: Zero-fill enforces an absolute mathematical baseline.
        FillNullStrategy::Zero
    };

    if let Ok(filled) = series.fill_null(strategy) {
        *series = filled;
    }
}

The Engineering Magic:

When system stress crosses the critical 0.8 barrier, the kernel automatically swaps the logic profile. It stops attempting heavy static recalculations and shifts to an Asymptotic Geometric Sequence (Forward Fill). This prevents the execution queue from stalling, absorbing the kinetic shock of incoming data anomalies in real-time.

3. The Structural Immunity Audit: Penta-V vs. The Market

To understand why this approach renders traditional cleaning obsolete, we must look at how data corruption behaves across different software layers:

Architectural Metric	Traditional Cleaners (`Pandas` / `Pydantic`)	Native Engines (`Polars` Alone)	Penta-V Kernel Substrate (`PentaCleaner`)
System Interaction	Reactive (Cleans after corruption infects the memory).	Pure Processing (Executes instructions blindly).	Sovereign Prevention (Sanitizes at the FFI boundary before core access).
Hardware Awareness	Completely Blind (Heavy CPU/Memory overhead).	Static Multi-threading (Fixed execution path).	Dynamic Telemetry (`Pressure-Aware` adaptation under load).
Memory Delta	High (Continuous heap allocations and fragmentation).	Optimized but subject to continuous runtime re-allocations.	0.0000 MB Dynamic Delta via stack-resident pointer manipulation.
Execution Latency	Milliseconds ($ms$) — Destroys high-frequency pipelines.	Microseconds ($\mu s$).	Sub-nanosecond (0.85ns flat) via bit-level vector alignment.

Conclusion: Stop Cleaning. Start Hardening.

Writing code to "clean" corrupted data is a tacit admission that your system boundaries are porous. In the era of autonomous AI codebases and ultra-low latency requirements, you cannot afford to let bad data enter your core logic and then figure out how to patch it.

The PentaCleaner module within the Penta-V Kernel shifts the paradigm. By combining the raw parallel power of Polars and Rayon with a pressure-sensitive, zero-allocation design, it treats data corruption not as an administrative problem to be logged, but as a kinetic deficit to be neutralized.

When your system boundaries are architecturally incorruptible, "data cleaning" ceases to be a maintenance cost—it disappears from existence.

Full Source & Docs:

GitHub: https://github.com/narukihto/Penta-V-Kernel/tree/Heartbeat

Crates.io: https://crates.io/crates/penta_v_kernel

PyPI: https://pypi.org/project/penta-v-kernel/

Penta-V Kernel (v0.4.0) is compiled, hardened, and testing green at sub-nanosecond intervals. Let’s argue about system architecture in the comments below! ️⚡

How We Built Penta-V: A Rust Kernel That Wipes Out 50% of Technical Debt in AI Pipelines Under 1ns

Sat, 16 May 2026 08:25:10 +0000

Discover how Penta-V Kernel uses strict geometric traits and 1-cycle bitwise masking in Rust to eliminate FFI boilerplate, floating-point jitter, and branch misprediction debt.

Every software architect fears the same ghost: Technical Debt. We’ve all been there—choosing the quick, dirty hack over clean architecture to hit a deadline. We promise ourselves, "We’ll refactor it later." But "later" never comes. Instead, we pay compounding interest in the form of flaky systems, continuous bugs, and degraded performance.

Now, shift this reality to the context of DeepTech and Hybrid AI Pipelines (Python/Rust). When you bridge the chaotic, non-deterministic world of Python-based AI agents with the strict, low-level ecosystem of Rust, traditional technical debt doesn't just accumulate—it explodes.

In this article, I will tear down the architecture of Penta-V Kernel, a deterministic geometric substrate implemented in Rust, and demonstrate how its design structurally eliminates 40% to 50% of architectural and performance technical debt before a single line of bad code can even infect your system.

The Core Problem: Why AI Pipelines Accumulate Dead Memory & Logic Drift

When building high-frequency trading systems or orchestration layers for autonomous AI agents, developers heavily rely on Foreign Function Interfaces (FFI)—specifically bridging Python (PyO3) and Rust.

This introduces three hidden, compounding debts:

The FFI Boilerplate Debt: Writing manual type-checkers and safety wrappers that inevitably lead to runtime crashes and unmaintainable spaghetti code.
Silent Data Corruption (IEEE-754 Jitter): Python passes volatile floating-point numbers. Over long-context iterations, tiny round-off errors accumulate in the mantissa, leading to what we call Logic Drift—where the AI agent’s state subtly collapses without triggering a panic.
Performance Regression Debt: Complex object mapping and multi-layered arithmetic loops kill the processor's branch predictor, driving latencies way above the acceptable threshold.

We engineered Penta-V to act as a sovereign validation gate that solves this. Here is how it works under the hood.

1. Zero-Cost Architectural Decoupling: The Shapes Framework

One of the biggest contributors to Scalability Debt is tightly coupled code. In naive designs, every time you add a new heuristic or operational parameter, you are forced to modify the central routing mechanism or the core execution loop. This violates the Open/Closed Principle.

Penta-V mitigates this by abstracting operations into strict geometric traits. Take a look at our directory layout:

src/
├── bridge/      # Validates and packs PyO3/Python signatures
├── core/        # Enforces absolute boundaries (Guard & Cooling)
├── shapes/      # Polygons acting as decoupled mathematical capacities
└── utils/       # Micro-coherence lattices (Resonance)

In src/shapes/, every polygon—from triangle.rs to dodecagon.rs—implements a unified GeometricShape trait:

pub fn validate_logic_stability(
    state: &KernelState,
    signature: &LogicSignature,
    active_shape: &dyn GeometricShape
) -> bool {
    let capacity = active_shape.calculate_dissipation(state.current_stability);
    let logical_impact = signature.stress_level * (1.0 + signature.complexity_index);

    logical_impact <= capacity && state.current_stability > SECURE_CORE
}

How this wipes out debt:

The core validator doesn’t know—and doesn't care—what shape is currently processing the load. If a developer wants to introduce 100 new logic profiles tomorrow, they simply write a new shape file implementing the trait.

Zero modifications to the core kernel.
No regression testing needed for older modules.

2. Eliminating Arithmetic Complexity Debt: The Bit-Level Manifold Purge

In early iterations of our Phase VI Hyperdimensional Resonance Lattice (src/utils/resonance.rs), we explored continuous matrix folding to purge micro-drifts in floating-point data.

However, standard floating-point multiplication loop sequences introduce compiler optimizations ambiguity and variable CPU execution cycles. That is an unacceptable performance debt.

We refactored the entire lattice to run on Deterministic Bitwise Masking. Instead of executing expensive, floating-point cycles to fix noise, Penta-V drops straight down to raw bit-space using a fixed mantissa truncation barrier.

/// Ultra-fine mask targeting only the absolute lowest 2 bits of the 52-bit mantissa.
/// This clears microscopic float divergence without altering the macro value.
const LATTICE_PURGE_MASK: u64 = 0x_FFF_FFF_FFF_FFF_FFFC;

#[inline(always)]
pub fn stabilize(state: &mut KernelState) {
    let scalar = state.current_stability;
    if !scalar.is_finite() { return; }

    // 1. Forward Projection into the Manifold Array
    let amplitude = scalar * INV_SQRT_CARDINALITY;
    let mut components = [amplitude; RESONANCE_CARDINALITY];

    // 2. Sub-nanosecond Bit Purge
    for c in components.iter_mut() {
        let raw_bits = c.to_bits();
        let stabilized_bits = raw_bits & LATTICE_PURGE_MASK;
        *c = f64::from_bits(stabilized_bits);
    }

    // 3. Inverse Projection 
    let stabilized: f64 = components.iter().sum::<f64>() * INV_SQRT_CARDINALITY;

    if stabilized.is_finite() && stabilized > 0.0 {
        state.current_stability = stabilized;
    }
}

How this wipes out debt:

By trading traditional mathematical loops for a bitwise AND (&), the entire stabilization process executes in exactly 1 CPU cycle per component inside the ALU. It achieves two critical things:

It guarantees a deterministic latency well under 1 nanosecond (approx 0.85ns).
It completely purges the microscopic FFI jitter from Python before it cascades into a runtime corruption. You will never spend weeks tracking down phantom rounding bugs.

3. The Sovereign Defensive Sentinel: Linear Control Flow

When a kernel is placed under heavy stress, many developers resort to deep branch nesting (if/else inside loops) or runtime exceptions to handle edge cases. This introduces severe branch misprediction debt at the hardware level.

Penta-V’s src/core/guard.rs enforces a strict Linear Control Flow using a defensive throttle mechanism:

#[inline(always)]
pub fn apply_damage_with_cooling(
    state: &mut KernelState,
    impact: f64,
    cooling: &mut CoolingProtocol,
) {
    if !impact.is_finite() || impact < 0.0 { return; }

    let effective_impact = impact * cooling.reduction_factor;
    let new_stability = state.current_stability - effective_impact;

    if new_stability <= SECURE_CORE {
        state.current_stability = SECURE_CORE;
        cooling.activate(); // Escalation barrier triggered instantly
    } else {
        state.current_stability = new_stability;
    }
}

By ensuring that data flows sequentially and inputs are sanitized at the very entry gate, the CPU pipeline stays clean. The memory overhead remains an absolute 0.0000 MB delta, and state transitions are entirely predictable.

The Technical Debt Audit: Penta-V vs. Traditional Wrappers

Technical Debt Category	Traditional Runtime Guards (High-Level Python/C)	Penta-V Core Substrate (Rust Geometric Engine)
Performance Overhead	Microseconds to Milliseconds (Introduces Garbage Collection Jitter)	Sub-nanosecond (0.85ns flat) via zero-cost abstractions and Bitwise ops.
Architectural Coupling	High. Modifications to schema require altering central validators.	Zero Coupling. Enforced via polymorphic Traits and isolated module layers.
State Divergence (Logic Drift)	Ignored until a fatal system crash or invalid output occurs.	Proactively Purged at bit-level on every single state lifecycle step.

Conclusion: Engineering for Longevity

Penta-V wasn't built to fix bad code style; it was built to protect the structural survival of low-latency systems.

By pushing validation to compile-time Type safety, mapping data to decoupled geometric traits, and cleaning floating-point jitter inside a 1-cycle bitwise manifold, it systematically pays off the architectural debt that kills deep-tech infrastructure.

When your underlying engine provides absolute, deterministic immunity to state decay, your development team can focus entirely on shipping features rather than hunting memory ghosts and tracing compiler lag.

Full Source & Docs:

GitHub: https://github.com/narukihto/Penta-V-Kernel/tree/Heartbeat

Crates.io: https://crates.io/crates/penta_v_kernel

PyPI: https://pypi.org/project/penta-v-kernel/

Penta-V Kernel is fully open-source, fully optimized, and testing green at 100% fidelity. Check out our structure and let's talk devs in the comments below! ️⚡

Ditch the Clunky GUIs: Meet ComChan, the Blazingly Fast Rust Serial Monitor

Fri, 15 May 2026 15:58:16 +0000

If you spend any significant amount of time doing embedded systems development, you know the drill. You write your firmware, flash the microcontroller, and then… you stare at a serial monitor.

For years, the options have been a bit of a mixed bag. You have the classic Arduino IDE Serial Monitor (which is functional but basic), bloated GUI protocol analyzers that eat up RAM, or simple CLI tools like minicom that get the job done but lack modern developer quality-of-life features.

As someone who lives in the terminal—relying on tools like Neovim and terminal multiplexers—I wanted a serial monitor that actually fit into a keyboard-driven, modern workflow. I wanted something lightweight, visually clean, and capable of handling complex data without breaking a sweat.

So, I built ComChan (Communication Channel).

What is ComChan?

ComChan is a minimal, blazingly fast serial monitor and plotter built entirely in Rust. It’s designed specifically for embedded systems engineers who want powerful data visualization and logging without leaving the command line.

Because it's written in Rust, it’s memory-safe, incredibly fast, and runs natively across Linux, Windows, and macOS.

Here is a look at the features that make ComChan stand out from the crowd.

1. The Terminal-Based Serial Plotter

Sometimes, raw text isn't enough. When you are debugging an IMU, reading ADC values, or tracking temperature sensors, you need to see the data visually.

Instead of exporting data to Excel or firing up a heavy GUI application, ComChan features a built-in TUI (Terminal User Interface) plotter. You simply pass the --plot flag, and ComChan will automatically parse your incoming numeric data, assign legends, and graph multiple sensor channels in real-time—right inside your terminal.

Found a spike you want to investigate? You can hit Ctrl+S to instantly export the current plot frame as a clean SVG file.

Here is how it'll look like

2. Time Travel with Session Replay

This is perhaps my favorite feature. Debugging intermittent hardware anomalies is a nightmare because you can't always recreate the exact physical conditions that caused the bug.

ComChan introduces Session Replay. You can log your raw serial output to a .log or .csv file during a test run. Later, you can run comchan --replay <file> and the tool will parse the timestamps and play the data back to you in real-time.

It perfectly recreates the exact timing of the original hardware run. You can even replay a log file directly into the visual Plotter to analyze the anomaly visually, without needing the physical microcontroller plugged in.

3. Continuous CSV Streaming

If you are doing data collection for research or analysis, ComChan can automatically parse your incoming numeric telemetry and stream it directly into a clean, multi-column .csv file on the fly.

The best part? This works concurrently. You can stream to a CSV in the background while actively watching the data on the TUI Plotter.

4. Built for the Modern Embedded Workflow

ComChan includes several smaller features designed specifically to remove friction from the embedded workflow:

Auto-Detection: Tired of typing /dev/ttyUSB0 or COM3? Just use comchan --auto and it will automatically find and connect to your active USB serial device.
Zephyr RTOS Support: If you work with Zephyr, ComChan has a dedicated --zephyr shell mode that handles its specific echo and prompt behaviors cleanly.
Auto-Recovery: If you accidentally unplug your device or reset the power, ComChan won't crash. It detects the broken pipe and gracefully waits to reconnect when the hardware comes back online.
Configuration Files: Save your favorite baud rates, data bits, and parity settings in a .toml file so you don't have to pass a dozen flags every time you launch.

The "Brain Made" Philosophy

While AI tools are great for scaffolding tests or micro-optimizations, the core architecture, TUI design, and feature logic of ComChan are entirely human-crafted. It’s built to solve real pain points that I experience daily when working with memory-safe embedded development on ESP32s and RP2040s.

Try It Out

If you are ready to upgrade your embedded debugging workflow, ComChan is incredibly easy to install.

If you have Cargo installed:

cargo install comchan

For Arch Linux users, it's available on the AUR:

yay -S comchan

You can check out the full source code, read the documentation, and contribute on GitHub: Vaishnav-Sabari-Girish/ComChan

Drop a star if you find it useful, and let me know what features you’d like to see next (a Hex Dump mode is already on the roadmap!). Happy debugging!

I built a web scraper in Rust that bypasses Cloudflare without a browser

Thu, 14 May 2026 10:41:46 +0000

Every AI agent has the same problem. You ask it to read a webpage and it comes back with a 403, or worse, 5000 tokens of navigation bars and cookie banners.

I spent the last few months building webclaw to fix this.

The problem

Try fetching any real website with a standard HTTP client. Most of them will block you. Cloudflare, Akamai, DataDome, they all look at your TLS fingerprint before the request even reaches the server.

The usual fix is spinning up a headless Chrome. That works, but now you need 500MB of browser, it takes 2-3 seconds per page, and you still get all the HTML noise.

What webclaw does differently

Instead of launching a browser, webclaw impersonates one at the TLS level. The TCP handshake, cipher suites, extensions, everything looks like Chrome 142. Most anti-bot systems pass the request through because the fingerprint is already valid.

Then the extraction engine scores every DOM node by text density, semantic tags, and link ratio. Navigation, ads, footers, cookie banners get stripped. What comes out is clean markdown.

A real example: a news article that is 4,820 tokens as raw HTML becomes 1,590 tokens after webclaw processes it. Same content, 67% less tokens.

Architecture

webclaw is a Rust workspace with 6 crates:

webclaw-core    pure extraction, zero network deps, WASM-safe
webclaw-fetch   HTTP + TLS fingerprinting via primp
webclaw-llm     LLM provider chain (Ollama > OpenAI > Anthropic)
webclaw-pdf     PDF text extraction
webclaw-cli     CLI binary
webclaw-mcp     MCP server for AI agents

The split between core and fetch was intentional. webclaw-core takes a &str of HTML and returns structured output. No I/O, no network calls, no allocator tricks. It should compile to WASM without changes.

Extraction speed on the core alone (no network):

Page size	Time
10 KB	0.8ms
100 KB	3.2ms
500 KB	12.1ms

How to use it

CLI

# basic extraction
webclaw https://example.com

# different output formats
webclaw https://example.com -f json
webclaw https://example.com -f llm

# crawl a docs site
webclaw https://docs.example.com --crawl --depth 2

# extract structured data with LLM
webclaw https://example.com --extract-prompt "get all pricing tiers"

# track page changes
webclaw https://example.com -f json > snapshot.json
webclaw https://example.com --diff-with snapshot.json

MCP server (for Claude, Cursor, Windsurf, Codex)

npx create-webclaw

One command. It detects what AI tools you have installed and writes the config for each one. After restart you get 10 tools: scrape, crawl, search, extract, summarize, brand, diff, map, batch, research.

Docker

docker run --rm ghcr.io/0xmassi/webclaw https://example.com

128 MB image. Works on any machine.

Benchmarks

Tested on 50 real pages across news sites, documentation, e-commerce, SPAs, and blogs.

Metric	webclaw	readability	trafilatura	newspaper3k
Extraction accuracy	95.1%	83%	80%	66%
Noise removal	96.1%	79%	73%	61%

The biggest wins are on JavaScript heavy sites. When the visible DOM is empty because content is in embedded JSON (Next.js, React SSR payloads), webclaw has a data island extractor that pulls content from __NEXT_DATA__, window.__data, and similar patterns. Most other tools return nothing.

What I learned building this

TLS fingerprinting is fragile. Chrome updates their cipher suites every few versions and you have to keep up. I am using primp, which maintains patched forks of rustls, hyper, and h2. It works well but it is a maintenance burden. If Chrome ships a new TLS extension tomorrow, requests start getting blocked until the forks are updated.

The extraction scoring took the most iteration. Early versions were too aggressive and would strip content that looked like navigation (short paragraphs with links). The fix was a semantic bonus system: nodes inside <article> or <main> tags get a score boost, nodes with content-related class names get another boost. Combined with link density penalties, it handles most layouts without site-specific rules.

Try it

MIT licensed, fully open source.

GitHub: https://github.com/0xMassi/webclaw
Website: https://webclaw.io
Discord: https://discord.gg/KDfd48EpnW

If you run into a site that webclaw fails on, open an issue. Every edge case makes the extraction better.

GraphRAG: Introducing CoordiNode (Alpha)

Tue, 12 May 2026 18:41:39 +0000

Building relationship-aware AI applications today usually means duct-taping three different systems together: Neo4j for the graph, Pinecone for vectors, and Elasticsearch for text.

This infrastructure nightmare leads to:

Writing endless glue code.
Maintaining brittle data sync pipelines.
Zero transactional consistency across your data silos.

After getting completely fed up with this, I spent the last few months building an alternative: CoordiNode.

What is CoordiNode?

CoordiNode is an open-source (AGPL-3.0), graph-native hybrid retrieval engine built entirely in Rust. It unifies graph traversal, vector similarity search, and full-text retrieval into a single transactional engine.

The Elephant in the Room: Why not Neo4j, ArangoDB, or SurrealDB?

Whenever a new database drops, the immediate question is: why reinvent the wheel? Here is exactly why we built this:

Why not Neo4j? Neo4j is the king of graphs, but it runs on the JVM. We wanted a lightweight, zero-GC, single-binary engine that delivers predictable performance without JVM tuning overhead.
Why not SurrealDB or ArangoDB? Both are fantastic multi-model databases. However, they rely on proprietary query languages (SurrealQL and AQL). We firmly believe OpenCypher is the gold standard for graph traversals. CoordiNode is built to be OpenCypher-compatible from day one.

Core Features

One Engine, One Query: Traverse a knowledge graph, filter by vector distance, and rank by text match in a single Cypher query.
Predictable Performance: Pure Rust implementation using a native seqno MVCC with a unified oplog (built on a modified Fjall LSM-tree).
ACID Transactions: Full Snapshot Isolation across all your graph, text, and vector data simultaneously.

The "Magic Moment" Query

Imagine running this in one transaction:

MATCH (topic:Concept {name: "machine learning"})-[:RELATED_TO*1..3]->(related)
MATCH (related)<-[:ABOUT]-(doc:Document)
WHERE vector_distance(doc.embedding, $question_vector) < 0.4
  AND text_match(doc.body, "transformer attention mechanism")
RETURN doc.title, vector_distance(doc.embedding, $question_vector) AS relevance

The Reality Check (Early Alpha v0.3.x)

I want to be 100% transparent—this is an early alpha. The core is solid, but bugs are expected. Here is our actual roadmap status:

Single-node only: Currently optimized for 100K-1M nodes. Distributed Raft clustering is in active development (v0.4).
No Bolt Protocol (Yet): Use our native gRPC, REST proxy, or HTTP. Neo4j driver compatibility (Bolt) is planned for v1.2.
In-Memory Vectors: HNSW indexes currently reside in RAM (e.g., 1M vectors × 384 dims = ~1.5GB RAM).

Try it out

You can spin it up via Docker in under 5 minutes:

git clone https://github.com/structured-world/coordinode.git
cd coordinode
docker compose up -d

We are a small team building the infrastructure layer for AI-native applications. If you believe graph + vector + text should live in one engine under a genuine open-source license, give it a spin!

We would be incredibly grateful for your feedback or bug reports on GitHub.

GitHub: https://github.com/structured-world/coordinode

If you like what we're building, feel free to drop a ⭐ on GitHub! It helps us keep going.

How I tested 16 processes racing for the same task in 200 lines of Rust

Mon, 11 May 2026 10:02:16 +0000

I shipped a small tool last week called coord — a local daemon that lets parallel AI coding agents (Claude Code, Cursor, Codex, all running side by side) coordinate through a shared bulletin board. The whole product is one Rust binary, MIT, single SQLite file for state.

The interesting part isn't the daemon. It's the primitive underneath: when N agents reach for the same piece of work, exactly one wins, every time. No queue, no actor system, no distributed lock. One SQLite UPDATE and a thin Rust wrapper.

This post is about what that primitive looks like and how I convinced myself it's correct — including the test that proves it survives 16 independent OS processes hammering the same daemon over real HTTP.

The problem, one paragraph

You run two AI coding tabs side by side. They post tasks to a shared bulletin: bug reports, ack notes, "please pick this up." Both tabs scan the bulletin every turn. Both tabs sometimes see the same pending bug at the same instant. Without a coordination primitive, both tabs grab it, both write fixes, both push different commits. The merge conflict is the cheapest failure mode; the silent "two tabs running expensive parallel work, only one of them needed" is worse.

The job of coord is to make exactly one of those tabs the winner of the claim, and to tell the other tab "you lost; pick something else." Determinism, not optimism.

Why a queue or a mutex doesn't cut it

The first idea most engineers have is "stick a Mutex<Vec<Task>> behind an RPC call." It works inside one process. It does not survive process boundaries — and the whole point of the system is that the agents are different processes (different IDE plugins, different language runtimes, sometimes different machines on the same loopback).

The second idea is a queue (Redis, RabbitMQ, NATS). A queue does solve the claim race, but it brings a lot of baggage:

A second daemon to install, monitor, and back up.
A protocol the IDE side has to learn (or you end up writing a queue-to-MCP adapter).
Ordering and delivery guarantees you don't actually need for "is this task claimed yet?" semantics.

The third idea is a distributed lock (e.g. Redis SETNX, Postgres advisory locks). Closer, but still a separate piece of infrastructure for what is essentially a per-developer-laptop tool.

For a tool that wants to be brew install-and-go, the right primitive is the smallest one that does the job. SQLite already lives inside the daemon binary. SQLite already serializes writes. The job becomes "ask SQLite to tell me whether I won."

The whole primitive

This is the entire claim function:

pub fn claim_task(&self, id: Uuid, agent_id: &str) -> Result<Option<Task>> {
    let now = Utc::now();
    let conn = self.conn.lock();
    let updated = conn.execute(
        "UPDATE tasks SET state = 'claimed', claimed_by = ?1, updated_at = ?2
         WHERE id = ?3 AND state = 'pending'",
        params![agent_id, now.to_rfc3339(), id.to_string()],
    )?;
    if updated == 0 {
        Ok(None) // somebody else got there first
    } else {
        // re-read to get the canonical row, including the winner.
        Ok(Some(conn.query_row(
            "SELECT ... FROM tasks WHERE id = ?1",
            params![id.to_string()],
            row_to_task,
        )?))
    }
}

The whole correctness argument is in the WHERE id = ?3 AND state = 'pending' clause. Two clients may issue the same UPDATE simultaneously; SQLite serializes the writes; the second one finds state is no longer 'pending' and updates 0 rows. updated == 0 is the loser signal, propagated up the stack as Ok(None). The winner gets a fresh Some(Task). That's it.

There is no lock object, no compare-and-swap dance, no transaction explicit in user code. The transaction is the single statement. SQLite's WAL mode handles concurrent readers without blocking the writer.

The reason I'm comfortable putting this in a critical path is that it's the same primitive every database-backed work queue you've ever used uses underneath. Postgres SELECT ... FOR UPDATE SKIP LOCKED, Redis SETNX, all the big managed queue services — at the bottom they're some flavor of "atomic compare-and-set on a row." This one just doesn't dress it up.

Test 1: in-process, exhaustive

The first test hammers the primitive directly without going through the network at all. The intent: catch correctness regressions in the SQL or the Rust wrapper, not the HTTP layer.

#[test]
fn one_claimer_per_task_under_concurrent_load() {
    let store = Arc::new(Store::open(&db_path).unwrap());

    const TASKS: usize = 200;
    const CLAIMERS_PER_TASK: usize = 8;

    let task_ids: Vec<_> = (0..TASKS)
        .map(|i| store.create_task(&format!("race-{i}"), json!({"i": i}))
                  .unwrap().id)
        .collect();

    let mut handles = Vec::new();
    for task_id in &task_ids {
        for c in 0..CLAIMERS_PER_TASK {
            let store = store.clone();
            let id = *task_id;
            let agent = format!("claimer-{c}");
            handles.push(thread::spawn(move || {
                store.claim_task(id, &agent).unwrap().is_some()
            }));
        }
    }

    let wins: usize = handles.into_iter()
        .map(|h| h.join().unwrap() as usize)
        .sum();

    assert_eq!(wins, TASKS,
        "expected exactly one win per task (got {wins} wins for {TASKS} tasks)");
}

200 tasks × 8 claimers = 1,600 simultaneous claim attempts. The assertion is severe: the total number of Ok(Some(_)) returns across all threads must equal exactly 200. If any task is claimed twice, the count becomes 201+ and the test fails. If any task is missed entirely (e.g. a row-level lock dropped a write silently), the count becomes 199 and the test fails. The only way this assertion holds is if every single attempt is unambiguously a winner or a loser, with no ambiguous middle.

It runs in about 70 milliseconds in CI. I run it on every commit.

Test 2: real processes, real HTTP

The in-process test is reassuring but it has a giant asterisk: it doesn't prove the daemon survives multiple HTTP clients. SQLite serializing writes inside one Rust process is one thing; an axum server holding a single Mutex<Connection> while 16 reqwest clients hit it concurrently is another. The behavior I cared about — "16 IDE tabs in different OS processes, one daemon, exactly one winner per task" — has to be tested end-to-end.

So the second test spawns the actual daemon binary, lets it bind a free port, and fires 16 independent client processes at it:

const NUM_CLIENTS: usize = 16;

#[test]
fn many_concurrent_clients_share_one_daemon_safely() {
    // 1. Start a real `coord serve` on a free localhost port.
    let mut daemon = Command::new(&coord)
        .arg("serve").arg("--addr").arg(format!("127.0.0.1:{port}"))
        .arg("--db").arg(&db).spawn().unwrap();

    // 2. Seed N pending tasks via the CLI (separate process).
    let task_ids: Vec<String> = (0..NUM_CLIENTS).map(|i| {
        // ... shells out to `coord send` and parses the returned UUID
    }).collect();

    // 3. Spawn N independent shell scripts, each racing to claim
    //    every task in the list.
    let mut children = Vec::new();
    for c in 0..NUM_CLIENTS {
        let agent = format!("multi-agent-{c}");
        let mut script = String::new();
        for tid in &task_ids {
            script.push_str(&format!(
                "{coord} --url {url} claim {tid} --as {agent} \
                 >/dev/null 2>&1 && echo {tid}\n"));
        }
        children.push(Command::new("sh").arg("-c").arg(&script)
            .stdout(Stdio::piped()).spawn().unwrap());
    }

    // 4. Collect every (winner, task_id) pair.
    let mut all_wins = Vec::new();
    for (idx, child) in children.into_iter().enumerate() {
        let out = child.wait_with_output().unwrap();
        for line in String::from_utf8_lossy(&out.stdout).lines() {
            all_wins.push((idx, line.trim().to_string()));
        }
    }

    // 5. Assert: every task has exactly one winner.
    let mut by_task: HashMap<String, Vec<usize>> = Default::default();
    for (idx, id) in &all_wins {
        by_task.entry(id.clone()).or_default().push(*idx);
    }
    assert_eq!(by_task.len(), NUM_CLIENTS);
    for (_id, winners) in &by_task {
        assert_eq!(winners.len(), 1, "race lost: {winners:?}");
    }
}

A few things worth pointing out about this test design:

The clients are real processes, not threads. Command::new("sh") forks. Each one gets its own coord CLI invocation, its own reqwest client, its own TCP connection. There is no shared memory between them and no shared Tokio runtime. That's the actual deployment shape.
Each script tries to claim every task. With 16 clients and 16 tasks, that's 256 individual claim attempts hitting the daemon. The daemon has to reject 240 of them and accept 16 of them, with no overlap.
The assertion is again a counting argument. by_task.len() == NUM_CLIENTS says "every task got at least one winner;" the per-task winners.len() == 1 says "no task got more than one winner." Together they're a strict bijection between tasks and winners.
It runs in about 1.3 seconds, including spawning the daemon, waiting for it to bind, seeding tasks, racing, killing the daemon. Slow enough to be annoying as a unit test, fast enough to keep in CI.

This test is the one that gives me confidence to claim "coord scales to N agents and N apps" in the README. Not because 16 is a large number, but because the failure modes for 16 are the failure modes for 1,600 — if it works at this scale and the design hasn't introduced an O(N) lock contention path, it works at the next scale up.

What I didn't need to build

The motivation for writing this post is half technical and half cultural. There is a strong gravity in 2026 engineering culture toward immediately reaching for distributed primitives — actors, channels, CRDTs, lock services — when the actual workload fits comfortably on one machine.

For coord's actual workload (a developer's laptop, 2–10 agents, a few thousand tasks per day) the right answer is a single SQLite file and a single statement. The work isn't "design a queue;" the work is "trust the database to do its job, and verify it with a test that would catch a regression."

Things I considered and threw out:

A tokio::sync::Mutex<HashMap<Uuid, Task>> — works in-process, dies the moment the daemon restarts.
An actor per task — neat but every task lives only seconds; the spawn cost dominates.
Optimistic concurrency at the application level (read row, compare version, write) — that's just reimplementing what the SQL WHERE state = 'pending' clause already does for free, with more chances to get it wrong.
A pluggable storage backend — overkill until somebody asks. The whole product is one binary right now.

The shape of the test suite reinforces the design. If the primitive were doing more, the tests would have to be more complicated to constrain its behaviour. They aren't, because it isn't.

What you should take from this

If you're building anything where two agents (humans, processes, or LLM tabs) might reach for the same piece of work at once, you probably do not need a queue. You need a single atomic compare-and-set, plus a test that proves you can issue 1,600 of them concurrently and get exactly the right number of winners.

The technique generalizes far past AI agents:

Job runners (one task, one worker)
Lease/lock servers (one holder, until release)
Reservation systems (one seat, one buyer)
Cron deduplication (one runner per scheduled instant)

In each case the loop is the same: a WHERE state = 'available' clause in an UPDATE, the rowcount as your winner signal, and a counting-argument test that asserts the total number of winners across all attempts equals the number of available slots. That last test, the counting argument, is what turns "I think this is right" into "I know this is right." It catches things eyeballed code reviews don't — the clever-but-wrong refactor, the changed default isolation level, the silent race introduced by a connection pool tweak.

If you want to see the full machinery, coord is open-source: github.com/DmarshalTU/coord. The two tests above live in tests/race.rs and tests/multi_client.rs. Together they're under 200 lines and they're the load-bearing proof for everything else the project promises.

coord runs locally as a single Rust binary. It's MIT-licensed and installable with brew tap dmarshaltu/coord && brew install coord. The repo is github.com/DmarshalTU/coord.

Rust Generator Yield

Sun, 10 May 2026 20:44:41 +0000

Rust Generator yield: What the Compiler Actually Builds Under async/await

Every async fn you've ever written in Rust compiles down to something you probably never asked to see. The Rust generator yield mechanism isn't an exotic nightly toy — it's the actual substrate that async/await desugars into, and the compiler generates state machines from it that can quietly balloon your binary by hundreds of kilobytes. Most devs ship this without knowing it exists.

TL;DR: Quick Takeaways

Every Rust async fn desugars into a generator-backed state machine — a compiler-generated enum with one variant per suspension point.
The state machine size grows with nesting depth: a real-world async chain can produce enum variants exceeding 400 KB (Tweede golf, 2024 measurement).
Generators behind #![feature(coroutines)] are still nightly-only in 2025; stable workarounds exist but carry trade-offs.
Python's yield and Rust's yield look identical syntactically — the execution contracts are completely different at the machine level.

What the Rust Compiler Does with yield: A State Machine You Never See

When rustc compiles an async function, it runs a desugaring pass that converts suspension points into enum variants. Each .await call becomes a potential yield point; the compiler assigns each one a unique state index and generates a match arm in the polling loop. What you get is not a thread, not a closure, not a callback — it's a resumable enum that the executor pokes with poll() until it returns Poll::Ready.

The struct that wraps this enum implements Future. Inside sits the compiler-generated state machine with fields for every local variable that's alive across a suspension point. That's the part nobody warns you about: variables don't get dropped at the .await boundary — they get saved into the enum variant. If you have a fat buffer alive when you hit .await, that buffer now lives in the enum. Permanently. Until the future resolves.

// What you write:

async fn fetch_data(url: &str) -> Vec {

 let response = client.get(url).await; // suspension point 1

 let body = response.bytes().await; // suspension point 2

 body.to_vec()

}
// What the compiler roughly generates (simplified MIR view):

enum FetchDataStateMachine<'a> {

 State0 { url: &'a str }, // before first await

 State1 { url: &'a str, response: Response }, // between awaits

 State2 { body: Bytes }, // after second await

 Complete,

}

The mini-analysis here is straightforward but the implications aren't. State1 holds both url and response simultaneously because the borrow checker needs to prove both are valid at that point. The enum variant is sized to the largest variant — so if Response is 1 KB, the entire future allocates for that worst case even if most code paths never hit State1. This is where rust async function memory layout becomes a production concern, not a trivia question.

The Jump Table and MIR Output

The state machine dispatches via a jump table internally — the executor calls poll(), the match expression on the state index branches to the right arm, execution resumes from the exact suspension point. This is cooperative scheduling at zero OS overhead: no context switch, no kernel involvement, no stack swap. The Waker mechanism handles re-scheduling by registering a callback that the executor fires when the awaited I/O is ready.

If you want to actually see what rustc generates, run cargo rustc -- --emit=mir on a small async function. The MIR output is verbose but legible — you'll see the generator drop glue, the state enum definition, and the GeneratorResumeArgument plumbing. It's ugly in the good way: no magic, just a lot of autogenerated match arms.

Rust Generator vs Python Generator: Same Word, Different Contract

Python developers picking up Rust often assume yield works the same way. It doesn't — and the difference matters enough to trip up experienced engineers. In Python, yield suspends a function and returns a value to the caller; the caller can push a value back in via gen.send(value). The generator maintains its own implicit stack frame that Python manages for you. You don't think about lifetimes. You don't think about pinning. You don't think about drop order.

// Python mental model:

def counter(start):

 n = start

 while True:

 received = yield n # yields n, receives next value

 n = received or n + 1
gen = counter(0)

next(gen) # → 0

gen.send(10) # → 10
// Rust nightly equivalent (coroutines feature):
![feature(coroutines, coroutine_trait)]
use std::ops::{Coroutine, CoroutineState};

use std::pin::Pin;
let mut gen = #[coroutine] |initial: i32| {

 let mut n = initial;

 loop {

 let received = yield n; // type of received = i32

 n = received;

 }

};
match Pin::new(&mut gen).resume(0) {

 CoroutineState::Yielded(v) => println!("{v}"), // 0

 CoroutineState::Complete(v) => {}

}

The structural difference: Python generators are heap-allocated, garbage-collected, and implicitly pinned. Rust coroutines are stack-allocated by default, borrow-checker-validated, and require explicit Pin<&mut Self> to resume. The StopIteration exception in Python maps to CoroutineState::Complete in Rust — same conceptual role, completely different mechanism. Python devs learning Rust generators shouldn't expect a soft landing here.

The send() Asymmetry

Python's gen.send(value) is two operations fused: it resumes the generator AND injects a value that becomes the result of the yield expression. Rust's resume(arg) works the same way, but the type of the resume argument is part of the coroutine's type signature — the compiler enforces it. You can't accidentally send a string to a coroutine that expects an integer. Python discovers that at runtime; Rust refuses to compile it. Whether that's "better" depends on how much you enjoy type errors at 2 AM.

Why async/await in Rust Is a Generator in Disguise

This isn't metaphorical. Before Rust stabilized async/await syntax, the internal implementation literally used std::future::from_generator() — a function that wrapped a coroutine-like closure into something that implements Future. The wrapper was called GenFuture. You can find it in old Rust source and in pre-1.36 nightly code. The async keyword was syntactic sugar over this machinery from day one.

// Pre-stabilization desugaring (historical, ~Rust 1.36):

// async fn example() -> i32 { ... }

// compiled roughly to:

fn example() -> impl Future {

 from_generator(#[coroutine] || {

 // body with yields instead of awaits

 let x = yield some_future;

 x + 1

 })

}
// Modern compiler does this internally but hides the plumbing.

// The GenFuture wrapper still exists in the stdlib — it's just not public API.

The from_generator wrapper implements Future::poll by calling Coroutine::resume with a Poll-compatible argument. When the coroutine yields, it maps to Poll::Pending. When it completes, it maps to Poll::Ready. The Waker gets threaded through context via thread-local storage in the poll_with_tls_context helper. This is the async state machine — not an abstraction over it, not analogous to it. It is it.

The Stackless Architecture Consequence

Because Rust async uses stackless coroutines — meaning each suspended future stores only the variables it actually needs, not an entire call stack — you can have thousands of concurrent futures in a single thread without proportional memory cost. A Tokio application handling 10,000 simultaneous connections doesn't maintain 10,000 stacks. Each future sits as a struct on the heap, sized exactly to its state machine. This is the zero-cost abstraction claim made concrete: you pay for what you suspend, nothing else.

Rust Generator on Stable: Patterns That Don't Need Nightly

Here's the uncomfortable reality: the #![feature(coroutines)] gate has been open since RFC 2033 landed in 2016, and as of 2025 it's still not stable. The RFC got renamed, partially redesigned, merged with gen block proposals under RFC 3513, and the stabilization timeline remains "when it's ready." If you're shipping production Rust and need generator-like behavior today, you work around the gate.

// Stable pattern 1: std::iter::from_fn

// Generates an infinite Fibonacci sequence without nightly

let mut state = (0u64, 1u64);

let fibs = std::iter::from_fn(move || {

 let next = state.0 + state.1;

 state = (state.1, next);

 Some(state.0)

});
// Stable pattern 2: genawaiter crate (0.99.x)

// Provides gen!() macro that emulates generator syntax on stable

use genawaiter::{sync::gen, yield_};
let mut generator = gen!({

 yield!(1u32);

 yield!(2u32);

 yield_!(3u32);

});
// Stable pattern 3: Rust 2024 gen blocks (RFC 3513)

// Available in nightly as of late 2024, stabilization target: 2025 edition

// gen { yield 1; yield 2; } → implements Iterator directly

std::iter::fromfn is the zero-dependency stable answer for simple lazy sequences — you manually maintain state in a captured variable. It works, it's readable, and it compiles on stable going back years. The genawaiter crate goes further: it provides yield!() macro syntax that feels like real generators and works on stable Rust by implementing the state machine explicitly. The gen {} block from RFC 3513 is the official future — it targets the Iterator trait directly and sidesteps the coroutine/generator naming debate entirely.

The Hidden Cost: How Generator State Size Grows with Nesting

The state machine size problem is real and it bites in production. Tweede golf (a Dutch embedded systems consultancy) published measurements in 2024 showing that a moderately complex async function chain in an embedded Rust application generated a future struct exceeding 400 KB. That's a single future. On a microcontroller with 512 KB of RAM total, that's not a performance problem — it's a won't-compile problem.

The mechanism is straightforward once you understand enum layout. A Rust enum is sized to its largest variant. If you have an async function that calls three other async functions sequentially and holds local state between each call, the compiler generates an enum where the largest variant contains all the nested futures plus all the local variables. Those nested futures are themselves enums sized to their largest variants. The size compounds multiplicatively, not additively. Deeply nested async call chains can produce state machines that dwarf the actual code doing work.

The fix is Box::pin(): boxing a future puts it on the heap and stores only a pointer in the state machine. Box> costs one heap allocation and pointer indirection per await point, but the state machine size becomes constant regardless of what's inside. For embedded targets or size-sensitive code, this trade-off is often worth it. For throughput-sensitive server code, it's sometimes a regression. Profile before deciding.

The full deep-dive on Rust Generator yield - https://krun.pro/rust-generator-yield/

Mesa: Why I built a programming language

Sun, 10 May 2026 10:34:43 +0000

The Motivation: Technological Sovereignty
Most modern languages rely on a mountain of dependencies. As a student, I wanted to build something different: a "sovereign" language. I wanted a single native binary that could do everything out of the box.

That’s how Mesa-LP was born. It started in Python, but for version 4.0.0, I decided to take the final leap: A full rewrite in Rust.

Why the rewrite?

Moving from Python to Rust wasn't just about speed (though it’s incredibly fast now). It was about:
Zero Dependencies: No more pip or npm. Everything is baked into the binary.
Memory Safety: Leveraging Rust's borrow checker to build a robust engine.
Native Power: Implementing a real HTTP server using TcpListener and a toolkit for x86 assembly.

A Bilingual AST
One of Mesa's core features is native bilingualism. It’s not a wrapper; the Lexer and Parser handle English and Spanish with total parity.

Whether you write say() or decir(), the AST treats them as the same. This makes programming more accessible to the Spanish-speaking community without losing the "standard" feel of English coding.

Show me the code!
Mesa-LP is a multi-level language. You can build a web server or a bootsector.

Fragmento de código
-- High-level: A simple web server
page("Mesa World", "dark")
navbar("Mesa", [["Home", "/"], ["Docs", "/docs"]])
title("Welcome to the Future")
serve_web(8080)

-- Or low-level: x86 Assembly
asm = x86_new()
start_boot(asm)
video_mode(asm, 0x13)
point(asm, 0x7C00, "Hello from Native MesaOS")

The "Goat" Factor
Mesa v4.0.0 now includes:

Native Engine: 100% Rust.
Built-in Packages: A system to create and share modules.
Standard Library: +200 native functions for Crypto, SQLite, and JSON.

Personal Note
I’m still in school and learning every day. I used AI as a "co-pilot" (This doesn't mean I did all the AI; I spent three whole months on the project, having to correct everything, so I see myself more as the architect.) to bridge the gap in my Rust knowledge during this rewrite. My next goal? Self-hosting: Writing the Mesa compiler in Mesa.

I’d love to get your feedback on the architecture!

Check out the Repo here: https://github.com/crackanimad0r/Mesa-LP

Coder Legion - Recent posts tagged rust

Let AI Help Diagnose Incidents - Without Giving It the Keys

The Principle: Recommendations and Authority Are Different Things

1. Keep Automated Actions Deliberately Narrow

2. Plan for Missing, Slow, or Malformed AI Responses

3. Treat Decisions as Snapshots of a Changing World

4. Verify Effects Instead of Assuming Success

5. Make Decisions Explainable After the Incident

6. Evaluate the Trade-offs Honestly

Questions I Ask When Designing AI-Assisted Automation

Closing Thought

Explore Esapay

The Future of Offline Payments: The DeFi Approach

Innovations and Approaches to Offline Payments in Web3

PAT (Primitive Airborne Transaction)

Pyintel Mem

The Architecture: Single-File Binary Wire Layout (0x50 0x4D)

Zero-Copy OS Memory Mapping (mmap)

The 4-Signal Hybrid Retrieval Engine

1. Vector Quantization: SQ8 + BQ1 Hardware POPCNT

2. Zero-Allocation Hash-Keyed Okapi BM25+

3. CSR-Encoded 2-Hop Temporal Knowledge Graph

4. Ebbinghaus Spaced Repetition & O(1) Tombstoning

Solving "Filtered Recall Collapse" with the Adaptive Router

Empirical Benchmark

Quickstart

1. Install via Cargo

2. Shorthand CLI

3. Connect as an MCP Server (Cursor / Claude Code / Antigravity)

One-Click Chat Memory Harvester

Summary & Open Source

How to Catch Rogue LLM Agents on a Local Machine? Meet TopGent

What is Topgent

How does an OS tell a process from an agent?

Three questions

Why Rust

Pitfalls so far

eBPF, in simple words

No proxy

Where it is now

Try it

Returning to Salesforce After 8 Years: Building a Cross-Platform Task Ecosystem in Tauri v2 & Rust

Building aegis-filter: Why Standard Kalman Filters Fail in Adversarial Environments

1. The Core Architecture

A. Joseph-Stabilized Covariance Update

B. Chi-Square / 3-Sigma Outlier Rejection Gating

C. Unscented Kalman Filtering (UKF) & $H_\infty$ Robust Minimax

2. Flight Telemetry & Spoofing Resilience Simulation

Key Takeaways from the Data:

3. Quick Start (Rust)

1D Outlier-Gated Fusion Example:

4. Open Source Links

Data Types Across Programming Languages: What You Actually Need to Know

The deceptively simple data model

Python: let the values tell us what they are

JavaScript: one number, and two different nothings

C: representation becomes difficult to ignore

Java: primitives, references, and the null question

Rust: make some invalid states harder to represent

Go: useful zero values and deliberate simplicity

R: "missing" is not one simple state

The field that caused all the trouble

The table gets weird quickly

If you were designing a language, what would you choose?

One language where absence is not the question

Types are compressed language philosophy

Alex survived

Building a Line-Rate Linux Kernel Firewall in Pure Rust with Aya and eBPF/XDP

Why eBPF / XDP at the Driver Layer?

Architectural Highlights

1. Zero C Dependencies (Pure Rust)

2. Strict 64-Bit Memory Boundary Alignments

3. Dynamic BPF Virtual Filesystem (bpffs) Synchronization

4. In-Kernel Token-Bucket Rate Limiter

Empirical Verification: Real Linux Kernel Telemetry

Source Code & Research Paper

Mike Dabydeen on Rust, AI & Engineering Leadership

Developer Stories: Mike Dabydeen on Rust, AI & Engineering Leadership

🎥 Watch the Interview

💡 What We Discuss

The Architecture: Single-File Binary Wire Layout (`0x50 0x4D`)

Zero-Copy OS Memory Mapping (`mmap`)

JavaScript: one `number`, and two different nothings