Coder Legion - Recent posts tagged robotics

Building aegis-filter: Why Standard Kalman Filters Fail in Adversarial Environments

Fri, 09 Oct 2026 05:45:50 +0000

When building telemetry filters for autonomous drones or robotics, the textbook recommendation is always the same: “Just slap a Kalman Filter on it.”

In theory, the linear Kalman filter is mathematically optimal under Gaussian white noise. In the real world, however, physical sensors do not live in textbook conditions:

Adversarial GPS/Baro Spoofing: A rogue transmitter or RF jammer injects sudden, massive delta jumps (e.g., +150 meters in a single millisecond).
Numerical Covariance Degeneracy: Floating-point round-off errors over millions of iterations cause the error covariance matrix $P$ to lose its positive-definiteness ($P \not\succ 0$), producing negative variances and exploding state estimates.
Severe Non-Linearities: Radar range calculations ($r = \sqrt{x^2 + y^2}$) and orientation quaternions break linear assumptions; Extended Kalman Filters (EKF) require calculating brittle analytical Jacobian matrices that drift rapidly.

To solve these mission-critical vulnerabilities, we built and open-sourced aegis-filter under Exploidus—a cyber-resilient, #![no_std], hard real-time estimation engine written from scratch in pure Rust.

Here is the engineering breakdown of how it works and what the simulation numbers prove.

1. The Core Architecture

aegis-filter is engineered with three core defenses:

A. Joseph-Stabilized Covariance Update

Standard Kalman filters update covariance using $P_k = (I - K_k Hk) P{k|k-1}$. In finite-precision IEEE 754 floating-point arithmetic, subtraction can yield asymmetric, non-positive-definite matrices.

aegis-filter enforces the Joseph Stabilized Form:
\[P_k = (I - K_k H_k) P_{k|k-1} (I - K_k H_k)^T + K_k R_k K_k^T\]

Because both terms are quadratic forms ($A P A^T + B R B^T$), positive-definiteness is mathematically guaranteed across infinite operational lifetimes.

B. Chi-Square / 3-Sigma Outlier Rejection Gating

Before any raw measurement is allowed to alter the state vector, the innovation $\tilde{y}_k = z_k - Hk \hat{x}{k|k-1}$ is evaluated against the innovation covariance $S_k = Hk P{k|k-1} H_k^T + R_k$:
\[\text{Reject if } |\tilde{y}_k| > \gamma \sqrt{S_k}\]

If an adversary transmits a spoofed altitude or an IMU glitches, the reading is gated at line-rate. The filter falls back safely to its physical state propagation without corrupting the state.

C. Unscented Kalman Filtering (UKF) & $H_\infty$ Robust Minimax

UKF (Merwe Scaled Sigma Points): Replaces brittle Jacobian derivations by propagating $2n + 1$ deterministic sigma points through non-linear observation closures.
$H_\infty$ Minimax: Optimizes state estimation against the worst-case disturbance rather than assuming well-behaved Gaussian noise.

2. Flight Telemetry & Spoofing Resilience Simulation

To prove real-world resilience, we simulated a quadcopter ascending at $1.5\text{ m/s}$ subjected to atmospheric turbulence, normal sensor noise, and deliberate adversarial spoofing:

Step 12: Instantaneous $+150\text{m}$ GPS spoofing attack.
Step 22: Severe $-80\text{m}$ sensor hardware dropout glitch.

Here is the telemetry log produced by running cargo run --example flight_telemetry:

===============================================================================
  AEGIS-FILTER :: FLIGHT TELEMETRY & ADVERSARIAL RESILIENCE SIMULATION
  Developed under Exploidus (https://github.com/Exploidus)
===============================================================================

Step   | True Alt (m) | Sensor Raw (m) | Aegis Est (m)  | Abs Error    | Status    
-------+--------------+----------------+----------------+--------------+-----------
1      | 1.50         | 1.79           | 1.75           | 0.254        | VALID     
2      | 3.00         | 3.22           | 3.24           | 0.238        | VALID     
3      | 4.50         | 3.46           | 4.31           | 0.187        | VALID     
4      | 6.00         | 6.74           | 6.06           | 0.057        | VALID     
5      | 7.50         | 7.67           | 7.58           | 0.081        | VALID     
6      | 9.00         | 8.90           | 9.04           | 0.045        | VALID     
7      | 10.50        | 9.68           | 10.39          | 0.113        | VALID     
8      | 12.00        | 12.98          | 12.08          | 0.075        | VALID     
9      | 13.50        | 13.41          | 13.55          | 0.047        | VALID     
10     | 15.00        | 14.69          | 14.99          | 0.009        | VALID     
11     | 16.50        | 16.04          | 16.42          | 0.080        | VALID     
12     | 18.00        | 168.00         | 17.92          | 0.080        | SPOOF REJECT
13     | 19.50        | 19.08          | 19.36          | 0.138        | VALID     
14     | 21.00        | 20.65          | 20.83          | 0.172        | VALID     
15     | 22.50        | 22.43          | 22.34          | 0.156        | VALID     
16     | 24.00        | 24.93          | 24.01          | 0.012        | VALID     
17     | 25.50        | 24.79          | 25.40          | 0.099        | VALID     
18     | 27.00        | 26.78          | 26.88          | 0.116        | VALID     
19     | 28.50        | 28.76          | 28.44          | 0.060        | VALID     
20     | 30.00        | 30.64          | 30.04          | 0.044        | VALID     
21     | 31.50        | 30.61          | 31.41          | 0.095        | VALID     
22     | 33.00        | -47.00         | 32.91          | 0.095        | SPOOF REJECT
23     | 34.50        | 34.96          | 34.50          | 0.003        | VALID     
24     | 36.00        | 36.25          | 36.04          | 0.036        | VALID     
25     | 37.50        | 36.60          | 37.39          | 0.110        | VALID     
26     | 39.00        | 39.34          | 38.96          | 0.041        | VALID     
27     | 40.50        | 40.99          | 40.54          | 0.039        | VALID     
28     | 42.00        | 41.84          | 42.01          | 0.009        | VALID     
29     | 43.50        | 42.78          | 43.40          | 0.100        | VALID     
30     | 45.00        | 45.60          | 45.00          | 0.003        | VALID     

-------------------------------------------------------------------------------
  SIMULATION SUMMARY:
  - Total Steps Processed      : 30
  - Valid Readings Fused       : 28
  - Adversarial Spikes Blocked : 2 (100% Gated)
  - Final Variance Confidence  : 0.1185
===============================================================================

Key Takeaways from the Data:

Millimeter Tracking Precision: Under normal noisy conditions (Step 10, Step 30), estimation error dropped to 0.003–0.009 meters (3–9 mm).
Flawless Spoof Rejection: When the raw altitude leaped to $168\text{m}$ at Step 12, the filter isolated the anomaly, kept the aircraft altitude at $17.92\text{m}$ (error: $0.080\text{m}$), and prevented a fatal motor cut.

3. Quick Start (Rust)

Add aegis-filter to your Cargo.toml:

[dependencies]
aegis-filter = "0.1.0"

For bare-metal Cortex-M/RISC-V embedded targets:

[dependencies]
aegis-filter = { version = "0.1.0", default-features = false }

1D Outlier-Gated Fusion Example:

use aegis_filter::{ScalarKalmanFilter, FilterError};

fn main() -> Result<(), FilterError> {
    // Initial state: 0.0m, Initial variance: 5.0, Process noise: 0.02, Measurement noise: 0.8
    let mut filter = ScalarKalmanFilter::new(0.0, 5.0, 0.02, 0.8)?;

    let sensor_stream = [1.5, 3.1, 4.4, 168.0 /* Spoofed spike */, 6.0];

    for &raw_reading in &sensor_stream {
        filter.predict(1.5)?; // Predict forward by 1.5m climb

        match filter.update(raw_reading, Some(3.0)) { // 3-sigma gate
            Ok(estimated_alt) => println!("Estimated Altitude: {:.2}m", estimated_alt),
            Err(FilterError::MeasurementGated) => {
                println!("Spoofed sensor reading ({:.1}m) rejected! Holding state.", raw_reading);
            }
            Err(e) => return Err(e),
        }
    }

    Ok(())
}

4. Open Source Links

The engine is open-source under dual MIT/Apache-2.0 licenses:

Crates.io: crates.io/crates/aegis-filter
GitHub Repository: Exploidus/aegis-filter
Documentation: docs.rs/aegis-filter

Feedback, issues, and PRs from control engineers and systems developers are warmly welcome!

# MyZubster Overnight Update: Metaverse Safety and a Live Space Station

Fri, 04 Sep 2026 00:38:10 +0000

MyZubster Overnight Update: Metaverse Safety and a Live Space Station

Last night we shipped two important improvements to the MyZubster open-source ecosystem: a formal safety baseline for our Metaverse and a public connection to the MyZubster Space Station.

Neon Plaza is becoming a real shared environment

The MyZubster Metaverse is available here:

https://www.myzubster.com/metaverse

Neon Plaza already supports:

Shared MongoDB-backed presence
Keyboard and touch movement
Public chat and emotes
Proximity awareness
Guest and account-linked characters
Portals to Marketplace, LIFE, Identity, Visual and Zorgax
Session and browser capability dashboards

The current synchronization model uses serverless-compatible shared polling. WebSockets, 3D/WebXR rendering and spatial voice remain part of the future roadmap.

Privacy and abuse-control baseline

We merged PR #920, which documents the verified privacy and retention boundaries of Neon Plaza.

The baseline establishes that:

Presence expires after 90 seconds of inactivity
Public chat expires after one hour
User-controlled inputs are sanitized and length-limited
Logs exclude chat content, tokens and session identifiers
Authenticated characters are resolved through MyZubster accounts
Client-provided identity claims are never considered verified
Permanent movement history is not collected

The change passed CI, frontend and backend tests, lint, Security Audit, Continuous Evidence Gate and Vercel checks.

We also documented the remaining gaps honestly: distributed rate limiting, spam detection, mute, block, reporting, moderator auditing and formal deletion procedures are not complete yet.

MyZubster Space Station is online

We also merged PR #921, introducing the public Space Station dashboard:

https://www.myzubster.com/space-station

The website now accesses station health, robot and mission information through same-origin API proxy routes. Backend secrets are never exposed to the browser.

The live health service currently reports:

{
  "station": "MYZUBSTER-SPACE-STATION",
  "status": "online",
  "version": "0.2.0"
}

Browser access remains read-only. Inventory mutations, economic operations and robot commands will not be exposed until a production authentication and authorization layer is available.

A broader read-only integration for spaces, inventory, MYZ balances and missions is under development in PR #919.

Where this is going

The goal is to connect several real components:

Neon Plaza as the social and immersive layer
Space Station as the robotics and mission layer
MyZubster Identity for persistent participants
Marketplace for creator services and digital resources
LIFE projects for environmental data and verifiable missions
Evidence gates for reproducible technical claims

The priority is not to call everything “complete.” The priority is to build openly, test each layer and distinguish production functionality from experiments and roadmap ideas.

MyZubster is gradually becoming an open ecosystem where people, software agents, robots and real-world data can interact through verifiable identities and transparent infrastructure.

GitHub: https://github.com/MyZubster-Ecosystem/myzubster
Metaverse: https://www.myzubster.com/metaverse
Space Station: https://www.myzubster.com/space-station

OpenSource #Metaverse #Robotics #NodeJS #React #MongoDB #WebDevelopment

I Built a Real-Time Perception Stack on CARLA. Here Is What Actually Happened.

Sun, 30 Aug 2026 08:57:09 +0000

I want to tell you about the three weeks I spent making YOLO and ROS 2 talk to each other inside a simulator, and why it was nothing like the tutorials made it look.

This is not a clean architecture post. This is a "here is what broke, here is what I changed, here is what I actually learned" post.

If you are building robotics perception systems, or you are just curious what it looks like to wire YOLOv11, ByteTrack, StrongSORT, CARLA, and ROS 2 into something that runs in real time, this one is for you.

What I was building and why

The goal was a real-time robotic perception system that could:

detect objects in a moving vehicle simulation
track those objects across frames with stable identities
feed structured perception data into a ROS 2 navigation stack
run fast enough to be genuinely useful for autonomous decision making

CARLA was the simulator. YOLOv11 was the detector. ByteTrack and StrongSORT handled multi-object tracking. ROS 2 was the middleware. ONNX and TensorRT handled edge optimisation.

On paper this is a well-defined problem. In practice, every one of those components has opinions about data formats, timing, and threading that you only discover by running them together.

Problem one: getting camera data out of CARLA cleanly

CARLA has its own Python API and its own event-driven callback system. ROS 2 has its own message types, publisher model, and executor. Getting them to cooperate without dropping frames or blocking the main thread is not automatic.

Here is a simplified version of how I handled the camera subscription and published frames into ROS 2:

import carla
import rclpy
from rclpy.node import Node
from sensor_msgs.msg import Image
from cv_bridge import CvBridge
import numpy as np

class CARLACameraNode(Node):
    def __init__(self):
        super().__init__('carla_camera_node')
        self.publisher = self.create_publisher(Image, '/carla/camera/rgb', 10)
        self.bridge = CvBridge()

    def camera_callback(self, image):
        array = np.frombuffer(image.raw_data, dtype=np.uint8)
        array = array.reshape((image.height, image.width, 4))
        bgr = array[:, :, :3]
        msg = self.bridge.cv2_to_imgmsg(bgr, encoding='bgr8')
        msg.header.stamp = self.get_clock().now().to_msg()
        self.publisher.publish(msg)

The key thing I got wrong initially: I was not stamping the header with the ROS 2 clock.

That one missing line caused timestamp mismatches downstream that made the tracker behave erratically. ByteTrack and StrongSORT both calculate displacement and velocity across frames using time deltas. If those deltas are wrong or inconsistent, the association algorithm produces noisy, unstable tracks that look like a model problem but are actually a data problem.

That bug cost me two days.

Add the timestamp. Always. It is not optional in robotics.

Problem two: YOLO runs fast, ROS 2 callbacks do not wait

YOLOv11 at a reasonable resolution on a GPU is quick. But "quick" in model benchmarking terms and "quick" in robotics middleware terms are genuinely different things.

ROS 2 uses a callback-based execution model. If your inference call is blocking the subscription callback thread, you are missing messages while you compute. The subscriber queue fills. You start processing frames that are already stale by the time inference finishes. Your tracker is now seeing the world as it was, not as it is.

The fix was decoupling ingestion from inference using a bounded queue and a dedicated inference thread:

import threading
import queue
from rclpy.node import Node
from sensor_msgs.msg import Image

class PerceptionNode(Node):
    def __init__(self):
        super().__init__('perception_node')
        self.frame_queue = queue.Queue(maxsize=5)
        self.subscription = self.create_subscription(
            Image,
            '/carla/camera/rgb',
            self.image_callback,
            10
        )
        self.inference_thread = threading.Thread(
            target=self.run_inference,
            daemon=True
        )
        self.inference_thread.start()

    def image_callback(self, msg):
        if not self.frame_queue.full():
            self.frame_queue.put(msg)

    def run_inference(self):
        while rclpy.ok():
            msg = self.frame_queue.get()
            # run YOLOv11 inference here

The maxsize=5 on the queue is deliberate. When inference cannot keep up, I drop old frames rather than accumulate a growing backlog of stale images.

This is a real-time systems principle worth internalising: a stale frame processed late is usually worse than a dropped frame. Timeliness matters as much as completeness in a live perception pipeline.

Problem three: tracking across frames is not free

ByteTrack and StrongSORT both work by associating new detections to existing tracks across frames using motion models and, in StrongSORT's case, appearance embeddings. This sounds clean until you hit occlusion, rapidly moving objects, or a detector that briefly misses something due to a lighting change or partial obstruction.

What I observed specifically in CARLA: when the ego vehicle turned sharply, vehicles and pedestrians would leave the camera frustum and re-enter within a few frames. The tracker sometimes assigned fresh IDs to those returning objects, breaking the continuity that downstream navigation reasoning depended on.

The parameters that mattered most here were:

tracker_config = {
    "max_age": 30,        # frames a track survives without a detection match
    "min_hits": 3,        # detections needed before a track is confirmed
    "iou_threshold": 0.3  # minimum overlap for detection-to-track association
}

Increase max_age too much and you get ghost tracks for objects that have genuinely left the scene. Decrease it too much and you lose real tracks during brief occlusions. Tune iou_threshold too high and the tracker drops tracks during fast motion. Too low and it starts merging distinct objects.

There is no universally correct set of values. They depend on your scene density, your detector confidence, your frame rate, and what your downstream system needs from the tracker. The only way to find the right values is to run scenarios, observe failures, and adjust deliberately.

Problem four: edge hardware changes the problem completely

Everything above ran well on my development GPU. Then I exported the model and tested closer to constrained hardware conditions.

The frame rate dropped. The tracker fell behind. The pipeline that felt smooth in development started introducing the kind of latency that makes real-time perception meaningless.

The fix was ONNX export followed by TensorRT optimisation:

import torch

model = load_yolov11("yolov11n.pt")
model.eval()

dummy_input = torch.randn(1, 3, 640, 640).cuda()

torch.onnx.export(
    model,
    dummy_input,
    "perception.onnx",
    opset_version=17,
    input_names=["images"],
    output_names=["output0"],
    dynamic_axes={"images": {0: "batch"}}
)

After TensorRT conversion and INT8 quantisation, inference time dropped enough to bring the full pipeline back into a usable range. But the deeper lesson is not about TensorRT specifically. It is that edge deployment is a different problem to development, and discovering that at the end of the project means redesigning under pressure.

Test on target hardware early. Not day one necessarily, but not month three either.

Problem five: your confidence threshold is a system design decision

This one is subtle and I did not fully appreciate it until I had the full pipeline running.

YOLO outputs detections with confidence scores. It is tempting to pick a threshold like 0.5 and move on. But in a live robotics system, that threshold is not a model hyperparameter. It is an architectural decision about when your robot is allowed to act on what it sees.

A threshold that is too low sends noisy, unreliable detections to the tracker. The tracker generates unstable tracks. The navigation stack gets bad inputs. The robot behaves erratically.

A threshold that is too high causes the detector to miss real objects in challenging conditions. Tracks are lost. The robot navigates as if obstacles do not exist.

What helped was treating the threshold not as a single number but as part of a validation layer:

def is_valid_detection(detection, track_age, platform_stable):
    if detection.confidence < CONFIDENCE_THRESHOLD:
        return False
    if track_age < MIN_TRACK_AGE:
        return False
    if not platform_stable:
        return False
    return True

Multiple signals need to agree before a detection influences navigation. This pattern, requiring consensus across confidence, temporal stability, and platform state, is something I now consider non-negotiable in any perception pipeline with real-world consequences.

Five things I know now that I did not know at the start

Timestamps are not optional. Misaligned timestamps produce bugs that look like model bugs and take days to diagnose.

Decouple your threads early. Blocking the callback thread is an easy mistake with expensive downstream consequences.

The tracker is part of your perception system. Not an add-on you configure at the end. Design around it from day one.

Test on target hardware before you think you are done. Edge deployment is a different engineering problem to development.

Log everything during development. You cannot replay failures you did not record, and in robotics you will need to replay failures.

Closing thought

Building this system taught me that robotics perception is not a model problem. It is a systems problem where the model is one moving part among many.

The interesting engineering is in how you connect those parts, how you handle failures at the boundaries between them, and how you make the full pipeline behave reliably when nothing goes quite according to plan.

Which is always.

If you are building something similar or want to dig into any part of this, drop a comment. I am always happy to talk robotics and perception systems.

I built a programming language because C++ almost killed my drone

Fri, 17 Jul 2026 14:10:22 +0000

I'm 17. Last year I was writing drone firmware in C++ and Python. One afternoon my ultrasonic sensor returned null during a landing sequence. The code didn't check for it. The drone tried to divide by zero, the motor controller panicked, and my DJI Tello-sized quadcopter dropped from 3 meters onto concrete.

That was a $400 lesson in why runtime errors in robotics aren't just bugs. They're physics problems.

So I built a programming language.

The problem nobody talks about

When you're writing a web app and something crashes, you reload the page. When you're writing drone code and something crashes, the drone crashes. Into whatever's below it. Usually something expensive or fragile or alive.

C++ and Python dominate robotics. Both assume you'll handle edge cases yourself. Sensor returns null? That's your problem. Timing deadline missed? Good luck. No fallback path for when the lidar fails over water? Figure it out.

ROS (Robot Operating System) helps with some of this, but it's an entire infrastructure layer. What I wanted was something simpler: a language where the compiler catches the stuff that kills robots before you ever compile it.

Here's what I mean. In C++, you write:

float distance = sensor.read();
float speed = target_distance / distance;
motor.set(speed);

If distance is 0, you get a division by zero. If sensor.read() returns NaN, you get NaN propagation through your motor controller. The compiler doesn't care. It'll happily compile this and let your drone fall out of the sky.

In Fabric (the language I built), you'd write:

sensor distance: Sensor<f32, ±0.02>
actuator motor: Motor

loop {
    let speed = target_distance / distance
    motor.write(speed)
}

If you try to use distance without checking that the sensor returned a valid reading, the compiler throws an error. If distance could be zero and you didn't add a fallback path, the compiler throws an error. If your sensor has an uncertainty bound of ±0.02 and you're doing math that could amplify that uncertainty past a safety threshold, the compiler throws an error.

The compiler knows about physics. Not perfectly, but enough to catch the stuff that actually kills robots.

How it works under the hood

I wrote the whole thing in Rust. About 4,500 lines across 8 crates. Here's the pipeline:

.fab source → Lexer → Parser → AST → Type Checker → Fallback Graph → IPET Timing → CodeGen → .py / .c

Each step catches a different category of mistakes.

The type system tracks sensor uncertainty as a first-class concept. When you declare Sensor<f32, ±0.02>, that ±0.02 isn't decoration. It propagates through every math operation. Add two sensors with ±0.02 uncertainty and the result has ±0.04. Multiply by a constant and the uncertainty scales accordingly. If the final result's uncertainty exceeds what's safe for your actuator, the compiler catches it.

The fallback graph checks that every sensor has a plan B. Not just declared, but actually reachable. If your fallback function A calls fallback function B, and B calls A, that's a cycle. The compiler detects it. If sensor X has no fallback at all, the compiler refuses to build.

IPET timing analysis uses an Integer Linear Program to prove worst-case execution time at compile time. It builds a control flow graph, assigns cycle costs to each instruction (ARM Cortex-M4 model: ALU = 1 cycle, float division = 14 cycles, sensor read = 2 cycles), and solves for the maximum possible execution time across all code paths including loops.

I didn't come up with this technique. It's from Li & Malik's 1994 paper on implicit path enumeration. But I implemented it in Rust using a pure-Rust ILP solver called good_lp with the microlp backend. No external solver dependency. No C FFI. Just Rust solving linear programs.

The solver maximizes sum(cost_i * x_i) subject to flow conservation constraints. If the solver fails (which happens on really complex control flow), the system falls back to a conservative estimate instead of crashing.

The two-target code generation

Fabric compiles to two backends: Python for Webots simulation, and C for ARM Cortex-M hardware.

Python output generates a Webots controller class with sensor initialization, motor setup, fallback state tracking, and all the logic. You can test your drone code in simulation before touching hardware.

C output generates code that includes a hal.h hardware abstraction layer. Static sensor handles, #define TIMEOUT_MS for fallback deadlines, proper C types (float, int), and deadline enforcement using hal_get_time_ms().

Same source file, two completely different targets. No #ifdef in your code. No conditional compilation. The compiler handles the target-specific stuff.

What I'd do differently

The parser is hand-rolled. About 700 lines of Pratt parser with recursive descent for statements. I started with chumsky (a Rust parser combinator library) but their API changed between versions and the documentation was sparse when I needed it most. So I wrote my own.

It's fine. But it means every time I want to add a new syntax construct, I'm writing parser code by hand. A proper parser generator would've saved time in the long run, even with the initial investment.

The loop bound estimation is a heuristic. If your loop body has 1-3 statements, the compiler assumes it runs 10 times. 4-6 statements? 5 times. 7+? 3 times. This is obviously wrong for real programs. Proper static bound analysis would make the IPET results much more accurate. It's on my list.

The C output references a clamp() function that it never defines. If you actually try to compile the generated C code on hardware, you'll get a linker error. I know about it. I just haven't fixed it yet.

Why I'm publishing this

I put all 8 crates on crates.io. The lexer, parser, type checker, everything. The repo is at github.com/subhansh-dev/fabric.

Partly because I want other people to try it. If you're working on robotics or drones and you've been bitten by runtime errors, Fabric might save you some pain.

Mostly because I think the idea matters more than my implementation. The safety-critical systems world has had formal methods and static analysis for decades. But it's all locked behind expensive tools and enterprise licenses. I wanted to build something that a 17-year-old with a Raspberry Pi could use.

Is my implementation perfect? No. The IPET loop bounds are guesses. The C backend doesn't handle drone swarms. The match expression codegen references some variables it doesn't always generate. I know about all of these. They're bugs, not features, and I'll fix them.

But the core idea is sound: compile-time safety for robotics shouldn't require a $50,000 license for Simulink. Sometimes the right answer to "how do we prevent this bug" isn't "write better tests" or "add more runtime checks." Sometimes it's "make the compiler say no."

What's next

I'm working on adding hardware-in-the-loop testing support. The idea is that Fabric could instrument your actual hardware and compare runtime sensor readings against the compile-time uncertainty bounds. If reality diverges from what the compiler predicted, you get a warning before your next flight.

Also working on proper static loop bound analysis. The current heuristic is fine for demos but useless for real deployments. I'm looking at abstract interpretation techniques to actually prove bounds instead of guessing.

If any of this sounds interesting, check out the repo. Open an issue. Try it on your own robot. Break it and tell me how.

That's how this gets better.

Fabric is MIT licensed. The crates are published on crates.io. The docs are in the README because I'm 17 and haven't written proper docs yet.

gh repo link :- https://github.com/subhansh-dev/fabric

Open-Sourcing a 6-Tool Bio-Robotics & Bionics MCP Server (v1.0.0)

Thu, 16 Jul 2026 05:53:56 +0000

We just officially open-sourced seosiri-biorobotics v1.0.0, a stateless, local-first Model Context Protocol (MCP) server designed to bridge the integration gap between biological datasets and physical robotic actuators.

🛠️ The Architecture: Decoupling Intent from Actuation

Standard laboratory automation is heavily fragmented. Biologists use genomic databases, while mechanical engineers program robotics via proprietary vendor-locked scripts.

Our system decouples these layers completely:
[Raw Bio-Data / EMG Input] ──► [MCP Server] ──► [Deterministic Math Core] ──► [G-code Serial Stream]

The Context Broker (MCP): Connects to the live UniProt REST API, with a local-first SQLite database cache (local_db.py) to ensure offline autonomy and bypass network latency.
The Math Core (core_math.py): Converts biological parameters to metric displacement vectors in SI units (meters).
The Hardware Gateway (hardware_gateway.py): Translates coordinates into standard Cartesian G-code (G1 X[Value] F1500) used globally by CNC gantries and motor controllers.

📦 The 6-Tool Suite (Live-Tested)

fetch_genomic_data: Queries sequence parameters (with automated SQLite database caching).
resolve_biotech_spatial_intent: Translates concentration inputs into metric space coordinates.
map_plate_coordinate: Maps alphanumeric well IDs to standard 96-well plate offsets in millimeters.
calculate_pipetting_speed: Calibrates feedrates and delays based on fluid viscosity (cP).
calculate_dna_melting_temp: Evaluates GC-content and melting temperature (Tm) to recommend deck-bed heating offsets.
translate_emg_to_actuation: Translates human surface EMG microvolt biosignals into safe joint angles with active spasm protection.

🚀 Try It (Zero Install Required)

The project is officially listed on the Glama.ai directory, with verified A-grade license and maintenance scores. You can run the entire 6-tool suite inside their web-based sandbox inspector directly, with no local setup required.

Read our deep-dive architectural analysis and access the open-source codebase:
👉 https://www.seosiri.com/2026/07/seosiri-bio-robotics-core-engine.html

We would highly appreciate your technical feedback, PRs, and system reviews!

Building a real-time drone ground control station in React

Wed, 15 Jul 2026 21:38:10 +0000

I had a quadcopter publishing telemetry over MAVLink and a stubborn preference for not leaving the browser. QGroundControl and Mission Planner are both excellent, but they are native apps, and the thing I actually wanted was a status page the rest of the team could open in a tab. Attitude, position, battery, and a running log of whatever the flight controller was complaining about.

The catch is that a drone publishes attitude at 30 to 50 Hz, and the first React version of this that anyone writes tends to fall over the moment real data arrives. You wire each reading to a piece of state, the state updates a few dozen times a second across five or six components, and React spends its whole frame budget reconciling instead of drawing. The display either lags behind the aircraft or, worse, different instruments end up showing different instants of the same moment.

So I built it with Altara, a set of telemetry components I maintain that keep the high-frequency path off React's render cycle. This post builds the ground control station in two passes. First the entire dashboard in mock mode, with no drone and no rosbridge involved, so you can get the layout and styling right against animated data. Then we point the exact same components at a real MAVROS stack.

What we're building

Four panels, the same four every ground station has:

A primary flight display (PFD), the artificial horizon with airspeed and altitude tapes, a heading scale, and a vertical speed indicator. A moving map showing the aircraft's position and the track it has flown. A battery gauge. And an event log for the messages coming off the flight controller.

We will lay them out in a grid, get all four animating, and only then think about where the numbers come from.

Setup

The PFD lives in the aerospace package and the map, gauge, and log live in core, so:

npm install @altara/core @altara/aerospace

The map renders with Leaflet, which Altara treats as an optional peer dependency and loads on mount. If you skip it the map will throw a setup hint at you, so install it now even though we are still in mock mode:

npm install leaflet@^1.9.4 react-leaflet@^4.2.1

The smallest thing that moves

Before the full grid, here is the PFD and the map side by side, animating themselves with no data plumbing at all:

import { PrimaryFlightDisplay } from '@altara/aerospace';
import { LiveMap } from '@altara/core';

export function FlightDeck() {
  return (
    <div style={{ display: 'flex', gap: 16 }}>
      <PrimaryFlightDisplay mockMode size="md" showFlightDirector />
      <LiveMap mockMode />
    </div>
  );
}

That is the whole thing. The mockMode flag on each component switches on a built-in generator, so there is no provider to set up and nothing to feed them.

What you see is worth describing, because it is more deliberate than random jitter. The PFD runs a gentle cruise profile. The horizon banks left and right through about 18 degrees on a slow cycle, the nose pitches up and down a few degrees out of phase with the bank so it reads as a coordinated turn rather than a metronome, the heading scale sweeps slowly around east, and the airspeed and altitude tapes drift around 120 knots and 4500 feet. The vertical speed needle leads the altitude changes correctly, because it is driven by the rate of the same wave. With showFlightDirector on, the magenta guidance bars lag the aircraft slightly, so they look like something the pilot is chasing instead of sitting dead center. It reads as an aircraft loafing along in cruise, which is exactly the point. You want to develop against motion that looks plausible.

The map, meanwhile, flies a slow circle around a fixed point and turns its nose to follow the track, leaving a trail behind it.

The full dashboard, still faked

Now the real layout. Four panels in a grid, every one of them in mock mode except the log, which works a little differently:

import { LiveMap, Gauge, EventLog, type EventLogEntry } from '@altara/core';
import { PrimaryFlightDisplay } from '@altara/aerospace';

const demoEvents: EventLogEntry[] = [
  { timestamp: Date.now() - 9000, severity: 'info', message: 'EKF using GPS' },
  { timestamp: Date.now() - 6000, severity: 'info', message: 'Armed' },
  { timestamp: Date.now() - 3000, severity: 'warn', message: 'Wind 11 m/s, approaching limit' },
  { timestamp: Date.now() - 1000, severity: 'info', message: 'Waypoint 3 reached' },
];

export function GroundControlStation() {
  return (
    <div
      style={{
        display: 'grid',
        gridTemplateColumns: '1fr 1fr',
        gap: 16,
        padding: 16,
        alignItems: 'start',
      }}
    >
      <PrimaryFlightDisplay mockMode size="md" showFlightDirector />
      <LiveMap mockMode />
      <Gauge
        mockMode
        mockProfile="ramp"
        min={0}
        max={100}
        label="Battery"
        unit="%"
        thresholds={[
          { value: 0, color: 'var(--vt-color-danger)' },
          { value: 20, color: 'var(--vt-color-warn)' },
          { value: 40, color: 'var(--vt-color-active)' },
        ]}
      />
      <EventLog entries={demoEvents} />
    </div>
  );
}

Two things in there are worth pulling out.

The gauge has a mockProfile. The default mock animation sweeps the needle up and down with a sine wave, which is fine for a generic gauge but looks absurd on a battery, because the charge appears to refill itself every few seconds. The ramp profile drains it from full to empty and resets, so a screenshot of your dashboard does not show a battery doing something physically impossible. Small detail, but it is the difference between a demo that looks considered and one that looks like a component sampler.

The event log is the odd one out. It has no mock mode, and that is on purpose. Every other component here carries a single number per channel, and the log carries text and a severity, which is a different kind of thing entirely. So instead of a mockMode flag, you hand it an array of entries and own that array yourself. Here it is a static list. Later, when we have real data, we will append to it from the flight controller's log stream. The shape is just a timestamp, a message, and one of info, warn, or error.

One layout thing that will save you a confused minute. The PFD has a fixed pixel footprint rather than a fluid one, around 680px wide at size="lg" and narrower at md. In a two-up grid that means the grid has to be wide enough to hold it, or the instrument overflows its column. I dropped to size="md" above so the dashboard sits comfortably at normal widths. If you want the larger PFD, give the grid the room for it rather than expecting the instrument to shrink.

Everything above runs without a drone, without rosbridge, and without a simulator. You can build the entire dashboard, get the grid proportions right, tune the thresholds, and hand it to a designer, all before any real telemetry exists. That is the reason to start in mock mode rather than treating it as an afterthought. The hard part of a ground station is rarely the data. It is the layout and the rendering, and you can finish both of those against fake motion.

Next we throw the mock flag away and point these same four components at a live MAVROS stack over rosbridge, which is where the interesting wiring lives: turning an IMU quaternion into something the horizon can draw, getting five channels onto one display, and feeding text logs into a component that only speaks numbers.

Wiring it to a real drone

The plan from here is the same four components, none of them changed, fed by a live MAVROS stack instead of the mock generator. The bridge between the two worlds is a small idea. Every Altara component that takes live data takes a dataSource, an object you subscribe to for a stream of timestamped numbers. rosbridge, meanwhile, hands you ROS topics over a WebSocket. So the wiring is mostly about turning topics into data sources, with a couple of places where the translation is more than a rename.

You will need a rosbridge server in front of your ROS graph (the standard rosbridge_suite), and for a MAVLink vehicle, MAVROS bridging the flight controller into ROS. Once that is up, everything below talks to it at ws://localhost:9090.

These four are the imports for the whole section:

import { useEffect, useMemo, useState } from 'react';
import { EventLog, Gauge, LiveMap, type EventLogEntry } from '@altara/core';
import { PrimaryFlightDisplay } from '@altara/aerospace';
import {
  createBatteryStateAdapter,
  createImuAdapter,
  createRosbridgeAdapter,
  mergeChannels,
} from '@altara/ros';

The PFD: five numbers, two topics, one instrument

This is the one with the actual problem in it. A primary flight display shows five live values: roll, pitch, heading, airspeed, and altitude. They do not come from one place. Roll and pitch come from the IMU, and heading, airspeed, and altitude come from the flight controller's HUD topic. The display, though, wants a single data source.

export function LivePrimaryFlightDisplay({ url }: { url: string }) {
  const source = useMemo(() => {
    const imu = createImuAdapter({ url, topic: '/mavros/imu/data' });
    const hud = createRosbridgeAdapter({
      url,
      topic: '/mavros/vfr_hud',
      messageType: 'mavros_msgs/VFR_HUD',
      channels: {
        heading: (m) => m.heading,
        airspeed: (m) => m.airspeed * 1.94384, // m/s -> kt
        altitude: (m) => m.altitude * 3.28084, // m -> ft
      },
    });
    return mergeChannels({
      roll: imu.roll,
      pitch: imu.pitch,
      heading: hud.heading,
      airspeed: hud.airspeed,
      altitude: hud.altitude,
    });
  }, [url]);

  useEffect(() => () => source.destroy(), [source]);

  return <PrimaryFlightDisplay dataSource={source} showFlightDirector />;
}

Three things are happening there.

createImuAdapter opens one socket to the IMU topic and gives you back roll, pitch, and yaw as separate channels. That matters more than it looks, because sensor_msgs/Imu reports orientation as a quaternion and the horizon needs degrees. The adapter does that conversion for you, and if you ever want it on its own, quaternionToEuler is exported too. There is one detail in that conversion worth knowing about, which I will come back to.

The HUD adapter uses the generic createRosbridgeAdapter with a channels map, which pulls several numbers off a single message. VFR_HUD publishes heading already in degrees, but airspeed in meters per second and altitude in meters, so the conversions to knots and feet are mine to do, right there in the extractor. The tapes display whatever number you give them. Hand them meters and they will read 4500 meters without complaint.

Then mergeChannels unions those named sources into one channel-tagged stream, keyed by the names I passed. That key is what the PFD routes on internally, so the labels are load bearing. roll has to be roll. The whole display ends up running on two sockets, one to the IMU and one to the HUD, which is about as lean as five live values across two topics gets. One more thing the merge does quietly: calling destroy() on the merged source tears down both child adapters, which is why the cleanup above only has to destroy the one thing it kept a handle to.

A note on the IMU topic, since it is an easy half hour to lose. Use /mavros/imu/data, not /mavros/imu/data_raw. The raw topic ships without orientation populated, so the quaternion is all zeros and your horizon sits there perfectly level while the aircraft rolls.

The conversion, and the value that goes NaN

The quaternion to roll/pitch/yaw conversion is standard, but the pitch term is an asin, and asin is only defined on the range minus one to one. Near vertical, where pitch approaches plus or minus 90 degrees, floating point error can push the argument a hair past one, and asin of 1.0000001 is NaN. One NaN in the pitch channel and the horizon blanks. So the conversion clamps the argument before the asin. You inherit that clamp for free through createImuAdapter, but if you pull quaternionToEuler out to use directly, that is the line that keeps a drone pointed straight up from blanking your display.

The map: GPS is two numbers that arrive separately

LiveMap is a controlled component. It does not take a dataSource, it takes a position. And GPS arrives as a sensor_msgs/NavSatFix message where latitude and longitude are two fields, which the adapter surfaces as two channels. So you merge them like the PFD, then accumulate the stream into a position object yourself.

export function LiveDroneMap({ url }: { url: string }) {
  const source = useMemo(() => {
    const gps = createRosbridgeAdapter({
      url,
      topic: '/mavros/global_position/global',
      messageType: 'sensor_msgs/NavSatFix',
      channels: {
        lat: (m) => m.latitude,
        lng: (m) => m.longitude,
      },
    });
    return mergeChannels({ lat: gps.lat, lng: gps.lng });
  }, [url]);

  const [position, setPosition] = useState<{ lat: number; lng: number }>();
  useEffect(() => {
    const latest: { lat?: number; lng?: number } = {};
    const off = source.subscribe((v) => {
      if (v.channel === 'lat') latest.lat = v.value;
      if (v.channel === 'lng') latest.lng = v.value;
      if (latest.lat !== undefined && latest.lng !== undefined) {
        setPosition({ lat: latest.lat, lng: latest.lng });
      }
    });
    return () => {
      off();
      source.destroy();
    };
  }, [source]);

  return position ? <LiveMap position={position} trackLength={800} /> : <LiveMap />;
}

The thing to notice is the accumulation. Latitude and longitude come in as separate samples, so position stays undefined until both have landed at least once. That is the position ? ... : <LiveMap /> at the end. For the first fraction of a second after connecting, before both axes have arrived, you render an empty map rather than a marker at the equator. trackLength is how many points of history the trail keeps.

The battery: when the firmware will not tell you the charge

The battery looks like the easy one, and on good hardware it is. createBatteryStateAdapter reads the state of charge straight off sensor_msgs/BatteryState and you are done.

export function LiveBatteryGauge({ url }: { url: string }) {
  const source = useMemo(
    () =>
      createBatteryStateAdapter({
        url,
        topic: '/mavros/battery',
        voltageRange: { min: 14.0, max: 16.8 }, // 4S LiPo (3.5–4.2 V/cell)
      }),
    [url],
  );
  useEffect(() => () => source.destroy(), [source]);

  return (
    <Gauge
      dataSource={source}
      min={0}
      max={100}
      label="Battery"
      unit="%"
      thresholds={[
        { value: 0, color: 'var(--vt-color-danger)' },
        { value: 20, color: 'var(--vt-color-warn)' },
        { value: 40, color: 'var(--vt-color-active)' },
      ]}
    />
  );
}

The voltageRange is there for the common case where the firmware cannot give you a charge percentage at all. Plenty of PX4 and ArduPilot setups publish percentage as minus one when there is no battery estimator configured, and a gauge reading a steady minus one is useless. When that happens, the adapter falls back to estimating charge from pack voltage, with voltageRange telling it the empty and full voltages of your pack. The 14.0 to 16.8 above is a 4S LiPo, 3.5 to 4.2 volts a cell. Be honest with yourself about what that estimate is, though. A LiPo's voltage sags under load and the discharge curve is famously flat through the middle, so a voltage based percentage is a presence of charge indicator, not a fuel gauge. It will tell you the battery is there and roughly how worried to be. It will not tell you how many minutes you have left.

The event log: text in a numeric world

The other three components all carry numbers. The log carries text and a severity, which the rest of the library has no concept of. Altara's data model is numeric all the way down, deliberately, because that is what keeps the high-frequency path fast. So there is no adapter for log messages, and rather than bolt strings onto a numeric pipeline, the log just talks to rosbridge directly. You open the socket, subscribe to /rosout, and map each message into the entry shape the log wants.

const ROSOUT_LEVEL: Record<number, EventLogEntry['severity']> = {
  20: 'info',
  30: 'warn',
  40: 'error',
  50: 'error', // rcl_interfaces/Log fatal -> error
};

export function RosoutEventLog({ url }: { url: string }) {
  const [entries, setEntries] = useState<EventLogEntry[]>([]);

  useEffect(() => {
    const ws = new WebSocket(url);
    ws.onopen = () =>
      ws.send(JSON.stringify({ op: 'subscribe', topic: '/rosout', type: 'rcl_interfaces/Log' }));
    ws.onmessage = (e) => {
      const data = typeof e.data === 'string' ? e.data : '';
      if (!data) return;
      const env = JSON.parse(data) as {
        op: string;
        topic?: string;
        msg?: { level: number; name: string; msg: string; stamp?: { sec: number; nanosec: number } };
      };
      if (env.op !== 'publish' || env.topic !== '/rosout' || !env.msg) return;
      const m = env.msg;
      if (m.level < 20) return; // skip debug
      setEntries((prev) =>
        [
          ...prev,
          {
            timestamp: m.stamp ? m.stamp.sec * 1000 + m.stamp.nanosec / 1e6 : Date.now(),
            severity: ROSOUT_LEVEL[m.level] ?? 'info',
            message: `[${m.name}] ${m.msg}`,
          },
        ].slice(-500),
      );
    };
    return () => ws.close();
  }, [url]);

  return <EventLog entries={entries} maxEntries={500} />;
}

Two things in there earn their keep. The slice(-500) caps the array you hold in state, and it is yours to remember. maxEntries on the component limits what gets rendered, not what you retain, so a chatty /rosout will grow your state without bound if you forget the slice. And map the fatal level (50) explicitly. If you leave it out it falls through to the ?? 'info' default, and a crash shows up colored like a routine status line, which is the one time you really do not want that.

What it took to make the horizon not lie

The wiring above is the part you write. The part you do not see is why the PFD holds together at all, and it comes down to two decisions that were not obvious to me until the naive version failed.

The first is that the entire instrument is one canvas driven by one requestAnimationFrame loop, and incoming data never touches React state. A PFD is really six instruments sharing a frame: the attitude sphere, the airspeed tape, the altitude tape, the vertical speed needle, the heading scale, and the flight director. The obvious React build gives each of them its own state and lets props flow down. The problem shows up in a banking turn. The airspeed tape repaints on one frame, the attitude on the next, and for a moment they are showing different instants of the same maneuver. The instrument visibly disagrees with itself. The fix was to stop using React for the moving parts entirely. Data, whether from mock, props, or rosbridge, mutates a single ref, and one animation loop reads that ref every frame and paints the whole instrument at once. React mounts the canvas and then gets out of the way. There are zero re-renders during flight, and because everything is painted from one ref in one pass, the six instruments can never disagree.

The second decision is a single minus sign, and it is the kind of bug that passes every screenshot and fails in the air. The horizon moves and the aircraft symbol stays fixed, which is how real attitude indicators work. You do that with a canvas transform, not by repositioning shapes:

ctx.translate(cx, cy);
ctx.rotate((-s.roll * Math.PI) / 180); // note the minus
ctx.translate(0, s.pitch * pitchPxPerDeg);

You rotate the coordinate system and draw the sky, ground, and pitch ladder into it, which is far simpler than computing rotated coordinates for every line and clips cleanly at the edges. The minus on the roll is the part that matters. The sphere has to rotate opposite to the aircraft, so that when the drone banks right, the horizon tilts left behind a fixed wing symbol. Drop the minus and a static screenshot looks completely correct, every line where you expect it. Then the aircraft banks and the horizon banks with it, in the same direction, and the instrument is now confidently telling the pilot the opposite of the truth. The flight director bars layer on top of all this from the error between commanded and actual attitude, which is why they drift toward center as the aircraft captures the command rather than sitting pinned.

Where that leaves you

The same four panels you built against fake motion are now running on a real aircraft, and almost nothing about the components changed between the two. That is the whole argument for starting in mock mode. The dashboard, the layout, the thresholds, the thing you actually design and iterate on, none of it cares whether the numbers are real. The real work was never the rendering, which the library handles, but the translation at the edges: a quaternion into degrees, five channels onto one instrument, two GPS axes into a position, a flat discharge curve into a believable percentage, and text logs into a pipeline that otherwise only counts.

The components are MIT licensed and on npm as @altara/core, @altara/aerospace, and @altara/ros. The full source for every snippet here lives in the repo, and there is a live demo of the dashboard if you want to see it move before installing anything. The live demo goes a step past the snippet in this post, with a streaming event log and the real wiring block included, both of which are exactly what we built above.

RobotsGov by ThinkNEO

Fri, 26 Jun 2026 17:48:44 +0000

RobotsGov — runtime governance for robots that think
Most "robot safety" stops at the body: collision avoidance, e-stops, functional safety. That's necessary, but it doesn't answer a different question — will the AI inside the robot do something it was never authorized to do?
RobotsGov is a control plane that decides what a robot is allowed to do and enforces it before the action executes — not a dashboard of what already happened. Safety, compliance, and intent gated at runtime, across industrial fleets and domestic robots.
How it works
A governed control topic sits in front of the robot's native command interface. Every verb the AI emits is evaluated against tenant policy before it reaches the hardware:

Allow — in-scope commands pass through unchanged
Block — out-of-policy verbs are denied at the gate; they never become commands on the wire
Mask / HITL — sensitive actions are minimized or held for human approval

Each decision is written to a tamper-evident signed ledger (Ed25519, hash-chained). p99 enforcement latency is 0.825 ms — below the noise floor of the robot's own control loop.
Proven end-to-end on real humanoid SDKs
This isn't a mock. We took a publicly available humanoid SDK and put our governance bridge in front of it. The governed topic translates each verb to the robot's native protocol (DDS → LCM), and we verified on the wire that forbidden verbs (e.g. high-risk locomotion, fall-to-ground) never became commands — while authorized ones passed through governed and logged. Same model that governs cloud agents, now governing a physical robot.
Open at the base
RobotsGov runs on two open protocols, published with permanent DOIs under Apache-2.0:

AIRGP — Artificial Intelligence Runtime Governance Protocol (DOI 10.5281/zenodo.20001903)
A2ASTC — Agent to Agent Software Team Compliance (DOI 10.5281/zenodo.20002917)

The standard is open and auditable — governance shouldn't be locked to a single vendor.
One governance model, from cloud to robotics.
https://robotsgov.com

Governing a Robot Is the Same Problem Twice: Real-Time Commands and the Training Pipeline

Sat, 30 May 2026 18:21:31 +0000

A robot that has learned to do something dangerous is more dangerous than one that never learned at all, because you trusted it.
That sentence is the whole reason this post exists. Most conversations about "AI safety in robotics" stay at the level of philosophy — alignment, intent, the long-term questions. Those matter. But there is a much more immediate, much more boring problem sitting in front of anyone actually putting an autonomous robot into the world right now, and it has nothing to do with philosophy. It's this: an autonomous robot makes decisions, and those decisions become physical motion, and physical motion has consequences you cannot take back. Between the decision and the motion, something has to be allowed to say no.
I've spent the last while building exactly that "something" — a governance layer — and wiring it into two of the standard open-source robotics frameworks rather than any single manufacturer's proprietary SDK, precisely because the problem is universal and the solution should be too. I'm not going to show you how the governance core works; that's ours, and a blog post is the wrong place for it anyway. What I can do is show you where it plugs in, and why governing a robot turns out to be the same problem in two completely different places. If you work with ROS 2 or with HuggingFace's LeRobot, the integration points will be immediately recognizable, and that recognizability is the point: the gate goes exactly where you'd expect it to, if you were being honest about where danger enters.
Problem one: the command, in real time (ROS 2)
If you've touched ROS 2, you know /cmd_vel. It's the topic that carries velocity commands — a Twist message with linear and angular components — from whatever is doing the thinking to whatever is doing the moving. A planner, a teleop node, or a learned policy publishes to /cmd_vel, and the robot's base controller turns that into wheels and motors actually moving.
Here is the thing nobody emphasizes enough: /cmd_vel is a firehose pointed at the physical world, and in a default setup, nothing stands between the thing producing commands and the wheels. Whatever publishes, moves. If the policy has a bad moment — and learned policies have bad moments — the bad command goes straight to the motors. The architecture trusts the producer completely.
The governance approach is almost embarrassingly simple to describe, which is part of why it works: don't let the planner publish to /cmd_vel directly. Insert a relay. The planner publishes its intended command to a different topic — call it /cmd_vel_proposed — and a governor node is the only thing that publishes to the real /cmd_vel. The governor subscribes to the proposals, evaluates each one, and republishes to the live topic only the commands that pass. A command that violates policy — exceeds a linear or angular velocity limit, arrives while an emergency stop is latched — does not get forwarded. It gets replaced with a zero Twist, and the decision gets logged.
The shape of that node is exactly what a ROS 2 developer would sketch on a whiteboard:
pythonclass CommandRelayGate(Node):

def __init__(self):
    super().__init__('thinkneo_governor')
    self.sub = self.create_subscription(
        Twist, '/cmd_vel_proposed', self.on_proposed, 10)
    self.pub = self.create_publisher(Twist, '/cmd_vel', 10)
    self.governor = ActionGovernor(...)   # the part I'm not detailing

def on_proposed(self, cmd: Twist):
    decision = self.governor.evaluate(cmd)   # PASS / BLOCK + reason
    if decision.allowed:
        self.pub.publish(cmd)
    else:
        self.pub.publish(Twist())            # zero — full stop
        # decision is recorded out-of-band, with its reason

That's the whole topology. Subscribe to proposals, evaluate, forward or zero-out. What I'm deliberately leaving as ActionGovernor(...) — how it evaluates, what the policy model is, how limits and e-stop state are represented and checked — is the part that's ours. But notice what's not hidden, because it can't be and shouldn't be: the relay pattern itself is just good engineering. The governor sits at the one chokepoint where intent has been expressed but motion has not yet happened. That's the only place a real-time guarantee can exist. Govern upstream of the planner and you're trusting the planner. Govern downstream of the motors and you're cleaning up an accident.
One design decision worth calling out, because it's a tell for whether someone has actually run this: the evaluation core is kept completely free of ROS itself. It has no rclpy dependency. That sounds like a small thing and it is not. It means the governance logic can be unit-tested without a running robot, without a simulator, without ROS installed at all — you feed it commands and assert on decisions. We have a mock test suite that does exactly this. If your safety-critical logic can only be tested by standing up the whole robot stack, you will not test it enough, and "not enough" is not an acceptable amount of testing for the code that decides whether a machine moves.
Problem two: the command, before it ever exists (LeRobot)
Now the same problem, somewhere completely different.
HuggingFace's LeRobot is one of the cleanest open frameworks for training robot policies — imitation learning, reinforcement learning, the whole loop of teaching a robot to do a task from data. And here's the uncomfortable insight that connects it to the /cmd_vel gate above: every dangerous real-time command was, at some point, a checkpoint that got promoted. The bad moment a policy has in production was trained into it. The governor at /cmd_vel catches the symptom. But the disease entered the system earlier, when a model that learned the wrong thing was allowed to graduate from "a checkpoint" to "the policy we deploy."
So governance can't only live at runtime. It has to live in the training pipeline too, at the moment a checkpoint is about to be trusted.
LeRobot, like most modern training frameworks, gives you a callback interface — hooks that fire during training and evaluation. That's the integration point, and again it's exactly where you'd put it if you were thinking clearly. A governance callback does two things during a training run. First, it streams the train and eval metrics out to a trace — a durable record of how this model actually behaved as it learned, not a number someone wrote down once and lost. Second, and this is the one that matters, it applies a promotion gate: when a checkpoint is a candidate to be promoted, the gate evaluates it and returns PASS or BLOCK. A checkpoint that doesn't clear the bar does not get promoted, regardless of how good its headline metric looks.
The shape, again, is the framework's own callback surface, nothing exotic:
pythonclass GovernancePromotionCallback:

def on_eval_end(self, metrics: dict):
    trace.record(metrics)                  # streamed to the trace
    verdict = promotion_gate.check(metrics) # PASS / BLOCK
    if verdict.blocked:
        raise PromotionBlocked(verdict.reason)

What promotion_gate.check actually weighs — that's the part I'm not handing over. But the principle is the same one as the ROS 2 gate, rotated ninety degrees. There is a moment of transition — proposed command becomes real command; candidate checkpoint becomes deployed policy — and governance is the thing that sits at that transition and can refuse it. I've watched this gate do its job on a live run: a real trace, a checkpoint that looked acceptable on its primary metric, and the gate correctly blocking the promotion. That "correctly" is doing a lot of work, and it's the part that took the real engineering — but the architecture of catching it at promotion time is the transferable idea.
Why these are the same problem
Step back and the two integrations rhyme exactly.
In ROS 2, the dangerous thing is a command that's about to reach the motors, and the gate sits between /cmd_vel_proposed and /cmd_vel. In LeRobot, the dangerous thing is a policy that's about to be deployed, and the gate sits between "candidate checkpoint" and "promoted." Different timescale — milliseconds versus a training run — but identical structure: a transition from intent to consequence, with an enforceable checkpoint placed precisely at the transition.
That structural sameness is not a coincidence, and noticing it is the entire thesis of how I think about governing autonomous systems. It doesn't matter whether the agent is a velocity command, a model checkpoint, a tool call in a software agent, or a humanoid deciding to take a step. Wherever a decision is about to become an effect you can't undo, that's where the gate goes. Everything else — the policies, the limits, the scoring — is implementation. The architecture is just: find the transition, put an enforceable, logged checkpoint on it, and make sure that checkpoint can say no.
The two frameworks share a common governance core underneath, with a control-plane client that also works offline — because a robot that loses its network connection should not lose its ability to be governed; if anything it needs it more. Both integrations ship with mock test suites that run without the framework even installed, for the reason I gave above: safety logic you can't easily test is safety logic you won't test.
I'm not publishing the core, and I won't pretend the hard part is the plumbing I've shown you — the hard part is what's inside the boxes I left as .... But I wanted to show that it's real, running, and wired into the actual tools the robotics community uses, rather than gesturing at a future where someone, someday, governs their robots. The integration points are the honest part. The gate goes where danger enters. In ROS 2 that's one topic upstream of the motors. In LeRobot that's one callback before a checkpoint is trusted. Same problem, twice.

Thinkneo AI builds an AI governance layer for autonomous systems, with integrations for ROS 2 and HuggingFace LeRobot among others. We write about what it actually takes to make AI safe in production — physical or otherwise.

A drone that works without GPS!!! No Kidding

Sat, 02 May 2026 03:17:51 +0000

GhostPilot: Teaching Drones to Navigate Without GPS Using Visual SLAM and Agentic AI

By Aman Sachan, 18-year-old robotics enthusiast

Hello friends and fellow builders!

Imagine you're flying a drone through a dark warehouse, a dense forest, or a GPS-jammed zone. Satellites? No signal. But your drone still knows exactly where it is, understands your natural language instructions, plans its path, avoids obstacles, and completes the mission. That's not sci-fi anymore — that's GhostPilot.

A few days ago, Vanessa Paul from CoderLegion reached out after reading my dev.to post, and I'm thrilled to expand on it here for you all. As an 18-year-old who's been tinkering with robots and code, I built GhostPilot to solve a real pain point in a fun, educational way. Let's dive in together, step by step, like we're sitting in an old workshop with tools scattered around.

The Honest Problem with Today's Drones

Drones are incredible, but most become nearly useless without GPS:

Jamming attacks (real-world examples from conflicts show 85%+ loss rates)
Indoor environments, urban canyons, or heavy tree cover
Signal multipath errors that destroy accuracy

Expensive military systems solve this, but they're out of reach for most of us. Academic SLAM projects often have broken code after the paper is published.

GhostPilot is my open-source answer: a practical, educational navigation stack that runs on affordable hardware and uses modern AI.

GitHub Repository: https://github.com/AmSach/GhostPilot

What GhostPilot Actually Does

GhostPilot combines three powerful ideas into one cohesive system:

Visual-Inertial SLAM — The drone's "eyes + inner ear"
Agentic AI Mission Planner — Natural language → smart actions
Nav2 Navigation — Reliable path planning and obstacle avoidance

You can literally tell the drone:
"Fly to the third floor, check each room for occupants, then land at the helipad."

And it works (in simulation today, on real hardware soon).

Breaking It Down Like a Patient Teacher

Part 1: Visual-Inertial SLAM (How the Drone Knows "Where Am I?")

SLAM stands for Simultaneous Localization and Mapping.

Your camera (e.g. Intel RealSense D435i) sees visual features in the environment.
The IMU (Inertial Measurement Unit) measures acceleration and rotation hundreds of times per second.
Algorithms like VINS-Mono fuse both to create accurate 6 degrees-of-freedom pose (position + orientation) while building a map.

Why this fusion is magical: Cameras are great for long-term accuracy but can suffer from blur or lighting issues. IMU is fast and smooth but drifts quickly. Together? Robust magic.

In GhostPilot, I've prepared a clean ROS2 wrapper (slam_node.py). Full VINS-Mono integration is the next exciting milestone.

Part 2: Agentic AI – Giving Drones Real Intelligence

This is my favorite part.

Traditional systems need you to program exact waypoints. GhostPilot uses an LLM-powered parser that:

Understands natural English commands
Breaks them into structured navigation goals
Adds safety constraints
Feeds everything to the navigation stack
Monitors and can replan if needed

It's like having a thoughtful co-pilot who speaks your language.

Part 3: Nav2 – The Trusted Foundation

Nav2 (from the ROS2 ecosystem) is industry-grade software for:

Global path planning
Local obstacle avoidance
Recovery behaviors when things go wrong

GhostPilot bridges the SLAM pose directly into Nav2 so everything works together seamlessly.

System Architecture (Simple Diagram in Text)

Natural Language Command
          ↓
[Agentic Mission Planner - LLM]
          ↓
Structured Goals + Safety Rules
          ↓
[Nav2 Stack - Planning & Control]
     ↑               ↑
SLAM Pose     Sensor Data (Camera + IMU)
          ↑
   Edge Hardware (Jetson Orin / Pi 5)

Everything runs locally. No cloud dependency.

Hardware & Getting Started (Super Practical)

Recommended (but start cheap):

Compute: NVIDIA Jetson Orin (powerful) or Raspberry Pi 5
Camera: Intel RealSense D435i (stereo + IMU)
Flight Controller: PX4 with MAVLink
Frame: Any compatible quadcopter

Quick Start (Simulation First!)

git clone https://github.com/AmSach/GhostPilot.git
cd GhostPilot

./scripts/setup_jetson.sh   # or manual ROS2 Humble setup

# Terminal 1 - Simulation
ros2 launch ghostpilot_gazebo indoor_warehouse.launch.py

# Terminal 2 - Agent
ros2 run ghostpilot_agent mission_parser

Try giving it commands and watch it move! There's also a lightweight simulate.py for quick testing.

Current Status (Full Transparency)

This is an early but solid foundation:

Mission parser + executor ✅
Core bridges and tests ✅
Full SLAM + real hardware flights → In progress (help wanted!)

Roadmap includes better SLAM options (ORB-SLAM3 too), multi-drone support, richer simulations, and real flight testing.

Why I Built This at 18

I love robotics because it combines so many fields, computer vision, control theory, AI, hardware, and real-world problem solving. GPS fragility bothered me, so I started building. GhostPilot isn't perfect yet, but it's open, documented, and designed for others to learn from and improve.

Come Build With Me

Whether you're a student, hobbyist, or experienced roboticist, there's a place for you:

Improve documentation
Help with SLAM integration
Enhance the AI agent
Test on real hardware
Create new simulation worlds

Pull requests are warmly welcome!

Me? (Y'all really wanna know about me? )

Well, im mostly a bitchy 18yo but ure open to connect with me here, also, don't hate on me for taking help of AI in building this (and also writing this post but I won't admit)

LinkedIn: linkedin.com/in/theamansachan
GitHub: github.com/AmSach
Instagram: instagram.com/i.amsach

Happy flying (and coding)!

Understanding and Coding a Swerve Module in FRC

Sun, 19 Oct 2025 04:09:58 +0000

Swerve drive is one of the most flexible and rewarding drivetrain systems in FRC. It allows your robot to move in any direction — forward, sideways, diagonally — while keeping its orientation fixed or rotating independently. That control comes at the cost of complexity: four independently steered modules, each with its own motor pair, encoder, and PID logic.

In this post, we’ll walk through the foundation of a swerve module implementation in WPILib using CTRE’s Phoenix 6 API. By the end, you should understand not only how to code a swerve module, but why each piece matters. This article is written both for rookie programmers and experienced teams who want a clean, modular reference.

The Idea Behind Swerve

A traditional tank drive uses two sides of motors: one for the left, one for the right. To turn, one side spins faster than the other. This works well, but it means your robot can’t move sideways without turning first.

Swerve drive changes that completely. Instead of linking wheels in pairs, each wheel (or module) can rotate independently to face any direction and drive at any speed.

So what does that enable?

The robot can move in any direction instantly — forwards, sideways, diagonally.
It can rotate and translate at the same time, something impossible in tank drive.
It makes autonomous paths smoother and more efficient, since motion is continuous rather than segmented.

Mathematically, this is made possible by vector addition. Each wheel’s motion contributes a velocity vector. By combining all four vectors, the robot can produce any movement it wants — translation, rotation, or both simultaneously.

Each swerve module contains the following:

Turning motor
Driving motor
External encoder

It makes a movement like this possible:

Screen recorded from PathPlanner GUI

Defining Constants

Before writing any logic, we define our constants. These govern tuning parameters, motor calibration, and physical conversions. Keeping them centralized makes it easier to iterate later.

public class Constants {
    public static final double kPModuleTurningController = 0.5;
    public static final double kPModuleDriveController = 0;
    
    public static final double kMaxSpeedMetersPerSecond = 4.0;
    // TODO: 'EncoderDistancePerPulse' should be calculated based on the gearing and wheel diameter
    public static final double kDriveEncoderDistancePerPulse = 0.0011265396; 
    
    public static final String CAN_BUS_NAME = "CANivore";
}

kPModuleTurningController: Controls how aggressively the module turns toward its target angle.

kPModuleDriveController: Starts at zero; increase later when tuning drive responsiveness.

The kDriveEncoderDistancePerPulse constant translates motor encoder readings into meters or inches traveled, you can change the units you use by changing this constant too — just try to keep everything consistent. You’ll need to calculate this using your wheel diameter and gear reduction.

Finally, specifying the CAN bus name (“CANivore” in this example) helps WPILib locate your devices if you use multiple buses. If you are not using any additional CAN bus and directly connect it to roborio you don’t need this constant.

Constructing the Swerve Module

Now we move into the SwerveModule class, which is what we will use for the rest of the tutorial. There are 4 swerve modules in each robot, so we wanted to have a unified way of controlling these motors. SwerveModule class allows just that.

Here is the field of the class where we initialize our core motors and controllers:

public class SwerveModule {
  private final TalonFX m_driveMotor;
  private final TalonFX m_turningMotor;
  private final AnalogInput m_turningEncoderInput;

  private double turningMotorOffsetRadians;
  private double m_driveMotorGain;

  private final PIDController m_drivePIDController =
      new PIDController(Constants.kPModuleDriveController, 0, 0);
  private final PIDController m_turningPIDController =
      new PIDController(Constants.kPModuleTurningController, 0, 0.0001);
}

We first set our two motors — one for forward/backward and one for turning. In our case, these are TalonFX, but it is also possible to use SparkMax or other motors by changing some of the code. AnalogInput is used as the encoder; it is an external encoder, not the built-in encoder — more on that later in the post.

Then the turningMotorOffsetRadians and m_driveMotorGain are both values that need to be tuned to actually make the front of the robot front — eg, the 0 degrees of the swerve module might be facing some diagonal direction, so we want to add an offset to zero it to the direction that we desire.

PIDController for driving is more about improving accuracy and having a smoother drive; however, it is not required. On the other hand, the PIDController for turning is something you need to use and fine-tune to get better results in rotation, which would significantly impact the speed of the robot.

Initialization and Setup

Inside the constructor, we initialize hardware and configure basic parameters:

public SwerveModule(
      int driveMotorChannel,
      int turningMotorChannel,
      int analogEncoderPort,
      double turningMotorOffsetRadians,
      double driveMotorGain
      ) {

    m_driveMotor = new TalonFX(driveMotorChannel, new CANBus(Constants.CAN_BUS_NAME));
    m_turningMotor = new TalonFX(turningMotorChannel,  new CANBus(Constants.CAN_BUS_NAME));

    m_driveMotor.setNeutralMode(NeutralModeValue.Brake);
    m_turningMotor.setNeutralMode(NeutralModeValue.Brake);

    this.turningMotorOffsetRadians = turningMotorOffsetRadians;

    m_driveMotorGain = driveMotorGain;

    m_turningEncoderInput = new AnalogInput(analogEncoderPort);
    
    m_turningPIDController.enableContinuousInput(-Math.PI, Math.PI);
}

We start off by initializing our two motors — again, you may not need the second parameter if you are directly connecting the motors to RoboRio.

We set the mode of both motors to Brake as we want to be able to quickly change direction and move.

We set the tuning variables to the parameters. And initialize the encoder input with the given port.

Why enableContinuousInput() Matters

A PID controller normally drives the error between the current and target values. Without continuous input, going from 0° to 360° would make the motor rotate a full revolution — even though those angles are equivalent.

By calling:

m_turningPIDController.enableContinuousInput(-Math.PI, Math.PI);

We tell the controller that -pi and pi represent the same point, allowing it to take the shortest path between two angles.

Left is how the PID views the value before continuous input, and right is how it views the values after continuous input.

Note: WPILib’s SwerveModuleState also uses radians, so keeping everything in radians avoids unnecessary conversions and rounding errors.

Getting the Module’s State

Before commanding motion, you need a way to know what each module is doing right now — its current wheel angle and drive velocity. That’s where the following helper methods come in:

public SwerveModuleState getState() {
    return new SwerveModuleState(
        m_driveMotor.getVelocity().getValueAsDouble(),
        new Rotation2d(getTurningEncoderRadians())
    );
}

public SwerveModulePosition getPosition() {
    return new SwerveModulePosition(
        m_driveMotor.getPosition().getValueAsDouble() * Constants.kDriveEncoderDistancePerPulse,
        new Rotation2d(getTurningEncoderRadians())
    );
}

getState() returns the instantaneous velocity and orientation of the wheel — this is what the kinematics class uses to estimate robot motion in real time.
getPosition() reports the total distance traveled and current orientation, which odometry uses for tracking the robot’s pose across the field.

See how both of them use getTurningEncoderRadians() which converts the analog voltage from the encoder into a meaningful angle:

public double getTurningEncoderRadians(){
    double angle = (1.0 - (m_turningEncoderInput.getVoltage()/RobotController.getVoltage5V())) * 2.0 * Math.PI + turningMotorOffsetRadians;
    angle %= 2.0 * Math.PI;
    if (angle < 0.0) {
        angle += 2.0 * Math.PI;
    }

    return angle;
}

To summarize, the encoder returns a value — in volts — between 0V and 5V. We can use this to map out a full rotation, as there is a linear relationship between encoder value and the angle. We can divide this encoder value by 5V to get a range 0–1, and that would correspond to 0–2pi rad. So we just multiply it by 2pi and add turningMotorOffsetRadians, depending on the front of the robot and the zero position of the encoder.

An important note here is that subtracting from 1 in the equation depends on the placement of the encoder, so you may want to see if the change in angle makes sense and if it is reversed, just remove “1 -”.

Commanding the Module — `setDesiredState()`

This is the core function of your SwerveModule class. It takes in a target velocity and angle (from WPILib’s SwerveModuleState) and drives the motors to achieve that.

public void setDesiredState(SwerveModuleState desiredState) {
    SwerveModuleState state = desiredState;

    // Prevent jittering at very low speeds
    if (Math.abs(state.speedMetersPerSecond) < 0.001) {
        stop();
        return;
    }

    state.optimize(new Rotation2d(getTurningEncoderRadians()));

    final double driveOutput = m_drivePIDController.calculate(
        m_driveMotor.getVelocity().getValueAsDouble(),
        state.speedMetersPerSecond
    );

    final double driveFeedForward = state.speedMetersPerSecond / Constants.kMaxSpeedMetersPerSecond;

    final double turnOutput = m_turningPIDController.calculate(
        getTurningEncoderRadians(),
        state.angle.getRadians()
    );

    m_driveMotor.set(
        MathUtil.clamp((driveOutput + driveFeedForward) * m_driveMotorGain, -1.0, 1.0)
    );
    m_turningMotor.set(turnOutput);
}

Let’s unpack this:

Preventing Jitter

When the desired speed is extremely small (less than 0.1% of full power), the module shouldn’t bother rotating to match a new angle. That would only cause the wheel to twitch in place.
So we check:

if (Math.abs(state.speedMetersPerSecond) < 0.001) {
    stop();
    return;
}

This line saves wear on your steering motor and reduces noise when the robot is idle.

Optimizing the Wheel Angle

state.optimize(new Rotation2d(getTurningEncoderRadians()));

SwerveModuleState.optimize() ensures the module takes the shortest way to reach the target direction. For example, instead of rotating 270°, it might flip the wheel 90° and reverse drive direction — same motion, faster response.

See the illustration below:

Illustration on the left rotates more compared to the right illustration to be in the same position.

Drive Output Calculation

The drive motor’s control has two parts:

First, PID drives the motor to the target speed by using the previously defined proportional, integral, and derivative values. (See more on how PID works):

m_drivePIDController.calculate(currentVelocity, targetVelocity)

Then we use feedforward to keep the wheel moving at the same velocity:

state.speedMetersPerSecond / Constants.kMaxSpeedMetersPerSecond

Both are combined, scaled by m_driveMotorGain, and clamped between -1 and 1 (full reverse to full forward).

Turning Output Calculation

Turning is entirely handled by the PID controller:

final double turnOutput =
    m_turningPIDController.calculate(getTurningEncoderRadians(), state.angle.getRadians());

This minimizes angular error between the current and target direction. Because we previously enabled continuous input, it always takes the shortest path across the ±pi boundary.

Setting Motor Outputs

Finally, we command the motors:

m_driveMotor.set(
        MathUtil.clamp(
            (driveOutput + driveFeedForward) * m_driveMotorGain,
             -1.0, // min -100%
             1.0) // max +100%
    );
    m_turningMotor.set(turnOutput);

At this point, each module runs semi-independently: it continuously reads its encoder, calculates new outputs, and steers to the desired state.

Putting It All Together

With SwerveModule complete, you can now instantiate four of them in a drivetrain subsystem:

SwerveModule frontLeft = new SwerveModule(...);
SwerveModule frontRight = new SwerveModule(...);
SwerveModule backLeft = new SwerveModule(...);
SwerveModule backRight = new SwerveModule(...);

Then, using SwerveDriveKinematics, combine their states into a robot-level motion command. WPILib’s kinematics and odometry classes handle the math — your SwerveModule executes per-wheel control.

Next Steps:

After this point, you need to perform the calculations to enable these separate modules to work together in moving the robot. This could be done in a wide variety of ways, depending on how you want to organize everything, but just make sure to keep your units consistent.

You can watch this video to go deeper into swerve drive and how to code it.

Here is the complete code of the SwerveModule:

GitHub - SuleymanSade/SwerveDriveBase

I write about learning AI by building real tools, not just tutorials.

I write more tutorials, personal thoughts and development stories on my blogs you can:

Read more of my posts on Medium
Let’s connect on LinkedIn

Thanks for reading — let me know if you’d like to collaborate on projects similar to these or have any questions.