Coder Legion - Recent posts tagged nvidia

Data Residency Isn't Sovereignty: F5 and NVIDIA Add a Control Layer for AI Agents

Mon, 26 Oct 2026 18:59:13 +0000

Europe and Asia want AI they control. Governments and enterprises are pushing for models and infrastructure that run locally, under local rules. But putting GPUs in-country doesn't answer the harder question: who controls what happens to the data, the models, and the agents running on them?

F5 is working on that problem with NVIDIA and two telecom operators, Telenor in Norway and Indosat Ooredoo Hutchison in Indonesia. I asked Ahmed Guetari, senior vice president of product management for service providers at F5, how the pieces fit together and what developers and architects should do now.

Residency is location. Sovereignty is control.

Guetari draws a clear line between the two.

"Data residency is about where data sits. Sovereignty is about who controls what happens to it," he said.

For a platform team, that means controlling where AI traffic goes, which models can process it, who can access it, how tenants are isolated, and how those policies are audited and enforced. In short: "Residency is where AI runs and where data sits. Sovereignty is who controls it."

That framing matters for anyone building on sovereign AI infrastructure. A local data center is a starting point, not a finish line.

Agents need enforcement they can't bypass

NVIDIA introduced OpenShell at GTC 2026 as part of its Agent Toolkit. It's an open-source runtime that sandboxes agents and controls what they can see, what they can do, and where inference goes.

F5's role in the new OpenShell reference architecture sits around that runtime. F5 AI Guardrails inspect supported prompts, responses, and tool calls. F5 BIG-IP Next for Kubernetes (BNK) handles traffic management from Layer 3/4 through Layer 7.

Guetari explained the split this way: "While OpenShell governs what an agent may intend, BNK governs what it may actually do on the network, at the only chokepoint neither the agent, nor an attacker who compromises it, can avoid: the wire."

The risk customers worry about most isn't a rogue agent. It's "a legitimate agent doing something the enterprise never intended," especially when it can reach tools, APIs, files, and external endpoints. That's why Guetari argues security has to combine permissions, inspection, and network enforcement rather than trusting the agent to police itself.

BNK 2.4 adds identity-aware security for agent traffic. For Model Context Protocol (MCP) and Agent2Agent (A2A) interactions, BNK can validate OIDC/JWT identity claims at ingress, before a request reaches an agent, tool, or enterprise service. Then it applies policy to what that identity is allowed to reach. For example, a specific agent identity can reach an approved MCP server, while other identities are blocked. BNK also adds egress and cross-namespace policy controls around those workloads.

The pattern looks something like this. This is illustrative pseudocode, not BNK configuration syntax:

identity:   agent "invoice-reconciler" (verified JWT, issuer = corp IdP)
allow:      mcp://erp-tools.internal   (read-only tools)
deny:       all other MCP servers
egress:     no external endpoints

"If an agent is compromised or behaves unexpectedly, it shouldn't get unrestricted access simply because it can generate a valid network request," Guetari said.

Why enforce on the DPU

F5 runs this enforcement on NVIDIA BlueField DPUs instead of in software on the host. Guetari gave two reasons: stronger Zero Trust isolation and better infrastructure efficiency.

"The DPU creates an enforcement point the workload doesn't control," he said. BlueField provides a hardware-isolated domain for identity, policy, and multi-tenant isolation that doesn't depend on the host. It also moves networking and some security processing off host CPUs, leaving that compute for AI applications.

There are performance numbers, too. Keep the context in mind: both come from F5-sponsored testing or F5's own lab.

Tolly Group (March 2026, commissioned by F5): On a Llama 3.1 70B cluster with half the GPUs artificially pre-loaded, BNK on DPU cut time to first token (TTFT) by about 61% compared with HAProxy, roughly 14.0 seconds versus 35.8 seconds. Throughput was about 40% higher. BNK used about 2 host CPU cores, compared with roughly 12 for HAProxy.
ServeTheHome (October 2026, F5 lab tour): Running Qwen3-32B on H100 servers, BNK and an Envoy-based AI gateway performed similarly at lighter loads. Under heavy, oversubscribed load, BNK delivered up to 3.24x the performance. The reviewer noted that figure is near the top of what they saw.

The takeaway for architects: GPU-aware load balancing matters most when clusters are busy. That's also when GPUs are most expensive to waste.

Telenor: a sovereign service plane

Telenor's AI Factory serves Norwegian and European organizations that need tight control over where AI data is processed, stored, and accessed. Guetari described the division of labor simply.

"Telenor provides the sovereign AI service. NVIDIA provides accelerated AI infrastructure. F5 provides the service plane controlling how that infrastructure is consumed."

BNK runs on BlueField and provides traffic control, tenant isolation, policy, and telemetry per request, per tenant, and per model. Guetari was careful to separate what's live from what's next. Token-aware governance and more advanced inference optimization are planned extensions, not deployed capabilities.

Indosat: governed access to 300+ models

The Indosat Intelligence Suite puts more than 300 open and closed models behind one interface, including Indosat's Sahabat-AI family, Indosat-hosted models, and connected global models.

Guetari said every AI request raises three questions: Which model should handle it? Which infrastructure should serve it? Is the interaction safe?

Routing can weigh cost, quality, residency, and latency. BNK handles GPU-aware load balancing. F5 AI Security protects against OWASP Top 10 LLM threats and data leakage. Sovereignty policies can decide which workloads stay on Sahabat-AI or other local models and which can reach global ones.

Cost governance is built in. Tokens convert into credits that can be allocated across models, applications, and teams. A CIO can set a company budget and split it across business units, and each team can see and control its own spending in real time.

"Observability tells you what AI cost," Guetari said. "Governance lets you control the cost before it happens."

Europe and Asia start in different places

In Europe, Guetari said, sovereignty usually starts as a control requirement, and the definition varies by operator. Orange, for example, prefers the word "trust." In Asia, control matters too, but sovereignty is also a way to make AI locally relevant and affordable. The Telenor and Indosat projects reflect that difference.

Three mistakes in moving agents to production

Guetari sees teams make the same three mistakes:

They secure the model, but not the agent. The agent's scope and the network it runs on go unprotected.
They trust controls inside the workload's own boundary. Zero Trust requires enforcement outside it, ideally isolated in hardware.
They measure AI consumption but don't govern it.

"Pilots prove functionality," he said. "Production requires security, isolation, auditability and predictable economics."

What to do now

His advice for developers and architects: build around control points, not around today's model. Make policy portable, not model-specific.

Models, agents, and clouds will change. Your policies for identity, data, egress, model access, tenant isolation, AI security, and consumption should stay consistent. Egress is a good place to start. If prompts and responses must stay inside the AI factory, outbound traffic needs explicit control.

The bottom line

Guetari sees AI infrastructure entering a second phase. The first was about building capacity: GPUs, models, and AI factories. The next is about running that capacity as a governed, trustworthy service.

"GPUs provide capacity. Models provide intelligence. Agents provide action," he said. What's missing is the control layer that makes all three safe to use at scale.

You don't need to buy F5 to take the core lesson. Don't ask an agent to enforce its own limits. Put identity, policy, and egress controls somewhere the agent can't reach. And remember that whoever builds and deploys the agent owns what it does.

Reverse Engineering NVIDIA CUDA Binaries with Data Clustering and AI

Fri, 09 Oct 2026 14:14:25 +0000

Motivation

I am not a data scientist/engineer, but I wanted to do a little experiment on NVIDIA Cubin binary code. I recalled my colleague's thesis on Data Clustering and wondered what would actually happen if we tried something like that on an NVIDIA binary?

My assumption was that, if we managed to find a good distance metric, we could categorize instructions and gain a higher-level insight into how NVIDIA CUDA binaries are constructed.

Hamming distance

The Hamming distance between two fixed-length binary words is the number of bit positions at which they differ. For our 64-bit words, we can calculate it using XOR followed by a population count:

dist = (insn1 ^ insn2).bit_count()

The XOR identifies the differing bits, and bit_count() counts them. The fewer bits two words differ in, the closer they are under this metric.

Mathematically, we can view our data as a subset of the 64-bit binary words, with the Hamming distance as the metric. In other words, our two instructions are closer the more bits they have in common. They are further apart the more bits they differ in.

Obtaining data

My own CUDA kernel code could provide very little data. It was about 24 instructions in length, and rather than adding random code, I decided to check out the cuda-samples repository:

https://github.com/NVIDIA/cuda-samples

My target architecture, sm_50, was too old to compile these samples using the documented instructions, so I simply picked one sample and compiled it manually:

cd ./cpp/5_Domain_Specific/recursiveGaussian
nvcc -arch=sm_50 -cubin recursiveGaussian_cuda.cu -I /home/stjepan/Develop/cuda-samples/Common/ -o kernel.cubin

This gave me the Cubin ELF, but I still needed to extract the raw binary code. I used the NVIDIA-provided tools for this:

$ cuobjdump -ltext ../kernel.cubin
SASS text section 1 : kernel-_Z24d_recursiveGaussian_rgbaPjS_iiffffffff.sm_50.elf.bin
SASS text section 2 : kernel-_Z22d_simpleRecursive_rgbaPjS_iif.sm_50.elf.bin
SASS text section 3 : kernel-_Z11d_transposePjS_ii.sm_50.elf.bin

To extract these sections, change -ltext to -xtext. My small database of NVIDIA binary data was now much more promising (and more statistically significant):

$ ls *.bin | xargs -I{} xxd -e -g 8 -c 8 {} | wc -l
2280

Subsequently, I compiled a lot more CUDA binaries and added more than 40000 instructions. Due to performance constraints, I opted to limit the dataset to 12000 instructions.

First attempt (DBSCAN)

For my first experiment, I was not at all disappointed. I installed the scikit-learn and numpy packages for my Python script. Scikit-learn provides the DBSCAN clustering algorithm, and my first attempt was something as simple as:

import numpy as np
from sklearn.cluster import DBSCAN
from pathlib import Path
import struct

def cluster(insns):
    dist = np.zeros((len(insns), len(insns)), dtype=np.float64)

    for i in range(len(insns)):
        for j in range(i + 1, len(insns)):
            d = (insns[i] ^ insns[j]).bit_count()
            dist[i, j] = d
            dist[j, i] = d

    clusters = DBSCAN(
        eps=5,                 # epsilon (how far points in a cluster can be)
        min_samples=10,        # minimum number of samples a cluster must have
        metric="precomputed"
    ).fit_predict(dist)

    c_dict = {}
    for i in range(0, len(clusters)):
        c = clusters[i]
        if not c in c_dict:
            c_dict[c] = [insns[i]]
        else:
            c_dict[c] += [insns[i]]

    return c_dict

Sample output

This already provided useful data. The following is output showing five samples per cluster (the original dataset contains 12000 sampled instructions):

_______[   0  ]_______  _______[   9  ]_______  _______[  18  ]_______  _______[  29  ]_______
  0x001cc400e22007f6      0xe2400ffff400000f      0x5c403200026720ff      0x1e03fb8aa3b7170c
  0x001fd801fec0075d      0xe2400fffe408000f      0x4c413008001722ff      0x1e03fb8aa3b71a08
  0x089fec02fc2007f1      0xe2400fffff87000f      0x4c413008001723ff      0x1e03fb8aa3b7120b
  0x001ff400ffa007f0      0xe2400fffff87000f      0x5c423200017714ff      0x1e03fb8aa3b7190a
  0x003fd800e3a007fd      0xe2400fffc889000f      0x5c423200019718ff      0x1e03fb8aa3b7140c
_______[   1  ]_______  _______[  32  ]_______  _______[  16  ]_______  _______[  30  ]_______
  0x4c98078000870001      0x32807fdf80070303      0x30cd83ff800722ff      0x37be03c2fc070c1f
  0x4cb8000005472a03      0x32807fdf80082222      0x30cd83ff80070eff      0x37be03c2fc070817
  0xf0c8000002570002      0x32807fdf80092323      0x30cd83ff800704ff      0x37be03c2fc070b07
  0xf0c8000002170004      0x32807fdf80080e0e      0x30cd83ff80070dff      0x37be03c2fc070a0f
  0x1c00180000070300      0x32807fdf80091010      0x30cd83ff800707ff      0x37be03c2fc070c27
_______[   2  ]_______  _______[  10  ]_______  _______[  17  ]_______  _______[  38  ]_______
  0x0407f80000070000      0x040007fffff7030a      0x36bd83ff80072397      0x5bdf7f81f0670506
  0x0423f80000070a05      0x040007fffff70806      0x36bd83ff80072287      0x5bdf7f81c0770705
  0x0427f80000072121      0x040007fffff72020      0x36bd83ff8007108f      0x5bdf7f81c0470404
  0x0427f80000072121      0x040007fffff70a0c      0x36bd83ff80070e87      0x5bdf7f81c0770705
  0x0407f80000070404      0x040007fffff70604      0x36bd83ff80070597      0x5bdf7f81c0470404
_______[   3  ]_______  _______[  36  ]_______  _______[  19  ]_______  _______[  33  ]_______
  0x4b68038800070007      0x38423000002709ff      0x3659038000a71514      0x537104080007060a
  0x4b6d038005470007      0x38403000001709ff      0x3659038000a70517      0x537104080007060a
  0x5b6403800ff70317      0x38423000002709ff      0x3659038000aa1617      0x5b710a000087060c
  0x5b20030800970605      0x38403000001709ff      0x3659038000a71618      0x5371020800070608
  0x5b28038000970607      0x38403000007700ff      0x3659038000a71519      0x5371020800070616
_______[   4  ]_______  _______[  11  ]_______  _______[  20  ]_______  _______[  34  ]_______
  0xe260000052000040      0xda1005affff70202      0x36b403c080071317      0x386800437f070404
  0xe29000002a000000      0xda1005affff70404      0x36b403c080071307      0x386800437f070505
  0xe290000020000000      0xda1005affff70202      0x36b403c080071707      0x386800437f070606
  0xe2600000fc800040      0xda1005affff70404      0x36b403c080071707      0x386800437f070707
  0xe290000015000000      0xda0006effff70303      0x36b403c080071707      0x386800437f070206
_______[   7  ]_______  _______[  12  ]_______  _______[  21  ]_______  _______[  35  ]_______
  0x1c00ffffffe70506      0x3868003f00070a12      0x328002c000071414      0x5be70b0780d70c0c
  0x1c0ffffffff70606      0x3868003f00070f0f      0x328002c000071313      0x5be7098780e70c0c
  0x1c0ffffffff7050d      0x3868003f0007060a      0x328002c000071313      0x5be7070780f70c0f
  0x1c0ffffffff70003      0x3868003f00070709      0x328002c000071313      0x5be7068780870f08
  0x1c0ffffffff70603      0x3858003f00070000      0x328002c000071313      0x5be70b0780970808
_______[  28  ]_______  _______[  13  ]_______  _______[  22  ]_______  _______[  37  ]_______
  0x5c4707000047ff04      0x0103f8000007f007      0x368403c010071e17      0x1e03f3504f370303
  0x5c6007800027ff03      0x0103f8000007f018      0x368403c010071e07      0x1e03f3504f370b0b
  0x5c2107800037ff03      0x0103f8000007ff03      0x368403c010071e07      0x1e03f3504f370e0e
  0x5c60078001a7ff17      0x0103f8000007f021      0x368403c010071e07      0x1e03f3504f370d0d
  0x5c6007800127ff09      0x0103f8000007f027      0x368403c010071e07      0x1e03f3504f370d0d
_______[  -1  ]_______  _______[  14  ]_______  _______[  23  ]_______  _______[  39  ]_______
  0x4cc0018005470404      0x4801038006670310      0x002c4800eee00711      0x36b403c37f070a17
  0x5cc002a000870705      0x4801038006670414      0x002c4800eee00711      0x36b403c37f071f07
  0x5b28030000470505      0x4801038006670515      0x002c4800eee00711      0x36b403c37f070b1f
  0x5c4707000032ff05      0x4801038006670615      0x002c4800eee00711      0x36b403c37f07090f
  0x32807fdf80070202      0x480103800667070e      0x002c4800eee00711      0x36b403c37f072607
_______[   5  ]_______  _______[  24  ]_______  _______[  25  ]_______  _______[  40  ]_______
  0x366c03800037050f      0x36f0058010171410      0x011c7801e2404776      0x5bae05000ff7200a
  0x3668038000170407      0x3670054000071416      0x011c7801e2404776      0x5bae0f800ff70808
  0x366803800fd72707      0x3670054000071416      0x011c7801e2404776      0x5bae05800ff7200b
  0x366820000fd72407      0x3670054000071416      0x011c7801e2404776      0x5bae04800ff71f09
  0x366c03800fe72007      0x3670054000071416      0x011c7801e2404776      0x5bae13000ff70808
_______[   6  ]_______  _______[  15  ]_______  _______[  26  ]_______  _______[  41  ]_______
  0xeed4200000070407      0xf0a81b8000070000      0x003ff401e3a00f06      0x338001437f070302
  0xeed4200000070408      0xf0a81b8000070000      0x003ff401e3a00f06      0x338003437f070c0a
  0xeedc200000070407      0xf0a81b8000070000      0x003ff401e3a00f06      0x338003437f070006
  0xeed4200000070805      0xf0a81b8000070000      0x003ff401e3a00f06      0x338003c37f070006
  0xeed4200000070c0b      0xf0a81b8000070000      0x003ff401e3a00f06      0x338003437f070006
_______[   8  ]_______  _______[  31  ]_______  _______[  27  ]_______  _______[  42  ]_______
  0xf0f0000034170000      0x30cc03ff80072326      0xd820056ff0470606      0x010437f00007f025
  0xf0f0000034670000      0x30cc03ff80070e0c      0xd82005aff0570b08      0x010437f00007f025
  0xf0f0000034470000      0x30cc03ff80071011      0xd82005aff0570609      0x010437f00007f025
  0xf0f0000034270000      0x30cc03ff80070406      0xd82005aff057070d      0x010437f00007f025
  0xf0f0000034670000      0x30cc03ff80070507      0xd82005aff0570a0e      0x010437f00007f025

DBSCAN has produced groups of binary words with similar bit patterns. The only exception is the -1 cluster, which essentially means "unassigned" or, as I prefer to call it, "noise". DBSCAN puts data there when it cannot assign it to a cluster.

Sorting things out further

In clusters 13 and 42, we can see the MOV-like instruction from our previous reverse-engineering exercise:

_______[  13  ]_______  _______[  42  ]_______
  0x0103f8000007f007      0x010437f00007f025
  0x0103f8000007f018      0x010437f00007f025
  0x0103f8000007f01d      0x010437f00007f025
  0x0103f8000007f021      0x010437f00007f025
  0x0103f8000007f027      0x010437f00007f025

Similarly, here is the second instruction we tinkered with in the previous article:

_______[   6  ]_______
  0xeed4200000070407
  0xeed4200000070408
  0xeedc200000070407
  0xeed4200000070805
  0xeed4200000070c0b

How could we sort these instructions out further? One thing we can do is apply bitwise AND to instructions like the ones in bins 13 and 42 to find out what they have in common:

TEST = [ 0x0103b8080817f003,
         0x0103f8000007f002,
         0x0103b8080817f000,
         0x0103f8000007f006,
         0x0103b8080817f000,
         0x010437f00007f025 ]

and_val = TEST[0]
for each in TEST:
    and_val &= each

print(f"{and_val:016x}")

For example, this gives us the following:

010030000007f000

So, except for the digit 3, we seem to be getting closer to the instruction's structure. If we could combine this with our clustering approach, we might be able to extract more information about the format.

Second attempt (adaptive HDBSCAN)

A little investigation led me to HDBSCAN, which seemed to give me more control over the clustering results:

clusters = hdbscan.HDBSCAN(
               min_cluster_size=min_size,
               min_samples=samples,
               cluster_selection_method="eom",
               metric="precomputed"
           ).fit_predict(dist)

It seemed to produce more specific clusters, but many samples still ended up in the "noise" cluster.

Sorting instructions

I had already described how we can use bitwise AND to get the common bit pattern of a cluster. Bitwise OR can also provide useful information about which bits vary across the data. My code combined these two operations while preserving cluster-size information:

c_dict = {}
noise = []
for i in range(0, len(clusters)):
    c = clusters[i]
    if c == -1:
        noise.append(insns[i])
    elif c not in c_dict:
        c_dict[c] = [insns[i]]
    else:
        c_dict[c] += [insns[i]]

res = {}
for (key, val) in c_dict.items():
    first = val[0]
    orval = val[0]
    for each in val[1:]:
        first &= each
        orval |= each
    orval ^= first
    if first not in res:
        res[first] = (orval, val)
    else:
        (old_orval, old_val) = res[first]
        res[first] = (old_orval | orval, old_val + val)

Later, I also added per-bit frequencies and a noise percentage:

total = 0
for (key, val) in res.items():
    total += len(val[1])
    print(f"{key:016x} {val[0]:016x} ({len(val[1]):4})")
    freqs = bit_frequencies(val[1])
    print_frequencies(freqs)
    count = 0
    for each in val[1]:
        if count > 5:
            break
        else:
            print(f"    {each:016x}")
            count += 1
    print("")

print(f"TOTAL: {total}")
noise_len = len(noise)
print(f"NOISE: {noise_len:4} {(noise_len/(total + noise_len)) * 100}%")

The results were OK, but not entirely satisfying. Here is one example:

5c9807800ff00000 00000000000f003f (  85)
 * 01011100100110000000011110000000000011111111____00000000000_____
    5c9807800ff7001e
    5c9807800ff7001b
    5c9807800ff70017
    5c9807800ff7000b
    5c9807800ff80020
    5c9807800ff00020

308c03ff80070000 0671800000003fff (  45)
 * 00110__01___110__000001111111111100000000000011100______________
    30cc03ff80072220
    30cc03ff80072326
    30cd83ff800722ff
    36bd83ff80072397
    36bd83ff80072287
    30cc03ff80070e0c

e34000000007000f 0000000000000000 ( 125)
 * 1110001101000000000000000000000000000000000001110000000000001111
    e34000000007000f
    e34000000007000f
    e34000000007000f
    e34000000007000f
    e34000000007000f
    e34000000007000f

3280004000070000 0102078140001f1f (  75)
 * 0011001_100000_000000____100000_0_00000000000111000_____000_____
    328002c000071414
    328002c000071313
    328002c000071313
    328002c000071313
    328002c000071313
    328002c000071313

5c47000000000000 0000060001f70f0f (  90)
 * 010111000100011100000100000000000000000_____0___00000___00000___
    5c47040000a70505
    5c47040000870202
    5c47040000870b0b
    5c47040000870f0f
    5c47000000c70e0e
    5c47000000470505

4880000800070000 05201f8000303f3f (  57)
 * 01001_0_10_0000000000____00010000000000000__011100______00______
    4980060800270909
    4980040800270b0b
    4ca004080027140c
    4ca0008800070f09
    4ca0008800070b0e
    4ca0000800070b0f

300000427c070000 0ffe1f8183003f3f ( 164)
 * 0011___________0000______100001__11111__0000011100______000_____
    37be03c2fc070c1f
    37be03c2fc070817
    37be03c2fc070b07
    37be03c2fc070a0f
    37be03c2fc070c27
    37be03c2fc071437

TOTAL: 6921
NOISE: 5079 42.325%

The bit fields indicate the per-bit frequencies. If the frequency of a particular bit in the cluster exceeds or falls below a threshold, it is represented as 1 or 0, respectively. If it varies across the cluster, it is represented as _.

The biggest problem, however, was the amount of noise. I could reduce it, but then cluster 0 grew:

0000000000000000 7ffbbf9017ff3f3f (1406)
 * 010_1100_____000000000000000000000000_0_____0111000_____000_____
    4cb8000005472a03
    1c00180000070300
    4e00020000270204
    5cb8010000370a05
    5cb0118000670a06
    5c10000000370004

Many instructions ended up in this cluster because all bits had equal weights. Bits representing instruction structure could be outweighed by immediate values or addresses. I spent a lot of time tinkering with DBSCAN and HDBSCAN parameters, trying to find a better representation, but eventually I got tired of it.

Employing the AI agent

I was concerned that the AI agent's existing knowledge of NVIDIA and Maxwell might influence an experiment intended to help me understand the binary format. I wanted to compare its conclusions with existing results, so I instructed the agent to approach the data without relying on NVIDIA-specific explanations.

I didn't ask the agent to decipher the binary independently, either. I still wanted to explore my clustering idea. So I left the agent running overnight, asking it to experiment with DBSCAN and HDBSCAN parameters, document any insights, and reduce the noise as much as possible.

I also instructed it to optimize my Python code if necessary and to try weighted Hamming distance.

Minimizing noise

The AI agent's report on unweighted Hamming distance suggested that the clustering approach was not necessarily being used incorrectly; rather, some of the data was too dispersed to form useful clusters:

| data        | best                         | clusters | noise |
|-------------|------------------------------|----------|-------|
| all words   | HDBSCAN `mcs=5, ms=1, eom`    | 572      | 28 %  |
| all words   | DBSCAN `eps=2, ms=2`          | 1886     | 14 %  |
| type A only | DBSCAN `eps=1, ms=2`          | 407      | 12 %  |
| type B only | HDBSCAN `mcs=3, ms=1, eom`    | 1318     | 27 %  |
| type B only | DBSCAN `eps=2, ms=3`          | 789      | 25 %  |

With equal weights, both algorithms performed best at relatively fine-grained settings in the configurations tested. High-entropy operand bits appeared to dominate the distance and break potential groups apart. Larger `min_samples` or `min_cluster_size` values tended to push more words into noise. HDBSCAN `eom` performed better than `leaf` in these tests, and `min_samples=1` gave the best results in the configurations compared.

The big breakthrough

One finding from the AI agent immediately caught my attention: I had overlooked the possibility that word position might matter.

All file sizes are multiples of 32 bytes. A comparison of words at offset 0 mod 4 (type A) with words at offsets 1 and 3 mod 4 (type B) showed different statistics: about 3,258 versus 5,300 distinct values per slot, and mean popcounts of 24.8 versus 18.6.

I had assumed that the binary contained only real instructions, rather than potentially including periodic non-instruction data. I checked the observation against the data:

$ xxd -e -g8 -c8 kernel-_Z9dwtHaar1DPfS_S_jji.sm_50.elf.bin | awk '{ if (NR % 4 == 1) { print $2 } }' | head -n20
001c4400fe0007f6
081fc401fec0073f
001f8400fec217f6
081fc400fea207f1
001fc000fec007f1
0002c440fe0007b2
081fcc00fe2007e1
003fd440fe2007f2
081fc400fea007e1
041fc400fee007f6
001fd400ffe007f1
001ffc00fcc00711
001fbc00feaa0ff1
001ffc00fe0007e1
0003d000fe0007f5
001fbc00fde007f1
001fd400ffe007e9
001fbc00fde007ef
083fc400ffa007e8
001fc400fe2007f1

The words at offsets divisible by four appeared to have a different pattern from the words at offsets 1 and 3 modulo four. The AI agent's analysis helped me notice this difference, which I then checked against the data. I haven't established what, if anything, the pattern tells us about the underlying binary format.

Final thoughts

This was my first experiment with data clustering for reverse-engineering. Although the clustering results were imperfect, the experiment showed me that Hamming distance can reveal structure in binary data.

The most useful outcome was an observation I had overlooked: words at different positions in the binary appeared to have different statistical properties. An AI agent helped me notice this pattern, which I then checked against the data.

I haven't established what this positional pattern means yet. For now, it's simply an observation worth investigating further.

An AI agent can be useful even when it doesn't solve the problem outright: it can run experiments, compare results, and help identify patterns that deserve a closer look.

Reverse-Engineering NVIDIA: Changing a register operand

Fri, 25 Sep 2026 14:41:55 +0000

Motivation

In the previous article I have
started reverse-engineering a simple CUDA binary by modifying the value of
a pointer variable.

After identifying where the value was stored and successfully running the modified CUDA binary, it was time to find out what the corresponding assembly instructions actually looked like at the machine-code level.

Finding the immediate size

Now, to be able to determine the size and format of the instruction (or at
least the largest constant the instruction from the previous article can hold),
I started to increase the value word-by-word. Instead of simple 0xcc from the
last time, I did:

#include <stdio.h>

__global__ void test(int *a) {
    a[0] = 0xecc;
}

int main(int argc, const char* argv[]) {
    cudaError_t err;
    int *a;
    cudaMallocManaged(&a, sizeof(int));
    *a = 0x01;
    test<<<1, 1>>>(a);
    if ((err = cudaGetLastError()) != cudaSuccess)
        fprintf(stderr, "launch: %s\n", cudaGetErrorString(err));

    cudaDeviceSynchronize();
    printf("*a = %x\n", *a);

    return 0;
}

And yes, it was there:

$ xxd -e out/test-old.sm_50.cubin | grep ecc
00000690: ecc7f000 01000000 05070002 4c980780   ...............L

Now to try a bit larger value like '0xdecc':

$ xxd -e out/test-old.sm_50.cubin | grep decc

Searching for the complete 0xdecc pattern didn't find anything. Searching for
the original 0xecc did, however, and the surrounding bytes had changed:

$ xxd -e out/test-old.sm_50.cubin | grep ecc
00000690: ecc7f000 0100000d 05070002 4c980780   ...............L

And there it was. If we compare this line with the hex dump from above, we can
see that we have an extra d in 0100000d. So I tried increasing the word
size in xxd:

$ xxd -e -g 8 out/test-old.sm_50.cubin | grep decc
00000690: 0100000decc7f000 4c98078005070002   ...............L

Now the bytes looked much more like a structured 64-bit value rather than an
arbitrary sequence of bytes. The next thing to do was simply increase the test
value and see how much the apparent immediate field could hold.
The result with the largest value was:

$ xxd -e -g 8 out/test-old.sm_50.cubin | grep ffffffff
00000690: 010ffffffff7f000 4c98078005070002   ...............L

After that, it was impossible to fit more into an int value as nvcc
complained about truncation. I naturally tried to find out what will
happen if I use uint64_t instead of int:

$ xxd -e -g 8 out/test-old.sm_50.cubin | grep ffffffff -C1
00000680: 001fc400fe4007f1 010eeeeeeee7f004   ..@.............
00000690: 010ffffffff7f005 eedd200000070204   ............. ..
000006a0: 001f9c00fde007ef 50b0000000070f00   ...............P

So we discovered something. We now had two apparent MOVs and I wasn't sure
whether they differed because they targeted different registers or because they
loaded the high and low parts of a value.

Adding a static variable

After seeing how the MOV behaved while changing the value, I tried a little
experiment. I added one more variable with a lot of bytes:

#include <stdint.h>

__device__ uint64_t x = 0xdaec;

__global__ void test(uint64_t *a) {
        x = 0xccccccccdddddddd;
        a[0] = 0xaaaaaaaabbbbbbbb;
}

After looking for my usual patterns, I got:

$ xxd -e -g 8 out/test-old.sm_50.cubin | grep bbbb -B5 -A7
00000780: 001fc400fe2007f6 4c98078000870001   .. ............L
00000790: 010000000007f002 010000000007f003   ................
000007a0: 001fc000fe4007f1 010dddddddd7f008   ..@.............
000007b0: 010cccccccc7f009 4c98078005070004   ...............L
000007c0: 001fc400fc2007f1 eedd200000070208   .. .......... ..
000007d0: 4c98078005170005 010bbbbbbbb7f006   .......L........
000007e0: 001fbc00fe2007f2 010aaaaaaaa7f007   .. .............
000007f0: eedd200000070406 50b0000000070f00   ..... .........P
00000800: 001ffc00fc4007ef 50b0000000070f00   ..@............P
00000810: 50b0000000070f00 e30000000007000f   .......P........
00000820: 001f8000fc0007ff e2400fffff07000f   ..............@.
00000830: 50b0000000070f00 50b0000000070f00   .......P.......P
00000840: 000000000000daec 0000000000000000   ................

Looking at this we can filter some of these out. We can filter by patterns I
created. Also, I noticed that the line beginning with 0xeed appeared a lot
in my dumps. Here it appeared two times, so I filtered useful stuff out:

$ xxd -c 8 -e -g 8 out/test-old.sm_50.cubin | grep 'bbbb\|cccc\|aaaa\|dddd\|eedd'
000007a8: 010dddddddd7f008   ........
000007b0: 010cccccccc7f009   ........
000007c8: eedd200000070208   ..... ..
000007d8: 010bbbbbbbb7f006   ........
000007e8: 010aaaaaaaa7f007   ........
000007f0: eedd200000070406   ..... ..

The interesting part was that the final byte changed systematically as I introduced different values. This made it look increasingly likely that the final byte represented a register operand.

This is current test output:

$ ./test-old 
*a = aaaaaaaabbbbbbbb

Can we make it display the value of variable x? Let's try hacking 0xeed2
instructions. My hypothesis was that the low bits of these instructions encode
the register operand, similarly to what we observed in the 0x10 lines:

$ xxd -c 8 -e -g 8 test-old | grep eedd2
0008a848: eedd200000070208   ..... ..
0008a870: eedd200000070406   ..... ..

I used bvi to change the final byte of the second instruction from 06 to
08:

xxd -c 8 -e -g 8 test-old | grep eedd2
0008a848: eedd200000070208   ..... ..
0008a870: eedd200000070408   ..... ..

And (not) surprisingly, I got:

$ ./test-old
*a = ccccccccdddddddd

We actually switched registers from which the value for a[0] was loaded.

Conclusion

For now we can identify (and modify to a certain extent) two instructions:

The first instruction appears to be a MOV-like instruction, with a format that
currently looks like:

010VVVVVVVV7f00R

The VVVVVVVV appears to represent an immediate value, while R appears to
identify the destination register.

The second instruction appears to select a register containing the value that is
ultimately written to a[0]. Its encoding currently appears to be:

eedX200000070Y0R

where R appears to identify the register. X/Y remain unknown; I suspect one
of them encodes information about the operand size, since I observed different
values when experimenting with 32-bit int versus 64-bit uint64_t values.

No Chance of AI Ending Humanity by 2030: NVIDIA CEO Jensen Huang

Tue, 22 Sep 2026 14:28:15 +0000

The rapid development of Artificial Intelligence (AI) has sparked growing debate about its potential impact on humanity and the future of civilization. While some AI researchers have warned about extreme risks, NVIDIA CEO Jensen Huang has strongly rejected predictions that AI could lead to the end of humanity within the next decade.

Jensen Huang Rejects AI Extinction Predictions

In an interview with the American broadcaster CBS, Jensen Huang said that there is a zero percent chance of AI causing the end of the world by 2030.

Huang described efforts to frighten people about the possibility of AI ending humanity as unnecessary and irresponsible. His comments represent a significantly more optimistic view of the future of AI compared with some researchers who have warned about the long-term risks of increasingly powerful AI systems.

Huang Criticizes Jacob Coxon’s Warning

Jensen Huang also criticized the warning made by former Anthropic researcher Jacob Coxon, who argued that AI could eventually develop extraordinary human-level capabilities and potentially cause the extinction of humanity within the next decade.

Huang described such a prediction as irresponsible, rejecting the idea that current AI development should be viewed primarily through the possibility of human extinction.

AI Development Should Move Faster, Not Slower

Huang also disagreed with calls to slow down or stop the development of artificial intelligence.

According to his position, the technology industry should continue advancing AI as quickly as possible while ensuring that unsafe products are not released into the market.

This approach places the focus on developing and deploying AI responsibly rather than significantly reducing the overall speed of technological progress.

Growing Debate Over the Future of AI

Jensen Huang’s comments further highlight the growing divide within the technology industry over how society should approach increasingly powerful AI systems.

Supporters of rapid AI development argue that progress should continue while companies focus on building safer products. Meanwhile, researchers concerned about AI risks have called for greater caution, stronger safety measures, and more attention to the potential consequences of increasingly capable systems.

The disagreement reflects one of the central debates surrounding AI today: Should the industry continue pushing AI capabilities as quickly as possible, or should development slow down to allow safety research to keep pace?

As AI technology continues to advance, the balance between innovation, safety, and responsible development is likely to remain one of the most important discussions in the technology sector.

Written by: Muhammad Yasir

Muhammad Yasir is a Full Stack Developer, tech and AI enthusiast who shares practical insights, tips and guides on the latest technology, AI tools, robotics, videos and engineering topics.

Explore the website: https://yasirawaninfo.vercel.app/

#AI #ArtificialIntelligence #JensenHuang #NVIDIA #AISafety #FutureOfAI #TechNews #MachineLearning #GenerativeAI #Innovation #Technology #AIEthics #ResponsibleAI #TechTrends #DigitalTransformation #MuhammadYasir #YasirAwan4831

Inside the Gemini-Powered Siri: Why Apple Rented Its AI Brain from Google

Wed, 16 Sep 2026 16:33:52 +0000

Apple officially shipped its flagship Siri overhaul, but the core reasoning engine isn't running on native silicon it's powered by a customized, 1.2 trillion parameter Google Gemini model.

For developers and system architects, this represents a massive structural shift in how tech giants handle frontier-scale AI infrastructure.

Here are the key technical takeaways from this release:

Hybrid Execution Pipeline: Simple system tasks run on-device via Apple Silicon, while complex multi-step reasoning routes to a cloud-based Gemini 1.2T model.

Private Cloud Compute (PCC): Requests bypass Google's public APIs. Queries run inside hardware-isolated enclaves with end-to-end encryption, ensuring zero data retention or exposure to third parties.

Nvidia Blackwell B200 Stack: The underlying cloud layer relies on Nvidia’s B200 GPUs with confidential computing primitives enabled for secure remote inference.

End of Vertical Integration?: By opening iOS 27 to third-party model orchestration and Foundation Models APIs, Apple acknowledges that controlling the interface and ecosystem outweighs owning the frontier model layer.

Regulatory Constraints: Due to strict data sovereignty and local privacy frameworks, this rollout remains unavailable in the EU and China.

We broke down the full engineering architecture, privacy proxy pipelines, antitrust context, and what this means for developers building on Apple's ecosystem.

👉 Read the full technical breakdown on TheFluxRead: [https://www.thefluxread.com/2026/09/the-worlds-most-valuable-hardware.html]

Apple #GoogleGemini #Siri #AI #SoftwareArchitecture #SystemDesign #CloudComputing #Nvidia #TheFluxRead

Reverse-Engineering NVIDIA: Modifying a CUDA binary

Sun, 06 Sep 2026 18:41:28 +0000

Motivation

For quite a while I wanted to find out how NVIDIA really works under the hood.
One of the greatest unknowns for me was how GPU code is compiled, packaged and
loaded. So I accepted my own challenge: let's try and decode machine code
running on GPU.

Test code

I needed to start really simple and to get as much data as possible. My first
CUDA program to try and decipher was simply:

#include <stdio.h>

__global__ void test(int *a) {
        a[0] = 0x00;
}

int main(int argc, const char* argv[]) {
        cudaError_t err;
        int *a;
        cudaMallocManaged(&a, sizeof(int));
        *a = 0x01;
        test<<<1, 1>>>(a);
        if ((err = cudaGetLastError()) != cudaSuccess)
                fprintf(stderr, "launch: %s\n", cudaGetErrorString(err));

        cudaDeviceSynchronize();
        printf("*a = %x\n", *a);

        return 0;
}

I won't get into too many details here (I am not a CUDA expert), but essentially, the function marked with __global__ is the kernel that will run on the device. The cudaMallocManaged() function takes care of allocating memory that can be accessed by both the host and the device, saving us some additional memory-management code. The test<<<1, 1>>>(a) syntax launches the test kernel with one block containing one thread, while the CUDA runtime takes care of loading the kernel for execution on the GPU. Finally, cudaDeviceSynchronize() waits for the GPU to finish executing the kernel, ensuring that the result written to a is available when we access it from the host.

Build

I also created a reusable build command that will also give me something to
start with:

nvcc -arch=sm_50 -lineinfo --keep --keep-dir out -o test test.cu

Note: for this to work, the out directory needs to be created before running
nvcc compiler. Now I got a folder full of really useful files:

stjepan@stjepan-Aspire-F5-573G:~/Develop/NVIDIA$ ls out/
test.cpp1.ii    test.cudafe1.cpp     test_dlink.fatbin    test_dlink.reg.c        test.fatbin.c   test.ptx
test.cpp4.ii    test.cudafe1.gpu     test_dlink.fatbin.c  test_dlink.sm_50.cubin  test.module_id  test.sm_50.cubin
test.cudafe1.c  test.cudafe1.stub.c  test_dlink.o         test.fatbin             test.o

The test.sm_50.cubin file sounded like exactly what I was looking for:

stjepan@stjepan-Aspire-F5-573G:~/Develop/NVIDIA$ file out/test.sm_50.cubin 
out/test.sm_50.cubin: ELF 64-bit LSB executable, NVIDIA CUDA architecture,, statically linked, not stripped

First hack

Now I wanted to try changing the value of a in the test function and see how
cubin file will behave. I had no idea what I was dealing with, so I started
checking the file with random small values to get the one with least repetitions
in the CUDA binary. After some random attempts I realized that the best one is:

stjepan@stjepan-Aspire-F5-573G:~/Develop/NVIDIA$ hexdump out/test.sm_50.cubin | grep bc
0000680 07f1 fde0 bc00 001f 02ff 0007 2000 eedc
00008f0 0000 0000 0000 0000 04bc 0000 0000 0000

So I decided to change my test function to:

__global__ void test(int *a) {
        a[0] = 0xbc;
}

If everything went well, I would see an additional line here containing my bc.
The reason I started with really small values is because I had no idea what kind
of byte ordering I will encounter. After rebuilding the binary, I now got:

stjepan@stjepan-Aspire-F5-573G:~/Develop/NVIDIA$ hexdump out/test.sm_50.cubin | grep bc
0000310 f1bc 06d4 02fc 0000 0209 0000 0000 0000
0000690 f000 0bc7 0000 0100 0002 0507 0780 4c98
00006a0 07f2 fe20 bc00 001f 0003 0517 0780 4c98

Well, I definitely got a bit more different hex dump. So I tried just changing
it again, this time to 0xcc (just randomly) to see what I will get. And I got:

stjepan@stjepan-Aspire-F5-573G:~/Develop/NVIDIA$ hexdump out/test.sm_50.cubin | grep cc
0000690 f000 0cc7 0000 0100 0002 0507 0780 4c98

This was promising. Notice that this is the same as address 0x0000690 in the
previous run, except we got 0cc7 instead of 0bc7. Now I could try and find
this kind of pattern in the real binary:

stjepan@stjepan-Aspire-F5-573G:~/Develop/NVIDIA$ hexdump test | grep 0cc7
000cc70 528b 4830 e789 5c89 1024 c748 2444 0008
008a700 0001 0087 0780 4c98 f000 0cc7 0000 0100

We can see our value on address 0x008a700. Right now, if we run the binary,
we get:

stjepan@stjepan-Aspire-F5-573G:~/Develop/NVIDIA$ ./test 
*a = cc

Now, I used bvi to quickly find the address and changed 0cc7 to 0dd7.
So the result was:

stjepan@stjepan-Aspire-F5-573G:~/Develop/NVIDIA$ hexdump test | grep 0dd7
000dd70 01bb 0000 e900 ff76 ffff 0f66 441f 0000
008a700 0001 0087 0780 4c98 f000 0dd7 0000 0100

Well, hopefully, I didn't corrupt the binary. When I tried running it:

stjepan@stjepan-Aspire-F5-573G:~/Develop/NVIDIA$ ./test 
*a = dd

Bingo. We didn't recompile the CUDA source, regenerate the CUBIN, or even touch
the host executable's source code. We changed bytes inside the compiled binary,
and the GPU executed the modified instruction.

Final thoughts

We managed to modify code running on the GPU without recompiling it. More
importantly, we have found what appears to be an instruction field containing
our constant. Given what the kernel does, it is likely part of a mov-like
instruction, although we haven't decoded it yet.

So the next question is obvious: what is the real encoding of the instruction?

CUDA, ROCm, and the GPU Software Stack: Where Do You Even Start?

Tue, 01 Sep 2026 07:59:33 +0000

If you've worked with PyTorch, you've probably written something like model.cuda() and watched your model start running on the GPU. But what actually happened after that? How does a line of Python eventually turn into thousands of threads doing computation on a GPU? What exactly is CUDA? Where does ROCm fit in? And if you're someone who wants to go beyond simply using GPUs and actually understand them, where do you even start?I started asking myself these questions recently, and that slowly pulled me into the world of GPU programming. At first, I thought CUDA was basically just a way of writing code for NVIDIA GPUs. But it's much bigger than that. CUDA is NVIDIA's platform and programming model for general-purpose GPU computing. It gives you the tools and abstractions needed to write GPU kernels, manage memory, launch work, and interact with the hardware. Once you start writing CUDA, you stop thinking of a program as something that simply runs from top to bottom. Instead, you start thinking about how thousands of pieces of work can execute in parallel.

That leads to one of the first important things to understand about GPUs: a GPU isn't simply a faster CPU. CPUs are built around a relatively small number of powerful cores, while GPUs are designed to keep huge amounts of parallel work running at the same time. In NVIDIA's programming model, threads are organized into blocks, and threads execute in groups called warps. Those warps are scheduled onto the GPU's Streaming Multiprocessors, or SMs. You don't need to understand every hardware detail immediately, but once you start caring about performance, these concepts become difficult to ignore.

Then there's memory, which is probably where GPU programming really starts getting interesting. GPU memory isn't one giant pool where every access costs the same. You have registers, shared memory, caches, and global memory, all with different sizes and access characteristics. This means that an operation can be doing relatively little computation but still be slow because it keeps moving data back and forth between global memory and the compute units.

That changed the way I started looking at GPU optimization. Instead of only asking, "How can I make the GPU calculate this faster?", I started asking, "Why am I moving this data in the first place?" If you can load something once, reuse it several times, and avoid unnecessary trips to global memory, you can sometimes get a much larger performance improvement than simply trying to squeeze more computation out of the GPU.

So where does AMD fit into all of this? If CUDA is NVIDIA's GPU computing ecosystem, ROCm is AMD's corresponding open software stack for GPU computing. ROCm provides the compilers, runtimes, libraries, profilers, debugging tools, and other components needed to develop software for AMD GPUs. One of the main programming technologies within ROCm is HIP, which provides a C++ programming model that is deliberately similar to CUDA.

This is useful because the two ecosystems aren't completely unrelated worlds. CUDA code and HIP code share many of the same fundamental ideas: kernels, threads, blocks, memory management, synchronization, and parallel execution. AMD also provides tools such as HIPIFY that can help translate CUDA source code into HIP. That doesn't mean you can blindly convert any CUDA program and expect identical performance, because hardware-specific optimizations still matter, but it makes the ecosystems much closer than they might initially appear.And this is why I think it's useful to understand CUDA and ROCm as part of a larger GPU software stack, rather than treating them as isolated technologies. At the top you might have your ML application and PyTorch. Underneath that are GPU libraries and kernels. Those eventually interact with CUDA on NVIDIA hardware or ROCm/HIP on AMD hardware, which then interacts with the compiler and ultimately the GPU architecture itself.

For someone coming from ML, this is where things get interesting. Most of the time, you don't need to think about any of this. You write torch.matmul(), torch.softmax(), or torch.layer_norm(), and highly optimized implementations take care of the underlying work. That's exactly what frameworks are supposed to do. But eventually you might encounter an operation that's a bottleneck, a sequence of operations that could be fused, or a workload where the default implementation isn't ideal. That's when understanding what happens underneath the framework becomes extremely useful.

It also explains why technologies like Triton are becoming interesting. Triton gives you a higher-level way of writing custom GPU kernels without forcing you to manually deal with every individual thread in the same way CUDA does. You can think more in terms of blocks or program instances, while the compiler handles much of the mapping to the GPU. For me, this became a much more approachable way of experimenting with GPU kernels while still having to think about things like memory access, tiling, fusion, and data reuse.
And this is probably the biggest recommendation I'd give: don't just learn CUDA or ROCm by reading about them. Build something. I'm planning to approach my own learning through a GitHub repository where each kernel becomes an experiment. Start with a naive implementation, benchmark it, identify the bottleneck, optimize it, benchmark it again, and document what changed. Over time, that becomes much more valuable than simply having "CUDA" listed on a resume because you have actual evidence that you understand what the GPU is doing.
And I think that's ultimately why CUDA and ROCm are worth learning, especially if you're interested in ML systems. You start with something as simple as model.cuda(), but underneath that is an entire stack connecting Python frameworks, libraries, compilers, kernels, memory systems, and GPU hardware. Once you start peeling those layers back, things like FlashAttention, custom kernels, inference optimization, ML compilers, and GPU performance stop looking like completely separate topics.

You don't need to become a GPU expert before starting. Pick one kernel. Understand what it does. Write it. Break it. Benchmark it. Make it faster. Then move to the next one.

I Tried Getting Closer to the GPU With Triton

Sat, 29 Aug 2026 16:49:42 +0000

I've been trying to understand what actually happens when a piece of ML code reaches the GPU. Not the usual model.cuda() and GPU go brrrr kind of understanding, but the actual stuff underneath it — threads, warps, memory, kernel launches, and why some operations are ridiculously fast while others suddenly become slow.

That rabbit hole eventually led me to Triton.

At first, I was wondering why anyone would even bother writing custom GPU kernels when PyTorch already gives us highly optimized operations. But the more I looked into it, the more I realized that GPU performance isn't always about how much computation you're doing. A lot of the time, it's about how you're moving data around.

CUDA gives you an insane amount of control over the GPU, but that control comes with a lot of complexity. You have to think about threads, warps, synchronization, memory access, and a bunch of other low-level details. That's great if you're comfortable with CUDA, but if you're coming from Python and PyTorch, it can be a pretty steep jump.

PyTorch makes this much easier through its higher-level abstractions. You can just write something like torch.softmax() and let the framework handle everything underneath. The problem is that when you're executing operations eagerly, individual operations can result in separate kernel launches and repeated reads and writes to global GPU memory. The code is easy to write, but there can be a lot of unnecessary movement of data happening underneath.

And this is where Triton becomes interesting.

Triton is a Python-based language and compiler for writing GPU kernels. What I found interesting about it is that instead of forcing you to think about individual GPU threads, it lets you think in terms of blocks of data. You basically describe the work that one program instance should perform, and Triton takes care of launching many of those program instances across the GPU.

That changed the way I started thinking about GPU programming.

Before getting into Triton, I had to understand the GPU itself a little better. A GPU is built around massive parallelism. Instead of having a small number of powerful CPU cores, GPUs have huge numbers of lightweight threads that can execute work simultaneously. Those threads are organized into warps and blocks, and the actual hardware that executes these warps is organized into Streaming Multiprocessors.

But the part that really matters for performance is the memory hierarchy.

GPU memory isn't just one big pool where everything costs the same to access. You have registers, shared memory, caches, and global memory. Registers are extremely close to the computation and very fast, while global memory is much larger but significantly more expensive to access. So if your algorithm keeps loading data from global memory, performing a tiny amount of work, writing it back, and then loading the same data again, you're potentially wasting a huge amount of time.

The basic idea I started taking away was pretty simple: move data as little as possible and reuse it as much as possible.

My first Triton experiment was just vector addition. Nothing fancy. Given two arrays, add them together. But even this simple example helped me understand the programming model. Instead of launching a thread for every individual element, I could define a block size and let each Triton program instance handle a chunk of the input.

For example, if I have 1024 elements and a block size of 128, I can think of the workload as eight program instances. The first handles elements 0 to 127, the second handles 128 to 255, and so on. Triton gives each program instance an ID, and I can use that ID to calculate which section of memory it needs to work on.

The next thing that confused me initially was pointers. Triton kernels don't work with tensors in the same high-level way normal PyTorch code does. They work with pointers to GPU memory. You calculate offsets from those pointers, load the values you need, perform the computation, and then store the results back.

This is also where masks become important. If your input size isn't perfectly divisible by your block size, the final program instance might be assigned elements that don't actually exist. You obviously don't want the GPU trying to access memory outside the tensor, so you use a mask to make sure only valid elements are loaded and stored.

After vector addition, I moved on to something more interesting: softmax.

A straightforward softmax implementation involves several operations. You find the maximum value, subtract it for numerical stability, calculate the exponential, sum the results, and finally divide by the sum. Each of these operations can involve reading and writing data.

So conceptually, you can end up with something like: load data, calculate something, write it back, load it again, calculate something else, write it again, and repeat.

The data is constantly travelling between global memory and the compute units.

Instead of doing that, we can fuse the operations into one kernel. The idea is to load the data once, keep it on-chip while performing the different operations, and only write the final result back to global memory.

This is one of the things that made Triton click for me. The code isn't necessarily about doing more computation. It's about avoiding unnecessary memory traffic.

Then came matrix multiplication, which made the whole concept of tiling much more obvious.

When multiplying two matrices, a naive implementation can repeatedly load the same pieces of the input matrices from global memory. But if you're computing multiple nearby output values, you're often reusing the same data. So instead of calculating one output element at a time, you divide the matrices into smaller tiles.

You load a small tile of each input matrix into faster on-chip memory, perform as much computation as possible using those tiles, accumulate the result, and then move on to the next tile.

The fundamental idea is basically: load less often, reuse more.

That sounds ridiculously simple, but it is one of the core ideas behind high-performance matrix multiplication on GPUs.

Once I understood tiling, some of the other Triton concepts started making more sense too. Program IDs can be used to determine which output tile a program instance is responsible for. Strides let you calculate where elements actually live in memory. Masks handle boundaries. The accumulator keeps partial results while working through different chunks of the reduction dimension.

Then there is scheduling and cache locality.

If multiple output tiles reuse the same pieces of input data, it makes sense to execute those tiles close together so that the data has a better chance of remaining in cache. Triton provides mechanisms for grouping program instances to take advantage of this kind of locality.

And then you get to autotuning.

Choosing the perfect block size, number of warps, pipeline stages, and other parameters manually isn't always easy. Different workloads and different GPUs can behave differently. Triton's autotuning system lets you provide multiple configurations and benchmark them to find a better configuration for a particular workload.

This is another thing I found pretty interesting. Instead of assuming there is one perfect configuration, you can basically let the system search through several possibilities and keep the one that performs best.

Eventually I started benchmarking the kernels instead of just looking at the code and assuming the optimized version was faster. And that was probably another important lesson.

GPU optimization isn't about writing code that looks complicated.

It's about understanding what the hardware is actually spending time doing.

Sometimes you're compute-bound. Sometimes you're memory-bound. Sometimes kernel launch overhead matters. Sometimes you're repeatedly moving data that could have stayed on-chip. And sometimes the library implementation you're trying to beat is already so optimized that your custom kernel isn't going to magically win.

That's probably the biggest thing I took away from learning Triton.

Before this, I used to think about GPU optimization mostly as “How do I make the GPU perform more computation?”

Now I find myself asking a slightly different question:

“Why am I moving this data in the first place?”

That shift in perspective is probably more valuable than any individual Triton kernel I wrote.

I'm definitely not claiming to be a GPU programming expert after doing this. If anything, the more I learn, the more I realize how much more there is to understand — occupancy, register pressure, memory coalescing, Tensor Cores, persistent kernels, FlashAttention, compiler behavior, and a lot more.

But at least now those topics don't feel completely alien.

And that's honestly why I wanted to write this.

I didn't want this to be another post that throws a complicated Triton kernel at you and says, "look how fast this is." I wanted to understand why the kernel is written that way and what the GPU is actually doing underneath it.

Because once you start thinking about GPUs in terms of data movement, reuse, parallelism, and locality, GPU programming starts making a lot more sense.

My CUDA/GPU Journey: From "What Even Is a GPU?" to Actually Fascinated

Sun, 23 Aug 2026 17:43:11 +0000

A few months ago, if you'd asked me what a GPU actually does, I would've mumbled something about "graphics" and changed the subject. I'm an undergrad, self-taught in most of what I know about programming, and until recently, GPUs were just... background noise. Something gamers cared about. Something that showed up in specs sheets I didn't understand.

The moment it stopped being background noise

It started the way a lot of rabbit holes start: I kept seeing people online talking about RTX this, RTX that, benchmarks, VRAM, tensor cores — and it all went completely over my head. I'd nod along in threads I didn't understand, half-following conversations about why one card was "better" than another, quietly bothered that I had no idea what any of it meant.

At some point that turned into an actual itch. I don't like not knowing things, especially when it feels like everyone else is fluent in something I can't even parse. So I decided I had to figure out what a GPU actually was — not just as a gaming accessory, but as a piece of hardware people were clearly doing serious computational work with.

That's when I started noticing something else: people weren't just gaming on these things. I kept running into posts and videos of people running genuinely heavy workloads on GPUs — training models, running simulations, crunching numbers at speeds that made no sense to me coming from a CPU-only mental model. That was the real hook. Not the marketing, not the specs — the fact that people were using GPUs as general-purpose computing beasts, and I had zero idea how that was even possible.

Falling into CUDA (and discovering it's not the only player)

Once I started digging, I learned that "using a GPU for computation" isn't some universal, plug-and-play thing — it depends entirely on who made your hardware. NVIDIA has its own ecosystem called CUDA, and AMD has its own competing thing called ROCm. That distinction genuinely fascinated me. Two of the biggest hardware companies in the world, each with their own language and toolchain for talking to their own silicon.

A bit of digging confirmed what I'd suspected: CUDA is NVIDIA's proprietary platform, built specifically for NVIDIA GPUs, and it's been around since 2006 — which is a big part of why it has such a mature ecosystem of libraries, tools, and community support today. ROCm, on the other hand, is AMD's open-source answer, built around HIP (Heterogeneous-Compute Interface for Portability), which is designed to closely resemble CUDA's syntax so that code can be ported between the two with less pain. There's even a translation layer called HIPIFY that converts CUDA code into HIP so it can run on AMD hardware.

What struck me most was that this isn't just a technical footnote — it's basically a walled-garden situation. CUDA's maturity and dominance, especially in AI and deep learning workloads, is a big reason NVIDIA has had such a strong grip on the space. AMD is playing catch-up with an open-source strategy, hoping that openness eventually wins over raw maturity. As someone who'd never thought about hardware ecosystems as competing programming languages before, that was a genuinely new way of seeing the tech industry.

Where I'm at now

I'll be honest — I'm still very early. Right now my "journey" is less about writing optimized kernels and more about building the mental model: understanding what a thread, block, and grid even mean in CUDA terms, why memory transfer between CPU (host) and GPU (device) is such a big deal, and why "just throw it on the GPU" is a lot more nuanced than it sounds from the outside.

But that initial confusion — the feeling of being completely lost in a conversation about RTX cards — has turned into something I actually look forward to learning more about. I went from not knowing what a GPU was for, to being genuinely curious about parallel computing as a discipline. That's a bigger shift than I expected.

Why I'm writing this

I'm writing this partly for accountability, and partly because I know there are other self-taught devs out there who've felt that same "everyone else understands this but me" feeling. If that's you — you don't need a CS degree or a research lab to start. You just need the willingness to feel lost for a while before things click.

If you're also early in your GPU programming journey, or you've been down this road already and have advice, tools, or resources that helped things click for you, I'd genuinely love to hear about it in the comments. This is very much a "learning in public" post, not an expert one.

More updates to come as I actually start writing CUDA kernels instead of just reading about them.

Everpure Data Stream Takes Aim at the AI Pipeline Problem

Wed, 17 Jun 2026 14:15:00 +0000

Enterprises aren't running out of data. They're running out of ways to make it useful.

The gap between raw enterprise data and production-ready AI is where most projects die. Data scientists spend months wrangling ingestion pipelines, cleaning unstructured content, building vector embeddings, and trying to keep everything in sync as source data changes. According to a June 2026 IDC survey of more than 1,300 organizations, 34% of companies report their data scientists spend more than half their time waiting on IT — waiting for datasets, waiting for GPU time, waiting for requests to be fulfilled — rather than actually building models.

At Pure Accelerate 2026, Everpure announced Everpure Data Stream, a capability designed to collapse that pipeline work from months to days. It's the company's most direct answer yet to what Rob Lee, Chief Technology and Growth Officer at Everpure, described during the press conference as the defining bottleneck of enterprise AI: not compute, not software, but data.

The Pipeline Problem

To understand what Data Stream addresses, it helps to understand what AI-ready data actually requires.

Data doesn't arrive AI-ready. It arrives as PDFs, Word documents, PowerPoint files, images, database records, and unstructured content scattered across on-premises systems, SaaS applications, cloud storage, and legacy environments. Before any of that can feed an AI model, it has to be ingested, curated, classified, embedded into vector representations, indexed for retrieval, and connected to a generation layer. That full sequence — Ingest, Curate, Classify, Embed, Index, Retrieve, Generate — is what Data Stream automates end to end.

The alternative is what most enterprises are doing today: armies of engineers building and maintaining custom pipelines, manually handling data preparation, and rebuilding that work every time source systems change or model requirements shift. The IDC research puts the cost of that approach in concrete terms: better workload scheduling (49%) and faster network data delivery (48%) are the top two improvements IT leaders say they need to improve GPU utilization. Both point directly to the data pipeline as the bottleneck starving compute clusters.

Idle GPUs are expensive. As Sabur Mian, CEO and Founder of STN, put it: "Idle GPUs are economically destructive."

What Data Stream Does

Data Stream brings AI capabilities directly to enterprise data where it already lives. It doesn't require moving data into a new system or copying it into a vendor-controlled environment. It runs on infrastructure the customer owns, meaning content never leaves their environment — a critical requirement for enterprises with data sovereignty, security, or compliance constraints.

The pipeline it automates handles three core jobs.

Converting unstructured data for AI use. Most enterprise data isn't structured. PDFs, documents, presentations, and images contain valuable information that AI models can't directly consume. Data Stream reads the formats data actually lives in and converts it into representations AI can work with — without manual intervention.

Maintaining strict governance controls. Stream-level access controls keep information within the corporate network. As AI agents increasingly need to access sensitive data to complete tasks — compensation records, financial data, personally identifiable information — the governance model has to be embedded in the pipeline itself, not bolted on afterward. Data Stream enforces those controls at the point of data flow.

Scaling storage and compute independently. AI workload requirements change. A model that works fine at one scale may need dramatically different infrastructure three months later. Data Stream's architecture lets storage and compute scale independently to match evolving model requirements, avoiding the permanent over-provisioning that fixed infrastructure typically demands.

The NVIDIA Partnership

Data Stream extends the NVIDIA AI Data Platform reference design, giving it a GPU-accelerated pipeline from ingestion through inference. The integration replaces manual data ingestion and manipulation with automated processing that gets from raw enterprise data to real-time AI results significantly faster.

Everpure is also developing next-generation AI solutions with NVIDIA STX — a modular foundation for AI-native storage using NVIDIA Vera and the NVIDIA BlueField-4 STX storage processor. The collaboration is focused on bringing acceleration, security, and intelligent data services closer to enterprise data as organizations scale agentic AI deployments. Timeline details weren't disclosed, but the direction is toward tighter integration between the storage layer and GPU infrastructure at the hardware level.

Jason Hardy, VP of Storage Technology at NVIDIA, described the goal: organizations need infrastructure that bridges secure, governed enterprise data with accelerated computing, allowing them to scale from AI experimentation to full production intelligence.

FlashBlade as the Performance Foundation

Data Stream runs on FlashBlade, Everpure's scale-out file and object storage platform. The performance numbers matter here because data pipeline throughput directly determines how fast AI models can train and how responsive inference workloads can be.

FlashBlade//EXA scales to 800 Gbps of throughput per node while maintaining low latency — the combination required to keep GPU clusters fed at line speed. Organizations start with FlashBlade//S and scale to FlashBlade//EXA as AI factories grow, with Everpure's Evergreen architecture enabling non-disruptive upgrades throughout.

Portworx, Everpure's container platform, handles deployment and orchestration across the pipeline — managing AI workloads from edge to core data center and providing the data portability layer that lets containerized AI applications move across infrastructure without losing access to their data.

Two customer deployments illustrate what that infrastructure combination produces in practice.

STN, a GPU cloud provider, standardized on Everpure FlashBlade after a proof of concept and had an end customer live within a week of completing migration. FlashBlade//EXA allows them to scale thousands of GPUs with consistent throughput, driving up to 20% performance improvement for customers.

Beyond, a private AI factory in Central and Eastern Europe, deployed 3 petabytes of FlashBlade//S alongside NVIDIA DGX SuperPOD infrastructure. The result: 80% faster AI workloads, a 1.2 PUE for energy efficiency against a regional average of 1.5, and on-demand scaling through Evergreen//One.

Where This Fits in the Larger Architecture

Data Stream sits within Everpure's Unified Data Plane — the shared storage foundation that spans cloud, core, and edge across every workload type. It's the execution layer for making data AI-ready, sitting alongside Everpure Data Intelligence, which handles the discovery, classification, and contextualization work upstream.

The relationship between the two is straightforward. Data Intelligence figures out what data you have, where it lives, what it means, and how it connects to everything else in the enterprise. Data Stream takes that understood, classified data and converts it into the pipeline format AI applications and agents actually need — vectorized, indexed, and ready for retrieval.

Together they form what Everpure describes as the AI-ready data path: find and catalog, classify and contextualize, optimize for AI, govern and secure, execute at scale.

For engineering teams, the practical implication is that neither product alone solves the problem. Data Intelligence without Data Stream gives you a semantics layer but no automated pipeline to AI consumption. Data Stream without Data Intelligence gives you pipeline automation but without the context that makes retrieval accurate and agents reliable.

The 94% Problem

The IDC survey found that 94% of respondents consider data quality important or very important to AI project success. The top contributors to poor data quality: redundant data, multiple storage silos, and obsolete data. All three are symptoms of the same underlying condition — data pipelines that are manual, fragmented, and out of sync with source systems.

Data Stream's bet is that automating the pipeline end to end, running it on infrastructure the organization controls, and keeping it synchronized with live enterprise data addresses those quality problems at the source rather than compensating for them downstream.

Sixty percent of organizations in the IDC survey said their storage infrastructure needs significant improvement or a total refresh to support AI workloads. For organizations actively evaluating AI infrastructure, Data Stream is Everpure's argument that the storage layer — not just the model layer — is where that investment needs to happen.

The full pipeline from raw enterprise data to production AI results is available now. Organizations that have been rebuilding custom ingestion work for each new AI project have an alternative worth evaluating.

Everpure (NYSE: P) provided press access to Pure Accelerate 2026. The IDC research cited in this article was commissioned by Everpure.

Has AI started to explore engineering spaces other than coding?

Sun, 24 May 2026 05:06:09 +0000

NVIDIA GPUs helped train LLMs that are now helping design the next generation of NVIDIA GPUs.

Yes, there is something recursive happening here.

Nvidia trained internal LLMs: ChipNeMo and BugNeMo on decades of proprietary chip-design data.

These models can:

• Generate chip-design code

• Answer hardware engineering questions

• Summarize bug reports

So, this goes beyond normal coding assistants.

Bill Dally recently shared how AI is becoming deeply involved in chip design, not just for automation, but for discovering solutions human engineers may never think of.

Nvidia also built an AI system called NVCell that redesigns thousands of chip layout blocks whenever a new semiconductor manufacturing process arrives.

Though we are still far from:

“Prompt → Fully designed GPU.” :) :)

But what do you think, has AI started to explore engineering spaces other than coding?

Beyond Vibe Coding: The Architecture Behind Autonomous AI Agent Systems

Sat, 28 Mar 2026 08:41:12 +0000

Beyond Vibe Coding: The Architecture Behind Autonomous AI Agent Systems

There's a ceiling to prompt-and-paste development that most people hit without realizing it. You prompt an AI, get code back, copy it into your project, fix what's broken, repeat. It works, but YOU are the orchestrator, the memory, the quality control, and the decision-maker. The AI is just generating text between your manual steps.

Agentic systems remove you from that loop. Not entirely. You're still the architect and the judge. But the AI gains autonomy to read files, run tools, chain decisions, invoke specialists, and verify its own output.

The architecture behind this is universal. Doesn't matter which LLM, which framework, which language. The patterns are the same, and they're pure Python. No proprietary magic.

The Three Layers

Every agentic system I've built runs on three layers.

Encoded Workflows

An encoded workflow captures HOW to approach a specific type of problem. Not a prompt template. A structured process with decision points, context loading, and branching logic.

I have one that fires when I start a new project phase. Forces a sequence: classify the problem type, list every failure mode, identify the bottleneck, estimate the cost. Another runs a coaching-style interview: loads research, teaches context, asks targeted questions, synthesizes drafts. 58 total. Each one encodes a process I figured out once and never want to re-derive.

Encode your best process, not your best prompt. A good prompt generates good output once. A good workflow generates good output every time, regardless of who's running it or which model is underneath.

Specialized Agents

Instead of one general-purpose model trying to be everything, you decompose expertise into focused units. A specialized agent gets domain knowledge, specific tools, and a defined role.

I have agents that know GPU memory budgets. Agents that audit cryptographic choices. Agents that understand message queue patterns. 87 across three model tiers. When I need a decision about inference configuration, I delegate to the agent that already has that context. I don't re-derive domain knowledge every session.

One generalist with a massive system prompt gets confused. Focused agents with clear boundaries get things done. You define them with structured prompts and tool access. That's it.

Automated Guardrails

Guardrails are event-driven checks that fire automatically. Before actions execute, after outputs are produced, when system state changes. They don't require your attention.

Mine validate that actions aren't destructive before execution. Run syntax and health checks after every build. When a long session loses context (any LLM will eventually compress or forget), a guardrail re-injects critical state so the system doesn't drift. 5,700+ production executions. They work at 2am when I don't.

Every mistake you catch yourself making twice should become an automated check. Pre-action validator. Post-build verifier. Context recovery trigger.

How They Compound

Workflow invokes agent. Agent produces output. Guardrail validates before it touches the file system. Fails? Routes back. Passes? Post-action guardrails verify.

Take any layer out and the others degrade. Guardrails can't protect directionless output. Agents without workflows just generate. Workflows without automated checks are one bad session away from shipping garbage.

What This Looks Like at Scale

Hekaton is ten LLMs on NVIDIA's GH200. 96GB HBM3e. $2 an hour on Lambda. All local for the core debate.

Formations are YAML-defined. Swap a config and the topology changes: 10 models at 87GB, 6 local with API support at 78GB, a 3-model strike team at 20GB. Same orchestrator, different firepower.

Core debate: Architect (Phi-4 14B) decomposes the objective. Sapper (Qwen2.5-Coder-14B) builds. Auditor (DeepSeek-R1-7B) tears it apart. PASS or FAIL. Up to 5 rounds. A second Sapper races the first on retries. ZeroMQ IPC. Sub-5ms latency. No HTTP.

Real cycle: Architect assigns a binary analysis task. Sapper does a first pass. Auditor catches a hallucinated function and a control flow gap. FAIL. Sapper fixes both, resubmits. PASS. That hallucination would have sailed through any single-agent pipeline.

That's the agents. The workflows run underneath. Before any mission starts, a Planning Swarm puts 10 models through structured discovery and debate just to figure out the approach. After each mission, an OPRO loop evolves the prompts based on what worked and what didn't. +20% PASS rate from prompt evolution alone. The system literally rewrites its own instructions.

The guardrails are quieter but they're load-bearing. A 6-tier memory system filters out failed approaches so the system doesn't repeat its own mistakes. +14% improvement just from remembering what NOT to do. Confidence-based routing keeps 98.9% of inference local and only hits Gemini when the models aren't sure. 15 gates total where the project stops itself if the evidence says stop. 9 passed so far.

And the part that still trips me out: Hekaton was DESIGNED by agents. Planning agents researched the architecture. Security agent audited the crypto. HPC agent validated the memory budget. Agents building an agent system. Same three layers all the way down.

Start With One of Each

You don't need 87 agents. You don't need a GH200.

One workflow that encodes a process you repeat. How you start a feature. How you review a PR. Write it as a structured sequence with decision points. Automate it.

One agent that holds domain knowledge you're tired of re-explaining. Your deployment process. Your database patterns. Your testing philosophy. Scope it tight, give it the right tools.

One guardrail that catches a mistake you keep making. A pre-commit check. A build validator. A context recovery mechanism.

They stack. Intuition -> experimentation -> result -> reflection -> bake it in -> repeat. Each cycle creates the infrastructure for the next one.

The ceiling in prompt-and-paste is visible. The ceiling in agentic architecture? I don't know yet. I keep looking back weeks or months and realizing I've 10x'd what I can do. I started with a CLI, a fresh Linux server, and a dream. (Yes it was in root. They say don't do that. Sure I locked myself out of SSH and had to factory reset once or twice. Just growing pains.) You don't have to be that reckless. Just experiment.

herakles-dev builds agent systems from a terminal in Chicago. No CS degree, no team — 87 specialized agents and a stubborn conviction that AI gets better when it argues with itself. herakles.dev | github.com/herakles-dev

HPE Brings NVIDIA Vera Rubin and Vera CPU to AI Factories and Supercomputers at GTC 2026

Mon, 16 Mar 2026 21:41:30 +0000

HPE used NVIDIA GTC 2026 to announce the most significant refresh of its AI infrastructure portfolio since the company started building exascale systems. The updates span two product lines — HPE's supercomputing platform and its AI Factory portfolio — and introduce both NVIDIA's new Vera Rubin architecture and the first Vera CPU compute blade from any vendor.

Here's what shipped and why it matters for developers and engineers building on large-scale AI infrastructure.

Supercomputing: First Vera CPU Blade and Quantum-X800 Networking

HPE is adding two new capabilities to the HPE Cray Supercomputing GX5000, its second-generation exascale platform designed to unify AI and HPC workloads.

The first NVIDIA Vera CPU compute blade. The HPE Cray Supercomputing GX240 is a liquid-cooled blade featuring up to 16 NVIDIA Vera CPUs per blade. The Vera CPU is NVIDIA's successor to the Grace CPU, built on custom ARM-based "Olympus" cores with 88 cores per chip. A single rack supports up to 40 blades — that's 640 Vera CPUs and 56,320 ARM cores per rack. HPE claims industry-leading density on the Vera platform.

For developers working on CPU-intensive AI workloads — data preprocessing, feature engineering, inference serving, simulation — this is the ARM density story reaching a new scale. The Vera CPU is purpose-built for AI data center workloads, not repurposed from another market.

NVIDIA Quantum-X800 InfiniBand networking. The GX5000 now supports Quantum-X800 InfiniBand switches with 144 ports of 800 Gb/s connectivity per port. For teams running distributed training or large-scale inference across thousands of GPUs, the networking fabric determines how efficiently you can scale. Quantum-X800 also includes power-efficiency features such as low-power link state and power profiling — relevant when your electricity bill is a meaningful line item.

HPE has built the three most powerful exascale supercomputers in the world (validated by the November 2025 TOP500). This update keeps the Cray platform up to date with NVIDIA's latest silicon.

AI Factory: Vera Rubin NVL72 and Double-Density GPU Servers

The AI Factory portfolio targets service providers, sovereign entities, and large enterprises running AI at scale. Two new systems stand out.

NVIDIA Vera Rubin NVL72 by HPE. This is the flagship: a rack-scale AI system engineered for frontier models exceeding 1 trillion parameters. Each rack includes 36 Vera CPUs, 72 Rubin GPUs, NVLink interconnects, ConnectX-9 SuperNIC, and BlueField-4 networking. HPE provides liquid-cooling integration, services, and data-center design expertise.

The Vera Rubin platform represents a generational leap. NVIDIA claims 3.3x to 5x inference performance improvement over Blackwell in FP4 workloads, a 10x reduction in inference token costs, and 4x fewer GPUs needed to train equivalent Mixture-of-Experts models. For developers building large-scale AI applications, the math changes — workloads that required massive GPU clusters on Blackwell become tractable on smaller Vera Rubin deployments.

HPE Compute XD700. This is an OCP-inspired AI server built on NVIDIA HGX Rubin NVL8. Each rack supports up to 128 Rubin GPUs — double the GPU density of the previous generation (the HPE ProLiant XD685). The design targets reduced space, power, and cooling costs while increasing training and inference throughput.

For teams that don't need a full NVL72 rack but want high-density GPU access for model training and inference, the XD700 is the more practical entry point.

Enterprise: Security, Multi-Tenancy, and Agentic AI

The enterprise announcements focus on HPE Private Cloud AI, the company's turnkey AI factory for organizations that need on-premises AI infrastructure.

Scaling to 128 GPUs. New network expansion racks let HPE Private Cloud AI deployments scale up to 128 GPUs while maintaining a consistent operational experience. Previously, Private Cloud AI topped out at lower GPU counts, limiting the size of models organizations could run on-prem.

Air-gapped configurations. For sovereign deployments and regulated industries, HPE Private Cloud AI is now available in an air-gapped configuration — fully isolated from external networks. If your data can't leave the building, this is the hardware option that makes AI possible under those constraints.

Confidential AI. HPE ProLiant DL380a Gen12 servers and Private Cloud AI systems are being certified for Fortanix Confidential AI, built on NVIDIA Blackwell Confidential Computing GPUs. Sensitive data gets processed without exposure.

CrowdStrike integration. CrowdStrike delivers agentic security for HPE Private Cloud AI — AI-powered threat detection protecting AI infrastructure, models, and the AI agents running across enterprise environments. As organizations deploy more autonomous AI agents, securing the infrastructure on which those agents run becomes a distinct problem.

NVIDIA AI-Q and Omniverse blueprints. Private Cloud AI ships with updated NVIDIA blueprints, including AI-Q for building customizable AI agents and Omniverse for digital twins. These are pre-configured stacks — hardware and software together — that reduce time to first deployment.

Agentic AI services hub. HPE Services is launching an agents hub for structured enterprise adoption of agentic AI, developing and validating agents powered by NVIDIA Nemotron models. This creates reusable patterns for organizations that want to deploy AI agents but need a structured approach rather than experimentation.

What Developers Should Know

Three things matter most from these announcements.

The Vera CPU is real and shipping in blades. HPE is the first vendor to put it in a compute blade product. If you're building for ARM-based AI infrastructure, the 88-core Olympus architecture, with 640 CPUs per rack, sets a new density benchmark.

Vera Rubin changes the economics. A 10x reduction in inference token costs and 4x fewer GPUs for MoE training means workloads that were cost-prohibitive on Blackwell become viable. For AI startups and research labs, the cost-per-token math is the number that determines what you can build.

Enterprise AI security is becoming a product category. Air-gapped configurations, confidential computing certification, CrowdStrike agentic security integration — these aren't afterthoughts. They're shipping alongside the compute hardware as part of the same product stack. As AI agents proliferate in enterprise environments, the infrastructure securing those agents matters as much as the infrastructure running them.

HPE's full GTC 2026 portfolio is available starting today. Initial Vera Rubin samples are expected to ship to tier-one customers by late 2026, with full production in early 2027.