Running a 56M-Parameter LLM on Just Three ESP32 Boards

Mon, 10 Aug 2026 08:44:29 +0000

When I was a kid, I loved the movie WALL·E. I often wondered if something like that would be possible in the future.

As I grew up, I learned how robots work and how much it costs to build one. I realized that building intelligent machines isn't just about software—it's also about overcoming hardware limitations.

Projects like this remind me that we're slowly bringing intelligence to even the smallest devices.

When people talk about running AI locally, they usually mean a laptop with a decent GPU, a Raspberry Pi, or an NVIDIA Jetson.

Microcontrollers rarely enter this type of conversation.

After all, an ESP32-S3 only has a few megabytes of memory. Running a language model on it sounds impossible.

I saw a post about running an LLM on an ESP32 by slvDev, and I was fascinated by it.

Then I came across another project by Wladimir Avila, which took the idea even further.

Instead of asking:

"How can we fit a bigger model on one ESP32?"

It asks:

"What if multiple ESP32 boards worked together like a tiny AI cluster?"

That's exactly what esp32s3-distributed-ai does.

The Problem

Modern language models are huge.

Even "small" language models often require tens or hundreds of megabytes of memory, which is far beyond what an ESP32-S3 can provide.

Buying more powerful hardware is the obvious solution.

But this project explores another idea:

Split the model across multiple microcontrollers.

Rather than forcing one board to hold the entire model, each board becomes responsible for a different part of the inference pipeline. The boards then communicate wirelessly to generate text together.

The Hardware

The setup is surprisingly simple.

3 × ESP32-S3 N16R8 boards
ESP-NOW for communication
One board hosts a Wi-Fi access point
A browser-based interface to interact with the model

No cloud.

No router.

No internet connection after flashing the firmware.

Everything runs locally.

How the Model Is Split

Instead of storing the entire transformer on one board, the project partitions it.

             Prompt

                │

         Board C (Web UI)

                │

         Board A (Embeddings)

                │

      Board B (Transformer)

                │

      Board A (Output Head)

                │

         Board C (Browser)

Each board has a specific responsibility.

Board A

Embeddings
Output head

Board B

Transformer layers
KV cache stored in PSRAM

Board C

Remaining embedding table
Wi-Fi access point
Web interface

The boards exchange intermediate activations over ESP-NOW until the next token is generated.

Why ESP-NOW?

One of the most interesting design decisions is the communication layer.

Instead of using MQTT, TCP sockets, or a router, the project relies on ESP-NOW, Espressif's lightweight peer-to-peer wireless protocol.

Advantages include:

No external infrastructure
Low communication overhead
Direct board-to-board messaging
Fully offline operation

According to the author, the boards communicate through a custom protocol built on top of ESP-NOW, with Board C simultaneously hosting the browser interface.

A 56 Million Parameter Model

The project runs a language model with approximately 56 million parameters.

To make this possible, it uses:

4-bit and 8-bit quantization
Split Per-Layer Embeddings (Split-PLE)
Flash memory for large embedding tables
PSRAM for the KV cache

These optimizations allow a model that would never fit on a single board to execute across three inexpensive microcontrollers.

Inspired by Previous Work

This project builds upon another impressive experiment by slvDev, who demonstrated that a 28.9 million-parameter language model could run on a single ESP32-S3.

That work introduced the use of Per-Layer Embeddings (PLE), inspired by Google's Gemma architecture, to dramatically reduce SRAM requirements by storing most embedding parameters in flash memory.

The distributed version extends that idea even further by partitioning the model across multiple devices.

Current Limitations

The project is still experimental.

Some of the current limitations mentioned by the author include:

Brute-force tokenizer implementation
Quality loss from aggressive 4-bit quantization
Generating relatively short responses (around 30 words)
Focusing on demonstrating distributed inference rather than competing with larger LLMs

Why This Matters

The most exciting part isn't that three ESP32 boards can generate text.

It's the engineering mindset behind it.

Instead of scaling up, this project scales out.

Rather than purchasing more powerful hardware, it distributes computation across multiple low-cost devices.

That idea could inspire future work in:

Offline robotics
Smart factories
Distributed IoT intelligence
Edge AI
TinyML research
Sensor networks

As embedded AI becomes more common, approaches like this could make sophisticated models accessible on hardware that costs only a few dollars.

Final Thoughts

This isn't about replacing ChatGPT.

It's about pushing the boundaries of what's possible on resource-constrained hardware.

Seeing three inexpensive ESP32 boards collaborate to run a 56M-parameter language model is a reminder that innovation often comes from clever system design—not just bigger GPUs.

Projects like this show that the future of AI won't exist only in massive data centers. It may also live on tiny devices working together at the edge.

Project Repository

https://github.com/wladimiravila/esp32s3-distributed-ai

Thanks for reading!

I'm Ganesh, and I'm building MakeSense, an AI tool that turns public GitHub pull requests into concise summaries, prioritized insights, and interactive quizzes. It's free, unlimited, and source-available. If you review open-source code, I'd love for you to give it a try and share your feedback.

MakeSense: https://makesensegithub.com/

Coder Legion - Recent posts tagged boards