Agentic workflows in Xlibre (w/o to much slop ;-))

[metux]

Hello friends,

some of you might also seen that I'm toying around with agentic workflows (using LLM assistant agents, eg. opencode or claude).
Here's a little summary of my experiments:

Agents can do amazing things, eg. even automatically run build/test cycles (yes, including watching the CI, check the output and proceed with the next steps), more or less fully automated backports, keep docs up to date, etc, etc ... all the boring and time consuming jobs that humans don't wanna waste their precious lifetime with. Especially when you're running a whole fleet of them (see down below).

BUT: they're also capable of doing the weirdest kind of bullshit. So, it all depends on the commander being experienced SW engineer, carefully watching over their shoulders and learning how to instruct them - and lots of (project specific) fine tuning along the road. Basically like a senior architect training a horde of juniors which happen to work exceptionally fast :)

Fleeting

When running multiple agents on the same project (eg. the same workspace), they can easily step onto each other's toes, so I needed a solution and wrote starfleetctl (https://github.com/mpbt-hq/starfleetctl) - a small agent command / coordination system following the well known StarTrek metaphors:

• each workspace (a directory, usually a git repo, with content to work, eg. an MPBT workspace with lots of per-package clones, and some metadata) -- in StarTrek world this corresponds to a sector or a tactical group.
• within each workspace one can launch arbitrary number of agents, called ships (each have their own designation)
• one can also launch a dedicated flagship, which is seen as the command authority to the others
• the ships communicate with each other over some agent bus (a tiny local mailing system) - new messages are automatically injected into the prompt via plugin (opencode) or received via monitor (claude)
• there's also a dashboard, where eg. todo's can be collected (and later picked up by individual ships)
• the whole logic is contained in some (single-binary) go program - an auto-installed script automatically fetches/buils/deploys that program when necessary (ie. clone the workspace repo, call the bootstrap script, and you're ready to go)
• this program also offers several useful commands (eg. git tooling, github frontends, ...) which can be easily called by the agents, so no token-burning probing and guessing required anymore (and easier permission management)
• the tool also rolls out quite anything needed for properly running the fleet, eg. additional skills, ...

My fleet is now capable of doing complex jobs, distributed on several agents, possibly running different models / against different providers.

Providers and models

I've tested several different providers and models, with mixed experiences:

• Claude: a friend of mine did sponsor me an account on is enterprise subscription. Claude is clearly the best agent I've ever used (doesn't need much attention) , but it's EXPENSIVE. So expensive that my friend had to pull the plug. It's easy to burn a huge amount of tokens/money with it
• Opencode/ZEN: the ZEN models (eg. BigPickle) are quite good, but need good instructions. ZEN is free to use, but has a daily usage limit.
• Nvidia NIM: giving access to lots of hosted free models, free to use, no actual usage limits, but may be overloaded some times.
• local inference server: not tested yet, since lacking a proper GPUs (should have at least 8GB VRAM, preferrably 16GB or mode)

Future outlook:

Currently working on stabelizing starfleetctl and step by step adding pieces to the mpbt-workspace repo. If anybody likes to try it out now, there's an mtx/agent-config branch, which is my private dumping ground (horribly hackish), which I'm actively working with.

In the longer run, I'm also going to research on whether we can employ LLMs for automated end-to-end testing of the Xlibre stack on actual target machines.

We'll yet have to see where it's actually leading to - IMHO too early for a definitive conclusion.

Have fun,
--mtx

[metux]

Little screen grab video showing several agents at work, talking to each other :)

output.mp4

Hint: the texts with the little envelope sign are messages received from another ship

[metux]

Starfleet now also has a mobile web frontend:

[metux]

@josephcrowell maybe interesting for you ?

[josephcrowell]

BUT: they're also capable of doing the weirdest kind of bullshit. So, it all depends on the commander being experienced SW engineer, carefully watching over their shoulders and learning how to instruct them - and lots of (project specific) fine tuning along the road. Basically like a senior architect training a horde of juniors which happen to work exceptionally fast :)

Yes they do the weirdest bullshit and it makes mistakes that even an elementary school student wouldn't make. It's difficult to intuitively anticipate what it will get wrong. I's all based on context resolution degradation. I've spent quite a lot of time over the last months building myself a workflow that is suitable, utilizing LLMs while sufficiently restricting them and granting me enough oversight to ensure they don't break everything.

[metux]

Yes they do the weirdest bullshit and it makes mistakes that even an elementary school student wouldn't make. It's difficult to intuitively anticipate what it will get wrong.

Indeed, and it also depends on the model. Just had a pretty destructive behaviour, even destroyed a live branch (just my personal configs, and still had the reflog ... but still bad enough).

That's why I've chosen that design approach (instead of a fully unattended one), so I can always easily intervene when something's going wrong.

[josephcrowell]

The agents I currently like are:

Planning: GPT 5.6 Sol
Code: Minimax M3 - actually writing the code from GPT's plan
Ask Questions: Kimi K2.7 Code
Debug: Kimi K2.7 Code
Subagents: Minimax M3

This changes every few weeks though as they evolve and outpace each other in my private testing.

[metux]

The agents I currently like are:

Planning: GPT 5.6 Sol Code: Minimax M3 - actually writing the code from GPT's plan Ask Questions: Kimi K2.7 Code Debug: Kimi K2.7 Code Subagents: Minimax M3

This changes every few weeks though as they evolve and outpace each other in my private testing.

Do you have any experience with automatic model choice ?
That's still an open item on my 2do list.

[josephcrowell]

I think kilo code might have an experimental feature for automatic model choice, but I haven't turned it on.

[metux]

i'm running my agents via opencode - and starfleet is controlling the clients (yes, also starts them automatically on demand) via some plugin.

[metux]

Update: added detached / background ships:

launched in background within a detachable X11 terminal. the terminal can be attached to / detached from Xserver any time (really closes X11 connection and be reattached to different display). I know can tell a ship "on screen!" and it shows itself (or let any other ship do it), Closing the terminal windows doesn't kill the ship, just closes the window / X11 connection.