Coder Legion - Recent posts tagged developer-experience

What even is a developer experience audit?

Wed, 28 Oct 2026 15:20:55 +0000

Since getting into the Developer Relations space, I take onboarding to new companies as an opportunity to evaluate what the overall product experience is like. This comes with all sorts of fun things like identifying broken links to fully broken internal processes. And everything in between

I am a strong believer that everyone is doing their best. I am also rational in understanding that priorities and focus change regularly. I want to make sure I approach this with curiosity and enthusiasm to propose a solution and fix it myself, if that's possible. After all, I probably have less meetings and less responsibilities than everyone else, simply by being new.

If I can't fix it myself, I propose a recommendation. A former manager of mine would be thrilled to see me bringing solutions, not problems. (To be fair to both of us, the platform I supported at the time didn't have a lot of solutions but certainly had a lot of problems.)

So with a use case in hand, I set off to see what the developer experience was like for Apryse.

But wait, where does Developer Experience fit in Developer Relations?

Before we get into the audit details, let's address the DevEx vs. the DevRel elephant in the room.

For me, Developer Relations is the broader umbrella that encompasses awareness (advocacy), enablement (developer experience), and engagement (community).

Some companies will bucket this a little different, including the one I'm at now, where Developer Experience is the umbrella but the core focus areas are still largely covered. At Arpyse, we also include AEO under Developer Experience which I think fits nicely in this era of AI-enabled or AI-assisted builders.

Now let's talk about audit process in terms of awareness, enablement, and engagement.

The process

I want to try to replicate, to the best of my ability, what the public facing developer experience is like. How do developers find the product, how can they try it out, and then how can they (or someone else in the company) start the purchasing and procurement process.

Doing this during onboarding should be the easiest time to ever do this because you probably have little to no deep product knowledge. you can literally approach it like an external developer because you are one. Unless your onboarding includes product training, which is cheating if it's only available to internals.

Discovery (awareness)

Can I find the product when I'm looking for it? Is it positioned well against competitors?

In today's world, discovery includes both a traditional search and AI tools, including LLMs like Claude, but also includes AI search summaries, like the ones you see at the top of an otherwise traditional Google search.

I approach this like any research project I did in school - lots of keywords, questions, and use case based researching. Then I see what kind of results I get.

If you think only your marketing website or your docs are your entry points for a developer audience, you have some work to do.

Consider other entry points, like if your product is available on a cloud or platform marketplace, as developers participating in that ecosystem may start there first - think a Salesforce or Appian developer starting in their marketplace rather than accidentally finding something not in the marketplace already and having to build it full custom. Is this information up-to-date? Well maintained and linked back to more granular information across your website and docs? Does it show up on the marketplace search with vague terms or only the company or product name?

Beyond marketplaces think of entry points like GitHub, package managers, or other places libraries and dependencies already live. Unless I'm starting from scratch or doing a re-platforming effort, the closer you are to where I'm already working, the easier it will be for me to pick it up. And here again, is this information up-to-date, well maintained, and linked back to more granular information? It's not enough to just be listed, it needs to look safe and maintained, often with very limited context.

Outside the scope of this audit, you can also evaluate broad, general awareness activities typically managed by marketing like events, webinars, blogs, etc. If I go to a developer conference, is Apryse there and how quickly can I understand what the product does in a conference setting?

All of this is part of the discovery process.

One thing I should mention before moving on - if your product doesn't come up when and where you expect it, that's a notable gap. Congrats, you have action items following your audit. To me, awareness gaps are the most critical to address immediately because if people (and AI) can't find you, they can't use you. Fix that!

Evaluation & trial (enablement)

I found the product I think I want to use, but I need to check on the features and eventually build a proof of concept (PoC).

You've been found, either by humans or AI, and they are moving forward with understanding and using your product, often in some kind of trial capacity.

Before they start building, they need to know if your features and capabilities will solve their problems. Maybe they've seen some marketing pages or blog describing this, but this is where they will start digging for more details.

This is probably where my process has changed the most in recent years. Using AI tools I can go from discovery to building really fast, sometimes even in the same chat.

And I did, with several LLMs on basic or free tiers with as little access to memories or prior knowledge as I could possibly give them. I didn't want them to taint my results by knowing I worked for Apryse because I wanted to experience what an external developer would experience, and a bias to Apryse because I was an employee was going to harm that.

During this evaluation phase is also where I check competitors and even PoC them too. You want to be choosy about this step because it's either time consuming or compute intensive (tokens), but it's a very realistic step for developers evaluating multiple options.

I'm evaluating the features, the ease of implementation, the clarity of documentation or other assets available for reference, and ultimately how much debugging did I have to do to stand up a working PoC. If this goes smoothly, I will literally go tell all my tech friends about it. (And this should be your goal, to have people outside the company talking about how great it is.)

Trial keys, licenses, and more

If your product has a trial process, it should be as frictionless as possible. I get that you want leads. You know what I, as an external developer, want? To figure out how many PoCs I have to do before I recommend a solution to my manager on a tight timeline. And know that when I build a PoC, I've timed it so it still works when I have to do a demo.

Your trial process should be super clear. How many days, weeks, credits, runs is the trial valid for? What features are included or not? Will I see uninformative error messages when using a trial key vs. a production key?

It does not matter if the friction experienced in this phase is 100% because of how trial keys or licenses are implemented. You need to fix that. External developers shouldn't have to absorb that.

A curated developer journey

This is the hill I'm willing to die on, sometimes daily. You cannot risk developers or AI defining their own developer journey. You need to own and nurture this. This stage of the audit is where this becomes alarmingly clear.

Taking a use case, I usually pick a programming language or framework along with it. In this case, I went Javascript (React + Node, with some Vite and Angular sprinkled in along the way. Remember, I ended up doing like 5 or 6 PoCs across Apryse and competitors.)

With both my own eyeballs and mouse pointer AND, separately, my crew of LLMs, I wanted to see if there was a clear, defined journey through Apryse for a Javascript developer. This was a well supported journey, but not necessarily a curated one depending on where I went or what questions I asked. For example, landing on the marketing website, we don't really link to a developer portal. In fact, we have about 4 pages I would consider someone external to Apryse calling a "developer portal," so we'll need to fix that.

A north star goal here would be for my various programming languages or frameworks, a developer in that space can move from discovery to building with clearly identified resources that we nurture, specific to their needs and use cases. For example, a Java developer would probably have different needs and use cases simply by living in the server space (Apryse Server SDK) compared to a developer using WebViewer (Web SDK). This could end up being the same developer, but they are solving different aspects of the problem and need an intro and understanding of our different SDKs.

Up and running, and beyond (engagement)

Is there anyone out there to support me? How can I get deeper knowledge?

Once I've successfully built a PoC, or like 5? 6...? in my case, I need to help whoever is actually going to do the purchasing make a final decision to make the purchase happen.

And in parallel, I need to make sure I know how and where to get support, either paid or community. Ideally, this is open and searchable because I'm both lazy and antisocial.

This is where I look for community support platforms, forums like Discourse, messaging platforms like Slack or Discord, GitHub communities, etc. I also evaluate their community health - is anyone actually posting? Are the only responses from employees or contributors? Does it look like a place I would feel safe asking a question?

If there are hundreds or thousands of people silently hanging out, this is a red flag. If only employees or contributors are responding, this is probably a yellow flag or maybe even green if you were looking for free (community) support. As a community grows, the community support should scale with it. If it doesn't, this might be something to look in to.

I also look at GitHub and forums outside of the direct control of the project or company. What's the sentiment here? Do issues remain open with no responses? Are PRs stale and neglected? Is there a whole subreddit bashing the project?

At this point, I also start looking for any signs of a champions program or certified developers in this space. This might come up on LinkedIn or a community support forum. These are usually really healthy indicators, that someone external to the company cares enough to go the extra mile with the technology. They may speak at conferences and share their experience too, which loops back to the general awareness activities.

With this information I can usually wrap up my recommendation and feel confident we are making a good choice.

The caveat

It's super important to stay as public and as focused on exactly what a given resource says. If you have access to internal experts or secret resources, this isn't a good replication of what an external user would have access to. If you overcome any and all friction with extensive research, that's not realistic with someone on a development team with a tight sprint timeline.

The AI caveat

I took this particular audit as an opportunity to use every LLM and AI chatbot I could reasonably get my hands on - Claude, Microsoft Copilot, GitHub Copilot, Gemini, ChatGPT + Codex. Occasionally I would use them a bit adversarially quoting one to the other. I also wanted to see what kind of project structure they delivered, what kind of experience I had pair programming or debugging with them, and what kind of AI-enabled or assisted experience they delivered.

Several times I had to reign them in because they could see my email in my directory file tree or path and infer Apryse as my employer. I get that they are being "smart" but this is not what I wanted. Microsoft Copilot had broad internal access and I actually ended up abandoning it because it would go find internal resources after I would explicitly tell it to only look on the public internet.

Speaking of public internet, GitHub Copilot out-of-the-box didn't have access to the public internet and neglected to share that with me. I wrote a whole thing about that, but TL;DR it was GUESSING old npm versions before I was finally like CITE YOUR SOURCES and it couldn't... because it didn't have access to the internet.

Wrapping up

So this was more info into my developer experience audit process. Have you done something similar? What does your process look like?

Again, and I cannot stress this enough, this isn't a finger pointing exercise. Some things may be accidentally neglected or forgotten, just fix them or recommend a fix and move on. Some things may be missing because it just wasn't the right time to do them, and maybe it's still not the right time.

The point is you now have a starting point for how and where to improve the developer experience, a great set of talking points to meet folks across the company and start building those cross-functional relationships, and you'll be able to directly impact the external developers trying to build a solution with your product. Lots of points. I think you get the point. 😉

_Photo by Agence Olloweb on Unsplash_

Stop Wasting Days on Dev Setup: How We Automated Our Microservices Environment for Every Developer

Thu, 24 Sep 2026 01:52:08 +0000

Stop Wasting Days on Dev Setup: How We Automated Our Microservices Environment for Every Developer

How we reduced new developer onboarding from 3 days to 15 minutes and eliminated configuration drift across 12 microservices.

The Problem That Made Us Snap

Picture this: A new senior developer joins your team. They're excited, ready to ship code on day one. Then reality hits.

Day 1: They spend 6 hours installing Node, Docker, configuring ESLint to match the team's style, figuring out which environment variables go where, and asking Slack questions like "which version of NestJS are we on again?"

Day 2: They finally get user-service running, but auth-service throws a CORS error because their local config is slightly different from everyone else's. They spend the afternoon comparing .env files with a teammate.

Day 3: They submit their first PR. The CI pipeline fails because their local Prettier config formats code differently from the shared config that was updated last month — but nobody told them.

Sound familiar?

We had 12 microservices, a team that was growing fast, and a configuration mess that was eating our productivity alive. Every service had slightly different ESLint rules, Docker setups, folder structures, and VS Code settings. Senior developers were spending hours every week answering setup questions instead of building features.

So we built a system that fixed all of it. Here's exactly how.

The Solution: One Repo to Rule Them All

We created a central configuration repository called dev-environment-config that acts as the single source of truth for every development environment across our entire microservices ecosystem.

The idea is simple:

One repository controls the setup, structure, linting, Docker config, VS Code settings, and shared code for every service. When a senior developer updates it, every developer's machine automatically syncs the next morning.

The Architecture at a Glance

dev-environment-config (Central Repo)
├── templates/          → Project scaffolding (NestJS, React)
├── shared/             → ESLint, Prettier, Git configs
├── vscode/             → Extensions & settings
├── docker/             → Base Docker configs
├── scripts/            → Automation (setup, sync, update)
├── common-lib/         → Shared NPM package
└── manifest.json       → Version tracking

         │  syncs to
         ▼
┌──────────┐ ┌──────────┐ ┌──────────┐
│ user-svc │ │ auth-svc │ │ pay-svc  │  ... 12 services
│ .eslint  │ │ .eslint  │ │ .eslint  │
│ .vscode  │ │ .vscode  │ │ .vscode  │
│ Docker   │ │ Docker   │ │ Docker   │
└──────────┘ └──────────┘ └──────────┘

Implementation: Step by Step

Step 1: The Central Config Repository

We created a new repository with this structure:

dev-environment-config/
├── templates/
│   ├── nestjs-service/    # Complete NestJS starter
│   └── reactjs-service/   # Complete React starter
├── shared/
│   ├── .eslintrc.js
│   ├── .prettierrc
│   ├── .editorconfig
│   ├── .gitignore
│   └── commitlint.config.js
├── vscode/
│   ├── extensions.json
│   ├── settings.json
│   └── launch.json
├── scripts/
│   ├── setup-new-developer.sh
│   ├── sync-config.sh
│   ├── auto-update.sh
│   └── create-new-service.sh
├── common-lib/            # Shared NPM package
└── manifest.json          # Version tracker

The manifest.json is the heartbeat of the system:

{
  "version": "1.2.0",
  "lastUpdated": "2024-01-15T10:00:00Z",
  "updatedBy": "senior-dev",
  "configs": {
    "eslint": "3.2.0",
    "prettier": "2.1.0",
    "docker": "1.5.0"
  },
  "services": [
    "user-service",
    "auth-service",
    "payment-service"
  ]
}

Every time a config changes, we bump the version. This is how the auto-update system knows something changed.

Step 2: The One-Command Onboarding Script

This was the game-changer. We wrote a bash script that a new developer runs once, and their entire environment is ready.

The new developer's experience:

Install VS Code ✅
Run one command:

curl -sSL https://raw.githubusercontent.com/company/\
dev-environment-config/main/scripts/\
setup-new-developer.sh | bash

Wait ~10 minutes. Done.

What the script does under the hood:

#!/bin/bash
set -e

WORKSPACE_DIR="$HOME/company-workspace"
CONFIG_REPO="https://github.com/company/dev-environment-config.git"

echo "🚀 Setting up your development environment..."

# 1. Check & install prerequisites
check_and_install "git" "Git"
check_and_install "node" "Node.js" "nvm install 20"
check_and_install "docker" "Docker"

# 2. Create workspace & clone everything
mkdir -p "$WORKSPACE_DIR" && cd "$WORKSPACE_DIR"
git clone "$CONFIG_REPO"

# 3. Clone all service repos from manifest
SERVICES=$(cat dev-environment-config/manifest.json | \
  python3 -c "import sys,json; \
  [print(s) for s in json.load(sys.stdin)['services']]")

for service in $SERVICES; do
  git clone "https://github.com/company/${service}.git"
done

# 4. Install VS Code extensions
EXTENSIONS=$(cat dev-environment-config/vscode/extensions.json | \
  python3 -c "import sys,json; \
  [print(e) for e in json.load(sys.stdin)['recommendations']]")

for ext in $EXTENSIONS; do
  code --install-extension "$ext" --force
done

# 5. Sync shared configs to all services
bash dev-environment-config/scripts/sync-config.sh

# 6. Install dependencies everywhere
for service in $SERVICES; do
  cd "$service" && npm install && cp .env.example .env && cd ..
done

# 7. Set up daily auto-update cron (8 AM weekdays)
CRON="0 8 * * 1-5 cd $WORKSPACE_DIR && \
  bash dev-environment-config/scripts/auto-update.sh"
(crontab -l 2>/dev/null; echo "$CRON") | crontab -

echo "✅ Setup complete! Open VS Code: code $WORKSPACE_DIR"

That's it. The developer opens VS Code, and everything is configured — extensions, linting, formatting, Docker, environment variables, the works.

Step 3: The Config Sync Script

This script copies the shared configs from the central repo into every service:

#!/bin/bash
# sync-config.sh

CONFIG_ROOT="$(dirname "$SCRIPT_DIR")"
WORKSPACE_DIR="$(dirname "$CONFIG_ROOT")"

for service in $SERVICES; do
  SERVICE_DIR="$WORKSPACE_DIR/$service"

  # Copy shared configs
  cp -f "$CONFIG_ROOT/shared/.eslintrc.js" "$SERVICE_DIR/"
  cp -f "$CONFIG_ROOT/shared/.prettierrc" "$SERVICE_DIR/"
  cp -f "$CONFIG_ROOT/shared/.editorconfig" "$SERVICE_DIR/"
  cp -f "$CONFIG_ROOT/shared/.gitignore" "$SERVICE_DIR/"

  # Copy VS Code settings
  mkdir -p "$SERVICE_DIR/.vscode"
  cp -f "$CONFIG_ROOT/vscode/settings.json" "$SERVICE_DIR/.vscode/"
  cp -f "$CONFIG_ROOT/vscode/extensions.json" "$SERVICE_DIR/.vscode/"

  echo "✅ $service synced"
done

Step 4: The Daily Auto-Update (The Magic)

This is the part that keeps everything in sync without anyone thinking about it.

Every morning at 8 AM (weekdays), a cron job runs auto-update.sh on every developer's machine:

#!/bin/bash
# auto-update.sh — Runs daily at 8 AM

cd "$CONFIG_ROOT"

# 1. Check for config updates
OLD_VERSION=$(cat manifest.json | jq -r '.version')
git pull origin main
NEW_VERSION=$(cat manifest.json | jq -r '.version')

if [ "$OLD_VERSION" != "$NEW_VERSION" ]; then
  echo "🆕 Config updated: $OLD_VERSION → $NEW_VERSION"
  bash "$SCRIPT_DIR/sync-config.sh"
fi

# 2. Pull latest code for all services
for service in $SERVICES; do
  cd "$WORKSPACE_DIR/$service"

  # Stash local changes safely
  git stash push -m "auto-update-$(date +%Y%m%d)" 2>/dev/null

  # Pull latest
  git pull origin $(git branch --show-current) --rebase

  # Update dependencies if package.json changed
  if git diff HEAD~1 --name-only | grep -q "package.json"; then
    npm install
  fi

  # Update shared library
  npm update @company/common-lib
done

# 3. Notify the developer
notify-send "Dev Environment" "✅ Updated to config v$NEW_VERSION"

The flow:

Senior Dev pushes config change
         │
         ▼
   GitHub Actions
         │
    ┌────┴────┐
    │ Slack   │ Notify team
    │ Publish │ common-lib to NPM
    └────┬────┘
         │
    Next 8 AM
         │
         ▼
  Every developer's cron runs
         │
    ┌────┴────────────────┐
    │ Pull config repo    │
    │ Sync shared configs │
    │ Pull service repos  │
    │ Update dependencies │
    │ Desktop notification│
    └─────────────────────┘

Step 5: The Shared Common Library

Instead of copying and pasting guards, interceptors, and DTOs across 12 services, we created @company/common-lib — a private NPM package:

// In any service, just import and use:
import {
  AuthGuard,
  RolesGuard,
  ResponseInterceptor,
  LoggingInterceptor,
  HttpExceptionFilter,
  PaginationDto,
  McpService,        // Qwen MCP integration
} from '@company/common-lib';

@Controller('users')
@UseGuards(AuthGuard, RolesGuard)
@UseInterceptors(ResponseInterceptor)
@UseFilters(HttpExceptionFilter)
export class UserController {
  constructor(private mcpService: McpService) {}

  @Get()
  findAll(@Query() pagination: PaginationDto) {
    return this.userService.findAll(pagination);
  }

  @Post('analyze')
  async analyze(@Body() dto: AnalyzeDto) {
    return this.mcpService.callTool('text-analysis', {
      text: dto.text,
    });
  }
}

When we update the common library (add a new guard, fix a bug in the exception filter), the auto-update system bumps the version in every service automatically.

Step 6: Service Scaffolding

Creating a new microservice used to take a day of copy-pasting and tweaking. Now it takes 30 seconds:

bash scripts/create-new-service.sh

# Prompts:
# Service name: order-service
# Description: Handles order processing
# Port: 3002
# Type: nestjs

# ✅ Done! Complete service with:
# - src/common/ (filters, guards, interceptors, DTOs)
# - Docker setup (Dockerfile + docker-compose)
# - VS Code settings
# - ESLint + Prettier
# - Health check endpoint
# - Swagger docs
# - MCP integration
# - Auto-update scripts

The Benefits (With Real Numbers)

1. Onboarding: 3 Days → 15 Minutes

Before	After
Manual setup following a 40-step Confluence page	One `curl` command
3 days to first productive commit	15 minutes to running all services
"Hey, can you help me with my env vars?" (daily Slack noise)	Zero setup questions

2. Configuration Drift: Eliminated

Before, every service slowly diverged. user-service had ESLint v8, auth-service had v7. Prettier configs had subtle differences. Docker setups were copy-pasted and never updated.

Now, every service has identical configs, and they stay identical because the sync script overwrites them daily.

3. Senior Developer Time: Saved ~5 Hours/Week

Senior devs were spending roughly an hour a day helping juniors with setup issues, reviewing PRs that failed CI due to formatting, and manually updating configs across services.

That time is now spent building features.

4. Consistent Code Quality

Every developer has:

The same ESLint rules (auto-fixed on save)
The same Prettier config (format on save)
The same commit message format (enforced by Husky)
The same VS Code extensions (auto-installed)

PR reviews went from "please fix your formatting" to "let's discuss the architecture."

5. Zero-Effort Config Updates

When a senior developer wants to:

Add a new ESLint rule → Push to config repo → Everyone gets it tomorrow
Update Docker base image → Push → Auto-synced
Add a VS Code extension → Push → Auto-installed
Fix a bug in the exception filter → Publish common-lib → Auto-updated

No Slack announcements needed. No "please run npm install" messages. It just happens.

6. Standardized Project Structure

Every service looks the same. A developer can jump from user-service to payment-service and immediately know where everything is. The src/common/ folder has the same patterns. The Docker setup is identical. The health check endpoint is at the same path.

What We Learned

Start Small

We didn't build this all at once. We started with just syncing ESLint and Prettier configs. Once the team saw the value, we added Docker, VS Code settings, the common library, and the auto-update system incrementally.

Don't Force Everything

We allow service-specific overrides. If payment-service needs a special ESLint rule, they can add a .eslintrc.local.js that extends the shared one. The shared config is the baseline, not a straitjacket.

Notifications Matter

The desktop notification ("✅ Updated to config v1.2.0") was a small touch, but it made developers aware of the system. Before that, updates happened silently and people were confused when things changed.

Version Everything

The manifest.json version tracking was critical. Without it, the auto-update script had no way to know if anything changed, and it would waste time syncing configs that were already up to date.

The Complete Stack

Layer	Technology
Backend	NestJS (TypeScript)
Frontend	ReactJS (TypeScript)
AI	Qwen MCP
Containers	Docker, Docker Compose
Runtime	Node.js 20 LTS
Database	PostgreSQL 16
Cache	Redis 7
CI/CD	GitHub Actions
Editor	VS Code
Config Sync	Bash scripts + Cron

TL;DR

If your team has more than 3 microservices and more than 5 developers, you need a centralized dev environment config. Here's the recipe:

Create a config repo with templates, shared configs, and scripts
Write a setup script that onboards new devs in one command
Write a sync script that copies shared configs to all services
Set up a cron job that auto-updates every morning
Create a shared NPM library for reusable code
Build a service generator for scaffolding new microservices

The result: developers spend their time writing code, not configuring environments.

Have you solved this problem differently at your company? I'd love to hear about it in the comments.

If you found this useful, give it a clap 👏 and follow for more microservices architecture deep-dives.

How I Built a Telegram Habit-Tracker Bot with No Programming Experience

Mon, 21 Sep 2026 12:26:13 +0000

Hi everyone!

This is my first post here. I am writing not for experienced developers, but for those who want to gain their first programming experience using AI models.

Here is how it went for me.

Recently, I had no idea about programming and had never built Telegram bots. Three months ago, as an experiment, I started "vibe coding," and now I have a working Telegram bot with several thousand lines of code, modules for selecting and saving time, passwords, a database, payment processing, 10 languages, and chart generation. You can launch the bot at https://telegram.me/I_promise_myself_bot (click the "Try free & Unlock" button).

I want to share my experience—how I used several different LLMs in a regular free chat, how I bypassed limits, and how I got out of dead ends when the AIs completely stalled, forgot context, and the bot started breaking.

Where It All Started

Like everyone else, I started with ChatGPT, then tried Grok, Mistral, Gemini, Qwen, and DeepSeek bit by bit. I wanted to figure out through trial and error what each could do and for how long. I quickly ran into token limits in free chats. Also, over long distances, they lose context at different times, start mixing up file structures, and produce non-working pieces of code.

As long as there were 200-300 lines of code, working with AIs was manageable, but when the code exceeded 1,000 lines, the problems began. To overcome bottlenecks, I couldn't paste or upload such an amount of code into the window for the AI to see the whole code, find the exact error, or add a new feature.

Another problem was that any AI, as a rule, forgets the conversation after a few hours or starts hallucinating heavily. After that, you have to kill the chat and create a new one, re-supplying the new chat with the general task, snippets of chat history and code, and almost the entire conversation so it understands what I want from it. I regularly read bloggers, and regarding this, I asked Kostya Doronin for advice on how to work with AIs when code accumulates to more than 1,000 lines across multiple files. Surprisingly, he readily suggested several free and paid workarounds, but I managed with free ones only. I fundamentally wanted to figure out if it's possible to make something working with AIs for free.

I had to change my approach a bit. First, I roughly figured out what can be squeezed out of this or that AI, when they start glitching, and how to bypass it. I stopped loading one AI with all tasks at once and started dividing tasks among models.

I settled on these:

1.Claude is the architect. It has the best logic and understanding of the general code and project structure. I used it for brainstorming and design, modularizing, and creating step-by-step instructions for other AIs. Also as a "rescuer" in dead-end situations when another AI started leading me in circles. The only downside is that free tokens run out very quickly, and you have to wait a few hours.
2.Qwen turned out to be the main "workhorse" over long distances. If you give it a detailed plan from Claude, Qwen handles it well and sticks to the plan. Claude has strict limits on free messages, but with Qwen, you could code all day long in the same chat, solving tasks one step at a time. It holds up for about 7 days before hallucinations begin, after which you need to kill the chat and start a new one.
3.Gemini is an additional tool for quickly double-checking individual pieces of code, like a "fresh perspective" on a problem or a new task. But in the free version, it is not particularly good at programming and architecture, and if you work with it for a whole day or two, it also starts hallucinating. Although there were periods when these same models worked for weeks without any hallucinations at all. And sometimes a day was enough for both Qwen and Gemini to start glitching, which I never encountered with Claude. Probably because it ran out quickly! From which I concluded that model owners for some reason release different quality versions of the same model into the wild.

In other words, all these LLMs are a very unreliable thing, and for serious projects, you should only work with people. But as assistants in simple coding tasks, they are very good.

Project Architecture

At the beginning of creating the bot, Claude guided me step-by-step to download and set up all these tools on my computer:

•Language and framework: Python 3.11+, aiogram 3.x.
•Database: Started with simple Google Sheets, but eventually switched to PostgreSQL, hosted on Supabase. Mistake: should have used PostgreSQL from the very beginning.
•Background tasks: APScheduler for push notifications.
•Graphics: matplotlib for reports.
•Payments: Paddle SDK integration.
•Localization: Multilingual system based on JSON (languages). Mistake: did it at the very end, but should have done it at the very beginning because I had to redo a lot of things.
•Security: Isolating tokens in a separate file.

Development Process

For each new bot feature, I tasked Claude with developing as detailed a step-by-step instruction plan for Qwen as possible. I coded in VS Code, but didn't use LLMs inside it directly, only in chats:

1.I initially split the entire project into logical modules: there is one main file up to 2,000 lines, and the rest are 200-700 lines. Everything is separate – handlers, database operations, notifications, buttons, and localization.
2.Working with Qwen: Before any changes, I saved all recent files with working code in a separate folder so I could easily roll back. I gave Qwen a step-by-step plan and changed or added functionality step by step. I inserted it into the project, ran the bot, and checked the operation and logs. If there was a problem, I gave the logs to all three AIs at once to find the issue. If the problem hit a dead end, I started over from the last working code, but mentioned in the task to Claude (if tokens remained) that there were such-and-such problems during the previous change and they needed to be avoided. If Claude was exhausted, I asked Gemini. And so I moved step by step.

How I Got Out of Dead Ends

Somewhere in the middle of the project, the code exceeded a thousand lines, and problems began: the AI forgot the context and broke what was already working when trying to fix a bug.

At that moment, I brought in Claude:

1.I described the essence of the technical problem and the bot's startup logs in VS Code.
2.I provided snippets of chat history where Qwen started going in circles.
3.I attached the problematic code.

Mostly, Claude found the error on the first try and provided a solution. If it didn't know, it asked leading questions and we ran additional checks. Unlike Qwen or Gemini, which immediately "poured out" a bunch of possible causes after which going in infinite circles began. In especially neglected cases, I "pitted models against each other": I drove the problematic piece in circles between Claude, Gemini, and Qwen until one of them found the bug. Their collective intelligence works much more reliably.

Conclusions

After a few months of casual work, I got a working Telegram bot with several thousand lines of code, written from scratch with no experience.

What I realized for myself:

1.On simple tasks, AI replaces a team, but you need to understand the project structure well yourself and at least superficially understand the code. If you have no knowledge, you can learn to think systemically and architecturally on such projects.
2.Use different models. Using a single AI for everything is undesirable. The "architect + coder + fresh perspective" bundle works better.
3.Small steps. Wrote a function – tested it. Broke – rolled back to the last working version.
4.Save chat history. Chat history with AI helps you not to lose the logic of the project yourself after a week or month of work, and to provide context to another AI for problem analysis.

I will be glad to answer questions in the comments!

Your Hiring Process Needs HTTP Status Codes

Wed, 26 Aug 2026 15:46:00 +0000

Because "we'll be in touch" is not an observable state.

I submitted a job application recently.

It was one of those applications where you actually put in the effort. I read the job description carefully, tailored the resume, wrote a cover letter specifically for the organization, and answered the optional questions rather than pretending they were optional.

Then I clicked Submit and got... nothing.

No confirmation email. No "we received your application." No indication that a candidate record had been created. Just a browser page telling me the submission had succeeded and the vague hope that somewhere, deep inside an applicant tracking system, my carefully assembled application had not been written directly to /dev/null.

As a developer, I find this troubling. Not because I expect an interview, or because I think every recruiter owes me personalized feedback. I just want a status code.

And I think that's a smaller ask than it sounds like, for a reason I'll come back to at the end: the states already exist. Somewhere in that ATS, my application is in one. The decision not to show me which one is a product decision, not a privacy constraint.

The Web Solved This Problem Decades Ago

When software sends a request to another system, we generally consider it useful for the receiving system to say what happened. Did it work? Is it still processing? Was the request malformed? Is the resource gone? Did something catch fire on the server?

HTTP gives us an entire vocabulary for this.

Hiring systems, meanwhile, have developed their own protocol:

POST /careers/applications

[no observable response]

This is sometimes followed, anywhere from three minutes to six months later, by:

After careful consideration...

Instead of inventing vague candidate statuses like "Under Consideration," perhaps hiring systems could adopt something developers already understand.

The 2xx Family: Success, Allegedly

`200 OK`

A human being has reviewed your application.

Nothing else is implied.

Honestly, this alone would be revolutionary.

`202 Accepted`

Your application has been received and accepted for processing.

This is the most important status code in the entire hiring protocol. The candidate does not need the hiring manager's notes or a walkthrough of the internal workflow. They need:

202 APPLICATION_ACCEPTED

Now we know the thing exists.

`204 No Content`

Your application was successfully received.

We have chosen not to acknowledge this in any way.

The 3xx Family: You Have Been Redirected

`302 Found`

We found an internal candidate.

Thanks for playing.

`304 Not Modified`

Your application has been "Under Review" for 47 days.

There has been no change since the last time you checked, or the 23 times before that. There will be no change the next time either. Please consider caching this response locally.

`410 Gone`

The position existed when you applied. It no longer exists. No additional information is available.

At least 410 would be useful. The current discovery mechanism is noticing that your bookmarked job posting has started redirecting to the careers homepage.

The 4xx Family: This One's On You

`401 Unauthorized`

You must create an account before applying.

Your password must contain at least 14 characters, uppercase and lowercase letters, a number, a symbol, and one ancient Sumerian glyph. It cannot match any password you have used since 1997.

You will then manually enter every field from the resume you just uploaded.

You will never use this account again.

`403 Forbidden`

You have the required experience and qualifications.

Unfortunately, you live 37 miles outside the geographic boundary within which we have determined this Zoom meeting can occur.

`404 Not Found`

Hiring manager not found. Recruiter not found. Job posting not found.

Nobody knows who owns this requisition.

`408 Request Timeout`

After seven weeks without communication, the candidate has accepted another position.

The hiring team will contact them tomorrow to schedule the second interview.

`409 Conflict`

The position requires 12 years of production experience with a technology introduced six years ago.

The conflict cannot be resolved.

`417 Expectation Failed`

You expected the phrase "we'll be in touch" to imply future communication.

It did not.

`422 Unprocessable Content`

Your experience is relevant, but your career path does not fit cleanly into the predefined boxes in our applicant tracking system.

Human intervention may be required.

Human intervention is unavailable.

`425 Too Early`

You applied 14 minutes after the job was posted.

LinkedIn reports 612 applicants.

We have questions too.

`451 Unavailable For Legal Reasons`

Your qualifications are not the issue.

The issue is a background check vendor, a non-compete of uncertain enforceability, or a sponsorship question nobody wants to be the one to ask about.

The 5xx Family: It Was Never You

`500 Internal Server Error`

Something has gone wrong internally. Your recruiter left, the headcount was frozen, the hiring manager changed, Finance reconsidered the budget, the organization restructured, or the position was posted by accident.

We will communicate all of these possibilities using the same message:

We have decided to move forward with other candidates.

`502 Bad Gateway`

The recruiter says they are waiting for the hiring manager.

The hiring manager says recruiting owns the next step.

Your request cannot be routed.

`503 Service Unavailable`

Your recruiter is no longer with the company.

Their calendar link remains active.

`504 Gateway Timeout`

"We expect to have a decision by Friday."

Friday has passed. The specification does not define which Friday was intended.

Candidates Don't Need Distributed Tracing

There's a serious point hiding under all of this. Hiring has an observability problem, and it's usually mistaken for a communication problem.

Companies understandably cannot expose everything happening inside a recruiting process. Candidate evaluations are private. Interview feedback may be sensitive. Hiring managers need room to compare people and make decisions.

That's fine. Nobody is asking for:

GET /hiring-manager/private-thoughts/about-me

But there is an enormous amount of territory between exposing confidential deliberations and providing no information at all.

A hiring pipeline already has states. An application was received. It entered review. Someone reviewed it. An interview was requested. The requisition was paused. The position was filled. The position was canceled. The candidate was declined.

Those state transitions already exist in the system. They are already recorded, timestamped, and reportable, because that's how recruiting teams measure their own funnel. The candidate is the one participant in the process who cannot see the record they're in.

Give Us the Boring Version

Imagine an applicant portal that reported nothing more than this:

202  APPLICATION_RECEIVED     Mar 04
200  REVIEWED                 Mar 19
304  NO_CHANGE                Apr 02
304  NO_CHANGE                Apr 30
410  POSITION_CLOSED          May 12

That's not radical transparency. It's barely transparency at all. But it tells you the system is functioning.

It removes the need to wonder whether an application disappeared. It reduces pointless portal refreshing. And it prevents candidates from reading silence as information, when silence might mean anything from "the hiring manager is on vacation" to "the requisition was canceled two weeks ago."

Mostly, it treats applicants like participants in a process rather than packets transmitted into an undocumented endpoint.

`200 OK`

I don't expect every application to result in an interview. I've submitted enough of them to have considerable empirical evidence supporting that position.

I don't expect detailed rejection feedback either. I understand why companies are reluctant to provide it. I don't even particularly mind 403 REQUIREMENTS_NOT_MET or 410 POSITION_FILLED. Those are answers.

What makes modern hiring uniquely frustrating isn't rejection. It's uncertainty about whether anything is happening at all.

Software engineers have spent decades building systems around a simple principle: when one system asks another to do something, the response should carry enough information to understand what happened.

Candidates aren't asking for distributed tracing of your recruiting infrastructure.

We'd settle for a status code.

Why Octopus invests in developer experience

Fri, 03 Jul 2026 12:47:04 +0000

When you join Octopus, you get a great laptop and your choice from a range of docks, monitors, webcams, and input devices. You also get a home office allowance, which lets you upgrade your space and get 50% back. You may want a sit/stand desk, an awesome chair, and portrait lighting so everyone can see how delighted you are when they demo their cool new feature.

What we ask in return is that you set yourself up to do your best work. Invest in your brain, bytes, back, and buns, as Scott Hanselman put it. The essential idea behind this is that when you win, Octopus wins. We all win!

Yet you can throw a soggy cat at a conference, and the chances are it will splash someone from an organization with a completely different approach to developer experience. One that doesn't value the sustainable delivery of their people's best work. Why is that?

A fork stuck in the road

As we've seen in the software industry, silos are the source of all kinds of chaos. DevOps was created when someone realized the scale of the problem caused by having two teams with conflicting goals working on opposite sides of the same problem. Leaders whipped developers up for feature throughput and wanted to hold operations teams to account for reliability.

In hindsight, it's obvious that this isn't going to work, but we did it for decades. You can get halfway to DevOps just by aligning the goals. When both teams share the same target, they'll work out how to get you the rest of the way.

If we zoom out on our organization, we're likely to find a similar silo/goal conflict between engineering managers and the finance team. We ask the engineering manager to build a high-performing team, while we direct finance to spend as little money as possible. This structural incentive problem leads developers to get laptops and tools that someone has negotiated to be "good enough," and trust me, they aren't.

When you look at developer experience through a FinBoss lens, where finance and engineering managers align around the return on investment, you'll find the optimal spend on developer tools shifts right rapidly.

FinCost organizations reduce costs through standardization, commoditization, and by delaying kit refreshes for as long as possible. They often spend less than 1,000 USD on a laptop without realizing the true cost of an under-spec'd machine. Meanwhile, FinBoss organizations are seeking the 30,000 USD annual return on a 3,000 USD investment.

To achieve this ROI, finance and engineering leadership work together to design policies that allow engineers to choose from a set of well-designed options. The combination of investment in great tools and individual choice in building the right setup is the path to success.

Tiny visible costs

FinCost organizations are driving decisions based on small, visible costs. Where there's a paper trail, there are savings to make: the purchase of a laptop, the training budget, and the conference invoice. These are part of the legible model, so there is no guesswork or experimental method needed to cost them. The numbers are right there on the page.

Once you're managing costs, it's not uncommon to dial up the control, too. You limit options, standardize across the organization, and negotiate in bulk on everything to drive costs even lower. Compared to competitors, you could be spending 75% less on laptops, and just look at the savings from those pesky licenses and SaaS tools various teams were using. Hooray.

Instead of letting engineers attend conferences they find relevant to their work, the organization negotiates a bulk purchase for a single conference. Instead of teams buzzing with diverse ideas and approaches, they all sit and watch the same talk. It's cheaper that way.

A wooden fence encloses my garden. A fence lasts a long time if you give it a protective coat of wood preserver every once in a while. A tin of wood preserve costs about 20 USD and a few hours of mindful application each year. Over 5 years, I can save 100 USD by skipping the treatment. If I do this, my fence lasts about 10 years instead of 30.

The cost of a tin of wood preserver is immediate and visible. The cost of replacing the fence is far higher, but it won't be visible for years. Additionally, if the fence provides value beyond privacy, like keeping a goat out of my vegetable garden, the cost of the fence failing could be the cost of my whole crop, which is even less legible to my cost control mindset.

If you're creating software, it's likely to be more valuable than a fence. In fact, it should be giving you returns multiple times your investment. It's like a golden goose. You feed it well, and you get regular, valuable deliveries. If you had a golden goose, you wouldn't try to save on food costs if it reduced the flow of golden eggs.

The same logic plays out with training. A FinCost organization cuts the training budget because the line item is visible and the return isn't. Individual engineers stop attending relevant courses, and teams miss out on the steady accumulation of skill that comes from ongoing, targeted learning. Nobody raises a purchase order for "capability that didn't grow this year."

Then something goes wrong. Delivery is slow, the codebase is a mess, or a competitor pulls ahead. Someone decides the answer is transformation, and that means Scaled Agile, or whatever flavor of desperation is in fashion in exec suites at the time.

So the whole department gets booked onto an external SAFe course. Not one or two people who need it. Everyone. The invoice lands, and it's eye-watering: many times what a thoughtful, ongoing training program would have cost across the same period. The organization that was too cautious to spend 500 USD on a course people wanted just spent 50,000 USD on one nobody asked for.

This is the hidden tax of FinCost thinking. Small, visible costs get cut. The problems that those small costs would have prevented quietly compound. Eventually, the pressure releases all at once, in an expensive, rushed, and hard-to-argue-against way, because now there's a crisis with a visible price tag.

The tin of wood preserver was the right investment all along.

The invisible people-hours

Developers work around 40 hours a week. Stripe's Developer Coefficient found that productivity friction can eat up roughly half of those hours. That's the time that evaporates before a single line of useful code is written. Across the global developer workforce, Stripe put a number on that loss: 300 billion USD. It's worth noting that only 85 billion USD of this is attributable to bad code. The rest is pure friction: slow tools, broken environments, waiting for builds, wrestling with under-spec'd machines. Productivity means more than writing code faster.

That friction has a texture. It's a developer losing flow because their laptop is thrashing memory, and they've lost the thread of what they were thinking.

It's waiting 10 minutes for a build that would have completed in 2 minutes on a faster machine. It's the workarounds you built because the approved tool wasn't good enough, which obliterates hours of your week for maintenance. It's the skilled engineers who start looking for a new role because the environment makes them feel like you don't take their work seriously.

None of this appears on a purchase order. None of it shows up in a variance report. It just quietly drains the value your software could be creating.

There's also an upside case, not just a loss case. Software that ships faster captures more market share. Developers who stay in flow write better code with fewer bugs, which means less rework and fewer incidents. A high-performing team attracts other high performers. The return on investing in developer experience isn't just "we lose less time"; it's sales opportunities, reduced churn, higher product quality, and a compounding advantage over competitors who are still buying 800 USD laptops and calling it responsible.

When you focus on cost control, there's a ceiling on how much you can save. When you look for return on investment, there's no equivalent ceiling on how much you can gain.

The CFO lens

When I asked Sammy, our CFO, about why we invest in people, powerful machines, and great tools, he said: "At Octopus, we think about equipment as a productivity tool. Developers are one of our most expensive and constrained resources. If spending a few extra thousand dollars on a fast machine and proper screen space saves time, reduces cognitive load, or keeps someone in flow more often, it pays for itself very quickly."

That means our finance team doesn't ask "What's the cheapest laptop we can buy?" Instead, they ask, "What setup helps this person do their best work?" From a finance perspective, this is about a high-ROI spend, not indulgence.

Octopus is running the FinBoss playbook, which aligns finance and leadership in empowering employees to do their best work. That means expanding what you're willing to measure so that you can balance the visible costs with traditionally invisible returns.

The responsibility for providing this environment for employees rests with all leaders, including Sammy, and is part of the deal employees have with the company; we'll give you the best environment and tools if you give us the greatest work of your career.

The reckoning

If we've ever met, I've likely mentioned DORA's research. I'm a super-fan of their State of DevOps reports, because they provide us techniques, practices, and outcomes that are all linked through complex relationships. The model shows that transformational leadership, lean product management, and Continuous Delivery drive software delivery performance and organizational outcomes.

While many of the capabilities in the DORA model are technical, many others relate to leadership. Particularly relevant to developer experience are generative organizational culture, transformational leadership, empowering teams to choose tools, team experimentation, and a learning culture. You can read about these and more on the DORA website. Together, they back up the FinBoss concept with tens of thousands of survey samples that support the case.

The question is: Which organization are you now, and which do you want to be? Is it FinCost, operating myopically against the bottom line, or FinBoss, sharing the responsibility for developer experience and employee productivity? The fork in the road was never a choice. We know FinCost is a dead end, and the benefits of FinBoss aren't soft claims; they're measurable outcomes.

Minha Experiência com o uso do SpecKit no desenvolvimento em Infra/Cloud

Fri, 03 Jul 2026 03:01:00 +0000

A engenharia de plataforma e o DevOps estão passando por uma transformação impulsionada por agentes de IA. Recentemente, em meus estudos de arquitetura, infraestrutura e governança, comecei a utilizar o SpecKit, uma ferramenta que atua desde a especificação técnica até a implementação de código.

Neste artigo, quero compartilhar minha experiência prática integrando o SpecKit no dia a dia para desenhar arquiteturas Cloud, escrever Terraform e automatizar fluxos, destacando o que funciona bem e os pontos de atenção.

O que é o SpecKit e sua Importância

O SpecKit funciona como um assistente avançado que ajuda a planejar e codificar soluções técnicas. A grande vantagem é que ele não apenas "gera código", mas segue um fluxo de engenharia estruturado que é vital para nós, Platform Engineers: Create the spec > Create a technical implementation plan > Break down into tasks > Execute implementation, isso integrado a minha IDE like Vscode ao chat de agente, já o utilizei com o Copilot e com o Antigravity.

Isso significa que, antes de cuspir dezenas de arquivos .tf, ele gera um documento de constituição (constitution.md) ou especificação técnica e um plano de ataque, permitindo que o arquiteto revise as premissas, a segurança e a modularização antes da execução.

Como eu configuro e utilizo (Hands-on)

A inicialização da ferramenta no repositório local é bem direta. Eu começo instalando o specify na pasta do projeto:

$ uvx --from git+https://github.com/github/spec-kit.git specify init --here

(Fonte de referência dos meus estudos de setup).

A mágica do SpecKit para DevOps acontece quando injetamos Skills específicas no agente. Dependendo do projeto, eu adiciono pacotes de habilidades voltadas para Terraform, GitHub Actions, ou skills de provedores de infraestrutura em nuvem:

npx skills add https://github.com/hashicorp/agent-skills --skill terraform-stacks
npx skills add https://github.com/jeffallan/claude-skills --skill terraform-engineer
npx skills add https://github.com/callstackincubator/agent-skills --skill github-actions

(Adicionando contexto técnico para o agente não alucinar com práticas ruins).

Com o ambiente preparado, eu utilizo o comando /speckit.constitution para passar meu prompt detalhado e iniciar o planejamento. Eu costumo preparar um prompt em outras ferramentas como Gemini ou ChatGPT, etc.

Casos de Uso Reais (e como o SpecKit se saiu)

Durante minhas validações técnicas, apliquei o SpecKit em variados cenários de infraestrutura. Aqui um exemplo ficticio:

Migração de Workloads para Subnets Privadas com Egress Controlado

O objetivo era definir uma estratégia de migração de aplicações containerizadas para uma arquitetura com isolamento de rede mais robusto, mantendo compatibilidade com integrações externas.

O prompt especificava requisitos como separação entre camadas públicas e privadas, roteamento de saída controlado, alta disponibilidade entre zonas de disponibilidade e impacto mínimo nas aplicações existentes.

Resultado: A ferramenta gerou uma proposta arquitetural completa, incluindo diagramas de fluxo, plano de migração, dependências técnicas e validações necessárias para garantir conectividade entre componentes. Durante a evolução do projeto, ajustes de segurança e requisitos de integração puderam ser incorporados sem reestruturar a documentação original.

Prós e Contras

Com em PoCs executadas, aqui está o meu veredito sobre o uso do SpecKit no contexto de plataforma:

Prós

Fluxo Racionalizado: A quebra do problema em "Especificação técnica" -> "Plano de tarefas" -> "Código" evita que a IA crie arquivos caóticos. Se a especificação arquitetural estiver errada (ex: usar EKS em vez de Fargate), eu posso corrigir antes que ele gere o Terraform.
Refinamento Contínuo: É muito fácil pedir alterações granulares, como aplicar testes HCL nativos (.tftest.hcl) ou injetar variáveis específicas de ambiente (variables.tfvars) nos módulos.
Adoção de Padrões: Usando as Skills externas, a ferramenta já herda boas práticas de módulos oficiais da comunidade.

Contras

Dependência de Prompts Extremamente Detalhados: O SpecKit só é incrível se você souber exatamente o que quer. Eu precisei especificar até restrições como "NÃO usar tecnologia X ou Y" e "manter tasks stateless" para garantir que a IA não tomasse decisões que ferissem os princípios de custos e ou anti-patterns.
Revisão Frequente: Arquiteturas baseadas em nuvem envolvem políticas sensíveis de IAM e rede (ex: s3:PutObject ou peering de VPCs). O código gerado exige um olhar crítico com ferramentas como Checkov ou Trivy para evitar misconfigurations geradas por IA.

Conclusão

O SpecKit se mostrou um copiloto poderoso para Engenharia de Plataforma. Ele não substitui o arquiteto de soluções — pelo contrário, ele exige que o arquiteto faça as perguntas certas e defina os guardrails corretos. Contudo, ao reduzir o tempo gasto na escrita de boilerplate (módulos Terraform, pipelines do GitHub Actions, Task Definitions do ECS), ele libera a equipe de plataforma para focar em segurança, governança e experiência do desenvolvedor.

Modern developer experience has deep roots

Tue, 12 May 2026 11:37:21 +0000

In his 1956 account of the SAGE program, Herbert Benington highlighted the opportunity to use computers to reduce the cost of programming, documentation, and testing.

The creation of utilities, compilers, and instrumentation accounted for about half of the programming effort for SAGE. Benington had recognized that writing programs to improve developer productivity was an essential investment.

At the time, computers were costly, so the idea of making programmers more productive had far less economic weight than it does today. Programmers earned approximately $15,000 per year, and computers cost $500 per hour to operate. Now programmers cost about 70 times the cost of the computers they use. That means the value of developer productivity is worth far more today than it was in the 1950s.

Betting on software

There has been a thread of developer experience all the way back to the earliest attempts to build software at scale. Savvy organizations know that money spent on providing the right environment and tools is worth more than simply “time saved”.

There are two ways to look at software development: Cost and value. It certainly costs money to build software, so the software must provide value that exceeds this cost to be viable. Software systems are built based on the anticipation of value and survive if they manage to meet or exceed that expectation. When an organization becomes cost-obsessed with software, it suggests a low anticipated value or a realization that the value won’t be realized. It’s better to bravely abandon attempts with such a thin payoff.

The sweet spot for software is where the value is highly likely to be obtained, or where there’s a chance of it providing a huge return on investment, so that in any 10 attempts to create value with software, a single success would pay for all the attempts and return a profit.

When you consider how software is a bet, it divides software delivery approaches into two categories: cost-focused or value-focused. Traditional project management works to keep the promise of cost and timeline. Modern agile methods try to increase the probability of the bet succeeding by adjusting course as you learn more about the problem you’re trying to solve.

And here’s the crucial insight. When you manage costs effectively, the best you can achieve is zero cost. When you seek out value, there is no absolute limit to how much value you could produce. It could be twice the cost, ten times the cost, or 1,000 times the cost. There is far more upside than downside if you’re creating valuable software.

Cost-first approaches to software delivery decrease the probability of success, and one (of many) reasons is that it damages developer experience.

DevEx is economics, not hugs

For whatever reason, the universe decided that you must treat people well, whether you like them or not. Even when you examine organizational culture through the lens of cold, hard business goals, you’ll find that unhealthy cultures are less successful than healthy ones. You can be a philanthropist or a capitalist; either way, you have to treat your employees well, or it will damage the thing you care about.

Here’s a simple way it plays out.

The developers need a bit more screen real estate so they can display more information in front of them without having to switch between background and foreground apps. Additional monitors incur costs, and a cost-focused organization will likely deny the request. Developers will have a lower fully loaded cost, but produce less value. A value-focused organization sees the potential returns and their developers will get more screen space, be less frustrated, produce better work in a shorter time, and produce a lot of value.

Having an extra monitor moves the needle a little, but it’s a strong signal. Once an organization chooses between the cost or value pathways, it tends to stick to that decision. That means it’s not just monitors; it’s also chairs, code editors, refactoring tools, test tools, and automation tools. The experience diverges further with each decision made based on cost rather than value.

Another individual who understood the concept of developer experience was Joel Spolsky. He created the Joel Test as a “highly irresponsible, sloppy test to rate the quality of a software team.” The Joel Test has items like “Do programmers have quiet working conditions?” and “Do you use the best tools money can buy?”

I haven’t met Joel, so I can’t speak for his motivation, but I don’t need to know if he was motivated by kindness or cash. The result was an excellent workplace for developers and phenomenal value creation; a win-win, as Stephen Covey called it. Spolsky’s most famous products, Trello and Stack Overflow, sold for $425 million and $1.8 billion, respectively.

You don’t need to make it easy

There’s a certain amount of inherent complexity to writing great software. You must fully grasp a problem, have a strong opinion about how to solve it, and be able to execute on your plans to make it happen. Developers don’t need protection from the difficulty of building software; they need minimal unnecessary complexity from tools, processes, and the workplace environment.

There was a trend that prioritized developer comfort above all other needs, which meant providing them with frameworks to tame complexity. The frameworks made development easier, but limited a developer’s options to the extent it damaged user experience. User needs were subverted to developer ease, which is wrong and somewhat patronizing to developers.

It’s not developer experience if you’re using frameworks that improve the ease of development while annoying those trying to use the software. Developer experience means providing the right environment and tools for developers to build valuable software. Software that doesn’t surprise people with a new paper cut every 5 minutes, pushing them ever closer to demanding an alternative solution.

Think instead of how we set up a surgeon for success. A sterile room, excellent lighting, high-quality equipment, and working with skilled individuals who can anticipate and respond as the situation unfolds. Surgeon experience is centered around a shared goal of achieving optimal patient outcomes. We don’t simplify the scenario by removing things like the need to prevent infection; we make it possible to handle it well.

Developer experience is the same. We don’t choose easier problems to solve; we set developers up to succeed at solving hard problems.

Modern DevEx and Platform Engineering

With the rise of Platform Engineering, developer experience has been largely absorbed. Your organization might have a DevEx team, a platform team, or even both. Across the industry, the two teams share more commonalities than differences. From a list of 30 features offered by platform and DevEx teams compiled by DX, only 4 were exclusive to a single discipline.

Platform only:

Certificate management
DNS
Networking

DevEx only:

Developer training and education

Everything else is along the scale from DevEx to Platform Engineering, where it may be more common in one or the other, but can be found in both.

Platform Engineering and developer experience both build on Benington’s early thoughts and Spolsky’s belief that if we provide developers with the right environment and the best tools, we can amplify their skills and generate lots of value. Forming teams around this idea helps standardize and scale the approach, rather than each team being subjected to differing views based on management styles or simply not knowing what they don’t know.

The Q1 2026 DevProd headcount benchmarking report from DX highlights how well this scaling works. Rather than costing a fixed percentage of your engineering organization’s headcount, developer productivity teams scale non-linearly, with their ratio shrinking as the number of engineers increases. This makes sense, as their work is being reused, unlike approaches that work within individual teams and depend on teams having access to the necessary skills and knowledge.

Engineers	Productivity headcount
200-600	5.1%
600-1000	4.2%
1000+	3.49%

That’s not to say the goal is to have the smallest possible teams. The goal is to unlock the value you create by developing software. If having half of all engineers in productivity roles resulted in the highest levels of value creation, that would be the right mix. There is likely a point of diminishing returns if you approach 10-15%, but you should be testing this by tracking meaningful outcomes for your organization.

Make sure developers have the right environment and the best tools so they can generate the most value for your organization.

Stop Treating Angular as a Second-Class Framework for UI Components

Thu, 16 Apr 2026 08:00:00 +0000

You’ve been promised “Angular support” more times than you can count.
What you got instead was a React library in an Angular costume.

A thin compatibility layer. A demo that technically works, but feels wrong the second you try to build something serious with it.

Angular support is not the same as Angular respect.

And the more you look at how Angular tooling is actually built, the less patience you have for ecosystems that treat Angular like an afterthought.

The Lie Behind “Angular Support”

You’ve seen it.

A UI library announces Angular support.
You click the docs.
You get excited for about 15 seconds.

Then you realize what “support” really means:

awkward APIs that feel imported from another ecosystem,
missing Angular ergonomics,
strange lifecycle edges,
styling and composition patterns that clearly weren’t designed for Angular first.

Technically, yes, it works.

But if your first reaction is “why does this feel like React with extra steps?” , then it’s not a native Angular experience.

”A wrapper is not a first-class developer experience.”

And Angular developers know the difference immediately.

React Gets Products. Angular Gets Adapters

This is the part nobody wants to say too loudly.

In a lot of modern tooling ecosystems, React gets the real product.
Angular gets the adaptation layer.

React gets:

the first-class API,
the best examples,
the cleanest DX,
the product decisions designed around its mental model.

Angular gets:

“integration”,
“compatibility”,
“support coming soon”,
or a wrapper that exposes just enough to close the sales objection.

That doesn’t mean every React-first library is bad.
It means Angular is too often treated as a market checkbox instead of a platform that deserves design respect.

And once you’ve felt that enough times, you stop reading “Angular support” as a positive signal.

You start reading it as a warning label.

Most multi-framework libraries are really one framework with export options.

The Problem Isn’t Just Pride. It’s Product Quality

This is not framework tribalism for its own sake.

Wrapper-heavy ecosystems create real problems:

worse developer experience,
inconsistent abstractions,
harder debugging,
awkward state integration,
weaker performance characteristics,
APIs that don’t fit Angular’s strengths.

When a library is born in one ecosystem and then “ported” into Angular without fully embracing Angular’s patterns, you pay for it everywhere:

in the template layer,
in reactivity,
in change detection assumptions,
in styling APIs,
in the mental overhead of using something that never really feels native.

That cost compounds fast in real products.

A bad wrapper is not just ugly.
It becomes architecture debt.

The ecosystem quality you tolerate today becomes the architecture debt you inherit tomorrow.

Angular Developers Are Not Asking for Special Treatment

The ask is much simpler:

If you say your library supports Angular, then build for Angular like it matters.

That means:

APIs that feel natural in Angular,
composition patterns that respect Angular architecture,
state handling that fits Angular’s reactive model,
docs written for Angular devs, not translated from another framework,
examples that look production-ready, not “we made this demo work once.”

Angular is not some fringe hobby framework.
It powers serious enterprise apps, internal tools, fintech platforms, compliance systems, and big teams that care about long-term maintainability.

So the expectation is not unreasonable.

If the framework is good enough to power critical systems, it’s good enough to deserve native tools , not compatibility leftovers.

If Angular is good enough for core systems, it’s good enough to deserve first-class tools.

Why Developers Are Spending Months on Angular-First Tools

At some point, complaining stops being interesting.

You either keep using wrappers and live with the friction, or someone builds the thing they wish existed.

That’s why you increasingly see stories of developers who spend months building Angular-first tools — including rich text editors — instead of gluing React-centric libraries into their apps.

Not because the world desperately needed yet another editor.
And not because anyone wanted a vanity side project.

They did it because they were tired of pretending Angular developers should be grateful for second-hand abstractions.

They wanted things that felt native from the inside out:

designed around Angular patterns,
built with Signals in mind,
not fighting Angular templates,
not leaking another framework’s assumptions into every integration point.

If your “Angular integration” feels like React with extra steps, it’s not Angular-native.

The deeper you look at these case studies, the clearer the pattern becomes:

Most “Angular ports” aren’t really ports. They’re translations with compromises.

Because once someone tries to build something properly for Angular, it becomes obvious how many “multi-framework” tools are really just one framework with export options.

Signals Changed the Standard

Angular’s ecosystem has a chance right now to stop copying old patterns and start building better ones.

Signals matter here.

Because an Angular-native future should not be:

old wrapper patterns with new branding,
React mental models translated into Angular syntax,
or generic abstractions that ignore what Angular is actually good at.

It should be:

signal-first APIs,
cleaner state flows,
better template ergonomics,
components designed for Angular’s architecture instead of awkwardly tolerated by it.

This is where Angular devs need to get more ambitious.

Not “Can we get this React thing working in Angular?”
But:

What would the best possible Angular-native version of this tool look like?

That’s a much better question.
And honestly, it’s long overdue.

The Angular Community Needs More Conviction

One of Angular’s biggest ecosystem problems is not technical.

It’s psychological.

Too many Angular developers have been trained to accept less:

fewer native libraries,
slower ecosystem momentum,
weaker mindshare,
and “good enough” wrapper support.

That creates a strange defeatism.

People start talking like Angular is lucky just to be included.

It isn’t.

Angular deserves builders who act like it’s worth building for.

Angular doesn’t need more compatibility. It needs conviction.

That means more developers creating:

Angular-first component systems,
Angular-native editors, tables, charts, and design tools,
signal-based primitives,
and products that don’t apologize for not being React.

Because if Angular developers keep waiting for React ecosystems to save them, they will keep getting what they’ve always gotten:

Adapters.
Wrappers.
Second-class support.

If you keep acting like Angular is second-class, don’t be surprised when the ecosystem treats it that way.

This Isn’t About Beating React

React is not the villain here.

React won the mindshare war for a reason:

huge ecosystem momentum,
lower barrier to experimentation,
enormous commercial gravity around component libraries and UI tooling.

Fine.

But Angular developers keep making the mistake of treating that as proof that Angular should just consume whatever React produces.

That’s not strategy.
That’s dependency.

The real question is not:

How do we get more React libraries into Angular?

The real question is:

What tools should exist because Angular exists?

That’s how you build a healthier ecosystem.

Not by begging for parity through wrappers, but by creating products that are unmistakably native.

Native Angular Tools Will Always Feel Better

A real Angular-native tool gives you things wrappers rarely do:

APIs that match Angular instincts,
predictable integration into Angular architecture,
cleaner mental models,
fewer translation layers,
less framework dissonance.

And when the product itself is designed for Angular from day one, the quality difference shows up everywhere:

in how fast teams onboard,
in how easy it is to debug,
in how natural it feels to extend,
in how much trust there is in the tool long-term.

That trust matters.

Because teams don’t just adopt components.
They adopt maintenance futures.

A wrapper always feels like borrowed infrastructure.

And borrowed infrastructure is the first thing people want to rip out when things go wrong.

The Manifesto

So here’s the position, clearly:

Stop calling wrappers “great Angular support.”
Stop rewarding ecosystems that treat Angular as a checkbox.
Stop acting grateful for tools that were never truly designed for Angular developers.
Stop lowering the bar for what “Angular-native” should mean.

Start building:

better Angular-first libraries,
signal-based primitives,
tooling that respects Angular architecture,
products that feel like they belong in Angular, not merely tolerate it.

If Angular is good enough for the hardest problems, it’s good enough to deserve first-class tools.

If you are building for Angular, build with standards high enough that nobody has to ask whether it was native.

They should feel it instantly.

Angular Deserves Better

This is bigger than one editor, one library, or one complaint.

It’s about whether Angular developers keep accepting second-class tooling in a first-class framework.

You can read plenty of stories of teams who inherited Angular apps full of awkward wrapper libraries and half-baked integrations.
Once those abstractions land in a codebase, they stay there for years.

That’s why this matters.

The shortcuts you accept at the library level become the nightmares you debug at the product level.

The ecosystem quality tolerated today becomes the architecture debt inherited tomorrow.

Angular deserves better than wrappers.

And if the ecosystem is not going to hand Angular better native tools, then Angular developers will have to build them.

That’s not just a technical decision.

It’s a standards decision.
A community decision.
A respect decision.

So yes — stop treating Angular as a second-class framework for UI components.

And start building like it isn’t one.

I fix the Angular apps that generalists break.

I’m Karol Modelski, senior Angular developer and frontend architect rescuing legacy B2B SaaS frontends.

If your Angular app is slowing your team down, start with a 3‑minute teardown of your current setup: https://www.karol-modelski.scale-sail.io/

Developer experience metrics

Mon, 03 Nov 2025 15:19:44 +0000

Developer experience (DevEx) is a key focus of platform engineering. It's also a crucial success factor for any organization developing software.

A recent paper by Abi Noda, Margaret-Anne Storey, Nicole Forsgren, and Michaela Freiler called DevEx: What actually drives productivity provides a framework for measuring and improving DevEx.

Developer experience

Developers tend to be happier when they can be productive. When a developer experiences friction, it’s a sign that they can’t deliver as much value as they feel they could.

The following can cause poor developer experience:

Working on low-value tasks
Interruptions
Poor tools slowing them down
Unrealistic deadlines

It’s hard to calculate the volume of an irregular object. Archimedes discovered that submerging an object in water provides a simple way to find its volume. Similarly, improving how developers feel and think about their work and its value can increase productivity.

Some benefits of improving developer experience are:

Increased productivity
Higher satisfaction
More engagement
Higher employee retention

Core dimensions

The 3 core dimensions categorize the friction developers experience:

Feedback loops
Cognitive load
Flow state

These dimensions are a valuable model for improvement.

Feedback loops

Software delivery involves many feedback loops:

Build and test times
Code review waits
Manual testing
Real user feedback

Each of these loops needs to be short to reduce friction. Ideally, all feedback loops should complete while the task is still active. If a task pauses during a feedback loop, it will interrupt the following work when it restarts, disrupting flow state and increasing cognitive load.

Organizations are twice as likely to exceed performance goals with short lead times and frequent deployments. Work in smaller batches to get shorter feedback loops.

Cognitive load

Writing and maintaining software involves a large amount of mental processing. This natural cognitive load for the task increases when developers have too many tools and technologies. The architecture of the software can also add to the burden.

You can reduce cognitive load if you:

Reduce toolchain friction
Keep documentation updated
Improve the system’s architecture
Remove delays in the process

Platform engineering is a is a strategy for reducing cognitive overload caused by tools and technologies.

Flow state

You can probably remember when you last felt in the flow. You may have had a good chunk of uninterrupted time and could focus on the problem. The code seemed to write itself and you made great progress. You enjoyed the work.

Psychologist Mihaly Csikszentmihalyi studied the concept. He described it as a fully immersed feeling of energized focus.

Flow happens naturally when you have:

Control over your work structure
Clear goals
Engaging work

It’s not a case of making flow happen but preventing disruption from interruptions and delays.

Studies show developers who enjoy their work perform better and produce higher-quality products.

You can make flow more likely by:

Using asynchronous communication to avoid interruptions
Clustering meetings to create large blocks of meeting-free time
Avoiding unplanned work
Batching questions

DevEx metrics

You can create DevEx metrics by measuring 3 core dimensions with 2 methods:

Perceptions — Gathered through surveys.
Workflows — Gathered from systems.

Many people question the value of perceptual measurements. They think they’re too subjective to be useful. However, as our white paper on metrics explains, they’re a vital reality check.

The whole point of DevEx metrics is to uncover ways to increase the 3 core dimensions. They’re not a measure of developers but for the benefit of developers.

The matrix below shows examples of the metrics you can collect. It’s intended to be a framework, so you can pick and change metrics as needed.

Dimension	Perceptions	Workflows
Feedback loops	How satisfied are you with (build times, test run speed, the time it takes to get a change live)	Build times, code review waits, deployment frequency, lead time for changes
Cognitive load	How complex is (the codebase, debugging the system, understanding the documentation)	The number of manual stages in the deployment pipeline, lead time for technical questions
Flow state	Are you able to (focus on work, avoid interruptions, understand your tasks)	Meeting free blocks, incident frequency, number of unplanned tasks

If you’re already collecting the DORA metrics, they’re helpful here.

DevEx key performance indicators

You must pick key performance indicators (KPIs) to track success. You can base these on the metrics you selected as part of the dimension and method matrix.

Perceived ease of delivering software
Perceived productivity
Employee satisfaction

The KPIs make sure you remember the purpose of measuring developer experience. They also help balance the trade-offs in the improvement process. You should measure the outcomes you want from your efforts.

Summary

Developer productivity measurements are traditionally elusive. However, you can use perceptual and workflow measurements to track and improve feedback loops, cognitive load, and flow state.

Using KPIs to improve these 3 dimensions will get you happier developers who help your organization achieve its goals.

GitOps Makes for Great DevEx, but Pragmatism Matters

Mon, 21 Jul 2025 11:52:53 +0000

At Kubecon Paris, we surveyed expert developers about their modernization initiatives, including what works well and where there’s pain.

KubeCon in Paris was an amazing opportunity for like-minded people to gather together to talk about Kubernetes, GitOps and cloud native computing. We took the opportunity to survey expert developers about their modernization initiatives, including what works well and where there’s pain.

Developer experience (DevEx) is the primary driver for modernization initiatives. More than two-thirds of organizations are moving to cloud native, Kubernetes and GitOps to improve DevEx. Despite these good intentions, there are two common trip hazards that must be addressed:

Developers want pragmatic GitOps, not pure GitOps.
Developers need better CD tools, not just a tool that runs custom scripts.

Developers Want Pragmatic GitOps

GitOps and Kubernetes are natural companions. Compared to virtual machines, Kubernetes has superpowers in container orchestration and dynamic scaling. The ability to automate Kubernetes is also a huge advantage, and GitOps is the natural way to do this.

Pragmatic GitOps stops short of saying everything should happen in version control. For example, a dedicated CD tool will handle your Kubernetes manifest file and handle the deployment process. It will apply the correct configuration for each environment and ensure software versions progress through each environment.

The problem with the pure-GitOps approach is that you end up with YAML files everywhere. Not just two or three files, but thousands. It only takes a few services and environments to generate YAML sprawl, as illustrated below. If you apply different configurations per customer or location, the problem is even worse.

We spoke to Kubernetes experts at KubeCon Paris. We found twice as many advocated for pragmatic GitOps over a pure approach. (Pragmatic GitOps was preferred by 67% of experts vs. 33% who opted for pure GitOps.) As DevOps teams apply GitOps at scale, the need for specialized tooling to handle things like configuration becomes more apparent.

Hammers are great at what they do, but they don’t do everything. Once you’ve experienced YAML sprawl, you’ll double-check your toolbox to see what else you have available for the job.

Developers Want Better Continuous Delivery

We know that DevEx is the №1 driver for modernization initiatives, but developers want better continuous delivery tools.

The current problem stems from the fact that too many teams are using their CI tools to run CD tasks (but CI is not CD). This means they are writing scripts they then store in strings inside of YAML files. CI tools don’t offer features for environment progression, deployment observability and configuration management for environments and tenants.

While these hand-cranked scripts run inside a CI tool, they are actually more like a homegrown scripted deployment solution.

You have likely encountered a shadow IT project at some point in your career, where a business user has created a critical application inside an Excel spreadsheet. These inevitably become urgent rescue projects when their creator moves on or becomes overwhelmed with what they created. Creating a CD solution inside a CI tool is basically do-it-yourself shadow CD.

Almost two-thirds of developers are using a CI tool or all-in-one solution for their CD pipeline, and it’s leading to an explosion in self-managed scripts as a basis for deployments. This is one of the reasons DevEx is a problem to be solved. More than 80% of developers we surveyed told us that CD is the most impactful investment that can be made for DevEx.

As organizations invest in improving DevEx and software delivery performance, an appropriate consideration of CD will be crucial.

Update Good Practices for Success

It has long been held that modern software development is fraught with cognitive overload. When you first pick up Kubernetes, the ecosystem of choices causes decision overload. Once you’re up and running, though, it’s YAML sprawl and DIY shadow CD that are limiting your software delivery and causing pain for developers.

In the “Leader’s Guide to Kubernetes Adoption,” Nikita Dergilev and Bob Walker explain that business goals need to drive your Kubernetes adoption to avoid being deprioritized by future initiatives. There are a number of decisions to make along the way that will benefit from a collaborative standardization process. The two crucial choices you can make are with respect to DIY shadow CD and YAML sprawl.

We’ve moved on from traditional infrastructure because modern cloud native solutions offer so many benefits. But unintentionally, we’ve lost finesse in the way we manage software versions. Updating our good practices to support modern software is critical if we want to solve the problems of cognitive load.

The Quest Continues: Porting the Word Game With AsyncSSH

Sun, 04 May 2025 23:33:09 +0000

Last week, we reimplemented our 20-questions variant, and it received quite a bit of attention. So far we built a web experience, as well as a command line interface for the game. However, unlike the web version, the command line interface is strictly local. There’s no way to share the game with a fellow friend unless we walk them through the setup. Coincidentally, a former coworker recently told me about hosting applications over secure shell protocol in Golang. That makes me wonder, how easily can we replicate it in Python?

A cute illustration from Copilot on the topic.

The Spark of an idea

The secure shell protocol (SSH) is usually what we use to log into another machine for work. Yet, there are a lot of interesting applications hosted over SSH. I remember there was one that would play back a Star Wars movie in ASCII art upon connecting successfully. Late last year, my coworker told me about a coffee shop selling freshly brewed coffee over SSH. We both found that idea intriguing.

He suspected the application was made with wish, a Golang library. That prompted me to see if there is an equivalent for Python, but I only found Paramiko at the time. After a quick glance at the documentation, I decided it was too much work and moved on. It does seem like it is capable of achieving the goal — serving a CLI application over SSH, though.

Yes, the em-dash is intentional.

Photo by Vitaly Gariev on Unsplash

Keeping in touch as usual, and he would bring up the wish project from time-to-time. Then fast-forward to last week, after I was done with the reimplementation of our little word game, I stumbled upon another library called AsyncSSH. Like Paramiko, it can be used to create an SSH server as well. The relatively more extensive examples also seemed helpful.

After some exchanges with Gemini, I received a generated proof-of-concept code that looked workable. After completing and publishing the article on the game, I finally had time to properly explore the topic. Read on to join the ride together in building an interactive application served over SSH.

Finding the Right Tool: Paramiko vs AsyncSSH

Like we briefly mentioned previously, Paramiko and AsyncSSH are libraries we can use to build SSH Server applications (they are also excellent as SSH client libraries). Though the former garnered more stars on GitHub, the amount of information on how to build an SSH Server application is surprisingly sparse. From what I read, it also required quite extensive setup.

The discovery of AsyncSSH was completely unintentional. It was buried deep in the search result when I was looking for a cohesive tutorial on Paramiko. While the design of the documentation site looked as if it was built in Python 2.0 era, the extensive collection of examples is extremely helpful. I particularly liked the simplicity offered by AsyncSSH in comparison.

Photo by Agence Olloweb on Unsplash

Our goal for this week is a fairly simple application sporting a simple REPL-like experience. The key of this exercise is to learn, hence efficiency isn’t a deciding factor. Therefore, the simplicity and developer friendly documentation make the library a sensible choice for our project this week.

Two Paths and a Relearning

Photo by Ross Sneddon on Unsplash

The quick proof-of-concept code I came out with the help of Gemini, by referring to the documentation, consists of 3 components. Firstly, we have the actual SSH Server class, subclassing asyncssh.SSHServer.

import asyncssh

class MySSHServer(asyncssh.SSHServer):
    def begin_auth(self, username):
        logger.info("logging in user:%s", username)
        return False

    def session_requested(self):
        return MyREPLSession()

This class is where we define all the authentication rules. For the purpose of this article, we are skipping all of them to maintain simplicity. Thus, we are returning False in begin_auth to indicate no authentication is needed. Do refer to the documentation for information on the topic before publishing applications in production.

The MyREPLSession is a class inheriting asyncssh.SSHServerSession, which defines the actual application logic we intend to serve over the server. The skeleton is shown as below:

class MyREPLSession(asyncssh.SSHServerSession):
    _chan = None
    _buffer = ""

    def connection_made(self, chan: asyncssh.SSHLineEditorChannel) -> None:
        self._chan = chan

    def pty_requested(
        self,
        term_type: str,
        term_size: tuple[int, int, int, int],
        term_modes: dict[int, int],
    ) -> bool:
        logger.info("pty received")
        return True

    def shell_requested(self) -> bool:
        logger.info("shell received")
        return True

    def data_received(self, data: str, datatype: int | None) -> None:
        # TODO The actual application
        pass

    def eof_received(self):
        assert isinstance(self._chan, asyncssh.SSHLineEditorChannel)

        self._chan.exit(0)
        return False

    def signal_received(self, signal):
        assert isinstance(self._chan, asyncssh.SSHLineEditorChannel)

        self._chan.exit(0)

    def break_received(self, signal):
        assert isinstance(self._chan, asyncssh.SSHLineEditorChannel)

        self._chan.exit(0)

There are a couple of key methods we need to implement. The first is the connection_made, in our setup we are expecting a asyncssh.SSHLineEditorChannel object. All the input and output through SSH would be handled by the channel object, hence it is crucial to keep a reference of it. Methods ending with the _received suffix are event handlers, and in our toy project, we simply close the channel when no more input is sent (upon EOF or on terminate/kill signals).

We will discuss data_received event handler later.

Lastly, we need to tie everything together through asyncssh.create_server. As implied by the library name, the function returns a coroutine object, and hence we would need to properly set up through AsyncIO. Feel free to revisit our previous article on how to write an asynchronous program. But for this example, we are adapting the example shown in the documentation.

AsyncIO Task Management: A Hands-On Scheduler Project

import asyncio

async def main() -> None:
    try:
        # Ensure ssh-host key exists before running
        await asyncssh.create_server(
            MySSHServer,
            "0.0.0.0",
            8022,
            server_host_keys=["./ssh-host"],
            line_editor=True
        )
        print("SSH server listening on port 8022...")
    except Exception as exc:
        print(f"Error starting server: {exc}")

if __name__ == " __main__":
    loop = asyncio.new_event_loop()

    loop.run_until_complete(main())
    loop.run_forever()

We previously used asyncio.run to run the main coroutine, and set up a ShutdownHandler working with an exit event object to end a program. On the other hand, this approach was likely written before asyncio.run was introduced, with the manual management of event loop being the telling sign. The run_forever() here also eliminates the hack where people put an infinite loop after asyncssh.create_server.

Essentially, the code snippet creates the server, where the server behaviour is defined by the MySSHServer, and we set line_editor=True to be explicit (as we are expecting user interaction in our session). Remember to create an SSH key for the application.

$ ssh-keygen -q -N "" -t ed25519 -f ./ssh-host

As mentioned earlier, we are building a very simple REPL-like application. Just as a refresher, REPL stands for read-evaluate-print-loop, which aptly fits our case here. We read user input, get DSPy to process it with the LLM in background, print the generated response, rinse-and-repeat. The skeleton of the flow is shown as below:

def main() -> None:
    game_setup_and_welcome_speech()

    while True:
        match input("> "):
            case command if command == "quit":
                break
            case attempt:
                process_and_print_reply(attempt)

Now we can populate our data_received handler. This method is called whenever our user is sending input through the SSH session. Unfortunately, we cannot assume the data is always ending with a newline (user pressing enter). Sometimes, the handler may have accumulated a couple of questions and send them in one go. The function may also get called while the user hasn’t done typing. For this, we would have to handle both cases, as shown below:

class MyREPLSession(asyncssh.SSHServerSession):
    _buffer = ""

    def data_received(self, data: str, datatype: int | None) -> None:
        assert isinstance(self._chan, asyncssh.SSHLineEditorChannel)

        self._chan.write(f"Welcome to my SSH server, {username}!\n\n")
        self._buffer += data

        lines = self._buffer.split("\n")
        for line in lines[:-1]:
            match line:
                case command if command == "quit":
                    self._chan.exit(0)
                    break

                case attempt if attempt:
                    process_and_print_reply(attempt)

        self._buffer = lines[-1]

Remember we talked about handling user input and output through the SSHLineEditingChannel object we received in connection_made? Output of our application is done through its write() method, just like how we output to stdout (to be displayed on the terminal emulator) through print() usually.

Let’s test the application.

A seemingly stalled game session

While the application worked, the experience wasn’t ideal. My modest GPU took quite a bit of time to generate a response for every user input, and somehow the cursor stalled and wouldn’t move to another line whenever the user pressed enter. It somehow gave an impression of the application just died after every enter.

Photo by Josh Redd on Unsplash

I was ready to move on, and start preparing to write, until I came across another example that fits the use-case. Now we have a code that looked a lot like the original skeleton.

async def handle_client(process: asyncssh.SSHServerProcess) -> None:
    username = process.get_extra_info("username")
    process.stdout.write(f"Welcome to my SSH server, {username}!\n\n")

    game_setup_and_welcome_speech()

    while True:
        match await process.stdin.readline():
            case command if command == "quit":
                break

            case attempt:
                process_and_print_reply(attempt)

    process.exit(0)

Previously we returned output through the SSHLineEditingChannel object, here we are doing it through the SSHServerProcess object. On the other hand, instead of input(), we read it through the readline() method in the stdin (an SSHReader object) property.

Then, we need to remove the session_requested method from MySSHServer, and define the process_factory argument in asyncssh.create_server to point to our new coroutine.

await asyncssh.create_server(
    MySSHServer,
    "0.0.0.0",
    8022,
    server_host_keys=["./ssh-host"],
    line_editor=True,
    process_factory=handle_client,
)

Re-run the application again, and to my surprise, it worked almost exactly like the original CLI version. For our project, the second approach definitely worked better. Nonetheless, the first attempt was also crucial, as it gives an idea on how input is being sent in a SSH session.

The choice of naming the property holding a SSHReader object as stdin, and the fact it reads input through a method named readline piqued my curiosity. I knew we could reach sys.stdin to read content redirected from the previous command (via the pipe | operator), or file content via the < operator (in Unix-like environments). That said, I didn’t know reading user input interactively in a running application is done through stdin as well.

My first experience writing a production CLI application started with PHP (very strange, yes). That was my first time learning about the relationship between the operators and stdin, stdout, and stderr through my supervisor. The scripts didn’t really need to collect input interactively, and given the similarity of PHP and C, I couldn’t find a scanf equivalent so I assumed PHP doesn’t have such feature.

I was wrong.

Photo by Mr. Bochelly on Unsplash

After some exchanges with Gemini, I realized I could use readline() in PHP for the task. Which itself is similar to scanf and input in Python. Alternatively, I could also use fgets(STDIN) to do the job, just that I need to figure out a way to print a prompt prior to the input collection. On the other hand, if I redirect some content into a Python application, if I call input(), it would read the content in stdin, until it encounters a newline character. Essentially, it roughly equates

import sys

sys.stdin.readline().strip()

Back to the topic, the toy example skipped many other aspects besides the authentication work we mentioned earlier. Unfortunately, due to the sheer scope and complexity, it is almost impossible for the developer to cover all use cases in the examples. Fortunately, the API documentation is very well-written, and I assume it could be useful for people who know more about the protocol.

We are lucky to live in an era where a LLM chatbot like Gemini is readily available, to provide help in navigating the documentation. Besides that, it also serves as a valuable companion to help gathering complementary information, like how I relearned stdin. Even so, critical thinking is still crucial while evaluating the response to avoid falling victim to AI hallucination.

Bridging Game and Server

Photo by Phil Hearing on Unsplash

We finally assembled all the pieces together and ported the game to be served over SSH. The concrete implementation can be found at the GitHub repository for those who are interested. Both the CLI version and SSH version (the handle_client variant) are very similar, as shown below

async def handle_client(process: asyncssh.SSHServerProcess) -> None:
    username = process.get_extra_info("username")
    process.stdout.write(f"Welcome to my SSH server, {username}!\n\n") # vs print()

    game_setup_and_welcome_speech()
    while True:
        match await process.stdin.readline(): # vs input()
            case command if command == "quit":
                break
            case command if command == "end" and answer is not None:
                answer = None
            case attempt:
                process_and_print_reply(attempt)

    process.exit(0)

The major difference between the two, is the use of process.stdout.write() to replace print(), and process.stdin.readline() to replace input (though a new coroutine called prompt_input is written to compose a process.stdout.write() to display prompt text).

async def prompt_input(process: asyncssh.SSHServerProcess, prompt: str) -> str:
    process.stdout.write(prompt)
    return await process.stdin.readline()

The REPL-like experience makes the word game a very synchronous experience. A response is only generated upon user submission of input. Therefore the await is necessary while expecting user input is needed to ensure order. Other than that, the asynchronous nature of AsyncSSH does not affect the game’s logic much. As each game session is independent, each time a new user connects, the library will run handle_client separately for each of them.

Once the log message shows “SSH server is listening …” after executing ssh.py, it is ready to be connected to with the following command.

$ ssh -p 8022 localhost

Considering we didn’t define any authentication methods in MySSHServer (which is definitely a bad practice), it should display a welcome message and the game is started right away.

In this article, we explored two different strategies to host our little word game over SSH. The two approaches have their own pros and cons, but it does show how AsyncSSH makes the development experience quite intuitive. Even though I do not have much experience doing this in Golang with wish (the documentation situation is also quite bad there), I enjoy the time spent doing this with AsyncSSH.

The Game Goes Live

Photo by Alessandro Erbetta on Unsplash

The point of the exercise is mainly on learning and discovery. If our game were developed with a text-based UI through Textual, or even Typer, some extra work is needed to ensure I/O is redirected properly.

The experience of porting the library to SSH is a series of happy coincidences. It started with a spark from a discussion with my former coworker, and it led to a series of discoveries. This article is a culmination of the whole journey, sharing what I learned throughout. It turns out building an application to be served over SSH just needed a change in how I/O is done. Gemini also helped immensely throughout, deepening my understanding.

We went through the whole process by configuring a server by subclassing a asyncssh.SSHServer, exploring two different ways to serve applications through process_factory and inheriting asyncssh.SSHServerSession, then getting it running via asyncssh.create_server.

Hopefully this article sparks an interest and we see more interesting applications hosted over SSH.

This article was written with assistance from Gemini, a large language model by Google. Gemini provided editorial help, fixing language errors and checking for article flow. However, the code and voice are entirely my own. If you’d like to collaborate or discuss job opportunities, please connect with me on LinkedIn or via Medium.