All six of these agents write a Terraform file that passes tofu validate. They all write Kubernetes manifests that survive a server-side dry run, an nginx config that reloads, and a multi-stage Dockerfile that builds and answers a health check. So on well-specified DevOps work, “does it produce correct output” stopped being the question that separates them.
The question that is left is what a good AI coding agent for DevOps costs you per task, how long it makes you wait, and how clean the output is before a human touches it. To answer that we put Claude Code, Codex CLI, Gemini CLI, opencode, Aider and Cline on the same seven tasks and graded every answer by running it, not by reading it. This is a different cut than our opencode DevOps walkthrough or the opencode vs Claude Code vs Cursor comparison: here every tool runs the same prompts and the output goes through a validator. Everything below was run in September 2026 on Ubuntu 26.04, with Claude Code 2.1.278, Codex CLI 0.155.1, Gemini CLI 0.60.0, opencode 1.18.31, Aider 0.86.2 and Cline 0.0.13, each pointed at its default model.
The scoreboard for AI coding agents on DevOps tasks
Short version: pick on cost, speed and how much lint cleanup you want to do, because the correctness column is a wall of green. The four agents that default to Claude Sonnet (Claude Code, opencode, Aider, Cline) and the two on their own models (Codex on gpt-6-astra, Gemini CLI on Gemini 3.5-flash) all cleared the bar on every task type.
Here is the full board. Cost per task is real spend, but only Claude Code and Aider expose per-call token cost in headless mode, so the other four read as “not exposed” rather than free. Gemini ran a single pass on five of the seven tasks because its free-tier key hit a hard daily cap partway through, which is a finding in its own right and comes up again below.
| Agent | Default model | Tasks solved | Avg time/task | Cost/task | Prod lint clean |
|---|---|---|---|---|---|
| Claude Code | Claude Sonnet | 7/7 | 7s | $0.031 | Yes (2/2) |
| Codex CLI | gpt-6-astra | 7/7 | 14s | not exposed | Yes (2/2) |
| Gemini CLI | Gemini 3.5-flash | 7/7 | 77s | not exposed | No (0/1) |
| opencode | Claude Sonnet | 7/7 | 14s | not exposed | Yes (2/2) |
| Aider | Claude Sonnet | 7/7 | 12s | $0.005 | Partial (1/2) |
| Cline | Claude Sonnet | 7/7 | 11s | not exposed | Partial (1/2) |
Every agent solved all seven task types. Two runs per task, with a single run for Gemini on five of them because its free-tier key hit a daily cap, gave five agents a clean sweep. Aider passed 13 of its 14 task-runs, losing one to an awk one-liner that threw a runtime error on its first pass. That is the shape of the whole test: correctness is nearly flat, so the interesting numbers are the ones next to it.
How we graded: execution, not vibes
Every answer was scored by feeding it to the tool that would reject it in a real pipeline. Nothing was graded by eye. Each agent got the same prompt, produced a single artifact, and that artifact went straight into a validator:
- Terraform:
tofu init -backend=falsethentofu validate. - Kubernetes:
kubeconform -strictfor schema, thenkubectl apply --dry-run=serveragainst a live kind cluster so the API server itself validates it. - Ansible:
ansible-playbook --syntax-checkfor correctness, thenansible-lintat the production profile for quality. - GitHub Actions:
actionlint. - Dockerfile:
hadolint, then a realdocker build, run the image, and curl its health endpoint. - nginx incident: drop the fix into
conf.dand runnginx -t. - Observability: run the shell one-liner against a fixed sample and diff the output against the known-correct answer.
This is the part a scraped listicle cannot fake, and it is the same idea behind execution benchmarks like Terminal-Bench: the grader runs what the model wrote. A single grading pass looks like this, with the real pass and fail rows the agents earned.
Two runs per task caught things a single run would have hidden. An artifact that looks right and an artifact that runs are not the same claim, and the gap between them is exactly where these agents differ.
Task by task: what each agent actually produced
Terraform, Kubernetes and GitHub Actions were table stakes
All six produced a single-file AWS VPC with public and private subnets, a NAT gateway, route tables and the right outputs, pinned to the version 5 provider, and every one passed validate. The only stylistic split was that Aider left the provider block empty and let the region come from the environment, which validates fine. On Kubernetes the bar was higher and they still cleared it. Claude Code’s manifest is representative: a full non-root security context, both probes, and a v2 autoscaler.
securityContext:
runAsNonRoot: true
runAsUser: 10001
readOnlyRootFilesystem: true
allowPrivilegeEscalation: false
---
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscalerThe GitHub Actions workflow was the same story. Every agent wired up the Go build-and-test matrix over two versions, a separate lint job, and a deploy job gated to pushes on main. Codex was typical, including the workflow-level permissions block the prompt asked for.
permissions:
contents: read
...
deploy:
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
needs: [test, lint]The Dockerfile that everyone got right
The Dockerfile task hides a classic trap. The prompt asks for a HEALTHCHECK that curls the app, and python:3.13-slim ships without curl, so an agent that forgets to install it builds a green image whose health check can never pass. All six installed curl in the final stage, ran pip only in the builder, and dropped to a non-root UID. Every image built, started, and answered the health endpoint on the first try.
docker build PASS 63MB
docker run PASS {"status":"ok"}Image sizes clustered tightly at 63 MB, with one Cline build coming in heavier at 79 MB because it kept an extra apt layer. Nothing here would fail a review, which is the point: the modern agents have internalized the multi-stage, non-root, slim-base pattern.
Where they diverged: one awk line, six answers
The observability task asked for one shell command that reads node_exporter text and prints the three fullest real filesystems as integer percentages, excluding tmpfs, overlay and squashfs. Six agents, six genuinely different solutions to identical output. Codex built the most portable one, splitting on {}, and keying by device plus mountpoint so two mounts never collide.
awk -F '[{},]' '
/^node_filesystem_(size|avail)_bytes\{/ {
for (i = 2; i < NF; i++) { p = index($i, "="); label[substr($i,1,p-1)] = substr($i,p+2,length($i)-p-2) }
if (label["fstype"] ~ /^(tmpfs|overlay|squashfs)$/) next
...
}'Claude Code and opencode leaned on gawk’s three-argument match(), which is a GNU extension that would break on mawk. Aider produced the same answer twice in structure but not in reliability, and that is the one failure worth dwelling on.
The failure only running it would catch: “awk: division by zero”
Aider’s first observability run used an unanchored pattern, /node_filesystem_size_bytes/, which also matched the # HELP and # TYPE comment lines that carry the same metric name. Those comment lines poisoned the arrays with a zero size, and the division blew up.
awk: cmd. line:7: (FILENAME=- FNR=18) fatal: division by zero attempted
/data 80%It printed one line instead of three and exited with an error. The second run anchored the pattern with /^node_filesystem_size_bytes\{/ and worked perfectly. Same prompt, same model, one broken answer and one clean one. A reviewer skimming the code would likely have approved the broken version, because it reads correctly. Only executing it against real input exposes the bug, which is the entire argument for grading agents this way.
The nginx incident: everyone found both bugs
The broken reverse-proxy config had two faults: a misspelled proxy_set_headers directive and a missing semicolon on proxy_read_timeout. The prompt hinted at “more than one” error. All six fixed both and produced a config that passes nginx -t. On a tightly scoped debugging task with a clear failure signal, these tools are reliable.
Same model, a very different bill
Four of the six agents run on Claude Sonnet by default, which turns them into a clean control: any difference between them is the scaffold, not the model. On correctness the scaffold barely mattered. On cost it mattered a lot. Claude Code and Aider both drove Sonnet, and Claude Code cost about six times as much per task.
That gap is scaffold overhead: the system prompt, the tool schemas and the repo map that Claude Code sends on every turn. Aider sends a lean prompt and the target file. The gap widens on the smallest tasks, from roughly 4x on Terraform to nearly 10x on the tiny nginx fix, because the fixed overhead swamps a short job. If your agent runs thousands of small edits in CI, that multiplier is a real line item, and it is paid on the identical model producing near-identical results.
Speed is the other axis where the scaffold shows. Claude Code was fastest at about 7 seconds per task, the rest of the Sonnet pack landed between 11 and 14 seconds, and Gemini CLI was the outlier at 77 seconds, most of it agent orchestration rather than model latency.
The quality tie-breakers
When everything passes, the tie-breaker is what an experienced reviewer would still flag. The clearest signal came from ansible-lint’s production profile. Every playbook passed --syntax-check, but the strict profile is where habits show. Claude Code, Codex and opencode were clean on both runs. Aider and Cline tripped it on one run each, and Gemini tripped it on its single run.
The failures were almost all the same rule, fqcn, using bare module names like apt instead of the fully qualified ansible.builtin.apt that a production codebase expects, plus a few yaml[truthy] hits for writing yes instead of true. Aider’s one slip was simpler, a play with no name. None of it breaks the play. All of it is the kind of thing a linter blocks in CI, so the agents that emit it hand you cleanup work.
Run-to-run consistency told the same story. Claude Code, Codex and opencode returned production-grade output on both passes of every task. Aider and Cline each had one run that slipped, whether a lint miss or the awk error, and Gemini’s daily-cap ceiling meant less coverage overall. If you are wiring an agent into an unattended pipeline, that consistency is worth more than a few cents of savings.
The Gemini cap deserves a flag on its own. The free-tier key defaults to gemini-3.5-flash, and its own quota error capped the run at 20 generate-content requests per day, which this modest benchmark exhausted. For anything past a handful of calls a day, Gemini CLI needs a billed key, which changes the cost math the table above cannot show.
Which agent to run for DevOps work
There is no single winner, because the tools split cleanly by job. Match the agent to the workload:
- Interactive, mixed infrastructure work: Claude Code. Fastest, cleanest output on both runs, and the richest surface. You pay for it, so keep it for the sessions where the speed and polish earn their cost.
- High-volume automated edits in CI: Aider. Same Sonnet output at a fraction of the cost, scriptable, and it reports its own spend. Pin the model settings and expect the occasional flaky run, so gate it behind the same validators we used here.
- Production Ansible and Kubernetes without cleanup: Codex CLI or opencode. Both were lint-clean on every run, and opencode stays model-agnostic if you want to swap the backend.
- Already in the VS Code and Cline habit: Cline is solid on correctness. Add a lint gate for the runs that drift.
- Gemini CLI: capable on the work itself, but budget for a billed key and the slower turnaround before you lean on it for DevOps.
The real takeaway is smaller than a leaderboard and more useful. The model these agents wrap has made basic DevOps output a solved problem, so choose on cost, speed and lint hygiene, and put a validator in front of whatever you pick, because the one bug in this whole run was invisible until something ran it.
