Coder Legion - Recent posts tagged opinion

Staying vigilant in the AI coding gold rush: from generation to delivery

Fri, 12 Jun 2026 22:12:23 +0000

Being developers and managers with boots on the ground, we owe it to our customers and users to be the voice of reason and stability when deploying AI-generated assets. Despite what is being asked of us from leadership.

For those who haven't worked inside the corporate world, the continuous tango of "look at metrics, adjust; look at metrics, adjust" is a never-ending cycle that we, as mid-level managers and directors, must both endure and champion for our reports. These metrics can be used to analyze almost anything about our personal performance and that of our teams. As the organizations we work for grow larger and more complex, context and awareness (of the application and situation) usually diminish. This awareness is replaced by a focus on delivering features and meeting financial and velocity goals.

What's ironic is "context" is one of the pillars that quality AI prompts and commands are built on. The more it knows, the better the result.

You might be asking yourself, "What metrics am I talking about?" I'm referring to productivity metrics, DORA metrics, but more specifically: lines of code generated. It would seem logical to equate lines of code written with productivity, right? The more code written, the more is delivered to the customer—making the application better, allowing more features to be sold, which means more money. Again, on the surface this makes sense and has been a point of emphasis in my experience. But the more I think about it, just because we are writing more code with AI are we actually more productive? I am not entirely convinced.

Let's dive into it.

A quick story

Recently I have attended a series of management meetings where the subject of AI metrics and its usage have been a topic of discussion. Further debates about which models work in which situation, and token-spend among others dominate the forum. Likely a pretty standard "corporate AI" meeting. However one point of discussion always seems to surface again and again: usage. The overwhelming common sentiment between each meeting is the ever-increasing lines of code being generated by our agents. The accompanying narrative being paired with this analysis is:

Our teams have never been more productive. We are writing more code than we ever have and we are shipping more features than any other point in the company's history.

Sounds like a pretty successful and celebratory conclusion eh? Initially I thought so. Code makes our applications function, functions create features, features means more money; so more code must be good right? But the more I got to thinking, there are other elements that go into code and features. Things like code reviews, security audits, unit tests, and so on. Technically you could use AI to perform these tasks, but (other than the unit tests) you really shouldn't. These things take time, more time to do it right, and even more time when you include multiple people in the process.

I recognize I am just one manager/developer, one lowly manager/developer. I also recognize that my opinion is just one small grain of sand in the desert that is the AI efficiency discussion. But I cannot help my skepticism of these MASSIVE gains surrounding the deployment of AI in software development practices.

More isn't always better

As a small disclaimer, I am basing a bulk of my argument with the assumption your company is following standard development processes, like writing unit tests, performing code reviews and such. If you aren't doing these things and just deploying AI-generated code to production without even the slightest review, then God's speed to you. Hope you have good insurance when your data gets ransomed.

Now let's continue.

When we leverage AI to rapidly generate code, it's hard to not be in awe of the speed and accuracy it is able to do so. Some outlets are reporting developers are seeing as much as a 55% percent increase of productivity, with a reduction of lead time for tasks by as much as half. More often than not, the code that is generated is pretty accurate and clean; especially if your agent guidelines are clear and actionable. The magic of typing a prompt and getting a working output will inevitably lull you into this result-driven comatose. It certainly has for me. The ease and luxury provided to you by your coding agent of choice makes it very easy to simply bypass the processes and protocols you would normally follow for your own code: unit tests, user-acceptance testing, quality assurance, security analysis, and so on. If the code works, why would I change it and why would I test it?

The "mechanism of structural decay"

If you have worked with AI long enough, you are likely to notice the trend of: write, create, write, append, write, refactor. The agent's immediate response to a prompt is to write code, regardless if you need it to or not. According to a study done by GitClear and Sonar citing:

an 8-fold increase in the frequency of code blocks containing five or more duplicated lines, confirming a significant decline in code reuse. Furthermore, 2024 was the first year where the number of copy/pasted lines exceeded the number of moved (refactored) lines.

What this crazy statistic ultimately points to is ever-growing code bases that are discarding the DRY (Do not Repeat Yourself) principles that were drilled into us as junior developers. What this also means is our code is growing larger and thus reducing efficiency and long-term maintainability. This quote from Sonar's article says it beautifully, "structural debt arises because LLMs prioritize local functional correctness over global architectural coherence and long-term maintainability." So despite the numerous additions and augmentations to our code, we could be only making it worse and thus damning our application to the pits of volatility.

Instability at the cost of insatiable productivity

According to a Google 2025 DORA Report, the folks over at Google found a 90% increase in AI-generated code also brought about a 9% increase in reported bugs and over a 150% increase in pull/merge request size. If you take these numbers at face value, there are a few big things that should jump out at you:

Our code bases are growing at an alarming rate. I recently reviewed a merge request in GitLab that included 388 lines of net new code (not changes or deletions). Let's assume the developer of this merge request does this four more times over the course of a week. In a month, this means a single developer will have introduced 13,580 net new lines of code into the project, in a single month. If that amount of new code is being written and merged into the repository at that rate, I would have some genuine concerns over the amount of duplicated code, and code that is just unnecessary.

The number of bugs will grow as well. On the surface, a 9% increase of reported bugs does sound like a lot, but I believe it is so much worse than that. All developers know all bugs are not created equal. They require time to analyze and troubleshoot, and even more time to fix regardless if you use AI or not. As the code base grows, the introduction of bugs is inevitable no matter the amount of testing or QA is done to prevent them (unless your company's name is Linear. Those guys know how to produce a clean product; props). But I digress.

Let's go back to the example we were using above. Let's assume for every three merge requests, one bug is introduced. In addition to this, another bug is introduced for every 5,000 lines of code. This means, keeping to the cadence of this developer's AI-powered productivity, in one month, our developer would have introduced around eight bugs. Now, as I was hinting at earlier, these issues could be very simple and depending your CI/CD process, a resolution could be deployed in a matter of minutes. No harm, no foul right? However some of these issues could be debilitating outages, causing lost progress and lost revenue for your business; all from a single developer.

Please know, the numbers that I used above are not actual numbers from me or any of my team. They are simply just examples.

This code needs to be tested and reviewed. If you are following prototypical development processes, we are probably aligned in the fact of code should be tested by both a human (QA) and a machine (automated unit/feature testing). All of these things takes time, even testing that has been automated. Depending on the size of the code base and testing coverage you have for your application, these tests take time to run. And if you are running these tests on dedicated resources, they also cost money. Code reviews by humans also cost money. Yes, it takes time for another developer to review, test, and smoke check the code. This has a cost. But what people don't think about is the productivity that is also lost due to that developer not coding. So in essence, every code review costs twice of what a developer costs during that same timeframe.

The cost of doing it right

For those of us in the tech and software industry, you are likely aware of the meme: "You can have it good, fast, or cheap, but it cannot be all three." Quality, at speed, comes with a cost. This also applies to testing, but even more so when it comes to AI generated code.

A 2025 study showed that senior engineers spend an average of 4.3 minutes evaluating AI-generated suggestions, versus 1.2 minutes for code written by humans. Meanwhile, teams that rely heavily on AI are delivering substantially more pull requests. Faros AI examined data from over 10,000 developers and found a 98% rise in PR volume. Consequently, PR review time increased by 91%, even though the code generation process itself became faster. A Qodo survey found that 68% of senior engineers report quality improvements from AI, but only 26% would ship AI-generated code without review (I'm surprised its that high).

So lets play this out. Lets assume you have a development team that makes the following hourly rates:

Junior/intermediate developers: about $50 per hour (or $0.83 per minute)
Mid-level developers: about $100 per hour (or $1.67 per minute)
Senior/lead developers or architects: about $180 per hour (or $3.00 per minute)

If you were to have these developers review human code at their hourly rate it would cost:

Junior/intermediate developers: about $1 per code review
Mid-level developers: about: $2 per code review
Senior/lead developers or architects: $3.60 per code review

If these same developers were to review AI-written code at their same hourly rate, it would cost:

Junior/intermediate developers: about $3.57 per code review
Mid-level developers: about: $7.01 per code review
Senior/lead developers or architects: $15.48 per code review

It is worth mentioning these numbers do not account for the lost productivity of the code reviewer as well, much like I mentioned above. Another item that is worth considering is the time added to timelines and lead times for features that can increase your company's bottom line. You will also need to keep an eye on the code bloat and potential regressions, as these can obviously affect these numbers too.

When you think about it...

Keep in mind, the numbers that I have provided are very much an approximation and estimate of your team's productivity. There are a fair amount of other caveats like language, code size, complexity, current technical debt, etc. But I think you get my point when you start to think of productivity and output in dollars and cents. As I also mentioned above, I myself have seen the benefits in my own work and undoubtedly you have as well. But with the security risks, bloat, and code duplication AI usage can introduce, AND you combine that with the increased code review time and associated costs with the potential delays and regressions; is it worth it?

As leaders, we have a direct responsibility to make decisions, and to put our teams into the best possible places/opportunities to succeed. This includes selecting tools and services to use. However when we are being pushed by our bosses and their bosses to use AI to optimize our work, I believe we owe it to our reports and the companies that we represent to at least take a second and really think about the outcomes of our decisions. We are very likely to not see the long-term effects for quite some time, but months and even years from now, are our AI-driven decisions going to really make our work better? Probably, but I am highly skeptical it will be this "slam dunk" we are being told it will be.

To put a bow on it

The AI coding gold rush is real — but so are the risks of mistaking generation for delivery. As organizations scale, context and awareness often get compressed into simplistic metrics. Lines of code and raw output from AI tools can feel like progress, but they are a poor proxy for actual customer value, reliability, and long-term maintainability.

To stay vigilant, treat AI as an amplifier, not a replacement, for disciplined software delivery. Anchor decisions in outcome-oriented metrics (customer impact, lead time, MTTR, change-failure rate), maintain strong feedback loops, and preserve human oversight across design, review, and deployment. Invest in observability, automated testing, security scanning, and provenance tracking so generated code is verifiable and safe to ship. Foster a culture that rewards quality, learning, and shared context—document assumptions, keep prompts and models auditable, and run blameless postmortems when things go wrong.

Practical steps:

Prioritize outcome metrics over LOC; use DORA metrics thoughtfully and complement them with product and user metrics.
Require tests, code review, and CI/CD for all AI-generated code before it reaches production.
Capture context: rich prompts, architectural notes, and intent annotations so AI output is traceable and maintainable.
Automate safety checks (security, license, performance) and monitor runtime behavior post-deploy.
Train teams on effective prompting, model limits, and how to validate AI outputs.

In short: harness AI to accelerate engineering, but measure and manage what actually matters—delivery, quality, and customer value. Vigilance, not hype, will determine who benefits from this gold rush.

The AI Content Invasion: AI-Generated Articles: Helpful Tool or Digital Ghost?

Thu, 30 Apr 2026 02:47:24 +0000

Let's talk about theelephant in the room. You've probably noticed it yourself. More and more articles on coding platforms read like they were written by a machine. Because, well, they were.

But here's the thing: this conversation is way more nuanced than "AI bad, human good." Let's dig into what's really happening.

Why AI ContenExploded?

Developers are busy people. We're shipping code, fixing bugs, attending standups that should have been emails. Writing articles takes time. A lot of time.

AI tools promised a shortcut. Need a quick tutorial? Generate it in seconds. Want to cover a trending topic? Let the machine handle it. Platforms like CoderLegion saw a surge in content, which is great for readers looking for answers. More articles, more solutions, right?

Well, sometimes!

The Honest Truth About AI Writing

Here's what AI does really well. It can explain well-documented concepts clearly. It can structure an article logically. It can generate basic code examples that work for simple use cases.

But here's where it falls apart. AI doesn't know what it feels like to debug a production outage at 2 AM. It never spent three days pulling its hair out over a weird caching bug. It can't share that moment of pure joy when you finally figure out why your tests were failing.

And that matters. Because those human experiences are what make technical articles actually helpful, not just technically accurate.

The Quality Problem Nobody Wants to Talk About

Let's get real. AI-generated code has about 1.7 times more issues than human-written code. When that code ends up in tutorials, it creates a chain reaction. Developers copy it, use it, and then wonder why things break in weird edge cases.

I've seen it happen. A junior dev follows an AI-generated tutorial step by step. Everything works on localhost. Then they push to production and everything falls apart. Why? Because the tutorial missed the nuance that only comes from real-world experience.

How This Changes Us as Readers

We're all becoming detectives now. We scan articles looking for clues that a real human wrote them. Personal stories. Specific mistakes. Those little confessions like "this approach failed for me initially" that prove someone actually tried the thing they're teaching.

Our trust radar is getting sharper. We can spot the AI tells: explanations that are technically correct but practically useless. Code examples that look perfect but would never survive contact with real users. Articles that answer the "what" but completely miss the why?

The Sweet Spot: Human + Machine = ...

The best content I've read lately isn't purely human or purely AI. It's a collaboration. Smart writers use AI to handle the boring parts: outlining, formatting, generating boilerplate examples. Then they add the magic that machines can't replicate.

Their own experience. Their own voice. Their own hard-won lessons from the trenches.

Platforms like CoderLegion have an opportunity here. They can be the place where this human-AI collaboration shines. Where articles are efficient but also authentic. Where readers get the best of both worlds.

What This Means For Writers

If you're writing technical content, your lived experience is your superpower. That bug that took you a week to fix? Write about it. That architecture decision that seemed wrong but turned out right? Share it.
AI can explain how a for loop works. It cannot explain why your team chose a particular state management solution after months of debate. That second thing? That's what readers actually want.

Don't try to compete with AI on speed or volume. Compete on depth and authenticity. Share your failures alongside your successes. Be specific about what worked and what didn't. Let your personality come through.

What This Means for Readers

Stay curious, but skeptical. When you read an article, ask yourself: does this include real-world insights? Does the author share specific experiences or just generic explanations? Would this actually help me solve a problem I'm facing?

The best articles will make you think, "This person has been where I am." That's something no AI can fake.

Moving Forward

AI-generated content isn't going anywhere. It's getting better, faster, and more convincing every day. But that's not a threat. It's an invitation.

An invitation to double down on what makes human writing valuable. To share our real experiences. To build trust through authenticity. To create content that doesn't just inform but actually helps.

The future of technical writing isn't human versus machine. It's humans using machines to be more helpful, more honest, and more human than ever.

What do you think? Have you noticed the shift toward AI content? Do you find it helpful or hollow? I'd love to hear your take in the comments.

AI is an industry-wide tech debt

Mon, 13 Apr 2026 20:21:33 +0000

Many years ago, I watched a talk about tech debt. The presenter argued about what counts as a tech debt and included any third-party dependency to the mix. His argument was the following - tech debt is created when we cut corners to deliver faster. If you adopt a library - a piece of code what you could write yourself, you do so to save time. You do so at the cost of a potential future liability. You often use open-source libraries where vulnerabilities get discovered and exploited. The maintainer might abandon the project, or they might do a licencing rug pull if they feel like everybody is using their work, and not giving them a dime.

I argue that AI is no different in this regard. It lets you rapidly "develop" code to meet business demands. However, there are fundamental limits to the current AI technology, limits that are very hard to circumvent, unless the technology itself changes:

data set poisoning - you cannot prove that the model you are using wasn't poisoned through a supply-chain attack on the learning infrastructure. The model can be trained to react to particular kinds of prompts to install backdoors or vulnerabilities into your code. Unless you know how to implement stuff properly, you have no way of validating the code.
lack of negative feedback loop - if your current business requirements contradict the previous business requirements, AI will happily oblige and silently break some edge case. Ask yourself - how many times have you tried to implement a feature, only to find out that trade-offs need to be made, or to find out that the requirements were poorly specified? AI won't tell you, or at the very least it won't tell you unless you ask. They only way to be sure is to implement the thing yourself.
inability to be consistent - somewhere I read that it is a good thing to first instruct the bot to formulate a plan of implementation, wait for approval, then carry on with the implementation. Claude Opus managed to follow this instruction for the first two requests and completely ignored it for the third time. Internet is full of stories where the bot suddently went haywire and went against the requirements in the prompt.
inability to be original - you tell AI to implement something. Do you believe it did it in the most safe and/or performant way possible? Think again. It wasn't trained on the "best practice" corpus (such thing doesn't exist), it was trained on the "common practice" one. It will give you mediocre, average outputs, not the great ones. Unless you know how to do it correctly, you have no way of telling.

They say that the definition of insanity is doing the same thing over
and over and expecting a different result. In that case, AI is
insanity.

All the techbros, CEOs, venture capitalists, opportunists, and influencers are trying to convince you that if you don't use AI, you're useless, antiquated, slow.

Sure, not using AI at all would be an extreme overreaction. AI can supercharge learning, analysis, and prototyping, if (emphasis on if) it is used responsibly. But many people don't use it responsibly. Agent AI which can do stuff "without any supervision" can also screw up anything.

What I am trying to say is - If you don't trust your junior devs and do code reviews after them, why would you blindly trusted something a machine randomly created?

If you are a developer

Use AI responsibly. I for example either use it to generate throwaway code, or I always manually refactor the code myself, just to force me to think about every single line. Just today, I was touching some CI pipelines and encountered a very long condition line in the YAML file.

I asked the bot if the line can be somehow pretty formatted. It obliged and used this weird >- operator. I asked back what it does. I asked for sources, and I went to those sources to study the operator. I don't want to be a passive prompter, I want to understand the code. Next time, I don't have to waste my time prompting the AI, I can just format it myself..

Generating scaffolding code is on the edge. I bet that you could write a deterministic tool that could do the scaffolding equally well, without fearing the bot just randomly screwing you over. If you let AI write the tests - do you read and reason about every single test case? Do you refactor the code so it doesn't just vomit tokens that would inflate your bills the next time you want to add a test case?

But for the love of god, if you let the bot generate production code and you don't even know what it produced - you are just tying a gun to your dick and giving somebody else the ability to pull the trigger.

If you are a manager

I know that most CEOs are just jacking off stakeholders and don't really care about the people they employ, but if you are a middle-to-bottom manager, you should care about your people. You should care about what drives them. And the reality is that most of those nerds are coding because they love the act of coding and they hate delegating the work and explaining the business requirements. So why would you force them into doing exactly that with AI? You're just going to kill their motivation.

And for those who are diving head first into the AI-driven submarine, keep an eye on them. Us humans are a very lazy bunch. The moment we are allowed to stop thinking for ourselves is the moment we would sell our souls to the devil. Why do you think there are so many autocracies around the world? People are lazy. People like when somebody else thinks for them. People only want change when the emperor makes their lives miserable.

I honestly fear that people who unapologetically like to use AI and see agentic development as something good will grow dumb. There are already studies out there saying that cognitive functions deteriorate when you rely on AI (and most of the facilities on the modern internet).

Also, do hire juniors

Anybody who refuses to hire junior developers because AI can replace them is a retard. Sorry, not sorry. Experienced developers don't grow on trees. They grow on battlefields. They learned by writing code. They learned by negotiating with customers, and product managers, and CTOs. If you refuse to hire juniors and teach them, you are creating a tech debt worse than AI-generated code. You are contributing to the lack of experienced developers down the line and I hope your business will bankrupt hard (I just pity people who will stick with you up until that point).

Also - maybe consider forbidding your juniors from using AI to generate code. Learning? Okay. Understanding your code base? Great. Writing boring documentation? Abso-fucking-lutely. But no code generation. They need to experience it for themselves. They need to learn why patterns exists, how to refactor code and how to notice code smells. If they don't learn it the hard way, they won't be able to direct anybody, let alone an agentic chatbot.

Feel free to tell me how backward-thinking loser I am down in the comments below.

From Simplicity to Complexity: My Love-Hate Journey with Next.js

Tue, 09 Sep 2025 07:37:16 +0000

I remember the day I started using Next.js like it was yesterday. It was version 8. And honestly? It was love at first sight. I bootstraped our new project in Nextjs despite my CTOs hesitation and made my teammates who never wrote a single line of code in React, learn Nextjs and ship with confidence.

File-based routing.
SSR. SSG. ISR.
Zero config.
You slapped a file in the pages/ folder, and boom—you had a route.
Simple, elegant, almost… magical.

// pages/about.js
export default function About() {
  return <div>About us</div>
}

It did what React should have done out of the box but didn’t. I was sold.
Deploy it, forget about webpack configs, no client-side routing hacks. Just productivity.

The Shift: Enter Next.js 13 & The App Router Madness

Then Next.js 13 hit, and things changed.
Suddenly, pages/ was optional. Enter the app/ directory. Layouts. Templates. Nested routing. React Server Components. use client directives.

It felt… like betrayal.

// app/layout.js
export default function Layout({ children }) {
  return <div className="layout">{children}</div>
}

Why?
Because the mental model shifted from “Page = Route” to “Segment = Route + Layout + Data Layer + Responsibility Split.”
Now I had to think in terms of route segments and where data should live: server component vs client component vs fetch cache vs what the hell.

The Mental Model Shift Nobody Talks About

Before, I thought:
“I add a file, it’s a route. Done.”

Now I have to ask:

Is this a server component or client component?
Should data fetching happen in generateStaticParams or inline in the component?
Where does this layout go?
Is this a template or a layout?

It’s not just about writing code anymore.
It’s about architecting for Next.js’s new ecosystem.

// app/products/[id]/page.js
export default function ProductPage({ params }) {
  const { id } = params
  // Do I fetch here? Should this be a server component?
}

Fun, right?

Love, But With Conditions

Let me be clear: I love the power.
Server Components are a huge win.
Nested layouts? Clean.
Improved routing flexibility? Great.
Performance optimizations baked in? Finally.

But…

It feels overengineered.
Every simple use case now requires reading three different RFCs.
Constantly wondering if you’re doing it “the Next Way™” or just making a hack that’ll break in the next release.

Should You Care?

If you’re building large, scalable apps or deeply care about performance, yes.
Next.js App Router is awesome when you’ve got the bandwidth to understand it.

If you just want to ship stuff, fast and simple…
Stick to pages/. Forget Server Components. Forget complicated layouts.
Use Next.js like the framework it used to be.

My Takeaway

Next.js is amazing at scale.
Next.js is terrible when you want simplicity.

It’s like loving a Swiss Army knife, but sometimes you just need a plain old screwdriver.
And now, the screwdriver has a dozen attachments you don’t care about.

Still, I’m in.
Because I know the payoff exists…
If I survive the learning curve.

One giant Kubernetes cluster for everything

Thu, 20 Mar 2025 08:05:13 +0000

The ideal size of your Kubernetes clusters is a day 0 question and demands a definite answer.

You find one giant cluster on one end of the spectrum and many small-sized ones on the other, with every combination in between. This decision will impact your organization for years to come. Worse, if you decide to change your topology, you're in for a time-wasting and expensive ride.

I want to list each approach's pros and cons in this post. Then, I'll settle the discussion once and for all and argue why selecting the giant cluster option is better.

The one giant cluster approach

Deciding on a single giant cluster has great benefits.

Better resource utilization

Kubernetes was designed to handle large-scale deployments, initially focusing on managing thousands of nodes to support extensive and complex containerized applications. This scalability was a key feature from its inception, enabling it to orchestrate resources across vast, distributed systems efficiently.

Thus, a Kubernetes cluster is a scheduler at its core: it knows how to run workloads on nodes according to constraints. Without constraints, it will happily balance workloads across its available nodes. If you split the cluster into multiple clusters, you lose this benefit. You can have situations where one cluster is idle while another cluster is close to resource starvation and must cancel workloads and kill pods.

Lower operational overhead

Good Kubernetes practices mandate that you back up your etcd data, monitor your cluster metrics, log your cluster events, provide security-related tools, etc. Size aside, it stands to reason that it's more time-effective to operate fewer clusters.

For example, regarding metrics, you'd set up a single Prometheus instance, potentially clustered to handle additional traffic, and be done with it. Automation can mitigate the repetitive aspect of installing and maintaining an instance for each cluster, but you'll still end up with as many instances as you have clusters (or more). Prometheus is just one example because many cluster admins have a long list of tools they run in every cluster.

Straightforward networking and service communication

Service-to-service communication inside a single cluster is straightforward. Point to <service-name>.<namespace>.svc.cluster.local and be done with it. Even better, you only need the service name part inside the same namespace.

You'll need a tool to help you with inter-cluster communication, from the simplicity of External DNS with LoadBalancer to the complexity of a full-fledged solution like Istio—both ends of the spectrum mandate time and operation costs.

Simplified governance

Since every object is part of the same cluster inside a single cluster, one can enforce a centralized set of policies with a standardized approach. For example, you can create a namespace per team and environment, restricting access to only that team's members.

Once you start having multiple clusters, even if you take the same approach, you'll duplicate the policy rules across clusters, with potential differences that will drift further with time.

Cost efficiency

A single cluster means a single control plane, simplifying management and reducing overhead. While a control plane is essential for orchestration, its value comes from enabling efficient workload execution rather than directly running business applications.

Additionally, many of the points above tie into cost optimization. With a single cluster, you only need to configure monitoring (e.g., Prometheus), logging, and security tools once, reducing duplication. In-place automation streamlines operations, helping manage costs without adding unnecessary infrastructure expenses.

Downsides of a one giant cluster approach

Unfortunately, the giant cluster option is not only unicorns and rainbows; there are definite downsides.

Larger blast radius

The larger the cluster, the more teams will use it. Unfortunately, this means that if something bad happens to the cluster, it will wreak havoc on the work of more teams. It occurs regardless of whether the outage results from a malicious actor, a wrong configuration, or resource starvation.

When an actual malicious actor does indeed breach access, a larger cluster exposes more workloads, and the actor can compromise more of them.

Even without malicious actors, every maintenance operation and upgrade on a cluster can affect its users; the bigger the cluster, the larger the potential impact. When planning an upgrade for a single cluster, you need to conduct an impact analysis that encompasses all users and teams in the organization.

Complex multi-tenancy management

Even within a single organization, multiple teams will use the Kubernetes cluster. Even if every team member behaves professionally, we must set strict policies to avoid issues. If the cluster resembles a building, you'd still put locks on your apartment even if you have friendly neighbors. Likewise, the cluster administrator must enforce strict rules to make sharing a cluster acceptable. At the very least, we need strict namespace isolation to avoid unnecessary access and resource quotas and enforce fairness across teams. The problems caused by using a single cluster by teams across a single organization multiply a hundredfold if the cluster is multi-tenant and shared across several organizations.

Scalability limits

Regardless of how exceptional Kubernetes is, it's still a physically based system with physical limits. For example, some objects have a clear limit, but even if you never reach them, getting close to them will require excellent system administration skills with some fine-tuning.

Even then, the more load you put on a Kubernetes API server, the more sluggish your system will be. If you're lucky, it will degrade linearly, but chances are it will hit some system limit and degrade all at once.

Cluster-wide objects

Kubernetes objects are namespace-scoped, but a couple of them are cluster-scoped. A cluster-scoped object can only have a single instance across the whole cluster. For example, a Custom Resource Definition is cluster-scoped.

It means that if a team wants to use v1 of a CRD, then every team on the same cluster is stuck with v1 if they wish to use this CRD. Worse, if any team wants to upgrade to v2, they must coordinate across all teams using the CRD to synchronize the upgrade.

What's the ideal size, then?

I could describe the pros and cons of very granular clusters, but they mirror the opposite of what we have just seen. For example, very granular clusters allow each team to work on their version of a CRD without stepping on another team's toes. For this reason, I'll avoid repeating myself.

Most, if not all, articles evaluating the pros and cons of each end of the spectrum advise a meet-in-the-middle approach: "a couple" of clusters to mitigate the worst aspects of each extreme approach. It's all well and good, but none of them, at least none I've read, tell precisely how many "a couple" is. Is it a cluster per environment, i.e., production, staging, or development? Is it a cluster per team?

I'll take a risk and advertise for two clusters: one for production and the other for everything else. How would you manage the cons mentioned above? Read on.

vCluster

vCluster is an Open Source product that allows creating of so-called virtual clusters. vCluster is part of the CNCF landscape, specifically a certified Kubernetes distribution. Being a certified distro means a virtual cluster offers every Kubernetes API you can expect, and you can deploy any application to it just like any other Kubernetes cluster.

vCluster operates by creating the virtual cluster in a dedicated namespace. You can specify the latter, or vCluster will infer it from the virtual cluster's name. By default, it creates a control plane using the vanilla k8s distribution, but you can choose another one, such as k3s. Likewise, by default, it stores its configuration in an SQLite database, which works particularly well for temporary and pre-production clusters, such as those you create for a pull request. Alternatively, you can rely on a regular etcd or even external databases such as MySQL or Postgres as a data store for more permanent usage and better resilience and scalability of the virtual cluster.

Once you've created a virtual cluster via the CLI or the Helm chart, you can connect to it. The client-side CLI creates a dedicated reusable kubeconfig context. From within a virtual cluster, users see no other virtual clusters.

If you need to access the host cluster resources from the virtual cluster or vice versa, vCluster uses a so-called syncer that syncs objects back and forth according to a configuration file. This way, you can set up an Ingress Controller on the host cluster and define your Ingress objects in the virtual cluster(s).

How vCluster mitigates the downsides of a giant cluster

Let's review each downside of a giant cluster and how vCluster handles it.

Larger blast radius: When using a virtual cluster, the blast radius is automatically contained inside its boundaries. If you want to be conservative, aim for a small granularity approach, such as a cluster per team and environment.
Complex multi-tenancy management: Gone are multi-tenancy problems since your tenants don't see each other and are isolated inside their respective virtual clusters.
Scalability limits: While the limits are still there, the chances of reaching them decrease with the number of virtual clusters. If your giant cluster had 100k services, they are now spread throughout all virtual clusters.
Upgrades and maintenance risks: Upgrade and maintenance tasks are limited to the scope of a single virtual cluster. You can do them in turn, and they will only affect the virtual clusters you target.
Cluster-wide objects: Finally, with virtual clusters, every team can install their version of a CRD, and its virtual cluster binds the CRD. It allows each team to be entirely independent of each other regarding the version of a CRD they use.

But I need different clusters!

While a single giant cluster provides compelling advantages, there are contexts in which a multi-cluster approach is justified. The most common reason is geographic distribution—specific applications require clusters in multiple regions to meet compliance requirements, reduce latency, or provide disaster recovery. For example, companies operating under GDPR or financial regulations may need strict data residency enforcement, which requires region-specific clusters. Similarly, organizations with stringent security postures may enforce complete isolation between environments or business units, making separate clusters a hard requirement.

However, even in these cases, vCluster remains relevant. It allows for minimizing the number of physical clusters while still enabling workload separation at a virtual level. Instead of creating a sprawling landscape of Kubernetes clusters, teams can deploy regional virtual clusters within a single host cluster, balancing isolation and operation complexity.

Conclusion

Kubernetes cluster topology decisions are critical and long-lasting. While many advocate for a middle-ground approach between a single cluster and many small ones, they rarely specify the exact setup. Instead of guessing how many clusters to create, consolidating everything into a single, well-managed giant cluster makes more sense. The benefits—better resource utilization, lower operational overhead, simplified networking, centralized governance, and cost efficiency—outweigh the downsides.

That said, the traditional downsides of a giant cluster, such as a larger blast radius, multi-tenancy complexities, scalability limits, upgrade challenges, and cluster-wide object constraints, are valid concerns. This is where vCluster changes the game. By using virtual clusters, you retain all the advantages of a single giant cluster while mitigating its worst drawbacks. vCluster isolates workloads, reduces operational risk, scales dynamically, simplifies upgrades, and removes conflicts over cluster-wide objects.

Enhanced with vCluster, one cluster for production and one giant cluster for everything else, is the best approach for long-term scalability, efficiency, and ease of operations.

Discover how to use AI writing tools without losing your authentic voice as a content creator.

Fri, 11 Oct 2024 21:16:13 +0000

Be Real: How to Use AI Writing Tools and Stay Authentic

Once you've read one ChatGPT article, it feels like you've read them all. The phrase "Let's dive in" is forever tainted, and the cheesy metaphors are a dead giveaway that you're reading AI-generated content. In this blog, I'd like to talk about my experience using AI writing tools in my own blog. I'll go over different methods I've tried, their pros and cons, and how I'm using AI to write now. My aim is to share my experience in a way that helps you keep your authentic voice so that you don't come across as just another AI blog writer.

If you’re like me, you've probably experimented with AI at some point and realized that while it can help you pump out blogs quickly, it also removes that human element from your content that it used to have. So why would anyone want to use AI-generated content in their blog? Firstly, it saves time and effort! But don't be fooled! Saving too much time and effort will tank not only the quality of your work but your reputation too! Now, I get it. When ChatGPT first came out, I did what most other people did: "ChatGPT, write me a blog about xyz." Then maybe you do a little proofreading, ask it to make some changes, and post it to your blog. Suddenly, producing new content is so easy that you're able to keep up with publishing a blog a day! Woohoo, right? Not really.

The obvious downside to this approach is what I mentioned in the first sentence of this blog. The cheesy metaphors and the overuse of phrases like "Let's dive in." If you're not a subject matter expert, or lazy, it's really easy to put out either inaccurate or misleading information. Many times, when reviewing AI-generated content, I’ve found much more efficient ways to achieve the same results when I’m trying to produce more technical guides. Additionally, people are tired of reading content generated by AI. They want something from a real person who’s an actual subject matter expert! They want something authentic. So what’s the solution? That’s right—it’s time to ditch AI-generated content!

AI-Generated Content vs. Human-Generated Content

Have you ever read a blog and thought, "This was for sure wrtten by AI"? People don’t want to read an article they feel like was created by an empethic robot. They can easily prompt an LLM to do that themselves. They’re looking for your unique human touch and expertise. When you’re discussing a subject you have years of experience in, there’s so much nuance you can add to your content that AI could never capture. That’s what people crave! They want to hear about your personal experiences. What are some of the real joys and pitfalls you’ve encountered along your journey? Have you run into an issue that’s caused you trouble more than once? It’s anecdotes like these that provide real value and build your authenticity. The brand is you. Sure, AI can pull the facts together, but it’s not living your life or walking in your shoes. It can't bring in that experience or those weird anecdotes that makes your work stand out. You just can’t fake those kinds of things. They have to come from you.

That’s the difference between AI-generated content and human-generated content. But just because it’s human-generated content doesn’t mean that AI can’t play a role in making your life easier. After realizing this, my next evolution was to write human-generated content that was AI-edited. This approach was better but still not perfect. So how can we use AI as a tool to help with our writing without letting it take over and drown out our voice?

Using AI To Edit Your Human-Generated Content

The first way I improved my terrible AI blog-writing workflow was to train a ChatGPT conversation on ALL of my blogs that I wrote before ChatGPT ever came out so that it could understand my “voice” as an author better. Then I would write the rough draft, let whatever I was thinking come out, give that to ChatGPT, and have it turn my loose rough draft into a bonafide blog. This was much better than before. Now I was able to capture my own experiences, nuance, and opinions and have ChatGPT use my “style” to turn it into a final draft blog. This definitely takes more effort, but being able to turn a 30-45 minute-long freewriting session into a final draft blog ready to publish is still a huge win and time saver! Plus, there’s this air of consistency in my blogs now that wasn’t there before.

However, it still wasn’t perfect. Getting comments like, “This feels very LLM generated” on my blog, Why Writing Your Own Tools is More Important Than You Think, really hurt. I took time out of my day to write my honest opinions about writing your own tools and the benefits and even drawbacks that come with it. Regardless of being edited by ChatGPT, that blog still reflects how I feel about the topic. However, it seems like my approach had put a stain on my reputation and damaged my authority as an author. That hurt. But sometimes, experiencing that kind of hurt is exactly what you need! So yes, it was something I wrote, edited by AI trained to sound like me, but it was obvious that the end product still didn't sound like me. The fact was, I was still trying to take a shortcut to the top, and people noticed. Having your real experience and nuance reduced to “This feels very LLM generated” is not a good feeling. So what’s a blogger, author, or content creator to do?

It’s time to stop being lazy!

Do I Still Use AI?

Yes, I do! However, I’m no longer letting AI write or rewrite my content anymore. Not only am I doing my followers and fans a disservice by giving my voice as an author over to AI, but I’m also doing myself a disservice by not continuing to hone my craft and writing style. Writing is a huge passion of mine. If I give that passion away to AI to do for me, what’s to stop me from giving away my other passions to AI? Should I let AI play and write music for me? Should I let AI skateboard for me? No! What makes passions so beautiful is that the joy doesn’t come from the end product, it comes from the process! If I let AI write my blogs for me, what joy do I get out of writing blogs? And for what? I want to build something real and authentic that means a lot, not just to me, but to other people as well. I want to find the joy in this blog and my writing as I go along. I’m not in this game for cheap tricks, and I’m not doing this so I can build some kind of cash cow where one day I’ll sit on the beach and sip on margaritas every day. I want to build something meaningful that lasts, and I’m in this for the long haul.

Using AI Moving Forward

So what’s my plan for using AI in my content generation moving forward? First of all, AI is great at grammar and spelling, which I’m very bad at. If programmers were good at spelling, we wouldn’t have invented spell check! So I plan to use AI to grammar-check my work. Additionally, there are three other key areas where I plan to use AI when it comes to content creation:

Feedback: Bugging friends to pre-read my blogs and give me good feedback isn’t feasible, and I can’t afford to pay people to do that for me. So having something I can ask, “Is this good?” or “What are some ideas for improving xyz?” is very nice. Then I can take that feedback and apply my own voice when implementing it.
Research: AI and LLMs are great research tools! I can have them do Google searches for me and summarize articles and videos I find through my own searches, helping me get the most meaningful data I need to create whatever blog I’m working on.
Blog banner creation: Let’s face it, MidJourney is cool. I love using it, and I plan to continue using it to create banners for my blogs moving forward.

Now, will I be able to produce a blog a day anymore? Not really. However, while the quantity of my blogs will go down, the quality and authenticity of my blogs will rise! I’m really looking forward to creating higher-quality content for you all than ever before. No more shortcuts, no more cheap tricks. Just me. :)

So what about you? How have AI writing tools shaped your writing? Share your thoughts in the comments below!

Join The Community

If you like what you’ve read, love to code, and are a fan of Linux, open source, and building your own projects, you’d probably have a great time in my Discord community, The Developers Lounge! We’re a rapidly growing community with lots of great people with a diverse set of skills among us. I’m super active in here, and we have a ton of cool people just waiting to chat. So what are you waiting for? Hop on in!