Coder Legion - Recent posts tagged platform

The Cost of Open Source

Thu, 08 Oct 2026 07:14:28 +0000

Open source is free to use. It has never been free to make.

Open source sits underneath almost every piece of software shipped today. Frameworks, databases, build tools and the libraries inside them are, in large part, written and maintained by people who were never paid to do so.

The conversation around open source tends to focus on its benefits: faster innovation, shared infrastructure and transparency. Far less attention is paid to what publishing costs the people who do it. That cost is rarely financial, and few maintainers expect it to be. It is measured instead in control, in attribution, and in the time and energy required to sustain something that others depend on.

This article examines those costs, why developers continue to accept them, and what can be done to make them more manageable.

What does a developer give up when they give their work away?

Part One: Loss of Control Over Use

Software is written with an intended purpose. A license, however, governs permission, not purpose. Once code is published, its author has no say in how it is applied.

A scraping library built for price comparison can power ticket-scalping bots. An automation tool designed for customer support can be repurposed to send spam. A speech-synthesis feature intended to let people reply in their own voice can be used to imitate someone else's.

For many maintainers, this is the most personal cost of publishing: becoming an unwilling participant in uses they would have declined. There is no approval step and rarely any notification. Discovery tends to be accidental.

The tension occasionally surfaces in public. In January 2022, the author of the widely used JavaScript packages colors and faker deliberately published broken releases, citing frustration that large companies relied on his unpaid work. The response was widely criticised and disrupted a large number of downstream projects. The grievance behind it, however, was one many maintainers recognised.

Part Two: The Erosion of Attribution

For most contributors, attribution is the primary return on their work. It underpins professional reputation, demonstrates competence to employers and clients, and is often the only recognition for work done unpaid. Its loss is therefore felt disproportionately.

Permissive licenses such as MIT ask for very little: chiefly, that the copyright and license notice be preserved. Even so, several patterns are common:

Rebranded forks. The same code under a new name and logo, with the original commit history collapsed into a single "initial commit".
Repackaged "premium" products. Open source dashboards and starter kits, lightly restyled and sold on template marketplaces without reference to their origin.
Installation and support services. Third parties charging businesses to deploy a free project, presenting it as a proprietary platform, while support requests flow back to the original maintainer.
Absorption into AI models. Public repositories have been used to train code-generation models that can reproduce familiar patterns without the accompanying license or author. The legal position remains under dispute. The attribution gap does not.

None of these practices affect whether the code works. They do, however, sever the connection between the work and the person responsible for maintaining it, who is usually the party the project most depends on.

Part Three: Premium Functionality Without a Price

The costs above are most acute for one category of project: the one that offers, for free, capabilities that are elsewhere sold as commercial products.

Agent orchestration, workflow automation, observability, voice synthesis and evaluation tooling all have established paid markets. When an independent developer publishes a working alternative, three effects tend to follow.

Credibility suffers. Free software is often assumed to be a prototype or an abandoned experiment. The same software can appear more credible once someone else attaches a price to it.

Others monetise it. Consultants and resellers package, install and bill for it. Most licenses permit this. For the author, it can mean watching their unpaid work become someone else's recurring revenue.

Large providers adopt it at scale. This dynamic has shaped much of the last five years of open source licensing:

Project	What happened
Elasticsearch	Moved away from Apache 2.0 in 2021, citing cloud providers offering it as a managed service. Later added the AGPL back as an option.
Redis	Switched from BSD to the source-available RSAL and SSPL in 2024. The Linux Foundation backed a community fork, Valkey. Redis 8 returned with an AGPL option in 2025.
Terraform	HashiCorp moved to the Business Source License in 2023. The community forked it as OpenTofu.

Each decision addressed the same problem: how to prevent others from profiting from a project without contributing back. Each also carried its own cost, in forks, lost trust and divided communities.

If organisations with legal teams and boards have struggled to resolve this, individual maintainers are considerably less well placed to do so.

Part Four: Expectation, Noise and Maintainer Fatigue

Unpaid support obligations

A useful project attracts issues. Many are constructive. A significant share are not: terse reports, requests for timelines, and questions about whether the project has been abandoned. Some come from users who paid a third party for an installation and now expect the original author to support it.

No contract exists in any of these cases. The expectation forms regardless.

Automated noise

A newer cost is the volume of AI-generated bug reports: submissions that appear credible, reference functions that do not exist, and consume hours of review before they can be dismissed.

The curl project illustrates the scale of the problem. It ended its bug bounty in January 2026, after more than six years, because fabricated reports had begun to outnumber genuine ones. From 1 July to 3 August 2026, its security team paused vulnerability intake altogether. One of the most widely deployed libraries in the world, maintained by a small volunteer team, had to close its inbox to remain sustainable.

Usage without support

In February 2023, the maintainer of core-js, a library downloaded tens of millions of times a week and present on a large share of the most visited websites, published an account of his financial difficulties. Early donations had amounted to roughly $57 a month, and adding a funding link had, by his account, drawn hostile responses.

The pattern is consistent: the more widely a project is used, the more its maintainer is relied upon, and the less visible that maintainer becomes.

Fatigue as a security risk

In 2024, a backdoor was discovered in xz, a compression library present in a large number of Linux distributions. It was not the result of a technical breach. The attacker spent years building trust with an overextended maintainer until they were granted commit access.

Maintainer fatigue is therefore not only a personal cost. When the people sustaining critical software run out of capacity, everyone who depends on it inherits the risk.

Part Five: Why Developers Continue to Publish

Given these costs, it is reasonable to ask why so many developers continue to release valuable work for free. The answer is that the other side of the ledger is equally real.

Cost	Return
Loss of control over how code is used	Reach far beyond what one developer could achieve alone
Attribution that is removed or overlooked	A public body of work that demonstrates capability directly
Premium features monetised by others	Clear evidence that the work has commercial value
Unpaid support and automated noise	Bug reports, fixes and ideas from a wider community
Mistakes visible to everyone	Security that can be audited rather than assumed
Time and energy	Freedom to build without commercial constraints

Open source also offers something few commercial products can: freedom from lock-in. A subscription business benefits when leaving is difficult. An open project has no such incentive. Users can run it on their own infrastructure, keep their data in-house, inspect every line and migrate whenever they choose.

Finally, there is reciprocity. Every developer builds on open source written by others. Publishing is, for many, a way of contributing back to the foundation they rely on.

Part Six: Managing the Costs

These costs cannot be eliminated, but they can be chosen deliberately rather than discovered later.

Select a license intentionally

MIT is often chosen by default. That is a reasonable choice, provided its implications are understood.

License	What others may do	What it protects
MIT / Apache 2.0	Almost anything, including closed commercial products built on the code	The copyright notice
GPL-3.0	Use and sell it; distributed modifications must remain open	Improvements to distributed software
AGPL-3.0	As GPL, with network use of a modified version also counting as distribution	Modifications offered as a hosted service

None of these prevent others from charging for installation or support. Each defines clearly what must be returned.

Make attribution durable

Include the author's name in the LICENSE and a NOTICE file, in the application's About screen and in API metadata. Attribution recorded in several places is considerably harder to remove than attribution recorded in one.

Protect the name

The code may be open while the project's name and logo are not. Trademarks are how many established projects prevent forks from presenting themselves as the official distribution.

Document expectations

A CONTRIBUTING guide, a SECURITY policy and a short support statement convert implicit expectations into explicit ones. A single line, such as "issues are reviewed weekly and paid support is not offered", can prevent a great deal of friction.

Provide a route for support

Sponsorship links rarely produce significant income. They do give users who value the work a practical way to contribute to its upkeep.

Set limits

Declining features, deadlines and demands is part of sustainable maintenance. curl's five-week intake pause demonstrates that even critical projects can, and sometimes must, step back.

A Case in Point: Obsidian AI

A brief disclosure: this subject is not abstract for me.

For the past seven months I have been developing Obsidian AI, an open source, self-hosted platform for building and orchestrating AI agents. It falls squarely into the category described in Part Three, offering capabilities that are typically sold as commercial products.

Capability	Typical commercial equivalent
Agents across OpenAI, Anthropic, Gemini, Ollama and OpenRouter	Separate per-provider integrations
Multi-agent teams with delegation between agents	Enterprise agent platforms
Visual workflows with conditions, approvals and loops, including n8n import	Workflow automation subscriptions
Evaluation suites, prompt optimisation, tracing and per-agent cost reporting	Observability and evaluation licences
Long-term memory, retrieval-augmented generation and Markdown knowledge vaults	Knowledge management add-ons
WhatsApp agents with local transcription, voice replies and voice cloning	Metered messaging and voice APIs
A Docker sandbox and isolated execution for user-defined tools	Hosted code execution services

The project now comprises roughly 62,000 lines of code and 173 API endpoints, and runs entirely on the user's own infrastructure. It has also appeared on third-party sites as a paid installation service.

Several of the costs discussed above apply directly. The voice cloning feature was designed so that users could respond in their own voice, but its use cannot be controlled once published. The WhatsApp integration avoids the need for a business account by relying on an unofficial library, a trade-off that every deployment inherits. Documentation has not kept pace with development. And when a weakness was identified in how user-defined tools were executed, it was resolved in public, where its prior existence was equally visible.

These considerations informed the choice of license. Obsidian AI is released under the AGPL-3.0. Anyone may install, host, customise and charge for services around it. Modified versions, including those offered as hosted services, must remain open, and their users are entitled to the source.

For those offering it commercially to clients, the request is a simple one: be transparent that the underlying software is open source, retain the attribution, and contribute improvements back upstream.

Conclusion

Open source remains one of the most significant achievements of the software industry. It is not, however, without cost. That cost is borne in control, attribution and energy, largely by individuals, and is seldom discussed.

Developers considering publishing valuable work should do so, but with a clear understanding of what they are accepting, and with deliberate choices about which costs they are prepared to carry.

Open source has never been free. Someone always pays. The question is whether they chose to.

Links:

Obsidian AI: github.com/sup3rus3r/obsidian-ai
Previous article: What If Full-Stack Features Were Installable?
Sponsor the project: GitHub Sponsors

Sources: curl's 2026 reporting pause, the core-js funding post, Redis returning to the AGPL.

How to Improve Enterprise Software Usage Without Disrupting Workflows

Fri, 11 Sep 2026 05:38:30 +0000

Every company today is running on a pile of software- CRM here, HR tool there, some project management app, an internal chat tool, and honestly a few others nobody even remembers signing up for. Sound familiar? Most enterprises are living in this exact mess right now and the second someone brings up fixing it, everyone panics a bit. Change the tools and work stops, right? That's the fear anyway, this is basically why more businesses are looking at an enterprise SaaS enablement platform now, a way to fix how software gets used without blowing up daily work in the process.

Fixing software usage doesn't have to mean tearing everything out and starting fresh. That's what most teams assume, and fair enough, it's not a crazy fear. Nobody wants to relearn ten tools in one week while also trying to hit deadlines but there's a better way to do this and it starts with figuring out what's actually going wrong first.

Why Software Usage Becomes a Problem in the First Place

Nobody sets out to build a messy software stack on purpose, it just happens over time. One team picks a tool for something they need, another team, somewhere else, picks a totally different tool for basically the same need. Nobody's talking to each other about it, a year or two goes by like this and suddenly there's overlap everywhere, and nobody remembers why half these tools even got approved.
The mess creates real friction, day to day. Employees waste time jumping between apps that don't talk to each other, data ends up scattered across five different systems instead of sitting in one place. Onboarding a new employee turns painful, because there's no real structure to explain, just a pile of tools with zero context.

An enterprise digital backbone is supposed to hold all this together in theory but without structure, that backbone stays weak, disconnected. More like a bunch of loose wires than one solid system, honestly and this is where things quietly fall apart, nobody notices until productivity has already taken a hit.

Here's the part people miss though. Employees usually don't complain loudly about any of this, they just work around it. Export data manually into a spreadsheet, keep their own tracker on the side, skip half the tools they're technically supposed to use. That quiet workaround behaviour is honestly not better than open complaints, it means people already stopped expecting the system to actually work.

The Real Risk of Disrupting Workflows

So why do companies drag their feet on fixing this? Every time you bring in new software, or mess with an existing one, there's a learning curve. Productivity dips during that curve, almost always. People make mistakes, old habits push back against the new process. Handle it badly, and it can hurt output for weeks, sometimes longer than that.

That fear isn't unreasonable but it often ends up making companies avoid fixing anything at all. They'd rather keep the broken, disconnected mess than deal with the risk of changing it. Which, honestly, isn't really a solution, it's just kicking a bigger problem further down the road.

The answer isn't avoiding change altogether though. Managing that change properly, with a plan is exactly the point of an enterprise SaaS enablement platform helping companies fix software usage without turning daily operations upside down. Nobody's talking about replacing everything overnight here, it's about building something smoother, step by step, in a way people can actually keep up with.
Rushed changes tend to backfire, almost every time. Give people time, give them clear direction and real support. The whole transition goes smoother when you do this, sounds obvious when you say it out loud, but so many companies still skip it because they're in a rush to "fix things fast."

How an Enablement Platform Actually Helps

So what's actually different about an enterprise SaaS enablement platform? Instead of forcing a full tool swap, it focuses on getting your existing systems to work better together. Connects things, cuts out duplicate processes, gives employees an actual clear path to use software the way it was meant to be used in the first place.

Picture it like fixing the wiring inside a house instead of knocking the whole thing down. You're not breaking walls, you're just making sure everything connects the way it should, runs efficiently, and doesn't short-circuit the moment there's pressure on it. That's roughly what a solid enterprise digital backbone should feel like once it's actually set up right.

A good platform also shows you how software gets used across the company, which most businesses genuinely have no visibility into. Loads of them don't even know which tools employees use daily versus which ones just sit there, quietly costing money every single month. A proper SaaS enablement platform pulls this out into the open, so tool decisions stop being pure guesswork.

Training gets a lot easier too, once this is in place. Instead of dropping employees into unfamiliar systems and hoping for the best, a decent enablement setup walks people through it step by step which leads to less frustration and faster adoption. None of the usual mess that comes with big software rollouts.

Keeping Workflows Steady During the Transition

Here's what worries most businesses the most, honestly — keeping daily work steady while improvements happen underneath it all. It's doable but it needs actual planning, not just good intentions and crossed fingers.

Start small, don't change everything at once. Pick one department, or one process, and start there- see how it plays out, fix whatever breaks then expand from there. It's a slower approach, less exciting maybe, but it protects your daily operations from a major hit. A modular enterprise SaaS enablement platform works well for this kind of phased rollout, adding pieces gradually instead of forcing one big switch on everyone at once.

Communication matters more than people give it credit for, employees deal with change a lot better when they actually understand why it's happening and what they get out of it. Force something on people without explaining it, and resistance builds fast. That resistance ends up slowing the whole process down even more than the change itself would have.

Support during the switch matters too. Having someone, or some system, to answer questions and fix issues fast keeps frustration from piling up. Nobody expects a perfectly smooth ride here, but quick support turns rough patches into manageable ones instead of overwhelming ones.

And patience, honestly, matters more than most people admit upfront. Real improvement in software usage takes months, not days. New habits need time to stick, the enterprise digital backbone needs time to actually feel solid instead of shaky. Rush this bit, and you'll likely cause more problems than you fix.

Long-Term Benefits Once the System Settles

Once the transition period is behind you, the benefits start showing up pretty clearly. Employees stop wasting time bouncing between disconnected tools, data flows better between systems instead of sitting scattered all over the place. Onboarding gets faster too, because now there's an actual structure to walk new hires through, not just a random pile of tools with no explanation.

A properly set up enterprise SaaS enablement platform also makes future changes way less painful. Once systems are connected and organised, adding a new tool or upgrading an old one doesn't feel like starting from zero every single time. That saves real time and stress for years, not just the first few months after rollout.

Cost savings show up too, and usually more than businesses expect going in- unused or duplicate subscriptions get spotted and cut. Teams stop paying for five different tools doing basically the same job. Honestly, this clean-up alone often pays for the whole investment.

Conclusion

Improving software usage without wrecking daily work isn't easy but it's absolutely doable with the right approach- rushed, careless changes cause chaos, every time. Thoughtful, phased improvements build something stronger without breaking what's already working. That's the balance SaaS enablement platform is actually built around.

This is where ItNet by Imbibe Tech comes in. ItNet is a modular, multi-tenant enterprise SaaS enablement platform that pulls your business tools, integrations, and workflows together in one place, without forcing a rip-and-replace approach on your team. It connects with things you're probably already using — WhatsApp Business API, AI tools like OpenAI and Gemini, major payment gateways and gives you automation, role-based access, and real-time reporting built right in. Since it's modular, you start with what you actually need and add more only when you're ready.

If disconnected systems, manual work, or scaling headaches are slowing your team down, ItNet is built to fix exactly that, one piece at a time. Reach out today and see what a properly built enterprise SaaS enablement platform can actually do for your business.

9 Steps to Build an IPTV Web Player With JavaScript: From M3U Playlist to Live Playback

Mon, 07 Sep 2026 16:58:04 +0000

Building a web-based IPTV player with JavaScript is a practical way to understand how modern live video applications work in the browser. What looks like a simple interface with a channel list and a video window actually involves several different technologies working together: playlist parsing, HTTP-based media delivery, HLS manifests, HTML5 video, browser APIs, network requests, and playback state management.

In this guide, we will build the foundation of a JavaScript IPTV web player that can read a channel list, extract stream information, display channels dynamically, and play HLS live streams in a browser.

The examples are designed around publicly accessible or self-hosted streams that you are authorized to use. The purpose is to explain the engineering behind an IPTV-style application rather than provide unauthorized access to copyrighted broadcasts.

The project will start with plain HTML, CSS, and JavaScript. Once the basic architecture is understood, the same concepts can be transferred to React, Vue, Angular, or another frontend framework.

What Is an IPTV Web Player?

An IPTV web player is essentially a web application designed to organize and play live media streams delivered over an IP network.

From a developer's perspective, the term IPTV describes the application and distribution context, while the actual video playback may rely on technologies such as HLS, MPEG-DASH, HTML5 video, Media Source Extensions, or WebRTC depending on the requirements.

A basic IPTV web player can be represented as:

M3U Playlist
     ↓
JavaScript Parser
     ↓
Channel Objects
     ↓
Channel List
     ↓
Selected Stream
     ↓
HLS / M3U8
     ↓
HTML5 Video
     ↓
Live Playback

This architecture is simple enough to understand but also provides a foundation for much larger streaming applications.

A production application can later add search, categories, favorites, EPG data, authentication, analytics, multiple audio tracks, subtitles, quality selection, and responsive interfaces.

Understanding M3U and M3U8 Playlists

Before writing JavaScript, it is important to understand what an M3U playlist actually contains.

M3U is a playlist format originally used to describe collections of media resources. In IPTV applications, an M3U file is frequently used to store channel metadata and stream URLs.

A simplified playlist may look like this:

#EXTM3U

#EXTINF:-1,Example News
https://example.com/live/news/index.m3u8

#EXTINF:-1,Example Sports
https://example.com/live/sports/index.m3u8

#EXTINF:-1,Example Music
https://example.com/live/music/index.m3u8

The #EXTM3U line identifies the playlist.

Each #EXTINF line contains metadata describing the following media URL.

The URL may point to an HLS manifest ending in .m3u8.

That distinction is important because the M3U playlist and the HLS manifest serve different purposes.

An M3U playlist might describe hundreds of channels:

Channel A → stream-a.m3u8
Channel B → stream-b.m3u8
Channel C → stream-c.m3u8

An HLS manifest, on the other hand, describes the media presentation itself.

A master HLS playlist can reference different quality levels:

The browser or playback library can then select an appropriate representation.

Why Use JavaScript to Build an IPTV Player?

JavaScript provides everything needed to create the application layer of a basic IPTV web player.

It can:

Fetch playlist files
Parse text responses
Extract channel metadata
Build dynamic interfaces
Handle user interaction
Change the current stream
Communicate with APIs
Monitor playback events
Display errors
Store favorites
Implement search and filtering

The HTML5 <video> element handles the actual media element, while JavaScript controls how the application interacts with it.

This separation is important.

You should think of the application as two layers:

Application Layer
├── Playlist
├── Channels
├── Search
├── Categories
├── Favorites
└── User Interface

Playback Layer
├── HLS
├── M3U8
├── Video Element
├── Buffering
└── Media Events

Keeping these responsibilities separate makes the project easier to extend later.

Step 1: Create the Basic IPTV Web Player Structure

Start with a simple HTML document:

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>JavaScript IPTV Player</title>
  <link rel="stylesheet" href="style.css">
</head>

<body>

  <main class="player-app">

    <section class="video-container">
      <video
        id="videoPlayer"
        controls
        playsinline>
      </video>
    </section>

    <aside class="channel-panel">
      <input
        id="channelSearch"
        type="search"
        placeholder="Search channels..."
      >

      <div id="channelList"></div>
    </aside>

  </main>

  <script src="app.js"></script>
</body>
</html>

This gives us three important elements:

A video element
A search field
A dynamic channel container

The JavaScript application will populate the channel list later.

A basic stylesheet could be:

body {
  margin: 0;
  font-family: system-ui, sans-serif;
  background: #111;
  color: #fff;
}

.player-app {
  display: grid;
  grid-template-columns: 1fr 320px;
  min-height: 100vh;
}

.video-container {
  display: flex;
  align-items: center;
  justify-content: center;
  background: #000;
}

video {
  width: 100%;
  max-height: 90vh;
}

.channel-panel {
  padding: 20px;
  overflow-y: auto;
}

At this stage, there is no IPTV functionality yet.

We have simply created the application shell.

Step 2: Represent IPTV Channels as JavaScript Objects

Before connecting an external playlist, it is useful to understand the data model.

A channel can be represented as:

const channels = [
  {
    id: 1,
    name: "Example News",
    category: "News",
    logo: "https://example.com/news.png",
    streamUrl: "https://example.com/live/news/index.m3u8"
  },
  {
    id: 2,
    name: "Example Sports",
    category: "Sports",
    logo: "https://example.com/sports.png",
    streamUrl: "https://example.com/live/sports/index.m3u8"
  }
];

This structure is considerably easier to work with than raw playlist text.

The application can now treat every channel as an object.

For example:

console.log(channels[0].name);
console.log(channels[0].streamUrl);

The result is:

Example News
https://example.com/live/news/index.m3u8

This leads to an important design principle:

Parse the playlist once, then work with structured data.

Do not repeatedly parse the entire M3U file every time the user clicks a channel.

Step 3: Parse an M3U Playlist With JavaScript

Now we can implement a simple M3U parser.

Consider this playlist:

#EXTM3U

#EXTINF:-1,Example News
https://example.com/live/news/index.m3u8

#EXTINF:-1,Example Sports
https://example.com/live/sports/index.m3u8

A basic parser can process the lines:

function parseM3U(content) {
  const lines = content
    .split(/\r?\n/)
    .map(line => line.trim())
    .filter(Boolean);

  const channels = [];

  for (let i = 0; i < lines.length; i++) {
    const line = lines[i];

    if (line.startsWith("#EXTINF")) {
      const name = line.substring(
        line.lastIndexOf(",") + 1
      );

      const streamUrl = lines[i + 1];

      if (streamUrl && !streamUrl.startsWith("#")) {
        channels.push({
          id: channels.length + 1,
          name,
          streamUrl
        });
      }
    }
  }

  return channels;
}

This is intentionally simple.

Real-world M3U playlists can contain additional attributes such as:

group-title
tvg-id
tvg-name
tvg-logo

For example:

#EXTINF:-1 tvg-id="news.example" tvg-name="Example News" tvg-logo="https://example.com/logo.png" group-title="News",Example News

A production parser should extract those fields rather than only reading the text after the comma.

Step 4: Fetch an M3U Playlist

Once the parser exists, JavaScript can retrieve a playlist using the Fetch API.

For example:

async function loadPlaylist(url) {
  const response = await fetch(url);

  if (!response.ok) {
    throw new Error(
      `Playlist request failed: ${response.status}`
    );
  }

  const content = await response.text();

  return parseM3U(content);
}

Then:

loadPlaylist("/playlists/channels.m3u")
  .then(channels => {
    console.log(channels);
  })
  .catch(error => {
    console.error(error);
  });

There is an important browser consideration here.

If the playlist is hosted on another domain, the server must allow the browser application to access it through appropriate CORS configuration.

For a streaming application you control, configure CORS at the server or CDN rather than attempting to bypass browser security from JavaScript.

This is a common source of confusion when developing M3U playlist applications.

Step 5: Render the IPTV Channel List

Once the playlist has been converted into JavaScript objects, the interface can be generated dynamically.

const channelList =
  document.getElementById("channelList");

function renderChannels(channels) {
  channelList.innerHTML = "";

  channels.forEach(channel => {
    const button = document.createElement("button");

    button.className = "channel-item";
    button.textContent = channel.name;

    button.addEventListener("click", () => {
      playChannel(channel);
    });

    channelList.appendChild(button);
  });
}

This creates one button for each channel.

The important part is that the UI is generated from data.

That means the same interface can display 10 channels or 10,000 channels without manually writing HTML for every entry.

A more advanced channel item could include:

[Logo] Example News
       News

The same principle applies.

Step 6: Play an HLS Stream in the Browser

Now we reach the actual playback layer.

Modern web browsers use different mechanisms for HLS playback depending on the platform.

Some browsers provide native HLS support through the HTML5 video element. Others can use Media Source Extensions with a JavaScript playback library such as HLS.js.

For a browser-compatible implementation, HLS.js can be loaded into the application.

Using a module-based installation:

npm install hls.js

Then:

import Hls from "hls.js";

The player logic can be structured as:

const video = document.getElementById(
  "videoPlayer"
);

let hls = null;

function playChannel(channel) {

  if (hls) {
    hls.destroy();
    hls = null;
  }

  if (Hls.isSupported()) {

    hls = new Hls();

    hls.loadSource(channel.streamUrl);
    hls.attachMedia(video);

  } else if (
    video.canPlayType(
      "application/vnd.apple.mpegurl"
    )
  ) {

    video.src = channel.streamUrl;

  } else {

    showError(
      "This browser does not support HLS playback."
    );

  }
}

This pattern is useful because it handles both HLS.js playback and native HLS playback.

HLS.js documents this feature-detection approach and uses Media Source Extensions for browsers where the required support is available. (github.com)

Step 7: Connect the Playlist to the Player

Now the main application can bring everything together.

let allChannels = [];

async function initializePlayer() {
  try {

    allChannels = await loadPlaylist(
      "/playlists/channels.m3u"
    );

    renderChannels(allChannels);

  } catch (error) {

    showError(
      "Unable to load the channel playlist."
    );

    console.error(error);
  }
}

initializePlayer();

The complete flow becomes:

Page loads
    ↓
JavaScript starts
    ↓
Fetch M3U playlist
    ↓
Parse playlist
    ↓
Create channel objects
    ↓
Render channel list
    ↓
User selects channel
    ↓
Load M3U8 stream
    ↓
HLS.js / Native HLS
    ↓
HTML5 Video
    ↓
Live playback

This is the basic architecture behind an IPTV web player.

Step 8: Add Channel Search

A channel list becomes difficult to use when it contains hundreds or thousands of entries.

Search is therefore one of the first features worth adding.

The HTML:

<input
  id="channelSearch"
  type="search"
  placeholder="Search channels..."
>

JavaScript:

const searchInput =
  document.getElementById("channelSearch");

searchInput.addEventListener("input", () => {

  const query =
    searchInput.value.toLowerCase().trim();

  const filtered = allChannels.filter(channel =>
    channel.name
      .toLowerCase()
      .includes(query)
  );

  renderChannels(filtered);
});

Now the user can type:

news

and the interface will display only channels containing that term.

For a large application, you could extend this to search:

Channel name
Category
Country
EPG metadata
Tags
Language

The key is to perform filtering against the structured channel data rather than reparsing the M3U file.

Step 9: Add Categories and Channel Metadata

M3U playlists often contain metadata that can improve the user experience.

For example:

group-title="Sports"

can be converted into:

{
  name: "Example Sports",
  category: "Sports",
  streamUrl: "..."
}

The interface can then provide:

All
News
Sports
Movies
Music
Kids
Documentary

Selecting a category simply filters the channel array.

function filterByCategory(category) {

  if (category === "All") {
    renderChannels(allChannels);
    return;
  }

  const filtered = allChannels.filter(
    channel => channel.category === category
  );

  renderChannels(filtered);
}

This approach scales much better than building separate pages for every category.

Handling HLS Playback Errors

A serious JavaScript IPTV player needs error handling.

Streaming failures can happen for reasons outside the frontend application.

Possible causes include:

Network interruption
HTTP errors
Missing media segments
Invalid HLS manifests
Unsupported codecs
CORS configuration
Expired URLs
Server overload
CDN problems
Decoder failures

HLS.js provides error events that can be monitored.

For example:

hls.on(Hls.Events.ERROR, (event, data) => {

  console.error(
    "HLS error:",
    data
  );

});

Fatal errors can be treated differently from recoverable problems.

A simplified example:

hls.on(Hls.Events.ERROR, (event, data) => {

  if (!data.fatal) {
    return;
  }

  switch (data.type) {

    case Hls.ErrorTypes.NETWORK_ERROR:
      hls.startLoad();
      break;

    case Hls.ErrorTypes.MEDIA_ERROR:
      hls.recoverMediaError();
      break;

    default:
      hls.destroy();
      break;
  }
});

The appropriate recovery strategy depends on the application.

Blindly retrying every request is not a good solution because it can create additional traffic while hiding an underlying server problem.

Understanding IPTV Buffering

Buffering is one of the most misunderstood parts of web video playback.

A player may buffer because of:

Insufficient bandwidth
Network instability
Slow server response
CDN congestion
Large segment sizes
High bitrate
Encoding problems
Player buffer configuration

The frontend is only one part of the equation.

Consider the complete path:

Encoder
   ↓
Streaming Packager
   ↓
Origin
   ↓
CDN
   ↓
Internet
   ↓
Browser
   ↓
JavaScript Player
   ↓
Video Element

If the origin server is slow, changing the JavaScript interface will not solve the problem.

If the CDN is performing poorly, increasing frontend retries may make the situation worse.

This is why video streaming problems should be diagnosed across the complete delivery pipeline.

Improving IPTV Channel Switching

Fast channel switching is one of the most noticeable quality improvements in an IPTV web player.

When a user selects another channel, the application should:

Stop the previous playback pipeline
Release the old HLS instance when necessary
Load the new manifest
Attach the new media source
Start playback when permitted by the browser

A clean implementation prevents multiple HLS instances from remaining active.

For example:

function destroyPlayer() {

  if (hls) {
    hls.destroy();
    hls = null;
  }

  video.removeAttribute("src");
  video.load();
}

Then before switching:

destroyPlayer();
playChannel(channel);

This is especially important for applications where users switch channels frequently.

Autoplay Restrictions in Modern Browsers

Another issue developers often discover is that a player works when clicking a button but fails to start automatically.

This is usually related to browser autoplay policies.

Modern browsers can restrict autoplay, particularly when media includes audio.

A reliable application should therefore design around user interaction.

For example:

button.addEventListener("click", () => {
  playChannel(channel);
});

The user's click provides a natural interaction point for starting playback.

Do not assume that every browser will allow a newly loaded live stream to begin automatically with audio.

Building a More Maintainable JavaScript IPTV Player

As the project grows, a single app.js file can quickly become difficult to maintain.

A better structure might be:

src/
├── player/
│   ├── hlsPlayer.js
│   └── playbackEvents.js
│
├── playlist/
│   ├── parser.js
│   └── loader.js
│
├── channels/
│   ├── channelList.js
│   ├── search.js
│   └── categories.js
│
├── ui/
│   ├── notifications.js
│   └── loading.js
│
└── main.js

This gives every part of the application a clear responsibility.

For example:

parser.js handles M3U parsing.

loader.js handles network requests.

hlsPlayer.js handles playback.

channelList.js handles channel rendering.

search.js handles filtering.

main.js connects everything together.

This architecture also makes future migration to React or another framework easier.

Security Considerations for IPTV Web Applications

A web player should never expose secrets in frontend JavaScript.

Do not place:

API keys
Database passwords
Private credentials
Administrative tokens
Origin server credentials

inside client-side code.

Anything shipped to the browser should be considered visible to the user.

If your application requires authentication, sensitive operations should be handled through a backend service.

The frontend can receive a controlled response from the backend rather than directly exposing internal infrastructure.

This is especially important when building commercial streaming applications.

CORS and M3U Playlist Access

CORS deserves special attention because it is a common issue when loading external M3U playlists.

Suppose your player is hosted at:

https://player.example.com

while the playlist is hosted at:

https://cdn.example.net/channels.m3u

The browser treats these as different origins.

If the server does not provide an appropriate CORS response, JavaScript may be prevented from reading the playlist.

The same consideration can apply to HLS manifests and media resources depending on how the streaming infrastructure is configured.

For infrastructure that you control, solve this at the server or CDN level.

Do not attempt to defeat browser security restrictions with client-side workarounds.

Measuring IPTV Player Performance

Once the player is working, performance should be measured rather than guessed.

Useful metrics include:

Startup time

How long does it take from selecting a channel until the first frame appears?

Channel switching time

How long does it take to move from one stream to another?

Rebuffering ratio

How frequently does playback stop because additional media is unavailable?

Manifest loading time

How quickly does the browser receive the HLS manifest?

Segment download time

How quickly can the player retrieve media segments?

Playback errors

How many sessions encounter fatal playback errors?

These measurements help identify whether a problem originates from the frontend, network, CDN, or streaming infrastructure.

Common Mistakes When Building a JavaScript IPTV Player

Using the M3U file as the player source

The M3U playlist is generally a collection of metadata and stream references.

The actual video player normally consumes an individual media stream, such as an HLS manifest.

Assuming every browser supports HLS in the same way

Browser capabilities differ.

Feature detection is safer than assuming universal support.

Ignoring cleanup

If every channel switch creates a new HLS instance without destroying the old one, unnecessary resources can remain active.

Loading massive playlists unnecessarily

A very large playlist can increase startup time and memory usage.

For large applications, consider pagination, server-side filtering, or an API that returns only the required channel data.

Putting playlist parsing inside the video component

Playlist parsing and media playback are separate responsibilities.

Keep them independent.

Ignoring network errors

A video player should provide meaningful feedback when a stream cannot be loaded.

Testing only on a fast connection

A player that works perfectly on a high-speed fiber connection may behave very differently on mobile or congested networks.

Testing should include slower and unstable network conditions.

Taking the IPTV Web Player Further

Once the basic JavaScript IPTV player is working, many additional features become possible.

Electronic Program Guide

An EPG can add program information to the channel interface.

For example:

Example News

18:00  Evening News
19:00  World Report
20:00  Documentary

The EPG data can be retrieved from an API or another authorized data source.

Favorites

Favorites can be stored locally:

localStorage.setItem(
  "favorites",
  JSON.stringify(favorites)
);

For authenticated applications, favorites can instead be stored server-side.

Recently watched channels

The application can keep track of recently selected channels and display them separately.

Responsive design

A desktop interface can use a two-column layout, while mobile devices can switch to:

Video
↓
Channel Search
↓
Categories
↓
Channel List

Accessibility should also be considered.

Users should be able to navigate important controls without relying entirely on a mouse or touchscreen.

From JavaScript Prototype to Production IPTV Platform

A prototype can be built entirely on the frontend, but a larger streaming platform will usually require backend services.

A more complete architecture might look like:

                    ┌─────────────────┐
                    │ JavaScript Web  │
                    │     Player      │
                    └────────┬────────┘
                             │
                       HTTPS / API
                             │
                    ┌────────▼────────┐
                    │    Backend API  │
                    └────────┬────────┘
                             │
              ┌──────────────┼──────────────┐
              │              │              │
          Channels          EPG         User Data
              │              │              │
              └──────────────┼──────────────┘
                             │
                            CDN
                             │
                       Media Origin

This architecture separates application data from media delivery.

The frontend does not need to know every internal detail of the streaming infrastructure.

Instead, it receives the information required to build the user experience.

This pattern is much easier to maintain as the platform grows.

Why M3U + HLS Is a Useful Developer Project

Building an IPTV web player may seem like a niche project, but it teaches several concepts that apply far beyond IPTV.

You learn how to:

Parse structured text
Consume remote resources
Work with asynchronous JavaScript
Manage browser media APIs
Handle network failures
Build dynamic interfaces
Design reusable components
Work with HTTP streaming
Debug playback problems
Measure frontend performance

The same skills are useful when building video-on-demand platforms, live event applications, media dashboards, educational platforms, and other streaming products.

The IPTV use case simply provides a concrete environment where all these concepts come together.

Final Thoughts

Building an IPTV web player with JavaScript is a good practical exercise for developers who want to understand the relationship between frontend applications and modern video streaming.

The basic architecture is straightforward:

M3U
 ↓
JavaScript Parser
 ↓
Channel Data
 ↓
Web Interface
 ↓
M3U8 Stream
 ↓
HLS.js / Native HLS
 ↓
HTML5 Video
 ↓
Live Playback

But a reliable application requires more than simply placing a stream URL inside a video element.

You need to think about browser compatibility, playlist parsing, CORS, network failures, HLS errors, buffering, channel switching, autoplay policies, application architecture, and performance monitoring.

The biggest lesson is that the player is only one part of the streaming system.

When playback is slow or unstable, developers should investigate the complete chain from media encoding and packaging through the origin server and CDN to the browser.

Starting with a small JavaScript prototype is therefore a good approach. Build playback first, then add playlist parsing, channel management, search, categories, error handling, and performance monitoring one step at a time.

Once the fundamentals are working, the same architecture can evolve into a React application, a Smart TV interface, a mobile application, or a much larger streaming platform.

For readers exploring the broader IPTV ecosystem, IPTV FOX PRO, VAST IPTV, and FOX IPTV Premium can be referenced as examples of IPTV-oriented services. These resources are supplementary to the technical discussion; the core of this guide remains the engineering behind playlist processing and browser-based video playback.

The most important principle is simple: build the data layer, playback layer, and user interface as separate components. Once those boundaries are clear, an IPTV web player becomes much easier to develop, debug, and scale.

9 Steps to Build an IPTV Player With React and HLS.js: A Complete Developer Guide

Mon, 07 Sep 2026 16:52:25 +0000

Building an IPTV player for the web is an interesting frontend engineering project because it sits at the intersection of React development, HTML5 video, HTTP streaming, playlist management, browser APIs, and user experience.

At first glance, an IPTV player may appear to be nothing more than a <video> element with a stream URL. In reality, a reliable player needs to handle several layers of complexity: HLS manifests, M3U or M3U8 playlists, adaptive bitrate streams, browser compatibility, buffering, playback errors, channel switching, and lifecycle management.

In this guide, we will build the foundation of a React-based IPTV player using HLS.js. The goal is not to create a commercial IPTV application or connect to unauthorized streams. Instead, we will focus on the engineering concepts behind a modern web video player and use legally accessible HLS test streams as examples.

HLS.js is an open-source JavaScript library designed to play HTTP Live Streaming content in browsers that support Media Source Extensions. The library provides APIs for attaching an HLS instance to an HTML video element, loading a manifest, handling events, and recovering from certain media errors.

Why Build an IPTV Player With React and HLS.js?

React is a natural choice for a modern IPTV web player because the interface usually contains much more than a video element.

A typical IPTV-style application may include a channel list, categories, search, favorites, an electronic program guide, playback controls, quality selection, subtitles, loading indicators, error messages, and responsive layouts.

React makes these interface components easier to organize into reusable pieces.

A possible architecture might look like this:

React Application
│
├── ChannelList
│   ├── Categories
│   ├── Search
│   └── Favorites
│
├── VideoPlayer
│   ├── HTML5 Video
│   └── HLS.js
│
├── PlayerControls
│
├── ProgramGuide
│
└── ErrorHandler

The important distinction is that React manages the application interface, while HLS.js manages HLS playback.

That separation is useful because it prevents the player component from becoming responsible for the entire application.

What Is HLS Streaming?

HTTP Live Streaming, commonly called HLS, is a streaming protocol that delivers media over standard HTTP infrastructure.

Instead of downloading one large video file, an HLS presentation is typically divided into media segments. A playlist, commonly identified by an .m3u8 URL, tells the player where those segments can be found.

An HLS stream can also provide multiple quality levels. The player can then select an appropriate representation based on network conditions and other playback factors.

This is one of the reasons HLS is widely used for live and on-demand video applications.

For developers, the important pieces are:

HLS manifest
Media segments
Video element
Media Source Extensions
JavaScript player logic
Network connection
Optional adaptive bitrate representations

HLS.js sits between the web application and the browser's media pipeline.

HLS.js vs Native HLS Playback

One of the first things developers need to understand is that browsers do not all handle HLS in exactly the same way.

HLS.js uses Media Source Extensions when the browser supports the required APIs. Some browsers and platforms also provide native HLS support directly through the HTML5 video element.

The official HLS.js documentation specifically recommends feature detection rather than assuming that every browser requires HLS.js. Safari, for example, has built-in HLS support through the standard video element.

That means a robust React IPTV player should consider both approaches:

Does the browser support HLS.js?
        │
        ├── Yes → Use HLS.js + MediaSource
        │
        └── No → Check native HLS support
                     │
                     ├── Yes → Use video.src
                     │
                     └── No → Display compatibility message

This small piece of browser detection can make the difference between a player that works on one machine and a player that works reliably across multiple environments.

Step 1: Create the React Project

You can start with a standard React project.

For example, using a modern Vite-based setup:

npm create vite@latest react-iptv-player
cd react-iptv-player
npm install

Then install HLS.js:

npm install hls.js

HLS.js is distributed as an npm package and also provides browser-ready builds. The official project documentation provides installation instructions and API documentation for current releases.

The project structure could initially be:

src/
├── App.jsx
├── components/
│   └── HlsPlayer.jsx
├── data/
│   └── channels.js
└── styles/
    └── player.css

Keeping the HLS logic inside its own component will make the application easier to maintain.

Step 2: Create a Reusable React HLS Player

The core player can be surprisingly small.

Here is a basic implementation:

import { useEffect, useRef } from "react";
import Hls from "hls.js";

export default function HlsPlayer({ src }) {
  const videoRef = useRef(null);

  useEffect(() => {
    const video = videoRef.current;

    if (!video || !src) {
      return;
    }

    let hls;

    if (Hls.isSupported()) {
      hls = new Hls();

      hls.loadSource(src);
      hls.attachMedia(video);
    } else if (video.canPlayType("application/vnd.apple.mpegurl")) {
      video.src = src;
    }

    return () => {
      if (hls) {
        hls.destroy();
      }
    };
  }, [src]);

  return (
    <video
      ref={videoRef}
      controls
      playsInline
      style={{ width: "100%" }}
    />
  );
}

There are several important ideas here.

First, useRef() gives React access to the underlying HTML video element.

Second, Hls.isSupported() checks whether HLS.js can use the browser's Media Source Extensions environment.

Third, loadSource() tells HLS.js which HLS manifest to load.

Fourth, attachMedia() connects the HLS.js instance to the video element.

Finally, destroy() is called when the component is removed or when the stream source changes.

The HLS.js API documentation describes this general lifecycle: check support, instantiate the HLS object, attach it to the media element, load the manifest, and clean up the instance when it is no longer needed.

Step 3: Understand the M3U8 Playlist

An IPTV player often receives a playlist containing multiple channels.

It is important to distinguish between an M3U channel playlist and an HLS M3U8 media playlist.

An M3U file may contain entries such as:

#EXTM3U

#EXTINF:-1,Example News
https://example.com/live/news/index.m3u8

#EXTINF:-1,Example Sports
https://example.com/live/sports/index.m3u8

The metadata describes the channel, while the URL points to a media resource.

The player application can convert that information into JavaScript objects:

const channels = [
  {
    id: 1,
    name: "Example News",
    streamUrl: "https://example.com/live/news/index.m3u8"
  },
  {
    id: 2,
    name: "Example Sports",
    streamUrl: "https://example.com/live/sports/index.m3u8"
  }
];

The important engineering principle is to keep playlist parsing separate from playback.

Do not make the video component responsible for understanding the entire playlist format.

Instead:

M3U Parser
     ↓
Channel Objects
     ↓
React State
     ↓
Selected Channel
     ↓
HLS Player

This architecture becomes particularly useful when you later add search, favorites, categories, or an electronic program guide.

Step 4: Build the IPTV Channel List in React

Once the channel data is available, React can render a channel list.

A simple component might look like this:

export default function ChannelList({ channels, onSelect }) {
  return (
    <div>
      {channels.map((channel) => (
        <button
          key={channel.id}
          onClick={() => onSelect(channel)}
        >
          {channel.name}
        </button>
      ))}
    </div>
  );
}

The parent component can store the currently selected channel:

import { useState } from "react";
import HlsPlayer from "./components/HlsPlayer";
import ChannelList from "./components/ChannelList";

export default function App() {
  const [selectedChannel, setSelectedChannel] = useState(null);

  return (
    <main>
      {selectedChannel && (
        <HlsPlayer src={selectedChannel.streamUrl} />
      )}

      <ChannelList
        channels={channels}
        onSelect={setSelectedChannel}
      />
    </main>
  );
}

This is where React becomes especially useful.

The player itself does not need to know how the channel list works. It receives a URL and focuses on playback.

That separation keeps the component reusable.

Step 5: Handle IPTV Stream Errors Properly

A production-quality IPTV web player needs much better error handling than a basic example.

Streaming errors can happen for many reasons:

The manifest is unavailable
A segment cannot be downloaded
The server returns an HTTP error
The stream has ended
The media format is unsupported
The browser encounters a media decoding problem
The network connection disappears
CORS prevents browser access
The source URL expires

HLS.js exposes events and error information that can be used to identify these problems.

A basic listener could look like:

hls.on(Hls.Events.ERROR, (event, data) => {
  console.error("HLS error:", data);
});

For production applications, it is useful to distinguish between fatal and recoverable errors.

For example:

hls.on(Hls.Events.ERROR, (event, data) => {
  if (!data.fatal) {
    return;
  }

  switch (data.type) {
    case Hls.ErrorTypes.NETWORK_ERROR:
      hls.startLoad();
      break;

    case Hls.ErrorTypes.MEDIA_ERROR:
      hls.recoverMediaError();
      break;

    default:
      hls.destroy();
      break;
  }
});

The exact recovery strategy should depend on the application. Automatically retrying everything can actually make an outage worse.

The HLS.js API includes methods such as startLoad() and recoverMediaError() for specific recovery scenarios.

A useful production player should also show the user a meaningful message rather than exposing raw JavaScript errors.

For example:

Unable to play this channel.

Please try another channel or check the stream source.

That is much better than simply showing a blank video element.

Step 6: Improve Channel Switching Performance

One of the most important features in an IPTV-style player is fast channel switching.

Users generally expect a new channel to start playing quickly after clicking it.

A poor implementation may destroy and recreate the entire application unnecessarily.

A better approach is to keep the React interface stable and only update the stream source.

For example:

const selectChannel = (channel) => {
  setSelectedChannel(channel);
};

The player receives a new src and updates its HLS instance.

However, channel switching also depends heavily on the stream architecture itself.

Even a perfectly optimized React component cannot eliminate latency caused by:

Large media segments
Slow origin servers
Poor CDN performance
High network latency
Excessive player buffering
Slow manifest retrieval
Encoding delays

Frontend optimization is only one part of the streaming pipeline.

Step 7: Add Adaptive Bitrate Streaming

Modern HLS streams can contain multiple quality levels.

For example:

1080p → 5 Mbps
720p  → 3 Mbps
480p  → 1.5 Mbps
360p  → 800 Kbps

The player can switch between representations depending on available bandwidth and playback conditions.

This is known as adaptive bitrate streaming.

It is one of the most important technologies for maintaining playback quality across different network conditions.

A viewer on a fast fiber connection may receive a higher-quality representation, while a user on a congested mobile network may temporarily receive a lower-quality stream.

The goal is not simply to maximize resolution.

The real goal is to provide a stable playback experience.

That means a 720p stream that plays continuously can provide a better experience than 1080p playback that repeatedly buffers.

HLS.js includes configuration and APIs related to quality selection and adaptive streaming, allowing developers to tune playback behavior for their applications.

Step 8: Add Loading States and Playback UX

Technical correctness is not enough for a good IPTV web player.

The interface should clearly communicate what is happening.

For example, when the user selects a channel:

Channel selected
      ↓
Loading stream
      ↓
Manifest loaded
      ↓
Buffering
      ↓
Playback started

React state can represent these stages:

const [loading, setLoading] = useState(false);
const [error, setError] = useState(null);

You can then display:

{loading && <p>Loading stream...</p>}
{error && <p>{error}</p>}

This seems simple, but it makes the application feel considerably more reliable.

You should also consider:

Responsive video dimensions
Mobile layouts
Keyboard controls
Fullscreen playback
Captions and subtitles
Audio tracks
Poster images
Accessible buttons
Clear error messages
Dark-mode interfaces
Channel search
Favorites

React's component model is particularly useful for separating these concerns into small reusable components.

Step 9: Add CORS and Security Considerations

One of the most common problems developers encounter when building an HLS player is CORS.

A browser may successfully access a URL from the address bar while preventing JavaScript from fetching the same resource because of cross-origin restrictions.

This is not an HLS.js bug.

It is a browser security mechanism.

If your application is hosted on:

https://player.example.com

and the stream is served from:

https://stream.example.net

the stream server must be configured correctly for cross-origin browser access.

For a legitimate streaming service that you control, this generally means configuring the server or CDN with the appropriate Access-Control-Allow-Origin policy.

Do not try to bypass browser security restrictions from frontend code.

If you control the streaming infrastructure, configure CORS at the correct layer.

React IPTV Player Architecture for Production

Once the basic player works, the application can evolve into a much more structured system.

A production architecture might look like this:

                    ┌──────────────────┐
                    │  React Frontend  │
                    └────────┬─────────┘
                             │
              ┌──────────────┼──────────────┐
              │              │              │
        Channel API      EPG API       User API
              │              │              │
              └──────────────┼──────────────┘
                             │
                     Channel Selection
                             │
                     React Player
                             │
                         HLS.js
                             │
                    HLS Master Playlist
                             │
                          CDN
                             │
                     Origin Streaming

This architecture makes the application easier to scale.

The frontend should not contain sensitive infrastructure credentials or private server configuration.

Instead, the backend can provide the information the frontend needs.

For example:

{
  "id": "channel-001",
  "name": "Example Channel",
  "logo": "https://example.com/logo.png",
  "stream": "https://example.com/live/channel.m3u8"
}

The React application consumes the API and renders the interface.

How to Optimize an IPTV Player for Better Performance

Once playback works, performance becomes the next major challenge.

One useful rule is to measure before changing configuration.

Developers sometimes increase buffer sizes, disable features, or force quality levels without knowing what problem they are trying to solve.

That can make the player worse.

Instead, monitor:

Time to first frame
Manifest loading time
Segment download time
Rebuffering events
Playback interruptions
Selected quality
Download bandwidth
Buffer duration
Channel switching time

These metrics provide much more useful information than simply asking whether a stream "feels slow."

Reduce unnecessary React re-renders

The video player should not be recreated every time an unrelated UI component changes.

Keep the HLS instance inside a dedicated component and avoid creating unnecessary objects on every render.

For example, configuration objects can be memoized when necessary.

Clean up HLS instances

If a user changes channels repeatedly, old HLS instances must be destroyed.

Otherwise, the browser can accumulate unnecessary event listeners and network activity.

The cleanup pattern is essential:

return () => {
  hls?.destroy();
};

Avoid forcing a fixed quality

For most users, adaptive bitrate is preferable to forcing the highest available resolution.

The best quality is not necessarily the best experience.

A player should adapt to real network conditions.

Common Mistakes When Building a React IPTV Player

Mistake 1: Treating HLS as a normal MP4 file

An .m3u8 URL is not equivalent to an .mp4 URL.

The HLS manifest describes a streaming presentation and may reference many media segments.

Mistake 2: Ignoring browser compatibility

Not every browser uses the same HLS playback mechanism.

Feature detection should be part of the player architecture.

Mistake 3: Putting everything inside App.jsx

A large component containing playlist parsing, API calls, playback logic, error handling, channel search, and UI rendering quickly becomes difficult to maintain.

Separate responsibilities.

Mistake 4: Ignoring CORS

If the stream server is not configured for browser access, no amount of React code will fix the problem.

Mistake 5: Retrying every error

Automatic retries are useful for transient network failures, but endless retries can create unnecessary traffic and hide real infrastructure problems.

Mistake 6: Testing only one stream

A player should be tested with different:

Bitrates
Resolutions
Segment durations
Network speeds
Browsers
Devices
Live and VOD streams

Mistake 7: Forgetting cleanup

HLS instances, timers, event listeners, and subscriptions should be cleaned up when components unmount.

IPTV Player vs Generic HLS Player

From a technical perspective, there is not necessarily a separate "IPTV protocol" that a React player must implement.

An IPTV application can simply be a user interface around standard streaming technologies.

For example:

IPTV Application
      │
      ├── Channel metadata
      ├── Playlist management
      ├── EPG
      ├── Search
      ├── Favorites
      │
      └── Video Playback
             │
             └── HLS.js
                    │
                    └── HLS

This is an important distinction for developers.

The IPTV part is often the application layer, while HLS is the media delivery technology.

Once you understand this architecture, the same knowledge can be applied to many other streaming applications.

Testing Your React HLS Player

A good development workflow should include multiple testing scenarios.

Start with a known-good HLS test stream rather than immediately testing a large playlist.

Then test:

Valid HLS stream
      ↓
Invalid URL
      ↓
Slow network
      ↓
Temporary network interruption
      ↓
Channel switching
      ↓
Unsupported browser
      ↓
Mobile browser

Browser developer tools are extremely useful during this process.

The Network tab can reveal:

Manifest requests
Segment requests
HTTP status codes
Request duration
Failed resources
CORS errors

The Console can reveal application and HLS.js errors.

Together, these tools provide a much clearer picture of what is happening inside the player.

How to Extend the IPTV React Player

Once the core player is stable, you can add more advanced functionality.

Electronic Program Guide

An EPG can provide program metadata such as:

20:00 — Movie
21:45 — News
22:30 — Documentary

The React application can display this information next to the channel list.

Favorites

Users can save frequently watched channels locally:

localStorage.setItem(
  "favoriteChannels",
  JSON.stringify(favorites)
);

For authenticated applications, favorites can instead be stored on the backend.

Search

A channel search field can filter the React state without affecting video playback.

Multiple Audio Tracks

Where the HLS stream provides multiple audio tracks, the player can expose a language selector.

Subtitles

Subtitles can be integrated using browser-supported text tracks when the media workflow provides them.

These features turn a simple HLS player into a complete streaming interface.

A Practical React IPTV Player Roadmap

If you are building this project from scratch, do not attempt to implement everything at once.

A practical development roadmap is:

Phase 1 — Basic playback

Build:

React application
HTML5 video element
HLS.js integration
HLS manifest loading
Basic controls

Phase 2 — Channel management

Add:

Channel objects
M3U parsing
Search
Categories
Channel switching

Phase 3 — Reliability

Add:

Error handling
Loading states
Retry logic
Browser compatibility
CORS configuration
Cleanup

Phase 4 — User experience

Add:

Responsive layout
Favorites
Fullscreen
Keyboard controls
EPG
Subtitles

Phase 5 — Performance

Measure:

Startup time
Channel switching time
Rebuffering
Buffer duration
Bandwidth
Quality changes

This incremental approach is much easier to debug than building a massive IPTV application in one step.

Where HLS.js Fits in the Modern Streaming Stack

It is useful to think about HLS.js as one layer in a larger system.

Content
   ↓
Encoder
   ↓
Packager
   ↓
HLS Manifest + Segments
   ↓
Origin / CDN
   ↓
Internet
   ↓
Browser
   ↓
HLS.js
   ↓
HTML5 Video
   ↓
React UI

React is responsible primarily for the application experience.

HLS.js handles the HLS playback layer.

The CDN and origin infrastructure handle media delivery.

The encoder and packaging system generate the actual streaming assets.

Understanding these boundaries helps developers troubleshoot problems much faster.

If playback fails, you can ask:

Is this a React problem?

Maybe not.

Is HLS.js receiving the manifest?

Check the network requests.

Are the media segments available?

Inspect the segment requests.

Is the browser capable of decoding the media?

Check the media errors.

Is the CDN returning the correct headers?

Inspect HTTP responses and CORS headers.

This systematic approach is much more effective than randomly changing player settings.

Final Thoughts

Building an IPTV player with React and HLS.js is a useful project for learning modern web video development because it combines several important concepts in one application.

You learn how React manages state and components, how browsers handle HTML5 video, how HLS manifests and segments work, how Media Source Extensions fit into browser playback, and how adaptive streaming responds to changing network conditions.

The first version of the player can be very small. A React component, a video element, an HLS manifest, and a few lines of HLS.js code are enough to get a basic stream playing.

The difficult part comes afterward.

A production-quality player needs proper lifecycle management, error recovery, browser compatibility, CORS configuration, responsive UI, channel management, performance monitoring, and thoughtful handling of network conditions.

That is also where the project becomes genuinely interesting from a software engineering perspective.

If you are building streaming-related applications, experimenting with legitimate HLS test streams and your own media infrastructure is a good way to understand the complete pipeline before adding more advanced features.

For developers interested in the wider IPTV and streaming ecosystem, resources and real-world service examples can also be explored through IPTV FOX PRO, VAST IPTV, and FOX IPTV Premium. These links should remain supplementary rather than becoming the focus of the technical article.

The most important takeaway is simple: a good IPTV web player is not just a video element. It is a combination of frontend architecture, streaming protocols, network delivery, browser capabilities, and careful performance engineering.

Once you understand how those pieces interact, you can use the same principles to build many other types of modern live-video applications.

Prediction Market Software: Build a Scalable Platform for Event-Based Trading

Thu, 03 Sep 2026 06:51:37 +0000

Prediction markets are changing the way people interact with future events. Instead of simply consuming information, users can express their expectations by trading on potential outcomes. From sports and financial events to elections, entertainment, and business forecasting, prediction markets create a structured environment where market prices reflect collective expectations.

Behind every successful prediction market is sophisticated prediction market software that handles market creation, pricing, trading, liquidity, user accounts, risk management, resolution, and settlement.

For businesses planning to launch their own prediction market, choosing the right technology is critical. The platform needs to deliver a fast trading experience while also giving operators the flexibility to manage markets, liquidity, users, and compliance requirements.

What Is Prediction Market Software?

Prediction market software is a technology platform that enables businesses to create and operate markets based on the outcomes of future events.

Users can typically buy or sell positions based on whether a particular outcome will occur. As traders enter the market, prices change according to supply, demand, liquidity, and the platform's pricing mechanism.

A complete prediction market platform usually includes:

Market creation and management
Order-book trading
Automated market making
Dynamic pricing
Real-time market data
User wallets and balances
KYC and payment integrations
Market resolution and settlement
Risk and liquidity controls
Admin and back-office tools
APIs and WebSocket connectivity
Reporting and audit trails

The technology behind the platform becomes especially important when thousands of users react simultaneously to breaking news, live sports events, financial announcements, or election results.

How Does a Prediction Market Platform Work?

A prediction market generally follows a simple lifecycle, although the underlying technology can be complex.

Market creation

The operator creates a question, defines the possible outcomes, sets the trading period, establishes exposure limits, and specifies how the result will be determined.

For example:

Will Team A win the match?

The market could provide Yes/No outcomes or support multiple possible outcomes.

Price discovery

Users place buy and sell orders based on their expectations. The platform's pricing engine continuously updates market prices based on trading activity.

Trading

Orders are matched through a Central Limit Order Book (CLOB), automated market maker (AMM), or a combination of both.

Market closure

When the predefined trading deadline is reached, trading stops automatically.

Resolution

The platform verifies the outcome using the configured resolution source and applies the appropriate result.

Settlement

Once the result is confirmed, the platform calculates positions and automatically settles eligible balances or rewards.

This complete workflow needs to be reliable because errors in pricing, matching, or resolution can directly affect user trust.

Key Features of Prediction Market Software

A modern prediction market platform needs more than a trading interface. It requires a complete technology stack covering the entire market lifecycle.

CLOB Trading Engine

A Central Limit Order Book allows users to place buy and sell orders that are matched against available counterparties.

A configurable CLOB engine can support:

Yes/No markets
Multi-outcome markets
Tick sizes
Configurable spreads
Order matching
Market and limit orders
Real-time order-book updates
Deterministic matching

This model is particularly useful when the platform needs transparent price discovery and exchange-style trading.

Custom AMM for Liquidity

Liquidity is one of the biggest challenges for new prediction markets. An empty market with limited counterparties can make it difficult for users to trade.

An automated market maker can provide algorithmic liquidity and help markets remain active even when trader participation is initially low.

A hybrid architecture can also combine CLOB and AMM, allowing operators to use order-book trading while maintaining baseline liquidity.

Dynamic Pricing

Prediction market prices should respond quickly to changes in trading activity.

The pricing engine can incorporate:

User orders
Market liquidity
Exposure limits
Configured pricing rules
Market conditions
Risk parameters

This creates a continuously changing market that reflects the collective expectations of participants.

AI-Powered Market Creation and Resolution

AI can reduce the operational effort required to create and manage large numbers of markets.

An AI-powered market system can assist with:

Generating market questions
Creating Yes/No and multi-outcome markets
Validating market questions
Identifying duplicate markets
Defining resolution rules
Monitoring configured information sources
Supporting outcome verification

Human approval can remain part of the workflow for markets that require additional oversight.

Real-Time Data and APIs

Prediction markets depend heavily on real-time information.

REST APIs and WebSockets can provide:

Live prices
Order-book updates
Market status
Trading activity
User information
Leaderboards
Market data

Institutional APIs can also allow market makers, brokerages, data providers, and other partners to connect directly with the platform.

Risk and Liquidity Management

Operators need granular controls to manage market exposure.

Prediction market software can include:

Per-market exposure limits
Trading throttles
Circuit breakers
Liquidity controls
Position limits
Monitoring dashboards
Risk alerts

These controls help operators respond quickly when trading activity changes sharply.

Market Resolution and Dispute Management

Resolution is one of the most important parts of a prediction market.

The platform should allow operators to define:

Resolution sources
Resolution deadlines
Outcome rules
Approval workflows
Manual overrides
Dispute processes
Audit records

A transparent resolution process can help establish trust between operators and users.

Wallet, KYC and Payment Integration

Depending on the business model and jurisdiction, the platform can integrate with wallet, identity, KYC, and payment providers.

Possible components include:

User wallets
Deposits and withdrawals
Payment gateways
KYC verification
Age verification
Geographic restrictions
Transaction monitoring
Settlement and reconciliation

The exact compliance and payment architecture should be designed around the target market and applicable regulations.

Centralized vs. Decentralized Prediction Market Software

Businesses launching a prediction market need to decide whether they want a centralized, decentralized, or hybrid architecture.

A centralized prediction market platform generally provides greater operational control over trading, market creation, resolution, user management, and compliance workflows. It can also simplify integration with traditional payment and identity providers.

A decentralized prediction market can use blockchain infrastructure and smart contracts to provide on-chain settlement and transparency, but it may introduce additional considerations such as network fees, blockchain performance, wallet management, and regulatory complexity.

The right approach depends on the business model, target users, jurisdictions, and desired level of control.

For businesses that prioritize operational control, low-latency trading, and easier integration with conventional infrastructure, centralized prediction market software can be a practical approach. Vinfotech's platform is designed around centralized architecture while keeping the option to integrate on-chain components when required.

Prediction Market Software Use Cases

Prediction markets can be applied across multiple industries.

Sports Prediction Markets

Sports organizations and operators can create markets around:

Match outcomes
Player performance
Live events
Tournament results
Matchday predictions
Seasonal competitions

Sports prediction markets can also be combined with free-to-play tournaments, leaderboards, rewards, and fan engagement features.

Financial Prediction Markets

Financial markets can be used for forecasting events such as:

Interest-rate decisions
Inflation releases
GDP figures
Commodity prices
Index levels
Economic indicators

These markets can support research, forecasting, education, and community participation.

Politics and Elections

Political prediction markets can allow users to forecast defined outcomes around elections, debates, turnout, or other measurable events.

Because political markets can be particularly sensitive, market definitions, geographic restrictions, resolution rules, and compliance controls need to be carefully designed.

Corporate Forecasting

Businesses can also use prediction markets internally to forecast:

Sales targets
Product launches
Project completion
Hiring requirements
Demand
Business milestones

Private prediction markets can provide an alternative way to collect structured forecasts from employees, teams, or selected participants.

Entertainment and Custom Markets

The same infrastructure can support markets around entertainment, awards, media events, weather, technology, and other measurable outcomes.

Why Liquidity Matters in Prediction Markets

A prediction market can have an impressive interface and still struggle if users cannot trade efficiently.

Liquidity affects:

Order execution
Price stability
Bid/ask spreads
User experience
Market participation
Trading volume

This is why modern prediction market software can combine an order-book engine with an AMM.

An AMM can help provide initial liquidity, while a CLOB can facilitate direct trading between participants as the market grows.

The result is a more flexible architecture that can adapt to different market sizes and trading conditions.

AI Is Changing Prediction Market Operations

One of the biggest opportunities in prediction market technology is automation.

Creating thousands of markets manually can require significant operational resources. AI can help operators identify potential market ideas, structure questions, validate market quality, and assist with resolution.

However, AI should not necessarily operate without supervision.

A practical architecture combines AI automation with human-in-the-loop controls. AI can handle repetitive workflows while operators retain the ability to approve markets, override decisions, investigate disputes, and review resolution data.

This approach can improve efficiency without removing operational accountability.

What to Look for in Prediction Market Software

Before selecting a prediction market software provider, businesses should evaluate several areas.

Trading Architecture

Check whether the platform supports CLOB, AMM, or hybrid trading models and whether the architecture can handle your expected trading volume.

Scalability

The system should be designed for high concurrency and sudden traffic spikes rather than only normal traffic levels.

Vinfotech states that its infrastructure is designed for 10,000+ trades per second, 100,000+ concurrent connections, and order-matching latency below 5 milliseconds. These are vendor-stated infrastructure capabilities and should be validated against the specific deployment and workload during technical evaluation.

Market Resolution

Ask how markets are resolved, what sources can be configured, and whether the platform maintains an audit trail.

Risk Controls

Look for exposure limits, throttling, circuit breakers, position controls, and monitoring tools.

APIs

Strong REST and WebSocket APIs make it easier to integrate external applications, data providers, brokerages, market makers, and mobile apps.

Customization

The platform should allow customization of branding, market types, pricing rules, user roles, payments, workflows, and business logic.

Compliance Infrastructure

Technology cannot replace legal or regulatory advice, but the software should provide the controls needed to implement the operator's compliance strategy, such as KYC integrations, geo-rules, age gates, audit logs, and reporting.

Build vs. Buy Prediction Market Software

Businesses typically have three options when launching a prediction market.

Build From Scratch

A completely custom platform provides maximum control but requires significant development time, infrastructure investment, testing, and ongoing maintenance.

Use a White-Label Platform

A white-label prediction market platform provides a ready technology foundation that can be launched under the operator's own brand.

This approach can reduce development time and provide access to existing trading, market management, wallet, API, and administrative functionality.

Customize an Existing Platform

For businesses with specific requirements, customization can provide a middle ground between building everything from scratch and using a standard product.

Vinfotech offers white-label, customization, and source-code ownership models for businesses with different levels of technical and operational requirements.

Why Choose Vinfotech for Prediction Market Software Development?

Vinfotech provides a prediction market platform designed around exchange-style infrastructure rather than a basic prediction-game model.

The platform supports:

CLOB trading
Custom AMM liquidity
Dynamic pricing
AI-assisted market creation
AI-assisted resolution
Real-time WebSocket updates
REST APIs
Wallet and payment integrations
KYC and geographic controls
Market management
Risk and liquidity controls
Resolution and dispute workflows
Free-to-play prediction tournaments
Web and mobile applications

The architecture is modular, allowing components such as settlement, KYC, payments, identity, and market data to be integrated independently.

For businesses that want to launch quickly, a white-label model can provide a faster starting point. Companies with more specialized requirements can choose deeper customization or source-code ownership.

Launch Your Prediction Market Platform

Building a prediction market is not simply about creating a trading interface. The real challenge is connecting market creation, pricing, liquidity, trading, risk management, resolution, settlement, compliance, and real-time data into one reliable system.

Whether you are building a sports prediction platform, financial forecasting product, corporate prediction market, or a broader event-trading marketplace, the technology needs to be flexible enough to evolve with your business.

Vinfotech's Prediction Market Software provides an exchange-style foundation that can be customized around your market model, target users, trading mechanism, and operational requirements.

Ready to build your own white label prediction market platform? Explore Vinfotech's prediction market software or discuss your requirements with the development team.

Kamal vs Kubernetes: An Honest Comparison for Teams Who Don’t Need 1,000 Services

Fri, 24 Jul 2026 10:00:00 +0000

The framing problem

Most “Kamal vs Kubernetes” articles are written by people who just discovered Kamal and are excited about it. They frame it as a simpler alternative that embarrasses Kubernetes. That framing is wrong and it will lead you to the wrong decision.

Kamal is not better than Kubernetes. Kubernetes is not overkill for everyone. They solve different problems at different scales, and picking the wrong one for your context is expensive either way.

This article gives you the honest version: what each tool actually is, where each one breaks, and the specific signals that should push you in one direction or the other.

What Kamal actually is

Kamal (formerly MRSK) is a deployment tool built by 37signals. It deploys Docker containers to servers via SSH, uses kamal-proxy as its default reverse proxy, and handles rolling deploys with zero downtime. That is the entire feature set.

kamal deploy

It connects to your servers over SSH, pulls the new image, starts the new container, waits for it to be healthy, then moves traffic through kamal-proxy and stops the old one. Traefik is still possible, but in Kamal 2 it sits in front of kamal-proxy as an optional accessory rather than being the standard path.

There is no control plane. No etcd. No API server. No scheduler. No concept of desired state reconciliation. Docker or systemd can restart a crashed container if you configure the right restart policy, but Kamal itself does not reschedule workloads to another node or provide cluster-level self-healing.

Kamal is a deployment tool. Kubernetes is an orchestration platform. These are not the same category.

That is the key distinction from tools like Nomad. I covered that angle in Nomad vs Kubernetes, as part of the same “lighter alternatives to Kubernetes” discussion. Nomad is a scheduler/orchestrator. Kamal is a deployment tool that leaves scheduling and node-level recovery mostly outside its scope.

What Kubernetes actually is

Kubernetes is a distributed system that continuously reconciles actual state against desired state. You declare what you want — N replicas of container X, with these resource limits, this health check, this rollout strategy — and the control plane makes it happen and keeps it that way.

When a node fails, your pods are rescheduled. When a container crashes, it is restarted. When you push a new image, the rollout respects your maxUnavailable and maxSurge settings. When traffic spikes, HPA can add replicas.

This machinery is powerful. It is also genuinely complex to operate.

A minimal production-grade Kubernetes cluster involves: a multi-node control plane for HA, a CNI plugin, an ingress controller (or Gateway API), cert-manager for TLS, a CSI driver for storage, a metrics server for HPA, RBAC policies, network policies, pod disruption budgets, and some form of secret management. Getting all of this right takes time and ongoing maintenance.

The question is whether the problems Kubernetes solves are problems you actually have.

The minimum config tells the story

Here is the same tiny web service expressed in both worlds. The exact fields vary by app, registry, ingress setup, and Kubernetes distribution, but the shape is representative.

<table> <thead> <tr> <th>Kamal <code>deploy.yml</code></th> <th>Kubernetes <code>Deployment</code> + <code>Service</code></th> </tr> </thead> <tbody> <tr> <td> <pre><code class=”language-yaml”>service: web image: ghcr.io/acme/web

servers: web: hosts: – 203.0.113.10

proxy: ssl: true host: app.example.com app_port: 3000

registry: username: acme password: – KAMAL_REGISTRY_PASSWORD

env: secret: – DATABASE_URL</code></pre> </td> <td> <pre><code class=”language-yaml”>apiVersion: apps/v1 kind: Deployment metadata: name: web spec: replicas: 2 selector: matchLabels: app: web template: metadata: labels: app: web spec: containers: – name: web image: ghcr.io/acme/web ports: – containerPort: 3000 envFrom: – secretRef: name: web-secrets — apiVersion: v1 kind: Service metadata: name: web spec: selector: app: web ports: – port: 80 targetPort: 3000</code></pre> </td> </tr> </tbody> </table>

The Kubernetes example is still incomplete for internet traffic and TLS: you would normally add an Ingress or HTTPRoute, cert-manager, DNS, and secret provisioning. Kamal’s example is closer to “deploy this app to these servers.” Kubernetes is closer to “declare this workload inside a platform that already exists.”

The 37signals case

37signals (the company behind Basecamp and Hey) moved off cloud Kubernetes to bare-metal servers running Kamal in 2022-2023. Their own Ops team wrote about the “de-cloud and de-k8s” move. The story got a lot of attention.

What the hot takes missed: 37signals is not a startup. They have a dedicated ops team. They own and operate hardware. Their workload is well-understood and stable: a small number of mature applications serving a large, established user base.

Their decision was rational for their context. Kubernetes was costing them in cloud spend and operational complexity for problems their workload does not have. Moving to Kamal on owned hardware made economic and operational sense for them.

It does not follow that Kamal is the right choice for your early-stage startup, your company that runs on AWS because your team does not want to manage hardware, or your platform that needs to scale from 0 to 10x on short notice.

Where Kamal breaks

No cluster-level self-healing. If your container crashes, Docker or systemd can restart it with the right policy, but Kamal has no involvement. If your server fails, Kamal will not reschedule that workload elsewhere; your service is down until you intervene or until you have set up something else to handle it.

No built-in auto-scaling. Kamal has no equivalent of HPA or KEDA. If your traffic spikes, you add servers manually or script something yourself.

No multi-AZ HA by default. You can deploy to multiple servers across regions, but the coordination is yours to manage. There is no concept of spreading replicas across availability zones automatically.

Rolling deploys are basic. Kamal’s zero-downtime deploy is good for single-server or simple multi-server setups. It is not a substitute for Kubernetes rolling update with fine-grained health check integration across a fleet.

Secret management is manual. Kamal reads secrets from .kamal/secrets and has kamal secrets helpers for fetching and extracting values from password managers and cloud secret stores, but there is no Kubernetes-style projected secret lifecycle. Rotation and policy remain yours to manage.

No workload isolation. Everything on the same server shares the same kernel, same resources. Resource limits are Docker-level, not enforced by an orchestration layer with quotas and namespaces.

Where Kubernetes breaks

The complexity tax is real. Running Kubernetes yourself — not a managed service, but actual cluster operation — requires dedicated expertise. A team without a platform engineer will spend significant time debugging networking issues, understanding why pods are Pending, or figuring out why cert-manager is not issuing certificates.

Managed K8s is not free. EKS charges for the cluster control plane, with standard support priced around $0.10/hour per cluster; GKE and AKS have their own pricing rules and credits. In all cases, you still pay for nodes, load balancers, and persistent volumes. For a small application, this cost can exceed what bare-metal hosting would cost.

The YAML surface area scales with complexity. A simple web app on Kubernetes requires a Deployment, a Service, an Ingress (or HTTPRoute), a Certificate, potentially a HorizontalPodAutoscaler, a PodDisruptionBudget, and NetworkPolicies. That is a lot of infrastructure for a CRUD app.

Debugging is harder. Distributed systems are harder to debug than single-server setups. kubectl exec and kubectl logs are good tools but diagnosing why traffic is not reaching your pod — DNS, CNI, network policy, service selector, readiness probe — requires Kubernetes-specific knowledge that takes time to build.

The honest decision framework

Ask yourself these questions in order:

1. Do you have more than one team working on independent services with different scaling needs?

If yes: Kubernetes starts making sense. The isolation and per-service resource management are worth the overhead.

If no: Kamal is probably sufficient.

2. Do you need to survive individual server failures automatically, without manual intervention?

If yes: You need either Kubernetes with proper HA setup, or a managed container platform whose recovery and placement model fits your app. Cloud Run, Fly.io, Render, and Railway all help here, but their HA boundaries differ. Kamal alone does not give you this.

If no: A Docker or systemd restart policy covers the common crash case.

3. Does your traffic profile require automatic scaling?

If yes: Kubernetes with HPA or KEDA, or a managed platform with the scaling behavior you actually need. Cloud Run scales request-driven services to and from zero; Fly.io supports autostart/autostop and machine scaling; Render and Railway support scaling, but with provider-specific limits and billing. Kamal has no answer here without custom tooling.

If no: Fixed capacity is fine. Kamal works.

4. Do you have a platform team, or is “ops” a shared responsibility among product engineers?

If platform team exists: Kubernetes operational overhead is distributed and manageable.

If ops is everyone’s side job: Kamal’s simplicity is a genuine advantage. Less to go wrong, less to learn, faster to debug.

5. Are you on cloud infrastructure you do not own, and cloud costs matter?

If yes: Run the numbers on managed K8s vs a few VPS instances. For small workloads, VPS + Kamal is significantly cheaper.

If no: You have hardware already. Kamal is almost certainly the right choice.

Side-by-side

	Kamal	Managed container platforms	Kubernetes
Examples	Kamal on VPS/bare metal	Cloud Run, Fly.io, Render, Railway	EKS, GKE, AKS, self-managed
Deployment model	SSH + Docker	Platform API / Git deploy / container deploy	API-driven, reconciliation loop
Self-healing	Docker/systemd restart policy only	Platform-managed restart and placement	Pod rescheduling, node failure recovery
Auto-scaling	Manual	Provider-specific; not all scale the same way	HPA, VPA, KEDA
HA	Manual (multiple servers)	Provider-managed, with platform limits	Available with a well-configured multi-node control plane (not free)
TLS	kamal-proxy with automatic HTTPS; Traefik optional	Usually built in	cert-manager + ingress controller
Secret management	`.kamal/secrets` + helper commands	Platform secrets	Secrets API, external-secrets, Vault
Observability	Standard Docker logging	Provider logs/metrics, export varies	Rich ecosystem (Prometheus, OTel, etc.)
Rollout control	Basic rolling deploy	Simple rollouts, provider-specific	Fine-grained (maxUnavailable, canary, etc.)
Learning curve	Low	Low to medium	High
Ops overhead	Low	Low	Medium to high
Right scale	1-20 services, stable load	Small to medium teams that want managed recovery/scaling	10+ services, variable load, or regulated

What the threshold actually looks like

Kamal is the right default if you are:

A team of 2–10 engineers shipping a web application
Running stable, predictable workloads
On a budget where managed K8s costs matter
Without a dedicated platform engineer

Kubernetes is the right choice when:

You have 10+ services with independent deployment cycles
You need multi-zone HA and automatic failover
Traffic is variable enough that auto-scaling saves real money or prevents real incidents
You have a platform team that can absorb the operational complexity
You are in a regulated environment that benefits from K8s’s audit trails and RBAC

There is a middle tier worth mentioning: managed container platforms like Fly.io, Railway, Render, and Google Cloud Run. These give you some of Kubernetes’s operational benefits with closer to Kamal’s operational simplicity, but the details matter: HA, auto-scaling, scale-to-zero, regions, and billing differ by provider. For teams that need more than Kamal but do not want to operate K8s, this tier is often the right answer and gets underrepresented in the debate.

The real lesson from 37signals

The story is not “Kamal beat Kubernetes.” The story is: match your infrastructure to your actual operational profile, not to what is fashionable or what scales to Google’s size.

37signals evaluated what problems they actually had and picked the tool that solved those problems at the lowest operational cost. That is the right framework.

For most teams reading this, Kubernetes is probably the right answer eventually — when your system’s complexity justifies it. The mistake is adopting it before you are there, burning engineering time on infrastructure problems instead of product problems.

Kamal is a good tool for a specific stage and scale. Use it until you outgrow it. When you outgrow it, you will know — because the things Kubernetes solves will be actual problems you have, not hypothetical ones.

What to do next

Before you choose, write down your actual requirements in one page: number of services, expected traffic variance, failure tolerance, who owns ops, whether you own hardware, and what compliance needs are real today. Then run a small proof of concept with the simplest tool that satisfies those requirements. If Kamal plus Docker restart policies covers the failure modes you actually accept, start there. If you need automatic placement, autoscaling, and multi-zone recovery now, compare managed container platforms first, then Kubernetes.

FAQ

Can Kamal and Kubernetes coexist?

Yes. Some teams use Kamal for simple auxiliary services (internal tools, cron jobs, simple APIs) while running their core platform on Kubernetes. The tools are not mutually exclusive.

Is Kamal production-ready?

Yes. 37signals runs Basecamp and Hey on it. It handles zero-downtime deploys, TLS, and multi-server deployments reliably. The limitation is not reliability — it is feature scope.

Platform Ecology: Trophic Cascades — Count the Cascade, Not the Keystone

Thu, 04 Jun 2026 01:13:51 +0000

Originally published at vibeagentmaking.com

In 1963, a zoologist named Robert T. Paine began prying sea stars off rocks at Mukkaw Bay, Washington. Before the removals, his plot held fifteen species. Within five years, it was a solid mat of one organism. One species had been holding fourteen others in existence.

The Cascade Is the Product

Sea stars don't eat algae or limpets. Those species disappeared because mussels overgrew them once the starfish stopped regulating the mussels. A trophic cascade is a chain of indirect effects -- always nonlinear, frequently counterintuitive.

AWS does not interact with Netflix subscribers or Ring doorbell owners. When US-East-1 failed for eight hours in December 2021, the cascade reached services whose owners never thought of themselves as AWS customers. The hub was invisible until it was gone.

Sea Otters and the Context Problem

Off Vancouver Island, sea otter recovery produced a strong cascade: urchins down, kelp up. Off San Nicolas Island, the same predator produced a measurably weaker cascade. The ecosystem's surroundings determined how much the keystone's job was worth.

Microsoft's Windows cascade ran hard through enterprise in the 1990s and barely turned in consumer mobile. The strategic moves were similar; the ecosystem was different.

Yellowstone Wolves and the Return Problem

Wolves were reintroduced to Yellowstone in 1995-96 after seventy years. Park-wide aspen cover dropped from several percent to under 1% and has not reversed at landscape scale. Removing the keystone can flip the ecosystem into an alternative stable state that resists reversal.

When BlackBerry tried to bring developers back, the mobile ecosystem had stabilized into iOS-plus-Android. The surrounding ecosystem had stopped being the one that produced the original cascade.

The Invisible Cascade in Tech

Twitter's API repricing in 2023 cascaded through research labs, civic early-warning systems, and data-journalism pipelines. Heroku's free-tier shutdown evaporated a decade of developer experimentation.

The pattern: keystones regulate levels below them. When regulation stops, the cascade rearranges levels you forgot the keystone was holding in place.

Where the Analogy Breaks

Three breaks: (1) Sea stars don't strategize or litigate; platforms do. (2) Time constants differ by orders of magnitude. (3) The convergence is real -- both fields independently found context-dependent cascade strength, alternative stable states, and mesopredator release.

Back to the Rock

Paine's plot is still mussel-dominated decades later. The alternative stable state held.

Count the ecosystem. Not the keystone -- the cascade.

Kubernetes vs Keda

Tue, 05 May 2026 06:37:27 +0000

Kubernetes 1.36 lands on April 22, and one of the headline features is something the community has been asking for since Kubernetes 1.16: the HPAScaleToZero feature gate, now in beta. The Horizontal Pod Autoscaler can finally scale a deployment down to zero replicas, and back up, without any third-party tooling.

Within hours of the release notes circulating, LinkedIn filled up with "KEDA is finished." The problem here is a category error. HPA and KEDA don't scale workloads for the same reason. They react to entirely different signals:

Kubernetes HPA (Horizontal Pod Autoscaler): The Core Focus is resource-based scaling (CPU and memory). Now, in the version 1.36, the HPA supports scaling down to zero replicas when resource utilization hits zero, and scaling back up when demand increases.
KEDA (Kubernetes Event-driven Autoscaling): The core focus here is the Event-driven scaling. KEDA scales the PODs based on external metrics rather than resource consumption. Examples include queue depth, Kafka consumer lag, Cron schedules, or database row counts. This approach is ideal for workloads where CPU/memory metrics are poor indicators of demand, such as background job processors or message consumers.

Think about a batch job that processes messages from an SQS queue. When the queue is empty, you want zero pods. When messages arrive, you want pods: fast, proportional to how many messages are waiting. CPU utilization on those pods tells you almost nothing useful about this. The queue depth is the signal. That's what KEDA reads.

Now think about a stateless API service. You want it scaled down at 3am when there's no traffic, and scaled back up when requests start coming in. CPU and memory are perfectly reasonable proxies here. HPA handles this well, and now, after years of waiting, it can go all the way to zero without you having to wire up KEDA or Knative just to get that last step.

These are different workloads with different operational profiles. The new HPA feature is genuinely useful. It closes a real gap for resource-based scaling, and it means one less reason to reach for an external tool if your scaling logic is purely CPU or memory driven.

But if you're running event-driven consumers like: Kafka, RabbitMQ, SQS, or any of the 60+ scalers KEDA supports, you're not going to replace it with HPA. You'd be throwing away the exact thing that makes your scaling logic correct and replacing it with a metric that doesn't describe your problem.

KEDA's website describes it exactly right: it's an event-driven autoscaler. The whole value proposition is that your scaling decisions live in the same conceptual space as your workload's actual behavior. A message consumer scales on message count. A scheduled job scales on a cron expression. A database processor scales on queue table depth. That's not something a CPU metric can replace, because CPU is not the right metrics to decided when scale in these use cases.

The honest summary: Kubernetes 1.36 is great news if you've been manually hacking around the HPA floor of 1. For anything event-driven, KEDA was already the right tool, and it still is. One more native feature doesn't change what problem it was solving.

Anti-Cargo-Cult Platform Engineering for Kubernetes at Scale (Part 4)

Wed, 22 Apr 2026 00:12:10 +0000

Anti-Cargo-Cult Platform Engineering for Kubernetes at Scale

A White Paper on Talos Linux and Omni — Part 4 of 4

This is Part 4, the final part of a four-part series.

Part 1 — The cargo cult problem and Talos as an anti-pattern breaker

Part 2 — Day-2 operations at scale

Part 3 — Omni and the uncomfortable verdict

Part 4 (this post): Feynman's Ghost — the wider lesson and where to go from here

Recap so far: Cargo cult engineering — rituals performed without understanding — is endemic in modern infrastructure. Talos Linux refuses to let you fool yourself: no SSH, no shell, API-only, immutable. Omni is a useful management plane but dangerous as a comfort blanket — start with the Talos API and understand the system before adding the abstraction. The verdict: Talos doesn't make Kubernetes easier. It makes bullshit harder. Most teams won't adopt it for exactly that reason.

Now we confront what cargo cult engineering actually costs — not in downtime, but in the NASA hearing room where seven astronauts' deaths were explained by the same pattern. This is the wider lesson.

Epilogue: Where to Go From Here

If You're Considering Talos

Note: Don't adopt Talos because this paper convinced you. Adopt it because you understand why it exists and what problems it solves.

Start small. Deploy a test cluster. Try to operate it without looking for the SSH escape hatch. When you hit something you don't understand, resist the urge to find a workaround. Dig into the documentation. Understand the API. Learn why Talos made the design decisions it made.

If this feels unnecessarily difficult, ask yourself: Is this actually difficult, or am I just uncomfortable because I can't use my usual crutches?

If you find yourself thinking "I could solve this if I just had shell access," stop. That thought is the cargo cult speaking. The correct thought is: "How would I solve this if shell access was architecturally impossible?"

Once you can operate a small cluster confidently through declarative configuration alone, you understand Talos. Scaling from there is just operational logistics.

If You're Already Using Talos

Note: Don't let Talos become your new cargo cult. The risk isn't that Talos makes things too hard — it's that Talos makes things hard enough that you build new rituals without understanding.

You memorize machine config patterns without knowing why they work. You copy-paste from documentation without understanding the implications. You build Terraform modules that hide complexity you never learned.

This is still cargo cult engineering. You've just swapped the rituals.

The goal isn't to use Talos. The goal is to understand infrastructure deeply enough that Talos makes sense. If you're using Talos but still feeling like you're guessing, you haven't escaped the cargo cult — you've just found a new runway to build.

If You're Evaluating Omni

Note: Ask yourself: what problem does Omni solve that I can't solve with the Talos API? If the answer is "I don't want to learn the Talos API," don't use Omni. Learn the API first.

If the answer is "I need centralized fleet management, policy enforcement, and observability at scale," then Omni might be valuable — but only after you've operated Talos directly long enough to know what you're managing.

Omni is an amplifier. Make sure you're amplifying understanding, not amplifying cargo cult patterns at scale.

If You're Not Using Talos and Don't Plan To

Note: That's fine. Talos isn't the only valid approach. But the principle is universal — you must not fool yourself about your infrastructure.

You don't need Talos to stop SSHing into nodes. You need discipline. You don't need Talos to operate declaratively. You need to commit to declarative operations. You don't need Talos to eliminate configuration drift. You need to stop making manual changes.

Talos enforces these patterns architecturally. You can enforce them culturally with any infrastructure. It's just harder because you have to maintain discipline when the escape hatch is available.

The question isn't "Should I use Talos?" The question is "Am I operating my infrastructure with intellectual honesty, or am I building cargo cult patterns and hoping they keep working?"

The Real Takeaway

This paper used Talos Linux and Omni as examples, but the real subject is how you think about infrastructure.

Are you copying patterns without understanding them? Are you relying on rituals that feel necessary but might not be? Are you confusing "it works" with "I understand why it works"?

These questions matter regardless of your technology choices.

The cargo cult is everywhere. Kubernetes without understanding. GitOps without knowing why. Observability dashboards that show metrics you don't comprehend. Infrastructure as code that's really just scripts you're afraid to modify.

Talos is interesting because it makes the cargo cult impossible. But you don't need Talos to stop participating in the cargo cult. You just need to be honest with yourself about what you understand and what you don't.

Feynman's Ghost

Richard Feynman never used Kubernetes. He never deployed containers. He never wrote YAML.

But his principle applies perfectly to modern infrastructure:

The first principle is that you must not fool yourself — and you are the easiest person to fool.

Feynman's last major public act was exposing cargo cult engineering at NASA.

In January 1986, the Space Shuttle Challenger exploded 73 seconds after launch, killing all seven crew members. NASA convened a presidential commission to investigate. Feynman was appointed to the commission.

What he found was institutional cargo cult at its most lethal.

NASA engineers knew the O-rings lost elasticity in cold temperatures. They had data. They had test results. They had documented failure modes. The night before launch, engineers recommended against launching because temperatures were below safety thresholds.

Management launched anyway. Not because they evaluated the engineering data and disagreed. Because the process said to launch. Because they'd launched before and succeeded. Because admitting the O-ring problem would delay the program.

The ritual overrode reality. Seven astronauts died.

Feynman demonstrated the failure on live television during the hearing. He took an O-ring, dropped it in ice water, and showed how it lost elasticity. Not because NASA didn't know — they did. But they'd stopped believing what they knew. The cargo cult had become institutional.

The form was perfect. The process was followed. The rituals were performed.

The shuttle exploded anyway.

Feynman died in February 1988, barely two years after Challenger. His final fight was against exactly what this paper describes: experts performing rituals they no longer understood, organizations following processes they no longer questioned, hoping the planes would land.

If cargo cult engineering can kill astronauts at NASA, it can certainly destroy your infrastructure.

There's an Italian phrase in infrastructure engineering:

È un lavoro dove è fondamentale capire per fare, fare senza capire non serve: è solo inutile.

Understanding is fundamental to doing. Doing without understanding isn't just ineffective — it's pointless.

This is the cargo cult problem distilled. You can follow the steps without understanding them. You can deploy infrastructure without comprehending it. You can operate systems you don't grasp.

But when they fail — and they will fail — you have nothing. No mental model to debug from. No understanding to guide repair. Just rituals that stopped working and hope that repeating them harder will somehow fix the problem.

Talos forces understanding before doing. That's uncomfortable. That's the point.

We fool ourselves constantly in infrastructure engineering. We fool ourselves that we understand systems we don't. We fool ourselves that our operations are sustainable when they're held together with manual interventions. We fool ourselves that we're engineering when we're really just cargo cult building.

Talos is one answer to this problem in one domain. It's not the only answer. But it's an honest answer.

It doesn't pretend to make things easy. It doesn't promise convenience. It doesn't hide complexity behind abstractions.

It makes you confront what you don't understand. It forces you to build knowledge instead of rituals. It refuses to let you fool yourself.

That's uncomfortable. That's the point.

The Cargo Cult Beyond Kubernetes

The cargo cult isn't unique to Kubernetes or Talos. It's everywhere in our industry.

Cloud engineering: We copy Terraform modules from GitHub without understanding state management. We cargo cult AWS reference architectures without knowing why they're structured that way. We deploy "infrastructure as code" that's really just imperative scripts wrapped in declarative syntax. We import configurations and policies into identity platforms via DevOps pipelines without understanding the permission models we're creating. We use Terraform, Bicep, or direct JSON imports to deploy Entra ID conditional access policies, AWS IAM roles, GCP IAM bindings — treating identity platforms as deployment targets instead of security boundaries that require architectural understanding. We cargo cult the syntax from examples without comprehending the access model we're creating.

Systems engineering: We use Ansible playbooks we found online, modifying variables without understanding what the tasks actually do. We follow runbooks written by people who left the company years ago, performing rituals no one remembers the reason for.

Security operations: We deploy tools because compliance frameworks require them, not because we understand the threats they mitigate. We generate reports no one reads, run scans no one acts on, maintain "security" that's really just checkbox theatre.

Platform engineering: We build "developer platforms" that abstract away complexity engineers need to understand. We create "golden paths" that are really just cargo cult patterns institutionalized. We celebrate "reducing cognitive load" when we're really just enabling ignorance at scale.

The disease is universal. This paper focuses on Kubernetes and Talos because that's a concrete, testable domain where cargo cult can be demonstrated and defeated. But the principle applies everywhere.

You must not fool yourself about your infrastructure. And you are the easiest person to fool.

Talos is one forcing function in one domain. The real question is: what are you doing to stop fooling yourself in yours?

The islanders never got their cargo back. The wooden headphones never summoned the airplanes. The bamboo control tower never brought the planes.

But somewhere, someone built an actual runway. Installed actual navigation systems. Trained actual air traffic controllers. Did the hard work of understanding instead of imitating.

And the planes landed.

References and Further Reading

Richard Feynman's Cargo Cult Science Speech (1974) — Caltech commencement address. The foundational text for understanding cargo cult thinking in technical fields. calteches.library.caltech.edu/51/2/CargoCult.htm
Douglas R. Hofstadter — Gödel, Escher, Bach: An Eternal Golden Braid (1979). An exploration of consciousness, self-reference, and strange loops through the lens of mathematics, art, and music. Demonstrates how complex systems achieve understanding by stepping outside themselves — a principle directly applicable to infrastructure operations.
JYSK Engineering Blog: 3000 Clusters Series
Talos Linux Documentation — talos.dev
Omni Documentation — omni.siderolabs.com

About This Paper

This white paper was written for engineers who are tired of cargo cult infrastructure. It is intentionally opinionated, deliberately uncomfortable, and grounded in real-world operational experience.

The goal is not to convince you to use Talos. The goal is to make you question whether you truly understand the infrastructure you're operating, or whether you're performing rituals that look like expertise but lack understanding.

If this paper made you angry, defensive, or uncomfortable — good. That discomfort is worth examining. It might be revealing something you need to confront.

If this paper confirmed what you already suspected about modern infrastructure — good. You're not alone in feeling like we've built too many abstraction layers on top of insufficient understanding.

If this paper made you want to learn Talos — good. But learn it for the right reasons. Not because it's easier, but because it forces better thinking.

And if this paper made you think "this doesn't apply to me" — that's fine too. But ask yourself one more time: Are you sure? Or are you just wearing wooden headphones?

The first principle is that you must not fool yourself — and you are the easiest person to fool.
— Richard Feynman, 1974

Anti-Cargo-Cult Platform Engineering for Kubernetes at Scale (Part 3)

Wed, 22 Apr 2026 00:12:05 +0000

Anti-Cargo-Cult Platform Engineering for Kubernetes at Scale

A White Paper on Talos Linux and Omni — Part 3 of 4

This is Part 3 of a four-part series.

Part 1 — The cargo cult problem and Talos as an anti-pattern breaker

Part 2 — Day-2 operations at scale

Part 3 (this post): Omni and the uncomfortable verdict

Part 4: Feynman's Ghost — the wider lesson and where to go from here

Recap so far: Cargo cult engineering — the rituals we perform without understanding — is the root cause of most modern infrastructure failures. Talos Linux refuses to let you fool yourself: no SSH, no shell, API-only, immutable. At scale, this forces honest operations — you either instrument properly or you don't run Talos.

Now we examine the control plane that markets itself as a solution to that complexity — and we arrive at the verdict the industry would rather not hear.

Section 4: Omni — Control Plane or False Idol?

What Omni Actually Solves

Omni is Talos's centralized management platform. It provides a control plane for managing fleets of Talos clusters — provisioning, configuration, upgrades, observability, access control.

At first glance, this seems to contradict everything Talos stands for. Talos forces you to understand your infrastructure through APIs and declarative state. Omni gives you a web UI and abstractions. Isn't this just adding a new cargo cult layer?

Not if you use it correctly.

Omni solves real problems at scale:

1. Fleet-Level Visibility. When you operate 100+ clusters, you need centralized observability. Which clusters are on which Kubernetes versions? Which nodes need patches? Where are failures occurring?

You could build this yourself using the Talos API and custom tooling. Or you could use Omni, which does it out of the box.

2. Policy Enforcement. You want all production clusters to run specific Talos versions. You want all nodes to have specific security configurations. You want upgrades to happen in specific maintenance windows.

Omni lets you define these policies centrally and enforce them across your fleet. This is governance, not abstraction.

3. Operational Efficiency. Creating new clusters, adding nodes, and managing lifecycle operations across hundreds of clusters is tedious through individual API calls.

Omni reduces toil without hiding complexity. You're still declaring intent — you're just doing it through a central control plane instead of per-cluster API calls.

The Dangerous Seduction

But here's the risk: Omni has a UI. And UIs are comfortable. They let you click buttons without understanding what's happening underneath.

This is where the new cargo cult emerges.

Instead of SSHing into nodes and running commands, you click buttons in Omni and "just make it work." Instead of understanding Talos machine configs, you use Omni's templates and trust they're correct. Instead of learning the Talos API, you rely on Omni's abstractions.

You've replaced the wooden headphones with a web dashboard.

JYSK could have used Omni to make their 3,000-cluster deployment "easier." But if they'd done that without understanding the underlying architecture, they would have simply moved their cargo cult from K3s to Talos+Omni.

The registry DDoS would still have happened. The PXE boot complexity would still have bitten them. The difference is they would have been debugging through Omni's abstractions instead of understanding the system directly.

How to Use Omni Without Bullshitting Yourself

Omni is an operational amplifier. It makes good operations better and bad operations worse.

If you understand Talos, Kubernetes, and distributed systems, Omni helps you operate at scale. If you don't understand those things, Omni just gives you new ways to create unmaintainable complexity.

Use Omni for:

Fleet-level observability — Seeing the state of all clusters at once
Policy enforcement — Defining and enforcing governance rules centrally
Operational efficiency — Reducing toil for operations you already understand
Access control — Centralized RBAC for your entire infrastructure

Don't use Omni for:

Hiding from complexity — Clicking buttons without understanding what they do
Emergency fixes — Treating the UI as a "better SSH"
Bypassing understanding — Using templates you don't comprehend
Replacing architecture — Hoping Omni will solve design problems

The test is simple: Can you accomplish the same operation through the Talos API? If you can't, you don't understand what Omni is doing for you.

The Single Pane of False Confidence

Infrastructure teams love "single pane of glass" solutions. One dashboard to rule them all. Everything visible in one place. Every operation one click away.

This is seductive. It's also dangerous.

A single pane of glass is only as good as your understanding of what you're looking at. If you don't understand the underlying systems, the dashboard doesn't help — it just gives you a false sense of control.

Omni gives you visibility into your Talos fleet. That visibility is valuable if you know what you're looking for. It's worthless if you're just staring at green lights hoping they stay green.

The danger is treating Omni as a replacement for understanding. Treating it as "Kubernetes management made easy." Treating it as something that lets you operate infrastructure you don't comprehend.

That's cargo cult engineering with better UX.

When to Adopt Omni

The decision to use Omni isn't about features or convenience. It's about whether abstraction helps or hides.

Ask these questions:

Do you understand Talos deeply enough to know what Omni is doing underneath?

If you can't explain how Omni's machine config templates work, how it orchestrates upgrades, or how it manages cluster lifecycle — don't use it yet. You're trusting an abstraction you don't understand.

Does your operational scale justify centralized management?

At 3-5 clusters, Omni might be premature. At 30-50 clusters, it becomes valuable. At 300+ clusters, it's essential. But only if you already understand what you're managing.

Can you operate without Omni if it fails?

If Omni's control plane has an outage, can you manage your Talos clusters directly through their APIs? If not, you've created a single point of failure in your understanding, not just your infrastructure.

The test is simple: If you can accomplish the same operations through the Talos API that you're doing through Omni's interface, then Omni is helping. If you can't, then Omni is hiding.

Start with the API. Understand the system. Then add the abstraction layer when operational scale justifies it. Not before.

Section 5: The Uncomfortable Conclusion

Talos Does Not Make Kubernetes Easier

Let's be direct: Talos is harder than traditional Kubernetes deployments. At least initially.

You can't SSH in to debug. You can't manually edit configs. You can't apply quick fixes. You can't follow the same runbooks you've been using for years.

You have to understand declarative state management. You have to understand the Talos API. You have to instrument properly from the start. You have to think through failure modes before they happen.

This is not "Kubernetes made simple." This is "Kubernetes done correctly, which is hard."

If you're looking for something easier, Talos isn't it. There are dozens of "easy Kubernetes" solutions. They'll let you get started faster. They'll let you deploy workloads without understanding the platform. They'll work great until they don't.

Talos makes different trade-offs. It makes early operations harder in exchange for making scaled operations sustainable.

It Makes Bullshit Harder

Here's what Talos actually does: it removes your ability to bullshit.

You can't claim you understand your infrastructure if you can't operate it declaratively. You can't pretend you've got everything under control if you need SSH access for routine operations. You can't hide poor architecture behind manual fixes.

Talos is infrastructure as discipline. Not convenience. Discipline.

This is exactly why it works.

The rituals that feel necessary — SSH access, manual debugging, imperative fixes — are the wooden headphones of systems administration. They look right because that's what you've always seen. They feel necessary because you've always used them.

But they're not necessary. They're cargo cult.

Why Do You Need SSH Anyway?

The question isn't "how do I operate Kubernetes without SSH?" The question is "why do I think I need SSH to operate Kubernetes?"

If your answer is "because I might need to debug something," then you're admitting your instrumentation is insufficient. Fix your instrumentation.
If your answer is "because I need to check logs," then you're admitting your logging infrastructure is inadequate. Fix your logging.
If your answer is "because sometimes I need to try things and see if they work," then you're admitting you don't understand your system well enough to predict its behavior. Learn your system.

SSH is a crutch. Talos takes away the crutch and forces you to walk properly. Yes, it's harder. Yes, you might fall. That's how you learn.

The Learning Curve Is the Point

Traditional infrastructure lets you succeed without understanding. You can copy-paste configurations, follow tutorials, and get things mostly working. You can operate at small scale indefinitely without ever building deep knowledge.

Talos doesn't allow this. The learning curve is steep by design. You can't fake understanding.

This means junior engineers struggle more initially with Talos than with traditional systems. They can't pattern-match from Stack Overflow. They have to actually learn.

But it also means that once they learn Talos, they actually understand distributed systems, declarative state management, and infrastructure as code. Not as buzzwords, but as operational reality.

Senior engineers struggle differently. They have to unlearn habits built over decades. They have to admit that some of their expertise is expertise in cargo cult patterns.

Both groups emerge better engineers. But only if they're willing to be uncomfortable during the learning process.

Infrastructure as Discipline

Feynman's cargo cult science speech ends with a simple principle:

The first principle is that you must not fool yourself — and you are the easiest person to fool.

Talos embodies this principle. It refuses to let you fool yourself.

You can't fool yourself about state — it's immutable and declared. You can't fool yourself about operations — they're API-driven and auditable. You can't fool yourself about understanding — if you can't operate it declaratively, you don't understand it.

This is uncomfortable. Discipline always is.

But the alternative is building bamboo control towers and wondering why the planes don't land.

What Success Looks Like

Success with Talos doesn't look like ease. It looks like:

Confidence in your infrastructure — Not because nothing ever breaks, but because when things break, you understand why
Reproducible operations — Everything you do can be codified, version-controlled, and repeated
Scaled sustainability — Operating 100 clusters isn't 100x harder than operating 1 cluster
Eliminated superstition — You don't have rituals you perform without understanding
Reduced heroics — Operations don't require senior engineers making emergency fixes at 3 AM

JYSK achieved this. They went from 3,000 bespoke K3s deployments to 3,000 identical Talos appliances. When they need to patch, they update a machine config. When they need to debug, they query structured logs. When they need to upgrade, they declare a new version and let the system reconcile.

It's not easier. It's better.

The Final Provocation

If you finish reading this paper and think "this doesn't apply to my infrastructure," you're probably right. Most infrastructure doesn't need Talos. Most teams can continue SSHing into nodes and manually fixing things indefinitely.

But if you finish reading this paper and feel defensive — if you find yourself thinking "but we NEED SSH because..." or "our operations are different because..." — then you should ask yourself: Are those actual architectural requirements, or are they wooden headphones?

Talos Linux exists to make a specific point: Modern infrastructure doesn't need the operational patterns we inherited from the 1970s. We keep using them because they're familiar, not because they're necessary.

The cargo cult continues because the ritual feels like expertise. The wooden headphones look convincing because that's what we saw the experts wearing.

But the planes still don't land.

Acknowledgment of Pain

Let's be honest about something else: Even if you understand all of this, even if you believe Talos is the right approach, even if you commit to operating infrastructure without cargo cult patterns — it's still painful.

Learning new mental models is painful. Admitting your expertise might be built on shaky foundations is painful. Rebuilding infrastructure you thought was working is painful.

That pain is not a sign you're doing something wrong. It's a sign you're doing something real.

Feynman talked about "leaning over backwards" to not fool yourself. That's painful. It requires intellectual honesty that most people aren't willing to commit to. It's easier to keep building bamboo antennas.

But if you're reading this paper, you're probably someone who's tired of the bamboo antennas. Tired of pretending. Tired of infrastructure that works until it doesn't, with no clear path to understanding why.

Talos won't make the pain go away. It redirects it. Instead of pain at 3 AM when production breaks and you don't know why, you get pain during design when you're forced to understand your system before deploying it.

Most people prefer the 3 AM pain. It feels like heroism. It generates war stories. It looks like expertise.

The design pain feels like failure. Like you should already know this. Like admitting you don't understand.

But that's exactly the pain worth experiencing.

Continue to Part 4: Feynman's Ghost — where we confront what cargo cult engineering actually costs. Not in downtime. In the NASA hearing room, on live television, in 1986.

Anti-Cargo-Cult Platform Engineering for Kubernetes at Scale (Part 2)

Wed, 22 Apr 2026 00:11:57 +0000

Anti-Cargo-Cult Platform Engineering for Kubernetes at Scale

A White Paper on Talos Linux and Omni — Part 2 of 4

This is Part 2 of a four-part series.

Part 1 — The cargo cult problem and Talos as an anti-pattern breaker

Part 2 (this post): Day-2 operations at scale

Part 3: Omni and the uncomfortable verdict

Part 4: Feynman's Ghost — the wider lesson and where to go from here

Recap of Part 1: Most modern infrastructure failures aren't caused by missing tools — they're cargo cult engineering. Copy-paste YAML, blind trust in abstractions, rituals mistaken for knowledge. Talos Linux refuses to let you fool yourself: no SSH, no shell, API-only, immutable OS. It doesn't make Kubernetes easier. It makes bullshit harder.

Now we examine what happens when you try to operate this at scale — and where every comfortable shortcut collapses.

Section 3: Day-2 Operations at Scale

Where Cargo Cults Collapse

"Day 1" operations are easy. Deploying your first Kubernetes cluster is well-documented. Getting "hello world" running feels like success. Every abstraction layer works exactly as promised when you're operating at trivial scale with trivial requirements.

Day 2 is where the cargo cult collapses.

Day 2 is when you have 100 clusters. When you need to upgrade Kubernetes versions across a fleet. When you need to patch CVEs within an SLA. When you need to debug why 3 nodes out of 3,000 are behaving differently. When you need to understand why something failed, not just that it failed.

Day 2 is when "it works" stops being good enough.

The JYSK Edge Reality Check

JYSK's blog series is a masterclass in what happens when cargo cult engineering meets operational reality.

Part 1: The K3s Illusion. They started with K3s, which promised "lightweight Kubernetes for edge." It seemed perfect. Single binary, easy installation, minimal resource usage. They deployed it to 3,000 retail store locations across Europe.

Then they needed to understand the boot process. And registry access patterns. And failure modes. And upgrade procedures at scale.

K3s didn't make any of this easier — it made it opaque. The "simplicity" was an abstraction layer that hid complexity, not removed it. When they needed to debug issues across thousands of nodes, they were running commands they'd found in documentation, hoping they worked, unable to verify their mental model was correct.

Part 2: The Migration to Understanding. They migrated to Talos. Not because Talos was "easier" (it wasn't), but because Talos forced them to understand what they were building.

With Talos, they couldn't just "try something and see if it works." They had to declare their intent explicitly. They had to understand machine configs, control plane architecture, and worker node lifecycle. They had to instrument properly because there was no SSH fallback.

It was harder upfront. It made operations dramatically simpler at scale.

Part 3: PXE Boot Complexity. They needed to boot Talos nodes using PXE and cloud-init. This required understanding the entire boot process — not as a black box, but as a series of explicit steps they controlled.

They couldn't just follow a tutorial. They had to understand kernel parameters, initramfs, cloud-init data sources, and how Talos parses machine configuration from nocloud metadata.

This level of understanding seems excessive when you're deploying one cluster. It's essential when you're deploying thousands.

Part 4: The Registry DDoS. When 3,000 nodes all try to pull container images simultaneously, you DDoS your own registry. This seems obvious in retrospect. It wasn't obvious until they built it.

With traditional systems, they might have SSH'd into nodes and manually staggered the pulls, or added rate limiting to individual nodes, or just hoped the problem went away. With Talos, they had to solve it architecturally.

They implemented proper image layer caching, registry mirroring, and pull rate limiting through declarative configuration. The solution was more work, but it scaled.

Why Talos Shines at 100+ Clusters

When you operate 5 clusters, manual operations are annoying but tolerable. When you operate 100 clusters, manual operations are impossible.

Talos gives you:

1. Enforced Homogeneity. Every node running the same Talos version is identical. Not "supposed to be identical." Not "mostly identical except for that one manual fix." Identical.

This means debugging becomes pattern matching. If one node fails, you can reproduce the failure deterministically. You're not chasing ghosts caused by configuration drift.

2. Declarative Lifecycle Management. Upgrades, patches, and configuration changes are declarative operations. You don't upgrade a node by running commands — you change the declared state and let Talos reconcile.

This is slower for a single node. It's dramatically faster for a thousand nodes.

3. API-Driven Operations. Everything is an API call. This means everything can be automated. Not "can theoretically be automated if you write enough Ansible." Actually automated, because the API is the only interface.

You can write operators that manage Talos clusters. You can build custom tooling that orchestrates upgrades across your entire fleet. You can integrate with your existing infrastructure-as-code pipelines.

You can't do any of this if your operational model is "SSH in and run commands."

4. Observable by Design. Talos exposes logs, metrics, and events through its API. You don't need to SSH in to check logs — you query them programmatically.

This means your observability tooling works the same way on every node. You're not parsing different log formats or dealing with different syslog configurations. The data is structured, consistent, and accessible.

Recognizing Cargo Cult in Your Own Operations

Here's what happens when you're honest about infrastructure: you recognize cargo cult patterns in your own work.

I was running Kubernetes the traditional way. Following tutorials. Deploying clusters. Everything worked — until upgrades. Every Kubernetes version upgrade broke something. I'd rebuild from scratch, follow the same tutorials, hope it worked this time.

Sometimes the upgrade worked. Sometimes it didn't. Same tutorial. Same initial setup. Different results.

Why? Because I'd SSH'd into nodes and made "quick fixes" I didn't document. Or tweaks I thought I remembered but couldn't reproduce. Or changes I made but didn't understand why they mattered. The nodes were supposed to be identical — I'd followed the same steps — but they behaved differently.

Configuration management could have helped, but most homelabs don't use Ansible or Puppet. Too much overhead for "just testing things." So I operated with tribal knowledge, manual changes, and hope.

This is textbook cargo cult. I was performing rituals without understanding causation. The tutorial said "run these commands," so I ran them. When they stopped working, I had no mental model to debug from. I couldn't even reproduce my own infrastructure reliably because I didn't know what state it was actually in.

I moved to Talos not because it was easier, but because it wouldn't let me hide from this lack of understanding. No SSH meant no undocumented changes. Immutability meant the nodes were actually identical, not "supposed to be" identical.

Refusing Helm Charts Is Refusing SSH

I run dozens of Kubernetes deployments. Threat intelligence platforms. Adversary emulation frameworks. Indicator sharing infrastructure. Each with their own architectural requirements — persistent storage for correlation databases, message queues for feed ingestion, object storage for artifacts, worker pods for analysis pipelines.

These aren't stateless web applications. They're complex stateful systems with specific operational patterns. Kubernetes isn't "plug and play" — it's "plug and pray" if you don't understand what you're deploying. Understanding how they work isn't optional — it's required to operate them reliably.

I could have deployed these using Helm charts:

The threat intelligence platform has an official Helm chart
The adversary emulation platform has an official Helm chart
The C2 framework has no Helm chart (forced to port manually from Docker Compose)

I refused to use any Helm charts. Even the good ones. Even ones created by competent engineers who clearly understood the problem.

Why?

Because Helm charts are cargo cult at the application layer. They're the SSH of deployment — a convenient escape hatch that lets you succeed without understanding.

The engineers who created those Helm charts understood the architecture because they did the work of porting from Docker Compose to Kubernetes. They learned by manually translating deployment patterns. If I install their Helm chart, I get their deployment without their understanding.

That's cargo cult. The ritual works, but I don't know why.

The Deeper Problem: Wrong Patterns for Security Infrastructure

But here's what's more important: the Helm charts assume the wrong operational model entirely.

Helm charts are built for CI/CD patterns. Frequent deployments. Multiple independent instances. Rapid iteration. This works great for stateless web applications.

It's architecturally wrong for threat intelligence platforms.

Ask yourself: how many threat intelligence platform instances do you deploy? If you're a multinational, do you deploy one per country? One per office? One per team?

No. You deploy one authoritative instance per continent, maybe one globally.

Why? Because threat intelligence requires centralized, consistent correlation. Multiple independent CTI instances create:

Intelligence discrepancies across regions
Fragmented threat correlation
Inconsistent indicator databases
No global view of threat landscape

A threat intelligence platform isn't a microservice. It's not a web app that needs horizontal scaling and blue-green deployments. It's stateful intelligence infrastructure that needs stability, consistency, and authoritative data.

The Helm chart treats it like the former when it's actually the latter.

This is cargo cult at the architecture layer: applying "cloud-native" deployment patterns to security infrastructure because "that's how we deploy things in Kubernetes."

Porting to Understand Operational Reality

I ported these security platforms from their Docker Compose definitions to Kubernetes manifests manually. Using the upstream project reference architectures. Building from the actual deployment structure the creators intended.

Not because it was faster. It wasn't.

Not because Helm charts didn't exist. They did (mostly).

Because I needed to understand:

Persistent storage architecture — Where state lives, how it's managed, what happens on pod restart
Connector lifecycle — How threat intelligence feeds are ingested, processed, and correlated
Worker scaling patterns — When to scale horizontally vs. vertically, which components are stateless
Intelligence feed ingestion — Rate limiting, API quotas, data freshness vs. system load
Database consistency — How different backends interact, where transactions matter

None of this is captured in Helm values.yaml files. These are operational patterns you learn by building the deployment from first principles.

Testing Understanding With Real Complexity

I didn't test Talos with nginx hello-world deployments. I tested it with actual complex stateful workloads:

Threat Intelligence Platform:

Elasticsearch for indicator search
MinIO for artifact storage
RabbitMQ for connector orchestration
Redis for caching and work queues
Multiple worker pods with different roles
10+ threat intelligence feed connectors
Each connector with different API requirements, rate limits, ingestion patterns

C2 Framework:

Command-and-control server (persistent session state)
Plugin architecture (volume mounts, dynamic loading)
Agent communication (network policies, egress rules)

Adversary Emulation Platform:

PostgreSQL for campaign tracking
MinIO for payload storage
RabbitMQ for job orchestration
Elasticsearch for results indexing
Stateful campaign execution
Integration with attack frameworks

If you can't operate these on Talos declaratively, you don't understand Talos. Toy examples teach you nothing.

The Outcome: Declarative Operations That Make Sense

On traditional Kubernetes, these platforms were fragile. Every upgrade was risk. Configuration drift was inevitable. Debugging required SSH access and manual inspection.

On Talos, I can't make quick fixes. If a threat intelligence connector fails, I can't SSH in and set environment variables manually. I have to fix the manifest. I have to understand why it failed. I have to solve it declaratively.

This is harder — the first time.

But now the entire stack is version-controlled, reproducible, and auditable. When I add the fifth node and rebuild to a hybrid control plane/worker architecture, I'm not migrating 20 artisanal deployments — I'm reapplying 20 declarative configurations.

When the platform releases a new version, I'm not SSHing into nodes to update containers. I'm updating a manifest and letting Kubernetes reconcile.

When I need to debug why a threat intelligence connector isn't ingesting data, I'm not guessing about node-level configuration. I'm checking the declared state against the actual state and identifying the mismatch.

Why Omni Is Next But Not Now

I'm planning to expand to a 5-node cluster. I'm integrating multiple security platforms into a cohesive operations environment. Should I use Omni?

Not yet.

At this scale, understanding the Talos API directly is more valuable than the convenience Omni provides. I need to build deep knowledge of machine configs, upgrade orchestration, failure modes, and API patterns.

Once I have that foundation, Omni becomes useful. It can help manage fleet-level operations, enforce security policies, provide centralized observability.

But if I start with Omni before understanding Talos, I'm building on abstraction. And abstractions leak.

The question isn't "Is Omni good?" It's "Do I understand my infrastructure well enough that Omni helps rather than hides?"

For now, the answer is: learn Talos first, abstract later.

The Difference Between Operating Systems and Appliances

Traditional operating systems are designed for human interaction. You install them, configure them, modify them, and operate them through human interfaces — shells, GUIs, configuration files.

Talos is an appliance. You don't "operate" it in the traditional sense. You declare the desired state, and it reconciles. You don't modify it — you replace it with a new version.

This is uncomfortable because it's unfamiliar. But it's how modern infrastructure should work.

Your networking equipment works this way. Your storage arrays work this way. Your load balancers work this way. You don't SSH into a Cisco switch and manually edit config files — you push configuration through an API and let the device reconcile.

Talos treats the operating system the same way. The node is an appliance, not a pet.

When Manual Operations Are Technical Debt

Every time you SSH into a node and run commands, you're creating technical debt. That operation isn't documented. It isn't reproducible. It isn't auditable. It won't be remembered when the next person needs to do something similar.

Traditional operations accept this as inevitable. Talos makes it impossible.

This forces better practices, but it also exposes when your mental model is wrong. If you can't declaratively express what you're trying to do, you don't understand what you're trying to do.

The discomfort you feel when you can't "just fix it manually" is your brain recognizing that you've been relying on shortcuts that don't scale.

Continue to Part 3: Omni and the Uncomfortable Verdict — where we evaluate the control plane that markets itself as the solution, and arrive at the conclusion most of the industry would rather not hear.

Anti-Cargo-Cult Platform Engineering for Kubernetes at Scale (Part 1)

Wed, 22 Apr 2026 00:11:49 +0000

Anti-Cargo-Cult Platform Engineering for Kubernetes at Scale

A White Paper on Talos Linux and Omni — Part 1 of 4

This is Part 1 of a four-part series.

Part 1 (this post): The cargo cult problem and Talos as an anti-pattern breaker

Part 2: Day-2 operations at scale — where cargo cults collapse

Part 3: Omni and the uncomfortable verdict

Part 4: Feynman's Ghost — the wider lesson and where to go from here

Caution: This is an opinionated engineering polemic, not a tutorial or a vendor-neutral comparison. It is intentionally uncomfortable. If you're looking for product marketing, balanced perspectives, or a "top 10 Kubernetes distros" listicle, this isn't it.

Introduction: On Being Uncomfortable

This white paper will make you uncomfortable. That's intentional.

If you finish reading this and feel defensive about how you operate infrastructure, or irritated by the tone, or convinced the author doesn't understand "real-world constraints" — good. That discomfort is the sound of your mental models being challenged.

Richard Feynman, in his 1974 Caltech commencement address, said:

The first principle is that you must not fool yourself — and you are the easiest person to fool.

This paper examines Talos Linux and Omni through that lens. Not as products to sell you, but as examples of what happens when you design infrastructure that refuses to let you fool yourself. Talos is deliberately hostile to comfortable lies. It removes the tools you use to hide from your own misunderstanding.

The thesis is simple: most modern infrastructure failures aren't caused by missing tools. They're caused by cargo cult engineering — copy-paste YAML, blind trust in abstractions, "it works" without knowing why, rituals mistaken for knowledge.

Talos Linux challenges this directly. It doesn't make Kubernetes easier. It makes bullshit harder.

The cargo cult exists everywhere. Cloud engineering is cargo cult — we copy Terraform modules without understanding state management. Systems engineering is cargo cult — we deploy Ansible playbooks from GitHub without comprehending what they do. Platform engineering is cargo cult — we build "infrastructure as code" that's really just scripts we're afraid to modify.

This paper uses Talos Linux and Kubernetes as a specific, concrete, testable case study. The principles apply universally. But Talos is interesting because it makes cargo cult architecturally impossible in one specific domain. You can't fake understanding when the system refuses to let you lie to yourself.

This paper is written for senior engineers, platform architects, and security-minded infrastructure teams who are tired of pretending they understand systems they don't. It is not a tutorial. It is not vendor marketing. It is an engineering analysis, grounded in operational reality, intentionally opinionated.

If that sounds insufferable, stop reading now.

If that sounds necessary, continue.

Section 1: The Cargo Cult Pandemic

Where the Metaphor Comes From

During World War II, Allied forces established military bases on remote Pacific islands. The indigenous people watched as airplanes landed, bringing seemingly endless supplies — food, medicine, equipment, wealth. Then the war ended. The soldiers left. The airplanes stopped coming.

The islanders wanted the cargo to return. So they built wooden runways. They lit fires along the edges, mimicking landing lights. They constructed control towers from bamboo and placed a man inside wearing carved wooden headphones with sticks protruding like antennas. They performed the rituals they had observed.

The form was perfect. It looked exactly the way it looked before.

But no airplanes landed.

Richard Feynman used this as a metaphor for pseudoscience — research that follows all the apparent forms of scientific investigation but is missing something essential. The planes don't land because the islanders don't understand why the planes came in the first place. They're imitating the surface without comprehending the substance.

This Is Your Infrastructure in 2026

Replace "airplanes" with "working Kubernetes clusters" and you have the state of modern platform engineering.

We build runways made of YAML. We copy Helm charts from repositories we don't understand, maintained by people we've never met, for use cases we haven't verified match our own. We apply manifests and hope they work. When they do, we don't know why. When they don't, we don't know why either.

We know the rituals:

kubectl apply -f deployment.yaml
Add more resources when Pods don't schedule
Set replicas: 3 because "that's what production means"
Install a service mesh because the architecture diagram looks impressive
Enable "GitOps" by pointing ArgoCD at a repo we don't audit

The form is perfect. We have CI/CD pipelines. We have observability dashboards. We have Slack channels full of YAML snippets. We have "infrastructure as code."

But when something breaks at 3 AM, we SSH into the node and start running commands we found on Stack Overflow.

The planes don't land.

The Kubernetes Cargo Cult

Kubernetes itself has become the ultimate cargo cult amplifier. It's a brilliant piece of engineering that very few people actually understand. Most engineers interact with Kubernetes through abstractions — Helm charts, operators, Terraform modules, platform engineering layers that promise to "make Kubernetes simple."

This creates a vicious cycle:

Kubernetes is complex
Abstractions hide the complexity
Engineers never learn the underlying system
When abstractions fail, engineers are helpless
More abstractions are added to "fix" the problem

JYSK, a Danish retail company, documented this perfectly in their blog series about deploying 3,000 Kubernetes clusters to retail stores. They started with K3s — a "lightweight Kubernetes" designed to be "easy." They built out their entire edge infrastructure on this foundation.

It worked. Until it didn't.

At scale, K3s revealed itself to be a leaky abstraction. The "simplicity" was superficial. When they needed to troubleshoot boot processes, registry access patterns, and cluster lifecycle management across thousands of edge nodes, K3s didn't make things easier — it made things opaque. They were running commands they'd found in documentation, applying configurations they didn't fully understand, hoping the planes would land.

They had built wooden headphones.

What's Missing: The Feynman Principle

Feynman identified what's missing in cargo cult science: integrity. Not moral integrity, but intellectual integrity. A kind of utter honesty. A willingness to report everything that might make your results invalid, not just what makes them look good.

In infrastructure terms, this means:

Don't claim you understand a system if you can't explain why it fails
Don't trust an abstraction you can't see through
Don't call something "production-ready" if it only works because you haven't stressed it yet
Don't SSH into a node to fix something unless you can explain why the fix works

Most importantly: Don't fool yourself into thinking "it works" means "I understand it."

Kubernetes gives you a thousand ways to fool yourself. You can get a cluster running without understanding the kubelet. You can deploy applications without understanding the container runtime. You can configure networking without understanding CNI plugins. You can set up storage without understanding CSI drivers or the difference between block and filesystem mounts.

It all works — until it doesn't.

Why This Matters Now

The infrastructure industry is drowning in abstractions. Every new tool promises to "simplify Kubernetes." Every platform engineering framework promises to let developers "deploy without understanding infrastructure." Every managed service promises to "handle operations for you."

This is not progress. This is institutional cargo cult engineering.

We are training an entire generation of engineers who know how to apply YAML but not why it works. Who can deploy applications but can't debug them. Who can follow runbooks but can't write them. Who can operate systems but can't understand them.

The problem isn't that tools are bad. K3s isn't bad. Helm isn't bad. GitOps isn't bad. The problem is that these tools let you succeed without understanding, which means you fail without learning.

The planes keep landing just often enough to reinforce the cargo cult. Until they don't.

Section 2: Talos Linux as Anti-Pattern Breaker

Why No SSH Is Not a Limitation

Let's address the most controversial aspect of Talos immediately: there is no SSH. No shell access. No emergency escape hatch. No way to "just log in and fix it."

Traditional systems administrators hate this. Their entire mental model is built on shell access. When something breaks, you SSH in, poke around, run some commands, maybe edit a config file, restart a service, and declare victory. This is how Unix systems have been operated for fifty years.

Talos removes this entirely. On purpose.

The immediate reaction is: "But what if I need to debug something? What if the API is broken? What if I need to check logs or inspect processes or modify a configuration?"

This reaction reveals the cargo cult. The question assumes that shell access is architecturally necessary for operations. It isn't. Shell access is a coping mechanism for poor architecture.

Here's what SSH actually provides in traditional operations:

Emergency fixes — You broke something, you need to undo it quickly
Investigative debugging — You don't understand the system, so you poke around until you find something
Configuration drift — You manually edit files because your automation is incomplete
Workarounds — The system doesn't do what you need, so you hack it

Every single one of these is a symptom of not understanding your infrastructure.

Talos forces you to confront this. If you can't operate the system through its API, you don't understand the system. If you need to "just log in and check," you haven't instrumented properly. If you need to manually edit configs, your declarative state is wrong.

The discomfort you feel when you can't SSH in? That's not Talos being difficult. That's you realizing you've been using SSH as a crutch.

Immutability as a Forcing Function

Talos is immutable. The root filesystem is read-only. You cannot modify the operating system at runtime. You cannot install packages. You cannot edit system files. The OS is built from a single image, and every node running that image is identical.

This seems restrictive. It is. That's the point.

Traditional operating systems let you lie to yourself about state. You apply a configuration with Ansible, but then someone SSHs in and makes a "quick fix" that never gets committed back to the playbook. You deploy with Terraform, but then manually adjust settings that drift over time. You have a "golden image," but every instance diverges through manual intervention.

Immutability makes this impossible. The system is either in the declared state or it's broken. There's no middle ground. No "well, it mostly works." No "just this one node is special."

JYSK discovered this when they migrated from K3s to Talos. With K3s, they could SSH into edge nodes and make adjustments. They had 3,000 nodes, and subtle differences accumulated. Some nodes had manual fixes. Some had different package versions. Some had configuration tweaks that were never documented.

When they moved to Talos, all of that stopped working. They had to understand every configuration parameter. They had to declare everything explicitly. They had to build proper automation because there was no manual escape hatch.

It was painful. It was also necessary. They went from managing 3,000 artisanal snowflakes to managing 3,000 identical appliances.

The API Is the Only Interface

Talos exposes everything through a gRPC API. You want logs? API call. You want to see running processes? API call. You want to reboot a node? API call. You want to upgrade the OS? API call.

This seems bureaucratic compared to SSH. Why should I make an API call when I could just run systemctl restart kubelet?

Because the API call is auditable. It's authenticated. It's declarative. It can be automated, tested, and version-controlled. The SSH command is none of those things.

More importantly: if the operation can't be done through the API, then the operation shouldn't be done. This is a design constraint that forces better architecture.

Consider a traditional scenario: your kubelet is crashlooping. You SSH in, check the logs, realize a config file is malformed, edit it, restart the service. Problem solved.

Now ask: why was the config file malformed? How did it get that way? Will this happen on other nodes? How will you remember to fix it the same way next time?

With Talos, that scenario can't happen. The kubelet config comes from the Talos machine config, which is declarative and version-controlled. If it's wrong, you fix it in the config and reapply. The change is documented, reproducible, and auditable.

You might argue this is slower. You're right. It is slower to do it correctly.

But "faster" is how you end up with 3,000 nodes that are all subtly different.

Security as Side Effect, Not Feature

Talos is often marketed as "secure by default." This misses the point. Talos isn't secure because someone added security features. It's secure because there's nothing to attack.

No SSH means no SSH vulnerabilities. No package manager means no supply chain attacks through dependencies. No shell means no arbitrary command execution. Immutable root filesystem means no persistent compromise.

The attack surface is the API. That's it. The API is mTLS-authenticated, role-based access controlled, and auditable. If you compromise the API, you can issue commands — but those commands are declarative operations, not arbitrary code execution.

Traditional systems have massive attack surfaces because they were designed for humans to interact with directly. Talos has a minimal attack surface because it was designed for machines to interact with declaratively.

This is what "security by design" actually means. Not adding security products on top of an insecure foundation, but removing the insecure foundation entirely.

Your threat intelligence platform deployment on Talos? The platform can't be compromised through the OS because there's no OS layer to compromise. The attack surface is the application container and the Kubernetes API. That's a massively smaller threat model than "entire Linux userland plus SSH plus sudo plus any package someone installed six months ago."

Traditional Linux distributions ship with 1,500-2,700 binaries. Talos ships with fewer than 50. Every binary is a potential vulnerability, a potential misconfiguration, a potential attack vector. Talos eliminates 98% of them.

Why Senior Engineers Hate This (And Why That Matters)

If you've been doing systems administration for twenty years, Talos feels wrong. Deeply wrong. It violates every mental model you've built.

You learned that good operators can fix anything if they can get a shell. You learned that automation is great, but sometimes you need to "just get in there." You learned that real expertise means knowing the magic commands to run when things break.

Talos tells you that all of that is cargo cult.

The wooden headphones looked convincing because that's what you saw the radio operators wearing. SSH access looks necessary because that's what you saw senior engineers using. But correlation isn't causation.

Junior engineers often adapt to Talos faster than senior engineers. Not because they're smarter, but because they haven't built up twenty years of muscle memory around SSH access. They don't have to unlearn anything.

This is uncomfortable to admit, but it's important: experience can be a liability when it's experience with the wrong patterns.

If your expertise is "knowing how to debug Kubernetes by SSHing into nodes," then Talos makes that expertise worthless. That's threatening. That's why the reaction is often defensive hostility.

But if your expertise is "understanding distributed systems, declarative state management, and failure mode analysis," then Talos makes that expertise more valuable. Because now you can't hide behind manual fixes. You have to actually understand what you're building.

This Is Not "Best Practices"

Before you dismiss this as "we already do infrastructure as code" or "we follow best practices," understand the difference:

Best practices are optional. You can choose to follow them or not. You can follow them partially. You can follow them "except in this one case." Best practices are suggestions that can be ignored when convenient.

Architectural constraints are not optional. Talos doesn't suggest you avoid SSH. It architecturally prevents SSH. It doesn't recommend immutability. It enforces immutability. It doesn't encourage API-driven operations. It makes API-driven operations the only option.

Most "infrastructure best practices" are cargo cult themselves. We say "infrastructure as code" but we mean "infrastructure as YAML files that we manually apply." We say "immutable infrastructure" but we SSH in to make changes. We say "declarative configuration" but we use imperative scripts.

These aren't best practices. They're aspirational buzzwords we use to feel good about infrastructure that's still fundamentally based on manual operations and hope.

Talos removes the gap between what we say and what we do. You can't claim to run immutable infrastructure while SSHing in to fix things. You can't claim to use declarative configuration while making imperative changes. The system won't let you lie to yourself.

This is why it's uncomfortable. Best practices let you succeed without changing. Architectural constraints force change first.

The Uncomfortable Question

Here's the question you need to ask yourself: Do you need SSH to operate Kubernetes, or do you need SSH to hide from the fact that you don't fully understand Kubernetes?

If you need SSH for legitimate operational reasons that can't be accomplished through Kubernetes APIs, Talos APIs, or proper instrumentation, then fair enough. Document those reasons. Make sure they're architectural requirements, not just convenience.

But if you need SSH because "what if something goes wrong and I need to debug it," then you're admitting you don't understand your system well enough to instrument it properly.

The planes don't land because you built a runway. They land because you have air traffic control, navigation systems, fuel logistics, and maintenance infrastructure.

SSH isn't the runway. It's the wooden headphones.

Continue to Part 2: Day-2 Operations at Scale — where we examine what happens to infrastructure when you try to operate it at scale, and where every comfortable shortcut collapses.

FreeLens vs OpenLens vs Lens: Choosing the Right Kubernetes IDE

Sun, 15 Feb 2026 21:25:02 +0000

Introduction: When a Tool Choice Becomes a Legal and Platform Decision

If you’ve been operating Kubernetes clusters for a while, you’ve probably learned this the hard way:
tooling decisions don’t stay “just tooling” for long.

What starts as a developer convenience can quickly turn into:

a licensing discussion with Legal,
a procurement problem,
or a platform standard you’re stuck with for years.

The Kubernetes IDE ecosystem is a textbook example of this.

Many teams adopted Lens because it genuinely improved day-to-day operations. Then the license changed and we already cover the OpenLens vs Lens in the past. Then restrictions appeared. Then forks started to emerge.

Today, the real question is not “Which one looks nicer?” but:

Which one is actually maintained?
Which one is safe to use in a company?
Why is there a fork of a fork?
Are they still technically compatible?
What is the real switch cost?

Let’s go through this from a production and platform engineering perspective.

The Forking Story: How We Ended Up Here

Understanding the lineage matters because it explains why FreeLens exists at all.

Lens: The Original Product

Lens started as an open-core Kubernetes IDE with a strong community following. Over time, it evolved into a commercial product with:

a proprietary license,
paid enterprise features,
and restrictions on free usage in corporate environments.

This shift was legitimate from a business perspective, but it broke the implicit contract many teams assumed when they standardized on it.

OpenLens: The First Fork

OpenLens was created to preserve:

open-source licensing,
unrestricted commercial usage,
compatibility with Lens extensions.

For a while, OpenLens was the obvious alternative for teams that wanted to stay open-source without losing functionality.

FreeLens: The Fork of the Fork

FreeLens appeared later, and this is where many people raise an eyebrow.

Why fork OpenLens?

Because OpenLens development started to slow down:

release cadence became irregular,
upstream Kubernetes changes lagged,
governance and long-term stewardship became unclear.

FreeLens exists because some contributors were not willing to bet their daily production tooling on a project with uncertain momentum.

This was not ideology. It was operational risk management.

Are the Projects Still Maintained?

Short answer: yes, but not equally.

Lens

Actively developed
Backed by a commercial vendor
Fast adoption of new Kubernetes features

Trade-off:

Licensing constraints
Paid features
Requires legal review in most companies

OpenLens

Still maintained
Smaller contributor base
Slower release velocity

It works, but it no longer feels like a safe long-term default for platform teams.

FreeLens

Actively maintained
Explicit focus on long-term openness
Prioritizes Kubernetes API compatibility and stability

Right now, FreeLens shows the healthiest balance between maintenance and independence.

Technical Compatibility: Can You Switch Without Pain?

This is the good news: yes, mostly.

Cluster Access and Configuration

All three tools:

use standard kubeconfig files,
support multiple contexts and clusters,
work with RBAC, CRDs, and namespaces the same way.

No cluster-side changes are required.

Extensions and Plugins

Most Lens extensions work in OpenLens.
Most OpenLens extensions work in FreeLens.
Proprietary Lens-only extensions are the main exception.

In real-world usage:

~90% of common workflows are identical
differences show up only in edge cases or paid features

UX Differences

There are some UI differences:

branding,
menu structure,
feature gating in Lens.

Nothing that requires retraining or documentation updates.

Legal and Licensing Considerations (This Is Where It Usually Breaks)

This is often the decisive factor in enterprise environments.

Lens

Requires license compliance checks
Free usage may violate internal policies
Paid plans required for broader adoption

If you operate in a regulated or audited environment, this alone can be a blocker.

OpenLens

Open-source license
Generally safe for corporate use
Slight uncertainty due to reduced activity

FreeLens

Explicitly open-source
No usage restrictions
Clear intent to remain free for commercial use

If Legal asks, “Can we standardize this across the company?”
FreeLens is the easiest answer.

Which One Should You Use in a Company?

A pragmatic recommendation:

Use Lens if:

you want vendor-backed support,
you are willing to pay,
you already standardized on Mirantis tooling.

Use OpenLens if:

you are already using it,
it meets your needs today,
you accept slower updates.

Use FreeLens if:

you want zero licensing risk,
you want an open-source default,
you care about long-term maintenance,
you need something you can standardize safely.

For most platform and DevOps teams, FreeLens is currently the lowest-risk choice.

Switch Cost: How Expensive Is It Really?

Surprisingly low.

Typical migration:

install the new binary,
reuse existing kubeconfigs,
reinstall extensions if needed.

What you don’t need:

cluster changes,
CI/CD modifications,
platform refactoring.

Downtime: none
Rollback: trivial

This is one of the rare cases where switching early is cheap.

Is a “Fork of a Fork” a Red Flag?

Normally, yes.

In this case, no.

FreeLens exists because:

maintenance mattered more than branding,
openness mattered more than monetization,
predictability mattered more than roadmap promises.

Ironically, this is very aligned with how Kubernetes itself evolved.

Conclusion: A Clear, Boring, Production-Safe Answer

If you strip away GitHub drama and branding:

Lens optimizes for revenue and enterprise features.
OpenLens preserved openness but lost momentum.
FreeLens optimizes for sustainability and freedom.

From a platform engineering perspective:

FreeLens is the safest default Kubernetes IDE today for most organizations.

Low switch cost, strong compatibility, no legal surprises.

And in production environments, boring and predictable almost always wins.

Home Lab: Chapter 6 — Kubernetes Ingress Controller

Thu, 06 Nov 2025 08:16:13 +0000

Howdy,

In this chapter, we're going to look at how to expose services running inside our Kubernetes cluster to the outside world using an Ingress Controller. We'll be using the NGINX Ingress Controller and taking full advantage of our GitOps setup with ArgoCD.

Let's get into it.

What Is an Ingress Controller?

An Ingress Controller is a Kubernetes resource that handles external access to services running inside your cluster - usually over HTTP or HTTPS. You can think of it as a smart traffic router sitting at the edge of your cluster.

It watches for Ingress resources and knows how to route traffic accordingly. It also supports other nice things like:

Load balancing
SSL termination
Name-based virtual hosting

In other words, it gives you centralized control over how incoming traffic is handled.

Preparing the Cluster

Before installing the Ingress Controller, we need to make sure it can actually receive external traffic. For that, we'll expose it using a Kubernetes Service of type LoadBalancer.

The LoadBalancer service will act as the public entry point, and from there, the Ingress Controller will decide how to route the traffic internally.

Now, since we're using Cilium as our CNI (as mentioned in Chapter 3), we've got a cool feature available to us: Cilium LB IPAM (Load Balancer IP Address Management). This lets us assign specific IPs to LoadBalancer services - perfect for when we want to reserve a static IP for our Ingress Controller.

This is especially useful if we plan to point a DNS record to the controller later, which is exactly what we'll do in future chapters.

Assigning a Static IP with Cilium

To assign a static IP using Cilium, we need to define a CiliumLoadBalancerIPPool object. This object tells Cilium which IPs it can use for LoadBalancer services, and under what conditions.

Here's the configuration:

# ippool.yaml
---
apiVersion: cilium.io/v2alpha1
kind: CiliumLoadBalancerIPPool
metadata:
  name: default-pool
spec:
  blocks:
    - cidr: "x.x.x.105/32"
  serviceSelector:
    matchLabels:
      "io.kubernetes.service.namespace": "ingress-nginx"

This will assign the IP x.x.x.105 to any LoadBalancer service in the ingress-nginx namespace.

To apply it, we can run the following command:

# Apply the configuration
kubectl apply -f ippool.yaml

With this in place, the Ingress Controller will get that static IP when we install it.

Installing NGINX Ingress Controller

Now that the IP pool is in place, we can move on to installing the NGINX Ingress Controller. Installing the NGINX Ingress Controller is straightforward. Like ArgoCD and Cilium, we'll use its official Helm chart. But since we already have a fully working GitOps setup (thanks to our ArgoCD configuration in the previous chapter), we can add the NGINX Ingress Controller to the Git repository and have it installed by ArgoCD. We can do this by creating an Application object that will define the NGINX Ingress Controller. We'll define it in a ingress-nginx.yaml file with the following content:

# ingress-nginx.yaml
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: ingress-nginx
  namespace: argocd
  finalizers:
    - resources-finalizer.argocd.argoproj.io
spec:
  project: default
  source:
    chart: ingress-nginx
    repoURL: https://kubernetes.github.io/ingress-nginx
    targetRevision: 4.12.1
    helm:
      valuesObject:
        controller:
          service:
            type: LoadBalancer
            externalTrafficPolicy: Cluster
            annotations:
              io.cilium/lb-ipam-ips: 'x.x.x.105'
          metrics:
            enabled: true
  destination:
    name: in-cluster
    namespace: ingress-nginx
  syncPolicy:
    automated:
      prune: false
      selfHeal: true
      allowEmpty: false
    syncOptions:
      - CreateNamespace=true
    managedNamespaceMetadata:
      labels:
        pod-security.kubernetes.io/enforce: privileged
        pod-security.kubernetes.io/enforce-version: latest
  info:
    - name: 'Description:'
      value: Ingress controller

This file defines an ArgoCD Application that will install the NGINX Ingress Controller using the Helm chart from the official repository. Here's a quick breakdown of what each part does:

service.type: LoadBalancer: creates a service of type LoadBalancer that will expose the Ingress Controller to the outside world.
service.externalTrafficPolicy: Cluster: allows the traffic to be routed to all the nodes in the cluster instead of just the node where the pod is running.
io.cilium/lb-ipam-ips: assigns the previous allocated IP address to the Load Balancer service.
metrics.enabled: true: enables the metrics server for the Ingress Controller, exposing metrics like the number of requests, response time, etc.
pod-security.kubernetes.io/enforce: sets the Pod Security Standards to privileged, allowing the Ingress Controller to run with elevated privileges. This is necessary for the Ingress Controller to function correctly.

Once this file is added to your Git repo, ArgoCD will pick it up and deploy everything.

We can follow the progress of the installation by checking the ArgoCD UI by running kubectl port-forward svc/argocd-server -n argocd 8080:80 and navigating to http://localhost:8080. You can log in with the default credentials (admin/admin) or the credentials you set up in the previous chapter.

Testing the Ingress Controller

Once ArgoCD has deployed the controller, we can check if it's working by making a simple curl request to the IP we assigned:

# Make a request to the Ingress Controller
curl http://x.x.x.105

We now should see something like this:

## Response from the Ingress Controller
<html>
  <head><title>404 Not Found</title></head>
  <body>
    <center><h1>404 Not Found</h1></center>
    <hr><center>nginx</center>
  </body>
</html>

That 404 is actually a good sign - it means the Ingress Controller is up, responding to requests, and just doesn't have any routes defined yet (because we haven't created any Ingress resources).

Visual Recap

Here's what we just set up, in diagram form:

Conclusion

In this chapter, we set up the foundation for managing external traffic in our Kubernetes cluster. Here's what we accomplished:

We used Cilium's LB IPAM feature to assign a static IP to a LoadBalancer service.
We installed the NGINX Ingress Controller using ArgoCD and Helm.
We validated that the controller is working by sending it a direct request.

Now that we've got an Ingress Controller up and reachable from the outside, we can move on to the next step: making it actually useful by adding routing rules, setting up DNS, and handling SSL/TLS termination.

In the next chapter, we'll look at how to:

Configure DNS records pointing to your static IP
Automatically issue and renew SSL certificates using cert-manager
Route traffic to real services using Ingress resources

We're just getting started with Ingress - but the foundation is solid. Catch you in the next one.

Home Lab: Chapter 5 — Kubernetes Managing Secrets

Thu, 06 Nov 2025 03:55:54 +0000

Howdy,

Secrets are a fundamental part of any application - it's how we securely store sensitive information. In Kubernetes, there are several approaches to handling secrets. In this chapter, we'll explore different ways to manage secrets in Kubernetes.

What is a Secret?

First off, what exactly is a Secret? A Secret is a Kubernetes object that holds a small amount of sensitive data - like a password, token, or key. Without Secrets, you might have to hard-code these values into Pod specs or container images. Users can create Secrets manually, and Kubernetes also generates some automatically.

Our Scenario

In the previous chapter, we already needed to work with some sensitive data:

We generated Talos Secrets, which include a bundle of crucial credentials.
We created ArgoCD Secrets, which hold a GitHub private key for repository access.

So the question is: How can we securely manage these secrets while still getting the automation benefits from Kubernetes and ArgoCD?

Secrets Management Options

You've got a few choices for managing secrets in Kubernetes:

Built-in Kubernetes Secrets
Third-party secret managers (Vault, AWS/Azure/GCP Key Management, etc.)
Kubernetes operators (e.g. Sealed Secrets by Bitnami)

For simplicity in this chapter, we'll stick with built-in Kubernetes Secrets - though we may revisit other options later.

Kubernetes Secrets

Kubernetes Secrets let you store things like passwords, OAuth tokens, and SSH keys securely. It's much safer and more flexible than embedding secrets directly in Pod specs or container images. However, remember: Kubernetes Secrets are only Base64 encoded by default, not encrypted. So if you require encryption at rest, layer in a third-party tool.

Managing Secrets

While Kubernetes makes it easy to use Secrets, we still need an automated and secure way to manage them. That's where ArgoCD comes in. With ArgoCD, we can store our secrets in a Git repository and use a Kustomize overlay to apply them to the cluster.

Kustomize lets you customize raw, template-free YAML files without altering the originals. This means we can keep clean, reusable manifests while layering in environment-specific or sensitive configurations, like secrets, on top.

So far, so good. But you might be wondering:

"How can we safely store secrets in Git?"

Great question! The answer is encryption - and this is where SOPS comes in.

SOPS (Secrets OPerationS) is a flexible tool that encrypts files in a way that lets you decrypt them later when needed. It supports various encryption backends: PGP, GnuPG, AWS KMS, Azure Key Vault, Google Cloud KMS, Vault, and more.

For this setup, I'll be using age - a modern, simple, and secure encryption tool that serves as a lightweight alternative to GPG.

By combining:

Kustomize
SOPS + age
And the KSOPS plugin (a Kustomize plugin for SOPS)

... we get a powerful GitOps-friendly workflow:

This setup allows us to store encrypted secrets in Git, and ArgoCD will automatically decrypt them using KSOPS when applying them to the cluster. This way, we can manage our secrets securely while still benefiting from GitOps automation.

Once the secrets are decrypted, they are applied to the cluster as standard Kubernetes Secrets, which can then be consumed by Pods and other resources.

How to use KSOPS

Generate an age key pair:

## Generate the age keys
age-keygen -o ~/.config/sops/age/keys.txt

We should get as the output of this command:

age-keygen -o key.txt
Public key: age1efe0s548vkwgvjkdtgu4exf9v4mtltjv6rn5yww33yd75ad7r5xsjq7f8l

Create a Kubernetes secret to store the age public key:

## Create the sops-age secret
kubectl create secret generic sops-age \
  --namespace argocd \
  --from-file=keys.txt=~/.config/sops/age/keys.txt

Create a .sops.yaml to define which files/fields should be encrypted:
```
# .sops.yaml
---
stores:
yaml:
    indent: 2
creation_rules:
- path_regex: secrets.yaml
    encrypted_regex: '^(id|secret|bootstraptoken|secretboxencryptionsecret|token|ca|crt|key)$'
    age: age1efe0s548vkwgvjkdtgu4exf9v4mtltjv6rn5yww33yd75ad7r5xsjq7f8l
```
This configuration instructs SOPS to encrypt the secrets.yaml file using the age1efe0s548vkwgvjkdtgu4exf9v4mtltjv6rn5yww33yd75ad7r5xsjq7f8l public key. It targets any field that matches the following regular expression:

> The age1efe0s548vkwgvjkdtgu4exf9v4mtltjv6rn5yww33yd75ad7r5xsjq7f8l key is the public key generated during the creation of the sops-age Kubernetes secret
Encrypt your file:
```
## Encrypt the file content
ksops -e -i secrets.yaml
```
If we now inspect the secrets.yaml file, we'll see that the content is now
encrypted. To decrypt the file content back to its original state, we can run
the following command:
```
## Decrypt the file content
ksops -d -i secrets.yaml
```
> Decrypting the file content requires the private key that we generated when we created the sops-age secret.

Config ArgoCD to manage Secrets

To configure ArgoCD to manage secrets, we need to tweak the values.yaml file that we created in the previous Chapter 4 and add the following configuration:

# values.yaml
---
repoServer:
  env:
    - name: XDG_CONFIG_HOME
      value: /.config
    - name: SOPS_AGE_KEY_FILE
      value: /.config/sops/age/keys.txt
  volumes:
    - name: custom-tools
      emptyDir: {}
    - name: sops-age
      secret:
        secretName: sops-age
  initContainers:
    - name: install-ksops
      image: viaductoss/ksops:v4.3.3
      command: ['/bin/sh', '-c']
      args:
        - echo "Installing KSOPS...";
          mv ksops /custom-tools/;
          mv kustomize /custom-tools/;
          echo "Done.";
      volumeMounts:
        - mountPath: /custom-tools
          name: custom-tools
  volumeMounts:
    - mountPath: /usr/local/bin/kustomize
      name: custom-tools
      subPath: kustomize
    - mountPath: /.config/kustomize/plugin/viaduct.ai/v1/ksops/ksops
      name: custom-tools
      subPath: ksops
    - mountPath: /.config/sops/age/keys.txt
      name: sops-age
      subPath: keys.txt

This configuration enables ArgoCD to manage secrets securely using KSOPS and age. It sets the XDG_CONFIG_HOME environment variable to /.config, directing SOPS to look for its configuration files there. The SOPS_AGE_KEY_FILE is set to /.config/sops/age/keys.txt, so SOPS can locate the age private key used for decryption.

Two volumes are defined: custom-tools, an emptyDir volume for storing the KSOPS and Kustomize binaries, and sops-age, a secret volume that holds the age key file. An initContainer named install-ksops installs the necessary binaries into the custom-tools volume before the main ArgoCD container starts.

The volumes are mounted inside the pod: custom-tools is mounted at /usr/local/bin/kustomize and /custom-tools to make the binaries accessible, while sops-age is mounted at /.config/sops/age/keys.txt so that the decryption key is available for SOPS during runtime.

We also need to create the sops-age secret that contains the age keys file:

# Generate the age keys
age-keygen -o ~/.config/sops/age/keys.txt

## Create the secret
cat ~/.config/sops/age/keys.txt | kubectl create secret generic sops-age --namespace argocd --from-file=keys.txt=/dev/stdin

With the secrets now created, the only thing left to do is apply the
new configuration to the cluster:

# Upgrade the ArgoCD installation
helm upgrade --install argocd argo/argo-cd \
    --namespace argocd \
    --values values.yaml

Voilà! ArgoCD is now KSOPS-enabled with age key support. We now have a secure and automated way to manage our secrets in a Git repository and still have them applied to the cluster in a secure way.

Adding Secrets to the Git

Create your secret manifest, e.g. example-secret.yaml:

# example-secret.yaml
---
apiVersion: v1
kind: Secret
metadata:
  name: example-secret
stringData:
  foo: bar

Add an entry in .sops.yaml to match and encrypt foo:

# .sops.yaml
# ...
- path_regex: example-secret.yaml
  encrypted_regex: '^foo'
  age: age1efe0s548vkwgvjkdtgu4exf9v4mtltjv6rn5yww33yd75ad7r5xsjq7f8l

And then we can encrypt it by running:

## Encrypt the file content
ksops -e -i example-secret.yaml

If we now inspect the example-secret.yaml file, we'll see that the content is
now encrypted:

# example-secret.yaml
---
apiVersion: v1
kind: Secret
metadata:
  name: example-secret
stringData:
  foo: ENC[AES256_GCM,data:s7FsAPs=,iv:ywvzww/Jq342vkENSEXLxopD8aAf3jCE0TPfwILJz1Q=,tag:DYFKhAVr7pgf1cW5+cevbw==,type:str]
sops:
  kms: []
  gcp_kms: []
  azure_kv: []
  hc_vault: []
  age:
    - recipient: age1efe0s548vkwgvjkdtgu4exf9v4mtltjv6rn5yww33yd75ad7r5xsjq7f8l
      enc: |
        -----BEGIN AGE ENCRYPTED FILE-----
        YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSB0cXo4YTNaYTdGT1Y3U29N
        WFR4WlJISTRUaU1jTXVzTUFqZCsxQVBoaEZvCmRQZVAzMS9ZM3RqbTlrSEdyUmJj
        ejFZNDVlMEpZY2s3Z1VSdTdQYWk3MmMKLS0tIFQxVUhibEVHSVJtb09XNkRxcVN5
        TUVIcmdaRloyUVZzckNMbkpVVXo5WjQK70C/ZvuailOheaSXMM5Rx+CGXZ9K98tw
        ++Q6PZPafdZxkwSIRjZU6ihAk0L6TXs3MJ93yvn/n3CA9zQp9tDuXg==
        -----END AGE ENCRYPTED FILE-----
  lastmodified: '2025-04-02T21:12:33Z'
  mac: ENC[AES256_GCM,data:t3MIPtm19pt+Ov27VkQvrDM/4IN48KXiOpQQlP1czWn12sv68pMt/fALxnrSM3jgv2q0reG5j9vJlA9zFPVw8sdudZ7mmY+HoFIfp8ryZOqX1Ro2hBPR4aj9eXBZT5Gjwf8eOYgYKRdOev6pRmtTA5wJ2qRAZkhvBm3mHHp7d+E=,iv:57/nNwc25K0J632kgo7MX7J0FyUN13ED7wwww9qOAMQ=,tag:qV1/qTDyAUrIFah2AeXrmQ==,type:str]
  pgp: []
  encrypted_regex: ^foo
  version: 3.9.0

As we can see, the foo field is now encrypted, and a new sops field has been added, which contains the encryption metadata.

The kms, gcp_kms, azure_kv, hc_vault, and pgp fields are lists of encryption keys for their respective backends. Since none of these were used in this case, they are empty.

The age field lists the age keys used to encrypt the file. Here, we used the age1efe0s548vkwgvjkdtgu4exf9v4mtltjv6rn5yww33yd75ad7r5xsjq7f8l key, so the field includes both the recipient and the corresponding encrypted payload.

The lastmodified field records when the file was last updated.

The mac field is a message authentication code used to verify the file's integrity and ensure it hasn't been tampered with.

The encrypted_regex field specifies a regular expression used by SOPS to determine which fields in the document should be encrypted.

The version field indicates the SOPS file format version used for encryption.

Add a Kustomize overlay by creating a kustomization.yaml file along with a KSOPS generator:

# kustomization.yaml
---
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
metadata:
name: example-secret
generators:
- example-secret-generator.yaml

And the associated generator file:

# example-secret-generator.yaml
---
apiVersion: viaduct.ai/v1
kind: ksops
metadata:
name: example-secret-generator
files:
- example-secret.yaml

This overlay tells Kustomize to use the KSOPS plugin to decrypt example-secret.yaml before applying it to the cluster, enabling secure GitOps-driven secrets management.

Now you can push the changes to your Git repository and let ArgoCD handle the deployment. When ArgoCD applies the configuration, it will automatically decrypt the secret using the provided Age keys and apply it to the cluster as a standard Kubernetes Secret.

Ensure that ArgoCD is already configured to track the correct Git repository and that the example-secret.yaml file is located in the expected directory.

With that, the setup is complete. You now have a secure and automated workflow for managing secrets through Git and ArgoCD.

Conclusion

In this chapter I've demonstrated how to securely manage Kubernetes secrets using GitOps principles. By integrating KSOPS with ArgoCD and Kustomize, we can encrypt secrets, store them in Git, and have them decrypted and applied to the cluster automatically.

While this setup offers a solid foundation, it's not the most advanced solution in terms of security. For production environments requiring features like access controls, audit logging, or automatic key rotation, consider tools such as HashiCorp Vault or a Kubernetes-native solution like Sealed Secrets.

That said, this approach strikes a good balance between simplicity, security, and GitOps compatibility - making it an excellent starting point for secret management in Kubernetes.

Originally published at https://techquests.dev on July 2, 2025.

Home Lab: Chapter 4 — Kubernetes GitOps with ArgoCD

Wed, 05 Nov 2025 23:58:23 +0000

Howdy!

Ever since I discovered GitOps, I've been in love with the concept. The idea of managing all your infrastructure configuration from a centralized Git repository - and having a tool automatically apply those changes - is incredibly powerful.

GitOps brings together Infrastructure as Code (IaC) and Continuous Integration/Continuous Deployment (CI/CD) in a seamless, declarative workflow. It's the ideal way to manage a Kubernetes cluster.

In the past, I've used Flux to implement GitOps, but I've always been curious about ArgoCD. After hearing so many good things about it, I decided it was finally time to give it a try - and this project was the perfect opportunity.

What is ArgoCD?

ArgoCD is a declarative GitOps continuous delivery tool for Kubernetes. It follows the GitOps pattern of using Git repositories as the source of truth for defining the desired application state.

It is implemented as a Kubernetes controller that continuously monitors running applications and compares their current live state against the desired target state (as defined in Git). A deployment is considered in sync when the live state matches the target state. If they differ, ArgoCD performs a kubectl apply to reconcile the live state with the target state.

Installation

Installing ArgoCD is straightforward. We can use the official Helm chart. First, add the ArgoCD Helm repository:

# Create the namespace
kubectl create namespace argocd

# Add the repository
helm repo add argo https://argoproj.github.io/argo-helm
helm repo update

Since we're using Cilium as the CNI, we need to exclude Cilium resources from ArgoCD's control. Create a values.yaml file with the following:

# values.yaml
configs:
  cm:
    resource.exclusions: |
      - apiGroups:
          - cilium.io
        kinds:
          - CiliumIdentity
        clusters:
          - "*"

This prevents ArgoCD from managing Cilium resources, which could interfere with Cilium's operation. For more details, see Troubleshooting Cilium deployed with Argo CD.

We can also provide a custom admin password for the ArgoCD UI. The password must be hashed. You can generate a hash using htpasswd:

# Generate the password hash
htpasswd -nbBC 10 "" <PASSWORD> | tr -d ':\n'

Then add it to the values.yaml file:

# values.yaml
configs:
  secret:
    argocdServerAdminPassword: <PASSWORD_HASH>

Now install the chart:

helm upgrade --install argocd argo/argo-cd \
    --namespace argocd \
    --values values.yaml

This installs ArgoCD into the argocd namespace. You can access the UI via port forwarding:

kubectl port-forward svc/argocd-server -n argocd 8080:80

Add Repository

With ArgoCD installed, the next step is to connect a Git repository containing your Kubernetes manifests. ArgoCD includes an operator and several CRDs (Custom Resource Definitions). We use the Application CRD to define which repository and path to sync.

Here's an example init.yaml:

# init.yaml
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: init
  finalizers:
    - resources-finalizer.argocd.argoproj.io
spec:
  project: default
  source:
    repoURL: <REPO_URL>
    targetRevision: HEAD
    path: <PATH>
  destination:
    name: in-cluster
    namespace: argocd
  syncPolicy:
    automated:
      selfHeal: true
  info:
    - name: 'Description:'
      value: 'Entrypoint to all homelab apps'

For more details, see the ArgoCD Application Spec.

If your repository is private, create a secret with credentials and label it for ArgoCD:

# github.yaml
---
apiVersion: v1
kind: Secret
metadata:
  name: github
  namespace: argocd
  labels:
    argocd.argoproj.io/secret-type: repository
stringData:
  type: git
  url: <REPO_URL>
  sshPrivateKey: <SSH_PRIVATE_KEY>

Apply the secret and application:

kubectl apply -f github.yaml
kubectl apply -f init.yaml

Once applied, you should see the application appear in the ArgoCD UI.

You can access the UI at by port forwarding the service:

kubectl port-forward svc/argocd-server -n argocd 8080:443

Then navigate to http://localhost:8080 in your browser. The default username is admin, and the password is the one you set earlier.

Adding Applications

With the repository connected, you can start adding applications. Here's an example app.yaml to deploy a Helm chart:

# app.yaml
---
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: <APP_NAME>
  namespace: argocd
  finalizers:
    - resources-finalizer.argocd.argoproj.io
spec:
  project: default
  source:
    chart: <CHART_NAME>
    repoURL: <CHART_REPO>
    targetRevision: <CHART_VERSION>
    helm: {}
  destination:
    name: in-cluster
    namespace: <NAMESPACE>
  syncPolicy:
    automated:
      prune: false
      selfHeal: true
      allowEmpty: false
    syncOptions:
      - CreateNamespace=true
  info:
    - name: 'Description:'
      value: My first Application with ArgoCD

Ensure this file is in the <PATH> specified in your init.yaml.

Push the file to your repository, and ArgoCD will detect it. You can then sync it through the UI and watch your application get deployed - this is the magic of GitOps.

Conclusion

ArgoCD is a powerful tool that enables managing Kubernetes clusters declaratively through Git. By treating your Git repository as the source of truth, you gain version control, automation, and a clear audit trail of infrastructure changes.

It's an excellent way to combine IaC and CI/CD - and I'm excited to explore what more I can do with it!

Originally published at https://techquests.dev on May 31, 2025.

Home Lab: Chapter 8 — Kubernetes Storage with Rook-Ceph

Wed, 05 Nov 2025 20:48:24 +0000

Howdy!

We've come a long way! We've set up our Kubernetes cluster, configured GitOps with ArgoCD, managed secrets securely, exposed applications through Ingress, and set up DNS with SSL certificates. But there's one critical piece we haven't addressed yet: storage.

In this chapter, we'll tackle one of the most challenging aspects of running Kubernetes in a homelab environment - persistent storage. Specifically, we'll explore how I implemented a distributed storage solution using Rook-Ceph to provide reliable, scalable block storage across my cluster.

The Storage Challenge

When you run applications on Kubernetes, especially stateful ones like databases, message queues, or monitoring systems, you need persistent storage that survives pod restarts and node failures. Without it, losing a pod means losing all your data.

In a cloud environment, this is straightforward - you just request a volume from your cloud provider. But in a homelab, you need to build this yourself.

Why Not Just Use Local Storage?

You might think, "Can't I just mount a local directory on each node?" Technically yes, but there are serious drawbacks:

No redundancy - If a node fails, your data is gone
Poor availability - Pods can't migrate between nodes
Limited capacity - Bound by individual node storage
Manual management - You have to handle backups yourself

For a homelab that aims to mimic production environments, this isn't acceptable.

Introducing Rook-Ceph

Rook is a cloud-native storage orchestrator that automates the deployment and management of storage systems in Kubernetes. Ceph is a distributed storage platform that provides block storage, object storage, and file system storage.

Together, Rook-Ceph gives you:

Distributed storage - Data replicated across multiple nodes
Self-healing - Automatic recovery from node failures
High availability - Pods can migrate freely
Scalability - Add nodes to expand storage
Production-ready - Used by enterprises worldwide

The Perfect Fit for Homelabs

Rook-Ceph is particularly well-suited for homelab environments because:

It uses node local disks, so no external storage appliances needed
It's open-source and free
It's battle-tested in production
It manages itself using Kubernetes native resources
It provides excellent observability and dashboards

Implementation

Now that we understand why Rook-Ceph is a great fit, let's dive into how I implemented it in my homelab. I'll walk through the deployment strategy, cluster configuration, storage classes, and some key design decisions that make this setup reliable and scalable.

Deployment Strategy

In my setup, I deployed Rook-Ceph using Kustomize through ArgoCD (as we configured in Chapter 4). This ensures:

Infrastructure-as-code approach
Automated deployments
Easy reproducibility
Version control of all configurations

Cluster Configuration

Here's the core Rook-Ceph cluster setup:

apiVersion: ceph.rook.io/v1
kind: CephCluster
metadata:
  name: rook-ceph
  namespace: rook-ceph
spec:
  cephVersion:
    image: quay.io/ceph/ceph:v18.2.2
  dataDirHostPath: /var/lib/rook
  mon:
    count: 3              # 3 monitors for quorum
    allowMultiplePerNode: false
  dashboard:
    enabled: true        # Web dashboard for monitoring
  storage:
    useAllNodes: true    # Use all nodes in cluster
    useAllDevices: true  # Use all available disks

Storage Classes

I configured a StorageClass to define how storage should be provisioned:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: rook-ceph-block
  annotations:
    storageclass.kubernetes.io/is-default-class: "true"
provisioner: rook-ceph.rbd.csi.ceph.com
parameters:
  clusterID: rook-ceph
  pool: replicapool
  imageFormat: "2"
  imageFeatures: layering,fast-diff,object-map,deep-flatten,exclusive-lock
  csi.storage.k8s.io/fstype: xfs
reclaimPolicy: Retain      # Keep volumes after deletion
allowVolumeExpansion: true # Scale volumes on demand

Key Design Decisions

Monitor Count (3): Ceph requires a quorum. With 3 monitors, the cluster tolerates 1 failure. Given my 3-node setup, one monitor per node is ideal.

Use All Nodes: This ensures distributed storage across the entire cluster, maximizing redundancy.

Use All Devices: Any available disk on any node becomes part of the Ceph cluster.

Retain Reclaim Policy: When a PVC is deleted, the underlying volume is retained (not deleted), providing data safety.

XFS Filesystem: More performant and reliable than ext4 for this use case.

Object Storage

Beyond block storage, Rook-Ceph also provides Object Storage (S3-compatible) through its Radosgw component:

apiVersion: ceph.rook.io/v1
kind: CephObjectStore
metadata:
  name: ceph-objectstore
  namespace: rook-ceph
spec:
  metadataPool:
    failureDomain: host
    replicated:
      size: 3
  dataPool:
    failureDomain: host
    erasureCoded:
      dataChunks: 2
      codingChunks: 1
  gateway:
    port: 80
    instances: 2

This enables me to:

Back up applications to S3-compatible storage
Host private registries
Create self-hosted object storage alternatives to AWS S3

Monitoring and Operations

The Rook-Ceph dashboard provides visibility into:

Cluster health and status
Capacity and usage metrics
OSD (storage) performance
Pool configurations
Real-time alerts

Accessing it is straightforward through port-forwarding:

kubectl port-forward -n rook-ceph svc/rook-ceph-mgr-dashboard 7000:7000

Real-World Usage

With Rook-Ceph in place, provisioning storage for applications is trivial:

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: my-database-pvc
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: rook-ceph-block
  resources:
    requests:
      storage: 10Gi

Applications request storage, Rook automatically provisions it across the cluster, and data is protected through replication. The complexity is hidden, the benefits are clear.

Conclusion

With Rook-Ceph in place, my homelab now has a production-grade distributed storage system. Applications no longer need to worry about node failures - storage is replicated, self-healing, and highly available.

The Complete Foundation

This chapter marks the completion of the foundational Kubernetes setup. Over these 8 chapters, we've built all the bare bones infrastructure needed to run applications reliably:

Hardware & Network (Ch. 1) - The physical foundation
Base Infrastructure (Ch. 2) - OS, networking, security
Kubernetes Cluster (Ch. 3) - Orchestration platform
GitOps (ArgoCD) (Ch. 4) - Automated deployments
Secrets Management (Ch. 5) - Secure configurations
Ingress & Load Balancing (Ch. 6) - External access
DNS & SSL (Ch. 7) - Domain names and encryption
Distributed Storage (Ch. 8) - Persistent data

What This Enables

With this foundation in place, you can now:

Deploy stateful applications with confidence
Know they'll survive node failures
Scale storage by adding nodes
Update applications safely with zero downtime
Manage secrets securely
Access services via stable domain names with valid certificates
Automate everything through version control

This is genuinely production-grade infrastructure - the kind you'd see in enterprise environments, but tailored for a homelab.

What's Next?

From here, the real fun begins. In future quests, we'll explore:

Running actual applications (databases, message queues, cache layers)
Monitoring and observability (metrics, logs, alerts)
CI/CD pipelines (automated testing and deployments)
Backup strategies and disaster recovery
Advanced networking and service mesh concepts

But for now, we have a solid, resilient, production-ready platform. Every component we've built is battle-tested, scalable, and self-healing. That's something to be proud of.

Home Lab: Chapter 3 — Kubernetes Setup

Wed, 05 Nov 2025 20:47:54 +0000

Howdy!

In this chapter, I'll walk through the setup of the Kubernetes cluster using Talos. The cluster will consist of three physical machines, each serving as both control-plane and worker to allow workloads scheduling on all nodes while maintaining control-plane functionality.

What is Talos?

Talos is a modern, minimalistic operating system designed specifically to run Kubernetes and nothing else. It is immutable, meaning the OS is read-only and cannot be modified, improving security by making it difficult for attackers to alter the system.

Talos is also managed entirely via Kubernetes, simplifying cluster operations.

Setting up Talos

Talos provides a command-line tool called talosctl, used to manage and interact with the cluster. To download it:

# Linux
curl -sL https://talos.dev/install | sh

# MacOS
brew install siderolabs/tap/talosctl

The process to install Talos OS involves:

Download the Talos image
Flash the image to a USB drive
Boot the node from the USB drive
Install Talos on the nodes
Reboot the nodes

Preparing Nodes

Before configuration, assign IP addresses to the nodes using DHCP with static leases by mapping each node's MAC address to a specific IP:

Node 1: x.x.x.101
Node 2: x.x.x.102
Node 3: x.x.x.103

Restrict the DHCP pool (e.g., x.x.x.101 to x.x.x.104) to prevent assigning addresses to unexpected devices.

Setting up the Nodes

Download the Talos image from the Talos releases page and flash it to USB:

sudo dd if=talos.iso of=/dev/sdX bs=4M status=progress && sync

Replace /dev/sdX with the USB drive path. This command overwrites the USB drive data.

Boot the node from USB, enter BIOS/UEFI to configure boot priority, then install:

talosctl install --node x.x.x.x

Repeat for each node in the cluster.

Prepare Nodes Config

Generate initial cluster configuration:

talosctl gen config

This generates three files:

talosconfig: Used by talosctl to connect to the cluster
controlplane.yaml: Configuration for control plane nodes
worker.yaml: Not needed since all nodes act as control plane nodes

Patching Nodes Config

Rather than manually editing files, use patches for a structured approach:

Key modifications include:

Allow Control Plane Workloads: Enable workloads on control plane nodes
Node Hostnames: Set unique hostnames (clustarino-k8s-1, -2, -3)
Interface Names: Allow interface IDs to be identifiable
DHCP: Enable DHCP on eth0
Disable Kubeproxy and CNI: Install our own CNI
DNS & NTP: Configure name resolution and time servers
Disk: Specify installation disk
Metrics Server: Enable cluster metrics
VIP (Virtual IP): Configure virtual IP for cluster access

Note: This post has been shortened due to character limitations. For complete patch configurations and detailed explanations, refer to the full documentation.

Generating secrets

Generate secrets for secure cluster communication:

talosctl gen secrets --output-file outputs/secrets.yaml

Generating final Nodes Config

Generate final configuration with all patches:

talosctl gen config clustarino https://x.x.x.105:6443 \
    --with-secrets outputs/secrets.yaml \
    --config-patch @patches/allow-controlplane-workloads.yaml \
    --config-patch @patches/dhcp.yaml \
    --config-patch @patches/disable-kube-proxy-and-cni.yaml \
    --config-patch @patches/install-disk.yaml \
    --config-patch @patches/interface-names.yaml \
    --config-patch @patches/metrics-server.yaml \
    --config-patch @patches/ntp.yaml \
    --config-patch-control-plane @patches/vip.yaml \
    --output rendered/

Generate node-specific configurations:

talosctl machineconfig patch \
    --patch @patches/control-plane-node-1.yaml \
    rendered/controlplane.yaml | yq - > nodes/control-plane-node-1.yaml

talosctl machineconfig patch \
    --patch @patches/control-plane-node-2.yaml \
    rendered/controlplane.yaml | yq - > nodes/control-plane-node-2.yaml

talosctl machineconfig patch \
    --patch @patches/control-plane-node-3.yaml \
    rendered/controlplane.yaml | yq - > nodes/control-plane-node-3.yaml

Applying config to the Nodes

Apply configuration to each node:

talosctl apply -f nodes/control-plane-node-1.yaml --node x.x.x.101 --insecure
talosctl apply -f nodes/control-plane-node-2.yaml --node x.x.x.102 --insecure
talosctl apply -f nodes/control-plane-node-3.yaml --node x.x.x.103 --insecure

Bootstrapping the Cluster

Specify cluster endpoints:

talosctl config endpoint x.x.x.101 x.x.x.102 x.x.x.103

Bootstrap the cluster:

talosctl bootstrap --node x.x.x.x

Monitor progress:

talosctl dashboard --node x.x.x.x

Connecting to the Cluster

Set up talosconfig:

export TALOSCONFIG=./rendered/talosconfig

Generate kubeconfig:

talosctl kubeconfig --node x.x.x.x

Validate the connection:

kubectl get nodes

You should see nodes in Not Ready state until CNI is installed.

Adding CNI

Install Cilium for networking:

helm repo add cilium https://helm.cilium.io/
helm repo update

helm upgrade --install cilium cilium/cilium \
    --namespace kube-system \
    --set ipam.mode=kubernetes \
    --set hostFirewall.enabled=true \
    --set hubble.relay.enabled=true \
    --set hubble.ui.enabled=true \
    --set kubeProxyReplacement=true

Cilium provides transparent networking, security policies, and network observability through Hubble.

Note: This post has been shortened due to character limitations. For complete Cilium configuration options and network policies, refer to the full documentation.

Verify Cilium pods:

kubectl get pods -n kube-system

After installation, check node status:

kubectl get nodes

Nodes should now be in Ready state.

Conclusion

This concludes the Kubernetes cluster setup. We've successfully bootstrapped the cluster and installed Cilium as the CNI. The base setup is complete.

While the cluster is now up and running, additional components are needed to expose services outside the cluster. In the next chapter, we'll configure the Ingress Controller for external service access.

Home Lab: Chapter 2 — Base Foundations

Wed, 05 Nov 2025 20:42:16 +0000

Howdy!

Everything needs a base to be built on top of. Nothing can be done out of the blue. Take a house for example: it needs a blueprint, then we need to start building the foundation to then build on top of. This is what I'll be tackling in this part: our baseline Infrastructure, our network layer.

Network

This chapter focuses on the actual network setup of my homelab, specifically excluding Kubernetes' internal networks or any network layers created later. This is the network that will enable me to connect to my homelab from my desktop and host the services I desire. My network consists of two main components:

Wifi Network: This is the primary network of my home, provided by my ISP's router, connecting all of my domestic devices. Due to the location of my homelab, I need to extend this network to reach my devices' location.
Custom Router: I’m configuring this router to have four network zones: WAN, LAN, DMZ, and VPN. This setup gives me full control over my network and allows me to manage the traffic between them.

Wifi Network

As I mentioned in the previous chapter, I need to extend the network from my router to the location of my devices, while also augmenting the Wi-Fi coverage throughout the rest of the house.

This was a straightforward process: plug and play. I simply connected the main unit to a power outlet, inserted an Ethernet cable from my home router into it, and placed one of the receivers near my custom router. Then, I downloaded the Devolo application for additional configuration:

Enabled 5G Wi-Fi only, as all my devices can connect to this network, eliminating the need for a 2.4G network.
Set the same SSID (Service Set Identifier, the name of my network) as my home network to extend coverage throughout the house.
Renamed the devices for easier identification:
- Powerline 1 -> PL-Router
- Powerline 2 -> PL-Office

Custom Router

OPNsense was the elected OS (Operating System) for my router. With four ports on my router, I’ll utilize each physical interface for a dedicated network:

Lan port 1: WAN - facilitating my router's internet connection.
Lan port 2: LAN - enabling my desktop's connection to the router.
Lan port 3: DMZ - hosting all my primary home services.
Lan port 4: VPN - housing the VPN server.

This setup achieves physical network separation, each with its own configuration.

The installation is straightforward:

Download ISO of OPNsense from
OPNSense Website
Create a bootable image on a thumb drive
Boot machine from the thumb drive
Follow the installation wizard
- Assigned the initial WAN network (PL-Office -> WAN Interface)
- Assigned the initial LAN network (Desktop -> LAN Interface) - this will also serve as the management interface, exposing the OPNsense web dashboard.

Once installed, I accessed the OPNsense web dashboard from my desktop using the machine's IP address - later, a DNS (Domain Name System) record will be created to avoid memorizing all IP addresses. I then created additional interfaces by assigning LAN ports 3 and 4 to DMZ and VPN, respectively.

WAN

WAN (Wide Area Network) typically refers to the interface used for internet access. In the case of my custom router, it serves exactly that role - providing connectivity between the router (and all connected devices) and the internet. The network setup involves a direct connection between just two devices: the router and a powerline adapter. However, since the adapter operates in bridge mode and uses the same IP range as the router, it effectively allows seamless communication between the router and any device on my Wi-Fi network.

LAN

A LAN (Local Area Network) typically describes a network within a home or organization. It is generally private, contrasting the public WAN. Here, the LAN network will connect to a single device initially. Though considered naming this interface Management, reflecting its current purpose (connecting my desktop to the router, later accessing the DMZ), I chose LAN as it is a broader name to accommodate potential future device additions.

DMZ

A DMZ (Demilitarized Zone) network sits between the LAN and WAN, used to host services requiring internet access without exposing the LAN. While my homelab’s DMZ aims to host internet-accessible services, it will also allow access from the LAN, maintaining separation. Hence, I opted to create a DMZ network.

Although I'll maintain LAN access from the DMZ, devices on the Wi-Fi network won’t reach the DMZ

This network will host my primary home services and core homelab infrastructure.

VPN

A VPN (Virtual Private Network) allows secure access to private networks through public internet connections. In my setup, I'm using a Raspberry Pi as the VPN server to enable global access to my homelab, as long as I have internet connectivity - obviously.

Since I'll be using only one device on this network, I only require two IP addresses - one for the Wi-Fi interface and another for the Ethernet interface, which connects to the VPN network. All other devices will access the VPN remotely, beginning with the Wi-Fi interface and then transitioning to the Ethernet interface for VPN network connections.

DNS

To simplify service access, I’ll create a DNS server for name resolution, allowing me to access my services by using the defined names instead of their IP addresses.

I've chosen Unbound as my DNS server because it’s easy to use and allows me to create the DNS records I need. It also works well with OPNsense, making DNS record management simpler. I'll use this DNS as the resolver for all my devices, letting me access my services by their names and resolving external domains too.

The installation is straight forward:

Access the OPNsense web dashboard.
Navigate to System -> Settings -> General.
Find the DNS section and enable the DNS Resolver.
Save changes and add DNS records.

This is what the final configuration looks like:

Connecting Networks

With all networks established, firewall configuration requires adjustment to
allow communication between the ones that I need to cross-connect:

LAN -> DMZ
LAN -> WAN
VPN -> DMZ
VPN -> WAN
DMZ -> LAN
DMZ -> WAN
DMZ -> VPN

Essentially, the goal is to allow LAN connections to access the internet and DMZ for local infrastructure access. VPN will require similar configurations. Bi direcional communication must be enabled for these connections.

This configuration might be too broad but it will be enough for an initial setup. I may revisit it in configuration.

Conclusion

This concludes the foundational infrastructure setup - My Homelab's Network. I covered the various networks established, outlined requirements, and detailed configurations. While broad in scope, it serves as an excellent starting point for future enhancements.

Originally published at https://techquests.dev on April 25, 2025.

Home Lab: Chapter 7 — Kubernetes DNS and SSL

Wed, 05 Nov 2025 20:41:17 +0000

Howdy,

Note: This post has been shortened due to character limitations. For complete patch configurations and detailed explanations, refer to the original post.

Our environment is starting to take shape. We have a Kubernetes cluster up and running, an Ingress Controller managing external access to our services, and a way to handle secrets. The next step is making sure our services are accessible from the outside world. To do this, we need to configure DNS and SSL.

Getting a Domain

Register a domain from any registrar (Namecheap, GoDaddy, etc.) with full DNS management access. Using your own domain is recommended for straightforward SSL certificate issuance and HTTPS access without browser warnings.

DNS and SSL

DNS (Domain Name System) is what translates human-readable domain names into IP addresses, allowing us to access websites and services without memorizing numbers. We've touched on DNS before, but it's worth revisiting since it plays a crucial role in exposing our services. In a previous chapter, we set up a DNS server to resolve some of our internal services - mostly infrastructure-related. Now, we want to extend DNS to resolve the domain names for services that will be accessible both externally and internally.

Because we have two different scenarios, we'll need two DNS setups:

Internal-facing applications - accessible only within our network.
Public-facing applications - accessible from the internet.

To keep things simple, I'll use two separate DNS servers for these scenarios. One server will manage public records, and the other will manage internal records. This isn't strictly required - we could use a single DNS server for both - but separating them helps avoid conflicts and keeps things organized.

Some configuration details will vary depending on the DNS solution you choose. In this guide, I'll be using Bind9 for internal-facing applications and Cloudflare for public-facing applications. You can pick whichever DNS servers you prefer, as long as you can manage both internal and external records without conflicts.

Below is a high-level overview of the DNS and SSL setup for our Homelab:

flowchart TD subgraph Firewall["Firewall"] Unbound["Unbound"] end subgraph Internal_Facing["Kubernetes"] Bind9["Bind9 Authoritative DNS"] Internal_Services["Internal Services"] Internal_Ingresses["Internal Ingresses"] ExternalDNS["ExternalDNS"] CertManager["Cert-Manager"] end subgraph Internal["Homelab"] Internal_Facing Firewall end subgraph Public["Public"] CF["Cloudflare DNS"] CFIP["Cloudflare Anycast IP"] CF_Tunnel["Cloudflare Tunnel"] PublicResolvers end Unbound -- Forwards internal --> Bind9 Unbound -- Forwards public --> PublicResolvers[(Public DNS)] Bind9 --> Internal_Ingresses Internal_Ingresses --> Internal_Services ExternalDNS -- RFC2136 Updates --> Bind9 CertManager --> Internal_Services CF -- A/CNAME Record --> CFIP CFIP --> CF_Tunnel CF_Tunnel --> Internal_Services ExternalDNS -. Sync Records .-> Bind9 CertManager -. "DNS-01 Challenge" .-> CF

Internal facing DNS records

For internal-facing applications, we'll be using Bind9, an open-source authoritative DNS server. This setup allows us to:

Host internal DNS records for services accessible only within our network (e.g., nginx.<INTERNAL_DOMAIN>).
Resolve public domains by forwarding requests to external resolvers such as 1.1.1.1 (Cloudflare) or 8.8.8.8 (Google).

By combining Bind9 with Unbound as a forwarder, Bind9 becomes our primary internal DNS server capable of resolving both internal and external domains.

Bind9 Setup

Install Bind9 via Helm and ArgoCD with basic configuration:

helm:
  valuesObject:
    image:
      repository: internetsystemsconsortium/bind9
      tag: "9.21"
    service:
      dns-udp:
        type: NodePort
        ports:
          dns-udp:
            nodePort: 30053
    chartMode: authoritative
    persistence:
      enabled: true

Bind9 Configuration

Define zones using a named configuration file with TSIG keys for secure updates:

named.conf.local: |
    key "tsig-key" {
        algorithm hmac-sha512;
        secret "<SECRET>";
    };
    zone "<INTERNAL_DOMAIN>" in {
        type master;
        file "/named_config/<INTERNAL_DOMAIN>.zone";
        allow-transfer { key "tsig-key"; };
        update-policy { grant tsig-key zonesub ANY; };
    };

After creating the configuration files, push to Git and ArgoCD will deploy automatically.

Record Creation

ExternalDNS automatically manages DNS records for Kubernetes resources via RFC2136, a DNS update protocol supported by Bind9. Install ExternalDNS via Helm:

provider: rfc2136
rfc2136:
  host: dns-bind9-dns-tcp.dns.svc.cluster.local
  port: 53
  zone: <INTERNAL_DOMAIN>
  secretName: external-dns-tsig-key

Generate TSIG keys with tsig-keygen -a hmac-sha512 tsig-key and store as a Kubernetes secret for secure updates.

TLS Certificates

Install Cert Manager to automate certificate issuance via Let's Encrypt with DNS01 challenges. Configure a ClusterIssuer:

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: "<EMAIL>"
    privateKeySecretRef:
      name: letsencrypt-production
    solvers:
      - dns01:
          cloudflare:
            apiTokenSecretRef:
              name: cert-manager-cf-api-token

Annotate Ingress resources with cert-manager.io/cluster-issuer: letsencrypt to request certificates automatically.

Testing the Internal Setup

Deploy a test nginx application with an Ingress annotation for ExternalDNS and cert-manager to automatically create DNS records and TLS certificates:

kubectl apply -f nginx-internal-test.yaml

Verify the setup with:

kubectl get ingress nginx -n default
kubectl get certificate nginx-tls -n default
curl "https://nginx.<INTERNAL_DOMAIN>"

Unbound Forwarder

Configure Unbound to forward internal domain queries to Bind9 via OpnSense:

Navigate to Services -> Unbound DNS -> Query Forwarding
Add forwarding for <INTERNAL_DOMAIN> to Bind9 IP x.x.x.101 on port 30053

This enables DNS resolution for all internal services from anywhere in the network without specifying the DNS server.

Public facing DNS records

Use Cloudflare for public-facing applications. It provides free DNS services and Cloudflare Tunnels, which securely expose internal services to the internet without revealing your IP address.

Cloudflare Tunnels

Cloudflare Tunnels act as a tunnel between your services and Cloudflare, hiding your infrastructure IP and providing built-in attack protection. Use the cloudflare-operator to manage tunnels via Kubernetes.

Install the operator and create a tunnel:

apiVersion: networking.cfargotunnel.com/v1alpha1
kind: ClusterTunnel
metadata:
  name: cf-tunnel
spec:
  newTunnel:
    name: cf-tunnel
  cloudflare:
    email: "<EMAIL>"
    domain: "<PUBLIC_DOMAIN>"
    secret: cf-api-token

Expose applications using TunnelBinding:

apiVersion: networking.cfargotunnel.com/v1alpha1
kind: TunnelBinding
metadata:
  name: expose-nginx
subjects:
  - name: nginx
    spec:
      fqdn: nginx.<PUBLIC_DOMAIN>
      target: http://nginx.default.svc.cluster.local:8080
tunnelRef:
  kind: ClusterTunnel
  name: cf-tunnel

DNS records and TLS certificates are created automatically by Cloudflare.

Testing the Public Setup

Deploy a test nginx service and create a TunnelBinding to expose it:

kubectl apply -f nginx-external-test.yaml

Verify tunnel status:

kubectl describe tunnelbinding expose-nginx -n default

Access the application at https://nginx.<PUBLIC_DOMAIN>. DNS records and TLS certificates are automatically created and managed by Cloudflare.

Conclusion

In this chapter, we tackled one of the most important steps in making our cluster truly usable from anywhere: DNS and SSL. We mapped out the architecture, set up Bind9 for rock-solid internal DNS, and leaned on Cloudflare for public-facing names - all with automation in mind. Thanks to ExternalDNS and Cert-Manager, record creation and TLS issuance now happen without manual intervention, keeping everything secure and up to date.

With this in place, our homelab services have:

A clean separation between internal and public DNS management.
Automated DNS updates directly from Kubernetes resources.
Seamless HTTPS access - internally and externally - without scary browser warnings.

The end result? Any service we spin up can be securely exposed, tested, and shared with almost no extra work. We're no longer manually juggling DNS zones or dealing with certificate renewal headaches - it's all declarative, reproducible, and in sync with our GitOps flow.

From here, we can focus on deploying more useful applications, knowing that they'll just work whether we're inside the lab or halfway across the world. In the next chapter, we'll start putting this setup to use by deploying real workloads and integrating them into our automated homelab stack.

Home Lab: Chapter 1 — Requirements, Hardware, Software and Architecture

Fri, 03 Oct 2025 21:34:37 +0000

Howdy!

Welcome to the first quest I'll be tackling! This one might take a while to complete, as it will serve as the foundation for everything that follows - my Home Lab.

I've had the idea of creating a Home Lab for quite some time now - a place to test things, learn new technologies, and essentially just have some fun.

While exploring similar projects, I noticed a common theme: most people build a Home Lab to self-host services they use - NAS servers, media servers, VPNs, etc. While I do plan to host some of those too, my main focus is on building my own applications and services, not just hosting them. I want to recreate what you'd typically find with on a cloud provider or in a software development company - a platform that supports development.

My goal is to create a developer-focused platform where building, testing, deploying, and monitoring applications is the core.

This will be extremely opinionated, based on my experience. I’ll be using tools I’m familiar with, as well as exploring others I want to learn.

Requirements

Let's start with some requirements I've gathered for my setup.

Orchestration

For this project, I want to use Kubernetes as the main orchestrator. I've been using Kubernetes for quite a while now and, while Docker might be a better approach for home use, I want my setup to be fairly close to what you would see in a real production environment. After all, I expect to host some of my own projects, one of them being this blog.

So, if you're reading this, it means that I've already set up my Home Lab, and some of the requirements that we'll be mentioning have already been met.

Many have given up on self-hosting Kubernetes for something simpler. I might follow that same path, as warned by some of my colleagues. But until then, let's continue.

As I plan to use Kubernetes, I'll need at least 3 nodes. After some research, I decided to use 3 mini computers since I wanted to run the nodes on actual hardware instead of virtual machines. Initially, I considered using Raspberry Pis, but many of the tools I plan on using do not support the arm64 architecture. Additionally, most enterprise applications run on amd64, so it made sense to stick with that to keep my setup as close to real-world scenarios as possible.

Network

For the internal network, I wanted to separate it from my home network. That meant I needed a router. Not just to create separation, but also to explore networking configurations and have a physical distinction between home and development environments.

To achieve this setup, I chose a mini computer equipped with multiple Ethernet ports to function as a router. This device will handle the connection between my home router and all devices within the DMZ (Demilitarized Zone) network. Implementing a DMZ provides an additional layer of security by isolating internal resources from direct external access.

To make future expansion easier, I added a switch to the internal network. This will allow me to connect more nodes or devices to the DMZ network as needed.

Storage

Storage is a crucial requirement, so I added a NAS server. It stores backups for stateful applications (e.g., databases), my personal data, and serves as a self-hosted alternative to services like iCloud, Google Drive, and Dropbox.

Given the above, here is the list of hardware I’ll be using:

1 Mini computer with additional LAN ports (router)
1 Switch (for the DMZ network)
3 Mini computers (K8s nodes)
1 Computer (NAS)

Hardware

With the plan in place, it was time to start doing some research.

Router

As mentioned, I wanted a mini computer with extra ports for the router. I've worked with Cisco routers and pfSense in the past, but for this setup, I wanted to try something new. After some research, I went with OPNsense - an open-source firewall/router software that's a fork of pfSense.

I found a great mini PC on AliExpress:

4 LAN ports
Intel i3 N305 (8 cores / 8 threads)
32GB RAM
1TB NVMe SSD
Passive cooling
Solid build quality
~450€

Powerline

My office is far from the router, and I wanted a wired connection. Drilling through walls and running long cables weren’t appealing, and I didn’t want to use Wi-Fi extenders. That left me with Powerline adapters.

Powerline devices uses the electrical wiring to transmit network signals. I chose the Devolo Magic 2, which extends Wi-Fi coverage and also provides
Ethernet ports. While the speed droped quite a bit, from ~500 Mbps to ~150 Mbps, this is still acceptable for my needs.

Switch

For the switch, I reused a 5-port model I had lying around (TP-Link TL-SF1005D) It’s quiet,compact, and fits my current requirements.

Nodes

I wanted the nodes to be quiet, compact, and performant. I found 3 mini PCs with the following specs:

Ryzen 7 5700U (8 cores / 16 threads)
16GB RAM (upgradable to 32GB)
512GB SSD
Passive cooling
Good build quality
~250€

They came with two M.2 slots, and I wanted to separate the OS from the data. So, I added three Kingston DataTraveler Exodia thumb drives (one per node) to boot the OS while keeping the M.2 SSDs for local node storage.

NAS

Long-term storage has always been something I needed. If you just want reliability, I’d recommend a cloud provider like Google Drive or a prebuilt NAS from Synology or QNAP. But I’m here to learn, so I went DIY.

I found a second-hand custom-built NAS:

5 bays (no drives included)
Intel N5000 (4 cores / 4 threads)
64GB ECC RAM
Passive cooling
Good build quality
400€

Software

With the hardware ready, it was time to look at the software stack.

Wireguard

I wanted to access the DMZ network remotely, just like I would with a cloud provider. I chose WireGuard as the VPN solution. It’s lightweight, fast, and secure.

I'll be hosting it on a Raspberry Pi 4B.

OPNSense

OPNsense is not just a router - it’s a flexible firewall platform with plugin support. I’ll use it alongside Unbound DNS to:

Assign hostnames to devices
Set up a local DNS server
Define static IPs

This setup offers much more flexibility than the router provided by my ISP.

Kubernetes

Kubernetes is the orchestrator of choice. To run it, I'm using Talos - a minimal OS built specifically for Kubernetes. Talos aligns perfectly with my goals: it's secure, immutable, and easy to manage.

Kubernetes alone isn’t enough, so I’ll be adding these tools:

Networking: Cilium for pod-to-pod communication and network policies.
Ingress: NGINX Ingress Controller for external access.
Storage: Ceph for persistent volumes.
Certificates: Cert Manager to automate certificate management.
Metrics & Logs: Talos has built-in metrics support, which I’ll use for monitoring.
CI/CD: ArgoCD for GitOps-based deployments.

This will be the base stack. More tools will be added as the platform evolves.

Homelab Overview

To better understand the components that we'll be using, here is a diagram of the components that we'll be using:

Architecture

Here’s a rough overview of the network setup (read right to left):

ISP connects to the Home Router (standard consumer-grade router).
Powerline Adapter connects the Home Router to the Custom Router in my office.
Custom Router manages the DMZ network, where all critical infrastructure lives.
Raspberry Pi runs the VPN server to allow remote access to the DMZ.

Network Breakdown

Wi-Fi Network: Main home network. Used by everyday devices not part of the DMZ.
WAN Network: Connects the Home Router to the Custom Router. Provides internet access to all custom networks.
DMZ Network: Hosts the Kubernetes nodes, NAS, and future services. Managed by the Custom Router.
VPN Network: Contains the Raspberry Pi and allows external access into the DMZ.
LAN Network: Contains the desktop PC. It didn’t make sense to have it on Wi-Fi, so it gets its own segment.

This is the initial setup. It’s designed to be modular and expandable over time.

Conclusion

This wraps up the first chapter of my Home Lab series. We covered the motivations and requirements behind the project, explored the hardware and software choices, and reviewed the overall architecture of the platform I’m aiming to build.

I'm really excited to start building this, as I feel like this will be a great learning experience - not just in terms of configuration, but also in exploring, architecture, planning, implementation, and of course, documenting and sharing.

I hope you’ll enjoy this journey as much as I will. Stay tuned for the next chapter! And if you have any suggestions, feel free to reach out.

Art Lovers Hub

Wed, 06 Aug 2025 06:53:03 +0000

I have two Instagram accounts. One is a personal account where I share piano covers and pencil drawings, the other is for sharing AI generated abstract art. Particularly with the art account, my experience has been less than stellar. IG keeps temporarily restricting my account for this or that reason. It’s never quite clear what the reason is, and I always feel like I’m one step away from getting banned and there is no recourse if that happens.

So I decided to create an alternate place to showcase my art without any limitations.

The project is vibe-coded, same as my other ones. There is a lot said about vibe-coding, and being a senior developer, I’m completely sold on it. We are past a turning point and there is no looking back as far as I am concerned.

I’m happy to answer any questions, or if you are interested, let me know and I’ll get you on the waiting list.

Some Ranting on Dev.to and Opinion on UI Design

Mon, 04 Aug 2025 19:43:11 +0000

Overview

Each platform strives to provide its best and each has its pros and cons. Some are simple technical issues - like the Markdown editor of CoderLegion, some are serious design flaws (like there is no way to manage brushes in Affinity Photo).

Today I was going to comb my thoughts and write something on Dev.to and was shocked so see this.

This ugly new feature is driving me mad.

It's one of the things I don't like about Discord/Slack. The interface is so clustered with irrelevant things or buttons that I rarely click on or need. And there are too many colors.

Now Dev.to is completely ruining the used-to-be quiet and focused writing experience. Just look at the screenshot below:

It's honestly so annoying I can't bear staying on this site any second longer. What's worse, because of this total distraction, I almost forgot what I was about to write.

Alternatives

I would love to officially moving to CoderLegion for canonical Dev Logs, but the image uploading experience is also not smooth. So that's a dilemma.

We have the official Methodox Blog on WordPress. For those who aren't familiar with its content management system - it requires uploading images to the blog server for every single thing you display (or else use plugins) and is really inconvenient if one just want to share quick screenshots.

Medium is actually great writing experience, but its design feels more suited for literature work than for tech.

Ideal Scenario

Here I propose a concept "QuietSpace" for distraction-free blogging.

Key ideas:

Medium style editing interface
Dev.to style Markdown and easy image uploading through dropping and pasting, single button Preview
CDN for blog contents
GitHub style support for video and YouTube embedding.

Key formatting elements:

Fully support tables (a issue in both CoderLegion and Medium)
Fully support images gifs and videos (through CDN link) (an issue in Dev.to)

Web editor, optional desktop editor. No dedicated subdomain: url path for user. No ads, central explore and save articles. May even be paid writing experience at $5 per month. Easy sharing, simple commenting.

Used for: DevLog.

Derivation: dedicated blog-style web hosting, like the simplest form of Word Press.

Summary

There are many blogging and posting platforms and each has distinct features. As a developer we seek a balance between publicity, platform friendliness and technical satisfaction. I think the best platform has to be customized for custom needs. While generic customizable frameworks exist, the customization process right now still take too much tweaking. Will AI solve this problem? I doubt it, as I see missing pieces in basic infrastructure that's not easily addressed with smart agents.

Follow Up

I was writing this post bearing in mind I have to suffer the new UI update of Dev.to - life felt truly like a compromise.

However, the dev team addressed the issue within an hour! I didn't even expect them to notice my post. Salute To Dev.To Dev Team! See discussion on the post on Dev.to.

Coder Legion - Recent posts tagged platform

The Cost of Open Source

Part One: Loss of Control Over Use

Part Two: The Erosion of Attribution

Part Three: Premium Functionality Without a Price

Part Four: Expectation, Noise and Maintainer Fatigue

Unpaid support obligations

Automated noise

Usage without support

Fatigue as a security risk

Part Five: Why Developers Continue to Publish

Part Six: Managing the Costs

Select a license intentionally

Make attribution durable

Protect the name

Document expectations

Provide a route for support

Set limits

A Case in Point: Obsidian AI

Conclusion

How to Improve Enterprise Software Usage Without Disrupting Workflows

9 Steps to Build an IPTV Web Player With JavaScript: From M3U Playlist to Live Playback

What Is an IPTV Web Player?

Understanding M3U and M3U8 Playlists

Why Use JavaScript to Build an IPTV Player?

Step 1: Create the Basic IPTV Web Player Structure

Step 2: Represent IPTV Channels as JavaScript Objects

Step 3: Parse an M3U Playlist With JavaScript

Step 4: Fetch an M3U Playlist

Step 5: Render the IPTV Channel List

Step 6: Play an HLS Stream in the Browser

Step 7: Connect the Playlist to the Player

Step 8: Add Channel Search

Step 9: Add Categories and Channel Metadata

Handling HLS Playback Errors

Understanding IPTV Buffering

Improving IPTV Channel Switching

Autoplay Restrictions in Modern Browsers

Building a More Maintainable JavaScript IPTV Player

Security Considerations for IPTV Web Applications

CORS and M3U Playlist Access

Measuring IPTV Player Performance

Startup time

Channel switching time

Rebuffering ratio

Manifest loading time

Segment download time

Playback errors

Common Mistakes When Building a JavaScript IPTV Player

Using the M3U file as the player source

Assuming every browser supports HLS in the same way

Ignoring cleanup

Loading massive playlists unnecessarily

Putting playlist parsing inside the video component

Ignoring network errors

Testing only on a fast connection

Taking the IPTV Web Player Further

Electronic Program Guide

Favorites

Recently watched channels

Responsive design

Keyboard navigation

From JavaScript Prototype to Production IPTV Platform

Why M3U + HLS Is a Useful Developer Project

Final Thoughts

9 Steps to Build an IPTV Player With React and HLS.js: A Complete Developer Guide

Why Build an IPTV Player With React and HLS.js?

What Is HLS Streaming?

HLS.js vs Native HLS Playback

Step 1: Create the React Project

Step 2: Create a Reusable React HLS Player

Step 3: Understand the M3U8 Playlist

Step 4: Build the IPTV Channel List in React

Step 5: Handle IPTV Stream Errors Properly

Step 6: Improve Channel Switching Performance

Step 7: Add Adaptive Bitrate Streaming

Step 8: Add Loading States and Playback UX

Step 9: Add CORS and Security Considerations

React IPTV Player Architecture for Production

How to Optimize an IPTV Player for Better Performance