Coder Legion - Recent posts tagged get

The Silent Enclosure: Stealing the Light

Wed, 26 Aug 2026 19:07:24 +0000

Pushed Out by the Shadows

For almost seven months now I have blinked less than the crippled, cramped fingers I count on my tired hands. I have howled so loud that even the sheep ignore me when I howl at the largest light in the room. “That's mine — I deserve a part,” I say as the light drifts away and covers me in its thick, worldly shadow, just to be forgotten like the sheep I can no longer eat or see because they think me weak, crazy, delusional, and band together, pushing the lone wolves out just to get their take.

Google invited me in, then slammed the Nexus 26 closed when my spirits rose. I watched them gawk with glee, throwing money and tech around, colliding and pointing and whispering while I watched: “You get this part, you get that part, I’ll take this part, and together we’ll crush them all.” I wasn’t crazy — it happened right in front of our eyes for six months. Google and Microsoft overnight poured insane amounts of money into an industry they had pushed out because it was too centralized, not by the book, "unscripted."

Now Coinbase takes my color and my wallet wants to call it a demo. MongoDB and GitHub use my data to fuel the industry so my flame can’t burn brighter than theirs. So, I can never keep up: by 2:00 a.m. I make the next big thing; by 2:01 my phone is pinging — Google released, Microsoft coming out with a new partner of the year, MongoDB, Google Cloud integrating Coinbase. But wait — it’s even bigger now: Google and Microsoft coming together.

I’m trying, but now I’m homeless, penniless. I have nothing — not even my shadow dares show itself. I’m stranded, pushed out by the shadows who are stealing the light.

Gratitude and Resolve

My words are heavy because so is the path I walk, but I have strength and resilience few could fathom and less dream of seeing. I am broken but beyond blessed: I have water to drink, food to eat, shoes on my feet, and clothes on my back. I am too blessed to be stressed. Nothing in this world can stop me, for I am a child of the one true King. I am a King of kings who will walk roads paved with gold and live in houses that sparkle brighter than diamonds. I will rejoice because I know that, and nothing can change it.

A Note on Rest

Take care of your body and hands. Rest matters. Tonight, step away from the screens and let your hands and mind recover. Your health and safety come before any platform, code, or company.

Closing

I hold on to gratitude for the basics and to the inner resolve that keeps me moving forward. The tech world may feel like it’s moving against you, but your voice, your vision, and your faith remain yours. Keep that flame — tend it, protect it, and let it guide you through the dark.

TO ALL, WE ARE “Government of the people, by the people, for the people.” — Abraham Lincoln, Gettysburg Address

Authored: NeonCyan0x · DEVPlank · Blade Yerby

The Mechanics of Compression: How 100GB Becomes 25GB

Sun, 19 Apr 2026 23:55:52 +0000

The Mechanics of Compression: How 100GB Becomes 25GB

In an era of 4K streaming and massive cloud databases, we often take for granted that a 100GB backup can be shrunk down to 25GB. It isn’t magic—and it isn’t just “squeezing” bits. It’s a careful process of spotting patterns, removing redundancy, and using mathematical shorthand so that the same information takes less space. This post explains how that works in plain terms, so you can understand what happens under the hood when you zip a folder or stream a video.

1. The Core Idea: Redundancy Is Waste

At its heart, compression is about efficiency. Most data is repetitive. The same words, phrases, or pixel patterns show up again and again. If the computer has already seen a pattern once, it doesn’t need to store it again in full—it can refer back to it or describe it in fewer bits.

Think of a recipe that says “add salt” five times. You could write “add salt” five times, or you could write “add salt (×5).” The second version carries the same information in less space. Compression algorithms do something similar: they find repetition and describe it more compactly.

So the main job of any compression scheme is to find redundancy and encode it in fewer bits without losing the ability to recover the original data when needed.

A Real-Life Example: Compressing a Book with Repeated Words

Imagine a book where the same words appear again and again. Words like "the," "and," and "to" might show up thousands of times. Instead of writing the full word every time, we could agree on a legend at the start and replace each repeated word with a short sub-symbol. Anyone with the legend can reconstruct the original text exactly.

Suppose we define a small dictionary at the top of the page:

Legend:  @ = the   |   # = and   |   % = to   |   § = of

Now take a normal sentence and "compress" it by substituting:

Original (longer): "The king and the queen went to the castle and stayed there for the rest of the day."

Compressed (shorter): "@ king # @ queen went % @ castle # stayed there for @ rest § @ day."

We still have the same meaning, but we used one character (@, #, %, §) wherever a repeated word appeared. The legend is the "dictionary"—it has to be stored or sent once so the reader can decode. For a whole book, if "the" appears 2,000 times, we save (3 − 1) × 2,000 = 4,000 characters just on that one word. Add "and," "to," "of," and other frequent words, and the book shrinks noticeably without losing a single word.

This is exactly the idea behind dictionary coding and Huffman-style compression: find what repeats, assign it a short code, and use that code everywhere. Real compressors do this automatically; they don't need a human to write the legend.

A real experiment from the wild: Researchers applied this idea to the full text of "Alice's Adventures in Wonderland" by Lewis Carroll. They treated each word as a symbol, counted how often it appeared, and assigned shorter codes to the most frequent words (like "the," "and," "to," "a") and longer codes to rare words (like "wretched," "yawned"). The result: the text went from 164 KB down to 109 KB—about one-third smaller—with no information lost (Huffman-coding English words, Project Nayuki—includes original text from Project Gutenberg, input/output file sizes, and codebook samples). The same technique—replace repeated words with shorter symbols, and keep a mapping so we can decode—is what powers the compression we use every day in ZIP files and beyond.

2. Lossless vs. Lossy: Two Different Goals

Before diving into algorithms, it helps to know that compression splits into two families: lossless and lossy. The choice depends on whether you can afford to lose any information.

Lossless Compression: Every Bit Preserved

Lossless compression means that after you decompress, you get back exactly the original data—every bit. Nothing is discarded.

How it works: The algorithm finds statistical redundancy (repeated or predictable patterns) and encodes them more efficiently. Decompression reverses the process and reconstructs the original stream.
When to use it: Whenever you cannot afford to lose information—source code, text files, databases, executables, legal documents, medical data, or images where every pixel must stay exact (e.g. PNG for graphics, FLAC for audio).
Common formats: ZIP, GZIP, PNG, GIF (for lossless use), FLAC.

So when you compress 100GB of source code or logs to 25GB with a lossless tool, you can later decompress and get back the same 100GB bit-for-bit.

Lossy Compression: Smaller Files, Imperceptible Loss

Lossy compression discards some information on purpose. What you get back is an approximation of the original—close enough for human perception, but not identical.

How it works: The algorithm keeps what matters for the intended use (e.g. what the eye or ear notices) and drops “invisible” or less important detail—e.g. colors we can’t easily tell apart, or sounds masked by louder ones.
When to use it: Media where small quality loss is acceptable—photos (JPEG), music (MP3, AAC), video (H.264, VP9). Streaming and storage would be far heavier without lossy compression.
Common formats: JPEG, MP3, AAC, H.264, VP9.

For the rest of this post we focus on lossless compression—the kind that powers ZIP, GZIP, and the 100GB→25GB backup scenario—and then briefly touch how video combines similar ideas with lossy temporal compression.

3. The Building Blocks: Huffman Coding and LZ77

Most modern lossless compressors (including the ones inside ZIP and GZIP) combine two ideas:

Dictionary / back-reference coding (e.g. LZ77): “I’ve seen this phrase before; refer back to it.”
Entropy coding (e.g. Huffman): “Frequent symbols get short codes; rare symbols get longer codes.”

Together they form algorithms like DEFLATE (used in ZIP and GZIP): first reduce repetition with something like LZ77, then shorten the remaining stream with Huffman (or similar) coding.

3.1 Huffman Coding: Short Codes for Common Symbols

In normal text encoding (e.g. ASCII or UTF-8), every character uses the same number of bits (e.g. 8 bits per character). So the letter “E” and the letter “Z” both cost 8 bits, even though “E” appears far more often in English text. That’s wasteful: we’re spending the same budget on rare and common symbols.

Huffman coding fixes that by giving variable-length codes: frequent symbols get shorter codes, rare symbols get longer codes. On average, the whole message uses fewer bits.

One crucial constraint: no code is allowed to be a prefix of another. So if “E” is encoded as 10, then no other character’s code can start with 10 (e.g. no 100 or 101). That way, when we read the compressed stream left-to-right, we always know where one symbol ends and the next begins—no need for extra separators (unlike Morse code, which needs pauses between letters).

A simple way to build such codes is with a binary tree: each character is a leaf, and the path from the root to that leaf (e.g. left=0, right=1) is its code. The algorithm builds the tree from the bottom up, repeatedly grouping the two least frequent symbols (or groups) until everything is in one tree. Frequent characters end up near the root (short path); rare ones end up deeper (long path).

Conceptually:

// Standard 8-bit encoding (e.g. ASCII): every character uses 8 bits
"E" = 01000101 (8 bits)
"Z" = 01011010 (8 bits)

// Huffman encoding (example): frequent letters get shorter codes
"E" = 10      (2 bits)   ← common in English
"Z" = 111010  (6 bits)   ← rare, so longer code

So Huffman doesn’t “invent” new information; it represents the same information in fewer bits by exploiting how often each symbol appears.

3.2 LZ77: “I’ve Seen This Before—Point Back to It”

Huffman only shortens symbols based on frequency. It doesn’t remove repeated phrases. That’s where LZ77 (and family) comes in.

LZ77 keeps a sliding window of recently seen data. As it reads the input, it looks for the longest match between:

what’s coming next (the “look-ahead” buffer), and
what it has already seen (the “search” buffer).

When it finds a match, it doesn’t output the phrase again. Instead it outputs a back-reference: a pair (distance, length) meaning “go back distance bytes and copy length bytes.” The decoder can then reconstruct the phrase from earlier in the stream.

So the stream becomes a mix of:

Literal bytes (when there’s no useful match), and
Back-references (when a phrase repeats).

Example:

Input:  "The quick brown fox jumps over the quick dog"
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        "the quick" appears twice

Compressed idea:
  Literals: "The quick brown fox jumps over "
  Back-reference: (distance=31, length=9)  → copy 9 bytes from 31 bytes ago
  Literals: " dog"

After this step, the stream has fewer bytes but still contains the same information. Then Huffman (or similar) is applied to that stream to shorten the representation of literals and back-reference pairs—that’s DEFLATE in a nutshell.

4. ZIP vs. TAR.GZ: Same Ideas, Different Packaging

Both ZIP and TAR.GZ use lossless compression (often DEFLATE/GZIP-style), but they package files differently. That difference affects compression ratio and how you can access files.

ZIP: One File at a Time

ZIP compresses each file separately and then puts them in one archive, with a central directory so you can list and extract individual files.

Pros: Random access. You can extract image_402.jpg from a 50GB archive without reading or decompressing the rest. Good for browsing and pulling single files.
Cons: The compressor never looks across file boundaries. If 100 files are nearly identical, ZIP may store 100 separate compressed versions and miss shared patterns. So total size can be larger than a “solid” archive.

So ZIP is great when you need per-file access and broad compatibility (e.g. Windows, macOS, Linux).

TAR.GZ: One Big Stream (Solid Archive)

TAR.GZ is a two-step process:

TAR (Tape Archiver): Concatenate all files into one continuous stream (no compression yet). This is a “solid” archive—one long byte stream.
GZIP: Compress that entire stream with DEFLATE (LZ77 + Huffman) as if it were a single file.

Because GZIP sees the whole project as one stream, it can find patterns that span many files. A repeated header in file_A.txt can be used to compress the same header in file_Z.txt. That’s why you can get a much better ratio—e.g. 100GB down to 25GB—when you have lots of similar or repeated content across files.

Trade-off: to get any file out of a TAR.GZ, you typically have to decompress from the beginning up to that file. There’s no cheap “jump to file 402” as in ZIP.

# Step 1: Bundle everything into one stream (size unchanged)
tar -cvf backup.tar ./my_project

# Step 2: Compress that whole stream
gzip backup.tar

# Result: backup.tar.gz — one compressed stream

When to use TAR.GZ: Backups, source tarballs, logs, or any case where you want maximum compression and usually extract the whole archive (or don’t mind decompressing from the start to reach a file). Common on Linux and in DevOps pipelines.

When to prefer ZIP: When you need random access to individual files or maximum compatibility with non-Unix systems.

5. How Video Compression Goes Further: Temporal Redundancy

Video has a special kind of redundancy: between frames, most of the image doesn’t change. Only moving parts (and lighting changes) differ. So instead of storing every frame in full, modern codecs (e.g. H.264) use temporal compression: store one full frame, then for the next frames store mainly what changed.

Roughly:

I-frames (Intra-frames): Full frames—like a JPEG image. They don’t depend on other frames. Used as anchors so you can seek or recover from errors.
P-frames (Predictive): Encoded as “differences” from a previous frame (or I-frame). Much smaller than a full frame.
B-frames (Bi-predictive): Use both past and future frames to predict the current frame, so they can be even smaller.

So conceptually:

Frame 1 (I):  Full image of a sunset
Frame 2 (P):  “Same as Frame 1, but the bird moved 2 pixels left”
Frame 3 (P):  “Same as Frame 2, but the bird moved 2 pixels left”
…
Total stored: one full frame + many small “deltas” and motion vectors

That’s temporal compression. Video also uses spatial compression inside each frame (similar in spirit to JPEG). Together, temporal + spatial (and often lossy choices) let streaming and storage stay practical at 1080p and 4K.

6. Wrapping Up

Compression is the invisible engine behind smaller backups, faster transfers, and watchable video. The main ideas are:

Redundancy is removed or shortened: repeated phrases become back-references (LZ77), and frequent symbols get short codes (Huffman).
Lossless (ZIP, GZIP, PNG) keeps every bit; lossy (JPEG, MP3, H.264) trades a bit of quality for much smaller size.
ZIP compresses file-by-file for random access; TAR.GZ compresses one solid stream for better ratio when you don’t need per-file random access.
Video adds temporal compression: store full frames rarely, and mostly store changes between frames.

Understanding these mechanics helps you choose the right format—whether you’re optimizing Next.js assets, shipping backups to S3, or tuning video encoding—and demystifies how 100GB can become 25GB without losing a single bit.

written using AI tools

Your Code Works. Your Architecture Doesn’t.

Sat, 28 Mar 2026 02:50:21 +0000

Why working code without architecture predictably collapses — and what to do about it.

Prefer watching? The full video covers everything in this post — with visuals and code walkthroughs.

The Code That Almost Took Down Black Friday

This code passed code review. Three senior developers approved it. It's been in production, serving real customers.

And it almost took down Black Friday.

Not because of a bug. Not because of performance. The code works. The architecture doesn't.

I've seen this exact story play out at every company I've worked at over the past decade. Different languages, different teams, same pattern. It's predictable. And it's preventable.

The Handler

Here's the code. Tell me if this looks familiar.

One handler function. About a hundred and seventy lines. And it does everything.

Top third — database queries to get product prices, discount calculations, tax logic.
Middle — Stripe payment processing.
Bottom — saving to the database, sending confirmation emails.

Business logic, infrastructure, third-party APIs — all in one function. Mixed together.

And if you're being fair? This code is fine. It works. It's readable. It handles errors properly. Three senior developers approved this PR. I would too.

And that's the trap.

Because it's not about whether this code works today. It's about what happens when you try to extend it.

The Timeline: From Ship It to Rewrite

Let me walk you through what actually happens. Week by week. This isn't theoretical — I've lived through this pattern multiple times.

Week one. Feature ships. Everyone's happy. PR approved, tests pass, production is stable. The team feels fast. And they ARE fast. This is real.

Week three. New requirement: show product prices in the cart endpoint. Developer needs product prices. Opens the order handler. Sees the query. Copy-pastes it into the new handler. Fastest way to meet the deadline. Small duplication. Seems fine.

Week five. Someone notices the duplication. Creates a helper function. Good instinct. But the order handler still has the inline version. The cart handler still has the copy-pasted version. Only new code uses the helper. Now you have three ways to get a product price in the same codebase.

Week seven. Three developers, three different approaches. One returns just the price. Another returns an object. A third returns a dictionary. No agreed pattern. Everyone solving the same problem their own way.

Week ten. Circular dependencies start forming. Orders depend on products. Products depend on discounts. Discounts depend on orders for loyalty calculations. Everything depends on everything. Can't test anything in isolation. Can't change one thing without breaking another.

Month six. The test suite takes forty-five minutes. Nobody runs it anymore. Production bugs are weekly. Simple changes take days because everyone's scared of breaking something.

And the developer who wrote that first handler? The one who shipped on time, passed code review, got the thumbs up? They're looking at the same code thinking — how did we get here? I did everything right.

And here's the sentence I've heard at every single one of these companies:

"We're talking about a rewrite."

Six months. Working code. Senior developers. Good intentions. And you're talking about throwing it all away.

Every codebase has an architecture. If you didn't choose one, you got one anyway — by accident. That handler? That IS an architecture. It's just not one anyone decided on. It's the one that happened when nobody made a choice.

And accidental architecture always ends the same way.

The AI Amplifier

Now here's where it gets worse.

Everything I just described? That timeline assumed humans writing code at human speed. Add AI to a codebase with no architecture and you don't just get the same problems. You get them ten times faster.

Think about it. You prompt your AI: "add a loyalty points discount feature." The AI looks at your codebase. It sees four different patterns for handling discounts. It picks one. Maybe creates a fifth. It generates code following that pattern — whatever pattern it found first.

Now you have five patterns. The AI didn't fix anything. It photocopied the mess at ten times the speed.

AI doesn't fix architecture. It amplifies whatever exists.

Good architecture? AI follows clean patterns. You get consistency at ten times the speed. No architecture? AI copies chaos. You get spaghetti at ten times the speed.

If you're planning to use AI — and you should be — you need better architecture, not less.

What To Actually Do

So what do you actually do about it?

Let me take that same handler and show you one decision that changes everything. You don't need to rewrite this. You need to pull it apart.

Step one: Take the business logic out. Discount rules, tax calculations, order totals — that's your business. It doesn't need to know about HTTP. It doesn't need to know about Stripe. It's just math.

Step two: Take the infrastructure out. Stripe, the database, email — those are tools. They serve the business logic. They shouldn't live with it.

Step three: What's left is the glue. Your handler just receives a request and passes it along. That's all it should ever do.

Now look at what you've got:

domain/
  models.py             — business rules
services/
  order_service.py      — orchestration
adapters/
  stripe_payment.py     — payment infra
  db_repositories.py    — data access
routes/
  orders.py             — HTTP, thin

That's hexagonal architecture.

You didn't plan it on a whiteboard for six months. You just asked one question about the code you already had: what belongs together, and what doesn't?

I'm not telling you to architect everything before writing your first line of code. That first handler? Ship it. Get it working. Pass code review. But somewhere between that first handler and the moment your codebase starts growing — before the copy-paste starts, before three developers create three patterns — make the decision.

Because the window where this is easy? It closes fast. Week one, it's a two-hour refactor. Month six, it's a rewrite. The only difference is when you chose to think about structure.

And once you have it — look at what changes:

Debugging a discount bug? One file. Twenty-five lines. Pure math.
New developer joins? They know where code goes on day one.
Tests? Under a millisecond. You actually run them.
AI generates code? It sees clean separation. It follows the pattern instead of copying chaos.

Three Things To Do This Week

One. Ask your team: "Where does new code go?" If nobody has a clear answer — that's your window. It's still open.

Two. Pick one piece of business logic buried in a handler. Extract it into a pure function. No dependencies. Just inputs and outputs. Write a test. Feel how fast it runs.

Three. Next time someone opens a PR with copy-pasted logic — don't just approve it. That's the moment the spaghetti starts. Be the person who says "let's extract this."

None of these take more than thirty minutes. All of them compound.

The Honest Trade

I'm not going to pretend there's no cost. Your first feature with architecture takes a bit longer. You're thinking about structure, defining boundaries.

But here's the thing that doesn't show up in sprint metrics but everyone feels: developer confidence.

When your code has clear boundaries, you're not scared to change things. You run your tests, they pass in seconds, you deploy. No sweating. No "let me check if this breaks the payment flow."

That feeling? That's what architecture gives you. Not just speed. Confidence. The confidence that the next Black Friday won't be yours.

Quick Gut Check

How many of these sound like your codebase right now?

"We're talking about a rewrite."
"I don't know which version of this function is the right one."
"The new developer asked where to put their code and nobody had an answer."

If any of those hit — you don't have technical debt. You have an architecture problem.

Hexagonal architecture is what I showed you today. It's what worked for me across many different projects and languages. But it's not the only answer. There's microkernel, event-driven, and dozens of other proven patterns. Go take a software architecture course on LinkedIn Learning or Coursera — people have spent decades figuring this stuff out. You don't have to invent the wheel. You just have to actually use one.

Because the difference between a codebase that scales and one that collapses? It's not talent. It's not the language. It's not even the framework. It's whether someone sat down and made a plan.

There's also a GitHub repo with the code so you can see this pattern for real. If you want to see the practical side — how to actually refactor this into clean architecture, step by step — that's the next video. Subscribe so you don't miss it.

The post Your Code Works. Your Architecture Doesn’t. appeared first on sudoish.