Coder Legion - Recent posts tagged images

Design Publishing Queues That Survive Image Provider Outages

Sun, 18 Oct 2026 11:52:14 +0000

A publishing queue should not forget an article because a photo API had a bad afternoon.

Quick solution:

if (providerError.isTransient) {
  return queue.defer({ state: "MEDIA_PENDING", retryAfter: "1h" });
}

The retry design must respect Cloudflare Workflow limits and the provider's Pexels API documentation; provider failure does not decide whether the article itself is valid.

Yet that is exactly what happens when a pipeline treats every missing hero image as "invalid content."

The article may be complete. The metadata may be valid. The internal links may pass. The only failure is an external media provider returning no usable result, a rate-limit response, or a temporary server error.

That is not the same class of failure as a malformed article.

The queue should preserve the article and defer the media stage.

Separate source validity from media availability

A clean pipeline asks two different questions:

Is the article publishable?
Is a valid hero asset available right now?

The first question is editorial.

The second depends partly on external systems.

If you merge them into one boolean, this happens:

article valid
photo provider unavailable
=> source rejected
=> next topic generated

Now a temporary outage has changed editorial history.

A better state model is:

VALIDATED
MEDIA_PENDING
MEDIA_SELECTED
READY_TO_PUBLISH
PUBLISHED

The article identity stays the same while the media stage retries.

Classify media failures

Not every visual failure deserves a retry.

Retryable

provider 429 rate limit;
temporary 5xx response;
network timeout;
provider returns zero acceptable results for one query;
asset download temporarily fails;
deployment check is still pending.

Deterministic repair

selected asset already used when uniqueness is required;
wrong dimensions;
missing attribution;
stale manifest hash;
unsupported media type.

Terminal for that candidate

license does not allow the intended use;
source URL cannot be established;
raster is corrupt;
asset content is clearly irrelevant;
provider explicitly rejects the request.

A terminal candidate is not necessarily a terminal article. Select another asset.

Bound provider retries

External APIs need retry budgets.

Pexels’ API documentation currently documents default API limits of 200 requests per hour and 20,000 per month. A publisher that retries without bounds can turn temporary scarcity into a quota problem. Cloudflare’s Workflow limits are a separate runtime ceiling, so keep provider quotas and orchestration limits distinct.

Use a fixed strategy such as:

query 1 -> retry transient failure up to 2 times
query 2 -> try one alternate search phrase
reviewed catalog -> use only if a genuinely relevant asset exists
otherwise -> MEDIA_PENDING

Do not keep changing the query forever inside one Workflow instance.

The purpose of retries is to absorb transient noise, not to search the entire internet before the run can finish.

Keep the source queued

When media is unavailable, persist the article in a state that makes the next run resume it.

Do not delete the source, mark it duplicate, move to a new topic, create another draft with the same intent, or reset the article ID.

A durable record needs enough identity to resume:

article_id: art_example
slug: example-article
state: MEDIA_PENDING
revision: 2
visual_attempts: 3

Whether this lives in Git, Workflow state, or another durable store depends on the architecture. The invariant is the important part: the article survives.

Reuse reviewed assets carefully

A local catalog can reduce provider dependence.

If the catalog already contains an unused or reusable raster that is genuinely relevant and its license permits reuse, it may be a better fallback than another external request.

But "available" is not enough.

Check topic relevance, provider/source URL, license, attribution, byte hash, dimensions, reuse rules, and whether the same visual is already overused.

Never reuse an unrelated image simply to make the pipeline green.

Bind provenance when selection succeeds

Once an asset is selected, record the evidence immediately.

Useful fields include:

provider: Pexels
source_url: https://www.pexels.com/photo/...
license_url: https://www.pexels.com/license/
attribution: Photo by ...
article_id: art_example

After downloading the final raster, add the MIME type, width, height, byte count, and SHA-256 of the transformed file.

The first block explains where the image came from. The second explains exactly which bytes the publication uses.

Do not invent file metadata before the file exists.

Avoid duplicate work after uncertain responses

Media providers and storage APIs can fail after doing the work.

Example:

download completes
network layer times out before caller records success
retry starts
same image downloaded again under another asset ID

Use stable stage identity.

If an asset selection has already been recorded for article A revision 3, the retry should inspect that record before searching again.

Likewise, if the final raster already exists and its hash matches the intended manifest, verify it instead of creating another copy.

This is the same idempotency principle used throughout Building a Reliable Batch Publishing Workflow.

Verify the actual raster, not provider metadata

A provider response can describe the original photograph, while your pipeline may crop or resize it.

The publication should bind to the final transformed bytes, not merely the provider metadata.

After transformation:

decode the file;
read actual dimensions;
compute byte length;
compute SHA-256;
update the asset manifest;
verify the article points to that asset.

The provider page remains provenance. The transformed file becomes publication evidence.

Keep provider outages out of editorial scoring

A photo API outage says nothing about whether an article has information gain.

Do not lower article quality, factual confidence, duplicate status, or rank-potential merely because an image API is unavailable.

Instead report the failure in the stage that owns it:

editorial: PASS
research: PASS
visual: DEFERRED_PROVIDER
publication: PENDING

This makes the blocker understandable to both humans and automation.

Verify after deployment

A queue is not finished when the image reaches Git.

After deployment, fetch the public hero URL and verify a successful response, correct content type, decodable bytes, expected dimensions, and that the expected article actually references it.

If the article is live but the asset is stale, keep the same publication pending until the intended raster is visible.

That exact-version discipline is part of Designing a Reliable Serverless AI Publishing Workflow.

Design for a provider-independent future

Do not let one provider's response schema leak across the whole publishing system.

Normalize a candidate:

type ImageCandidate = {
  provider: string;
  sourceUrl: string;
  downloadUrl: string;
  attribution: string;
  licenseUrl: string;
  width?: number;
  height?: number;
};

Then the rest of the pipeline can perform relevance checks, dedupe, transforms, manifest binding, and deployment verification without caring where the candidate came from.

That makes adding a second provider a local adapter change instead of a workflow rewrite.

A practical recovery algorithm

When an article reaches the visual stage:

1. inspect existing bound asset
2. if healthy -> continue
3. inspect reviewed catalog for relevant candidate
4. if found -> bind, transform, verify
5. otherwise query provider within request budget
6. reject duplicates or invalid provenance
7. transform final raster
8. validate bytes and dimensions
9. if provider unavailable -> MEDIA_PENDING
10. retry same article later

The crucial line is number 10.

When maintaining these pipelines, teams should also review policies for preventing search intent duplicates in automated systems to ensure safe retries.

Verification and execution checklist

Inspect the active article state to confirm MEDIA_PENDING is recorded instead of deleting the draft.
Verify that external API calls respect rate-limit thresholds and orchestration timeouts.
Confirm that transformed asset hashes match the recorded manifest before deployment.
Check public staging endpoints to ensure the final raster is accessible and correctly linked.

60 % of our image decode failures were HEIC files named .jpg

Mon, 28 Sep 2026 05:02:05 +0000

For a few days our analytics showed a class of failure that made no sense: images rejected by the decoder in well under a second. Not slow failures, not out-of-memory on a huge photo — instant refusals. Between 2026-09-06 and 09-10, 60 % of every decode_failed event came from the same shape of file: an iPhone photo called photo.jpg, reporting image/jpeg, whose actual bytes were HEIC.

We had a HEIC path. It was never tried, because the gate that decides whether to load the HEIC decoder was reading the filename.

Disclosure: this is from LensUp, which is ours. The fix is the boring one — read the bytes — but the interesting part is where naive byte-sniffing then went wrong, and the precedence rule we ended up needing.

How a HEIC ends up called .jpg

Nobody renames these on purpose. The path is mundane: an iPhone shoots HEIC, you share the photo through a messenger or copy it to a PC, and somewhere along the way the transport relabels it. The extension becomes .jpg, file.type becomes image/jpeg, and the container is still HEIC.

From the browser's side there is no hint of this. File.type is not derived from content — it is whatever the OS or the transport asserted. So:

// Cheap, synchronous gate so the page can decide whether to lazy-load the HEIC decoder
// module at all. Byte-level truth lives elsewhere — names and MIME lie often enough
// that this is only a candidate check, never a verdict.
export function isHeicCandidate(file) {
  const mime = file?.type?.toLowerCase();
  if (mime === 'image/heic' || mime === 'image/heif') return true;
  const extension = file?.name?.split('.').pop()?.toLowerCase();
  return extension === 'heic' || extension === 'heif';
}

That function is fine — as a hint for whether to lazy-load a wasm decoder you do not want to ship to everyone. It is not fine as the thing that decides which decoder runs. That was the bug: a hint had been promoted to a verdict.

Sniffing: twelve to sixteen bytes, never the file

The fix reads a small slice. Not the file — a slice:

const FTYP_READ_BYTES = 64;
export async function sniffInputSignature(file) {
  let head;
  try {
    head = new Uint8Array(await file.slice(0, FTYP_READ_BYTES).arrayBuffer());
  } catch { return null; }
  if (head.length < 4) return null;
  const ascii = (from, to) => String.fromCharCode(...head.subarray(from, Math.min(to, head.length)));

  if (head[0] === 0xff && head[1] === 0xd8 && head[2] === 0xff) return 'jpeg';
  if (head[0] === 0x89 && ascii(1, 4) === 'PNG') return 'png';
  if (ascii(0, 4) === 'GIF8') return 'gif';
  if (ascii(0, 4) === 'RIFF' && head.length >= 12 && ascii(8, 12) === 'WEBP') return 'webp';
  if (ascii(0, 2) === 'BM') return 'bmp';
  if (ascii(0, 4) === '%PDF') return 'pdf';
  // … TIFF (II*\0 / MM\0*), then the ISO-BMFF family below
}

Sixteen bytes covers every format the importer accepts, and file.slice() means the read is one small range request against the blob rather than a load of a 12-megapixel photo.

Where "just read the bytes" is not enough

ISO base media files — HEIC, HEIF, AVIF, MP4 and friends — all start the same way: bytes 4–8 spell ftyp, bytes 8–12 carry the major brand. So the naive version is: read the major brand, map heic → HEIC decoder, avif → native.

Then review found the hole. Two of those brands are generic:

const HEIC_BRANDS = new Set(['heic', 'heix', 'hevc', 'hevx', 'heim', 'heis', 'hevm', 'hevs']);
const AVIF_BRANDS = new Set(['avif', 'avis']);
const GENERIC_HEIF_BRANDS = new Set(['mif1', 'msf1']);

mif1 and msf1 mean "this is a HEIF container" and say nothing about the codec inside. An AVIF file may carry mif1 as its major brand and declare avif only further down, in the compatible-brands list. Treat mif1 as HEIC and you hand a file the browser decodes natively — losslessly, instantly — to a lossy wasm transcode. You have recompressed somebody's photo for no reason.

So a generic major brand gets resolved from the compatible brands, and the ftyp box is walked using its own declared size:

if (head.length >= 12 && ascii(4, 8) === 'ftyp') {
  const major = ascii(8, 12).trim().toLowerCase();
  if (HEIC_BRANDS.has(major)) return 'heic';
  if (AVIF_BRANDS.has(major)) return 'avif';
  if (GENERIC_HEIF_BRANDS.has(major)) {
    // ftyp box: [size:4]['ftyp'][major:4][minor version:4][compatible brands:4 each …]
    const boxSize = ((head[0] << 24) | (head[1] << 16) | (head[2] << 8) | head[3]) >>> 0;
    const end = Math.min(head.length, boxSize || head.length);
    const compatible = [];
    for (let offset = 16; offset + 4 <= end; offset += 4) {
      compatible.push(ascii(offset, offset + 4).trim().toLowerCase());
    }
    if (compatible.some((b) => AVIF_BRANDS.has(b))) return 'avif';
    if (compatible.some((b) => HEIC_BRANDS.has(b))) return 'heic';
    return 'heif';   // a HEIF container that names no codec
  }
}

Two details in there that are easy to skip. The box size is clamped with Math.min(head.length, …) so a bogus length cannot walk past the slice we actually read. And when no compatible brand is specific, the answer is the honest 'heif' — "a HEIF container, codec unknown" — rather than a guess.

The precedence rule: bytes win, the label breaks ties

"Always trust the bytes" is the slogan, and it is almost right. The generic-HEIF case is where it breaks: a file whose bytes only say heif could be a HEIC the browser cannot open, or something it can. There is no byte that settles it.

So the rule that shipped is narrower and, I think, the actually correct one: bytes win over the label; the label breaks ties only when the bytes say nothing.

export async function classifyInput(file) {
  const signature = await sniffInputSignature(file);
  const labelPdf  = isPdfCandidate(file);
  const labelHeic = isHeicCandidate(file);
  const labelTiff = isTiffCandidate(file);
  const isPdf  = signature === 'pdf'  || (labelPdf  && signature === null);
  const isHeic = !isPdf && (signature === 'heic' || (labelHeic && (signature === null || signature === 'heif')));
  const isTiff = !isPdf && !isHeic && (signature === 'tiff' || (labelTiff && signature === null));
  return { signature, isPdf, isHeic, isTiff, isImage: !isPdf && !isHeic && !isTiff,
           mislabelled: labelPdf && IMAGE_SIGNATURES.has(signature) };
}

Read the three interesting rows:

scan.pdf whose bytes are a JPEG is an image, not a broken PDF. Before this, those went to the PDF parser and produced pdf_invalid: 39 of them between 09-06 and 09-11, most from Windows desktops retrying the same file over and over — the user's file was fine, our routing was not.
IMG_0001.jpg whose bytes start with %PDF is a PDF. Same rule, other direction.
A generic HEIF goes to the HEIC decoder only if the label agrees. signature === 'heif' && labelHeic → HEIC. A .jpg with those bytes stays on the native path, because that is the case where the label is the only evidence that exists.

One more lie: application/octet-stream

Worth its own line, because it is not a type:

// application/octet-stream is what browsers say when they do not know — treat it as
// undeclared so the extension map (and downstream byte sniffing) can decide.
if (declared && declared !== 'application/octet-stream') return declared;

Windows file pickers hand over .pdf files with an empty or application/octet-stream type often enough that treating it as a real MIME type poisons every branch downstream. It means "no opinion", and the code should say so.

The same file also needs a small alias table, because the same format arrives under several names: image/pjpeg, image/x-ms-bmp, image/x-png, image/jpg. Normalise first, decide second.

What I would take from this

A cheap label check and an expensive byte check are different functions and should be named differently. Our bug was one function doing both jobs — a lazy-load hint named like a verdict. isHeicCandidate and isHeicFile now sit next to each other with comments saying which is which, and that naming is doing more work than the sniffer is.

"Read the bytes" needs a tie-break rule, not just a preference. Some formats genuinely do not identify their own contents. Write the precedence down explicitly — bytes, then label, then a documented default — rather than letting it emerge from the order of your ifs.

Failures that are too fast are a routing smell. A decoder that gives up in 200 ms did not struggle with your file; it was handed the wrong file. That timing was the clue that pointed at the label, and it is the one I will look for first next time.

If you want to see the byte path do its thing, drop a HEIC — renamed or not — into a browser-based document scanner. It runs in the tab and files are never uploaded.

The transparency bug you cannot see in a thumbnail

Mon, 14 Sep 2026 08:01:17 +0000

There is a failure mode in image generation that survives every visual check you
are likely to run, ships to production, and only surfaces when a designer drops
the asset onto a coloured background. It is worth knowing about because the
thing that makes it dangerous is not the model — it is the preview.

What actually goes wrong

Ask an image model for a subject "on a transparent background" and you can get
back one of two very different files.

The first is what you wanted: an RGBA image where the pixels around the subject
have an alpha value of zero. Nothing is drawn there. Composite it over red and
you see red.

The second is an RGB image, fully opaque, in which the model has painted a
light grey and white checkerboard around the subject. Composite that over red
and you see a chequered rectangle with your subject in the middle.

The second one happens because the checkerboard is a convention, not a fact.
Photoshop, GIMP, Figma and every image editor written in the last thirty years
draw that grid to represent absence. So the training data contains an enormous
number of images in which "transparent background" and "a grid of light grey and
white squares" are the same picture. A generative model has no privileged access
to the alpha channel; it predicts pixels. Asked for the look of transparency, it
sometimes produces exactly that — the look.

Why every preview lies about it

This is the part that catches people, and it is pure interface design.

Your file browser, your image viewer, your CDN's thumbnailer and the preview
pane in whatever tool generated the image all render genuine transparency as a
checkerboard. That is the standard way to display an empty alpha channel.

So both files look identical in the preview:

	Real alpha	Painted checkerboard
Thumbnail in Finder / Explorer	grey-and-white grid	grey-and-white grid
Image viewer	grey-and-white grid	grey-and-white grid
Dropped onto a red `div`	red	grey-and-white grid

The failure is invisible in exactly the three places you would look, and visible
only in the one place you look last. If your pipeline generates assets in bulk —
product cut-outs, icon sets, sprite sheets — you can ship hundreds of broken
files before anyone notices, and the person who notices will be a customer.

Checking it properly, in about ten lines

You do not need a library and you do not need to upload anything. The check runs
on the file the browser already has. Read the pixels, look at the alpha channel,
and ask one question: does any pixel have alpha below 255?

async function hasRealAlpha(file) {
  const bitmap = await createImageBitmap(file);
  const canvas = new OffscreenCanvas(bitmap.width, bitmap.height);
  const ctx = canvas.getContext("2d", { willReadFrequently: true });
  ctx.drawImage(bitmap, 0, 0);

  const { data } = ctx.getImageData(0, 0, bitmap.width, bitmap.height);
  for (let i = 3; i < data.length; i += 4) {
    if (data[i] < 255) return true;       // at least one non-opaque pixel
  }
  return false;                            // fully opaque: no alpha channel in use
}

Two notes on getting this right.

Check the corners before you check everything. A cut-out almost always has
transparent corners, so sampling the four corner pixels answers the question for
the common case at a fraction of the cost. Fall back to the full scan only when
the corners are opaque — that is either a genuine edge-to-edge image or the bug.

A JPEG cannot pass this test, ever. JPEG has no alpha channel. If your
pipeline re-encodes to JPEG anywhere — a thumbnailer, an upload handler, an
overeager optimiser — you will destroy real transparency and produce a file that
is indistinguishable from the painted-checkerboard failure. Check the format
before you blame the model.

The stronger version of the check

Detecting some transparency is not the same as detecting correct
transparency. A file can have a real alpha channel and still be wrong: a halo of
semi-opaque pixels around the subject, left over from a soft matte, shows up as
a grey fringe on dark backgrounds.

The honest test is the one that mirrors how the asset will be used: composite it
over a solid colour and look. Two colours, ideally — one much darker than the
subject and one much lighter. Fringing that is invisible on white is obvious on
black, and the reverse is true for dark subjects.

// draw the result over a flat colour, then look at it like a human would
ctx.globalCompositeOperation = "destination-over";
ctx.fillStyle = "#c0392b";
ctx.fillRect(0, 0, canvas.width, canvas.height);

That is three lines and it turns an invisible failure into an obvious one.

Why this is worth building into the tool rather than the test suite

A test suite catches this after generation, which means after you have paid for
the image. Running the check at the moment the file appears means the failure is
reported next to the thing that failed, while the prompt that produced it is
still on screen and still editable.

That is the reasoning behind the
transparent background generator
on the interface I work on: it runs the alpha check in the browser on the file
that is already there, composites the result over a solid colour, and shows you
that before the download button does anything. It costs nothing per image
because nothing is uploaded and nothing is re-encoded — it is the same
getImageData call as above, run on every result rather than on the ones you
remember to check.

The broader tool is the
GPT Image 2.5 Image Generator, an independent
third-party interface — not affiliated with OpenAI — that prints the model id,
quality tier and size under every result. Same principle as the alpha check:
the things you would otherwise have to take on faith are shown to you instead.

The short version

"Transparent background" is a visual convention in the training data, so a
model can produce the appearance of transparency rather than the thing.
Every preview surface renders both cases identically, which is why this
reaches production more often than it should.
The check is one getImageData pass over the alpha byte; the useful check
is compositing over two solid colours and looking.
If any stage of your pipeline emits JPEG, the question is already answered and
the answer is no.

What Developers Get Wrong About Free Images

Fri, 14 Aug 2026 02:48:44 +0000

Most of us treat images as a design problem, paste a URL from a stock site into a template, and move on. That is the part of the stack that quietly creates both legal exposure and slow pages, so here is what actually matters about free images from an engineering point of view.

Free Is Three Different Licenses

The word free covers three situations that are not the same thing. Public domain and CC0 let you do anything with no conditions. Attribution licenses let you do almost anything as long as the credit line exists and is correct. Non commercial and share alike licenses carry conditions that are legally binding even though the download cost nothing, and share alike in particular can reach into what you wrap around the image.

There is a fourth question that has nothing to do with the license file. If a recognizable person, a piece of artwork, or a branded product appears in the frame, you may need a release regardless of what the photographer granted you. The photographer can license their copyright. They cannot license someone else's face.

The Metadata Nobody Automates

The failure mode is not picking the wrong license, it is being unable to prove which license you were given. Stock sites reorganize, accounts get deleted, and terms pages get rewritten, so two years later the evidence is gone and the burden is on you.

Treat it like any other dependency. When an image lands in the repo, drop a sidecar file next to it holding the source URL, the license name and version, the uploader, the download date, and an archived copy of the terms as they read that day. It is the same instinct as a lockfile: pin what you actually got, not what the page says today.

Image Prep Belongs In The Build

The other half of the problem is delivery. Serve AVIF or WebP with a fallback, generate the responsive sizes at build time rather than shipping one 4000px original, set width and height attributes so nothing shifts while loading, lazy load anything below the fold, and write alt text that describes the image instead of repeating the filename. All of that is mechanical, which means it belongs in CI, not in a checklist someone follows by hand on a Friday.

The Takeaway

Images are the one asset class most teams handle with less rigor than a dependency, and both failure modes, a takedown notice and a two second render, arrive long after the decision was made. The libraries worth using, the free AI generators, the license rules and the prep steps are collected in this guide to free images, which is a good place to point whoever on your team is pasting stock URLs into templates.