Coder Legion - Recent posts tagged ffmpeg

How I Built My Own Opus Clip Alternative With Claude Code

Tue, 27 Oct 2026 10:03:19 +0000

Opus Clip, Submagic and Vidyo all work the same way: you upload a long video, the AI scores every moment for "virality", and you get the clips it likes. That is fine until you want something specific. When I am clipping a podcast for a client I don't want "the most viral 30 seconds". I want "every time they talk about pricing" or "the strongest hook in the first ten minutes".

So I built my own Opus Clip alternative, called Hive Clips AI, with Claude Code. You give it a long video and a plain English prompt, and it transcribes the video, uses an LLM to find the moments that match your prompt, reframes each clip to keep the speaker in shot, burns in word-by-word captions and renders vertical clips ready to post. The prompt drives the selection, not a fixed score. That is the whole difference.

The Hive Clips AI home screen. Drop in a long video or paste a YouTube link.

Here is how it works, stage by stage, and what I would tell you before you try building one.

What does an AI clipping tool actually do?

Strip the marketing away and every clipping tool is the same nine-step pipeline. I built mine so each step runs on its own and saves its result, which matters later.

Stage	What happens	What I used
1. Ingest	Upload a file or paste a YouTube link	yt-dlp, FFmpeg
2. Transcribe	Speech to text with a timestamp on every word	Whisper API, or faster-whisper locally
3. Select	Find moments that match your prompt	GPT-4o with structured JSON output
4. Review	You pick which candidates to keep	Next.js UI
5. Configure	Choose aspect ratio and caption style	Next.js UI
6. Reframe	Track the face and move a virtual camera	OpenCV, PySceneDetect
7. Caption	Build animated word-level captions	ASS subtitles, libass
8. Render	Crop, burn captions, encode	FFmpeg
9. Deliver	Play, download or regenerate	FastAPI + Next.js

The backend is Python with a FastAPI wrapper. The frontend is Next.js 15 with TypeScript and Tailwind. There is also a command line version, which is where I proved the whole pipeline worked before touching the UI.

The nine stages. Select and Reframe are where most of the work went.

How does the AI pick the right moments?

This is the part people assume is magic. It is mostly careful plumbing.

A one hour podcast transcript is tens of thousands of tokens. Sending all of it in one request gives you worse picks, so the transcript is split into 9 minute windows with a 15 second overlap. The overlap is there so a great moment that sits across a window boundary doesn't get cut in half.

Each window goes to GPT-4o with the creator's prompt and a strict instruction: return 2 to 5 clips, each between 20 and 90 seconds, and only clips that genuinely match the intent. The model also rates each clip's hook from 1 to 10, judged on the first three seconds, because that is what decides whether someone stops scrolling.

The output is forced into a JSON schema, so I never have to parse loose text:

{
  "clips": [
    {
      "start_time": 412.3,
      "end_time": 468.9,
      "title": "Why he stopped charging hourly",
      "reasoning": "Direct answer about pricing with a strong opening line",
      "hook_strength": 8
    }
  ]
}

After every window is done, the candidates get merged and re-ranked, and you see them in the review screen. Nothing renders until you pick. That review step is the thing I missed most in the paid tools.

Why is auto-reframing the hardest part?

Turning a 16:9 video into a 9:16 clip means cropping a narrow window out of a wide frame and moving that window to follow the speaker. The obvious way is to detect the face on every frame and centre the crop on it.

That looks terrible. Face detection is noisy, so the crop shakes constantly, and when the video cuts to a different camera the crop slides across the cut like a drunk camera operator.

The fix is to treat the crop as a virtual camera with physics, not a box that snaps to the face:

A critically damped spring. The camera moves toward the face but never overshoots. I ported Unity's SmoothDamp function for this. Smooth time is 0.55 seconds, which feels calm without lagging behind.
A deadzone. If the face moves less than 6% of the crop width, the camera ignores it. This alone kills most of the jitter.
A speed limit. Pans have a maximum speed, so a false detection can't whip the frame across the screen.
Scene cut resets. PySceneDetect finds the hard cuts. At a cut the camera drops its speed and repositions instead of carrying a pan across the edit. Between cuts it must move smoothly.
Headroom. The subject sits about 10% above centre, the way a real camera operator would frame a person.

The camera path becomes a crop track that FFmpeg applies during the render. Face detection itself is a basic OpenCV detector that needs no model download. I kept the smoothing separate from detection on purpose, so I can swap in a better detector later without touching the part that makes it look good.

Real output from the Hive Clips smoother on noisy test data. The camera ignores small jitter and makes one calm pan when the speaker moves.

Picking the output canvas. 9:16 for Reels, Shorts and TikTok, with a live preview on the right.

If you build one of these, spend your time on the smoothing, not the detector. The smoothing is what makes it look professional.

How do the word-by-word captions work?

Most tutorials use SRT subtitles. SRT can't do the captions people actually want on Shorts and Reels, where each word lights up as it is spoken.

I used ASS subtitles instead. ASS supports per-word timing, colours and animation, and FFmpeg burns it into the video through libass. Because the transcript has a timestamp for every word, each word can change colour exactly when it is said.

The tool ships with 24 caption presets. The original 8 cover the basics: a karaoke style, a minimal style, a news bar and an outline style. Then I added 16 styles based on what is working on short-form right now, including a Hormozi-style highlight where the full line shows and the active word recolours, a one-word-at-a-time punch style, and a word reveal where words pop in and stay.

Part of the caption preset grid. Each preset is a starting point you can fine tune per clip.

What broke while building it?

A few honest notes, because build logs that only show the happy path aren't useful.

The web app broke when the CLI didn't. When I added caption overrides (custom colours and sizes per clip), the command line kept working but the web API failed. The database was missing the new column. The fix was a one-line migration, but it taught me to test both entry points every time the data model changes.

Saving every stage paid off. Each of the nine stages writes its result and a job record to the database. So when a render fails, I re-run the render only. I don't wait for the transcription again or pay for the AI selection twice. On a long podcast that is the difference between seconds and many minutes.

Mock mode made testing free. Without an OpenAI key, the tool runs with a stand-in transcript and a simple moment picker, but reframing, captions and rendering are fully real. I could test the hardest parts of the pipeline all day without spending anything on API calls. The UI labels mock mode clearly so stand-in output is never mistaken for real picks.

How did Claude Code fit in?

I wrote a product spec first: the nine stages, the five screens, and the rules for each step. Then I built it with Claude Code one stage at a time, starting from the command line and proving each stage worked before moving on.

Two habits made the biggest difference:

Interfaces before infrastructure. Storage, the job queue, the database and the AI provider each sit behind a simple interface. Right now it runs on SQLite, local files and a local queue. Moving to Postgres, Cloudflare R2 and a proper job queue means swapping one piece at a time, not rewriting the app.
Pure maths in its own file. The camera smoother has no video code in it at all. That made it fast to test and easy for Claude Code to reason about, which is exactly where you want precision.

If you are new to building with Claude Code, my beginner's guide to building an app with Claude Code covers the setup, and Claude Code pricing and free options covers what it costs to run.

What is next for the tool?

The current build is the full pipeline plus the web UI. On the list:

Sharper cut handling. Right now the reposition at a cut uses the last face position from before the cut, so on a hard camera change the frame catches up a beat late. The fix is to look ahead to the first detection after the cut.
Live progress updates instead of polling every 1.5 seconds
Regenerating only the captions or only the reframe, instead of the whole clip
Speaker detection, so two-person podcasts can get a split-screen layout
Running transcription and selection fully locally, with no API costs
Auto-posting finished clips to TikTok, Instagram and YouTube Shorts

Frequently asked questions

Is there a free Opus Clip alternative?

Yes, if you are willing to run it yourself. Open-source pieces like Whisper (transcription), FFmpeg (rendering) and PySceneDetect (scene cuts) are free, and that is most of a clipping tool. The paid part is usually the AI moment selection, and you can run that on a local model to bring the cost close to zero.

How does Opus Clip choose which moments to clip?

Opus Clip and similar tools score moments with their own virality model and give you the top picks. You can't tell them what to look for beyond basic settings. A prompt-driven tool like mine lets you describe the moment you want in plain English.

Can GPT-4o pick good clips from a transcript?

Yes, if you give it structure. Split long transcripts into overlapping windows, ask for a fixed number of clips with strict start and end times, and force a JSON schema. Then review the picks yourself before rendering. It is good at finding topics, less reliable at judging delivery, which is why the review step matters.

What is the hardest part of building an AI clipping tool?

Reframing. Detecting a face is easy. Making the crop move smoothly, without jitter, without overshooting and without sliding across scene cuts, is where most of the work goes.

Do I need a GPU to run it?

No. Local transcription uses faster-whisper, which runs on a CPU, and the face detector is lightweight. A GPU makes encoding faster, and the renderer will use one if you have it.

If you want a clipping pipeline like this built for your channel or agency, get in touch.

Originally published on danishsofi.in.

Prepare a driving clip for Wan Animate 2 with ffprobe and ffmpeg: trim, crop, re-check

Mon, 26 Oct 2026 04:17:13 +0000

Character animation from a still image has a simple contract: one image says what the character looks like, one video says how it moves. Most failed attempts are not about the model at all. The driving clip is too long, the aspect ratio does not match the character, or the file breaks a size rule the upload form will reject anyway.

This post walks through a small Python script that checks both inputs against the published rules and prints the exact ffmpeg commands that fix what it finds. Everything below was run locally on a synthetic character image and an ffmpeg test pattern. No generation job was submitted.

Diagram made for this article from the rules on the page.

The rules worth encoding

The Wan Animate 2 page lists the input requirements in its upload panel and step-by-step section:

Character image: one JPG, JPEG or PNG, up to 10 MB, both sides greater than 340 px, aspect ratio between 2:5 and 5:2.
Driving video: one MP4 or MOV, up to 100 MB, both sides greater than 340 px, same aspect range.
Duration depends on orientation: following the video's orientation accepts 3–30 second clips; following the image's orientation accepts 3–10 seconds.
Other controls: background from the image or the video, 720p or 1080p output, and an optional prompt of up to 2,500 characters.

The orientation rule is the one people miss. It changes the allowed clip length, so the checker takes it as an argument instead of assuming one value.

Diagram made for this article.

The script

It needs Pillow and ffprobe/ffmpeg on your PATH. Save it as animate_prep.py:

#!/usr/bin/env python3
"""animate_prep.py - check a character image + driving video against the
input rules published on the Wan Animate 2 page, and print ffmpeg fixes.

Rules (edit when the page changes):
  image: JPG/JPEG/PNG, <= 10 MB, both sides > 340 px, aspect 2:5..5:2
  video: MP4/MOV, <= 100 MB, both sides > 340 px, aspect 2:5..5:2
         3-30 s when orientation follows the video, 3-10 s when it follows the image
Needs Pillow and ffprobe/ffmpeg on PATH. Nothing is uploaded.
"""
import argparse, json, os, subprocess, sys
from PIL import Image

MB = 1024 * 1024
IMG_EXT, VID_EXT = {".jpg", ".jpeg", ".png"}, {".mp4", ".mov"}
MIN_SIDE, AR_MIN, AR_MAX = 340, 2 / 5, 5 / 2
DUR = {"video": (3.0, 30.0), "image": (3.0, 10.0)}

def probe(path):
    out = subprocess.run(["ffprobe", "-v", "error", "-select_streams", "v:0",
                          "-show_entries", "stream=width,height:format=duration",
                          "-of", "json", path], capture_output=True, text=True, check=True).stdout
    j = json.loads(out)
    s = j["streams"][0]
    return s["width"], s["height"], float(j["format"]["duration"])

def check_dims(tag, w, h, errs):
    if min(w, h) <= MIN_SIDE:
        errs.append(f"{tag}: {w}x{h} - both sides must be > {MIN_SIDE} px")
    if not AR_MIN <= w / h <= AR_MAX:
        errs.append(f"{tag}: aspect {w / h:.2f} outside 2:5..5:2")

def main():
    ap = argparse.ArgumentParser()
    ap.add_argument("image"); ap.add_argument("video")
    ap.add_argument("--orientation", choices=["video", "image"], default="image")
    ap.add_argument("--start", type=float, default=0.0, help="where the useful motion begins (s)")
    a = ap.parse_args()
    errs, fixes = [], []

    ext = os.path.splitext(a.image)[1].lower()
    if ext not in IMG_EXT: errs.append(f"image: {ext} not in {sorted(IMG_EXT)}")
    if os.path.getsize(a.image) > 10 * MB: errs.append("image: larger than 10 MB")
    with Image.open(a.image) as im: iw, ih = im.size
    check_dims("image", iw, ih, errs)

    ext = os.path.splitext(a.video)[1].lower()
    if ext not in VID_EXT: errs.append(f"video: {ext} not in {sorted(VID_EXT)}")
    if os.path.getsize(a.video) > 100 * MB: errs.append("video: larger than 100 MB")
    vw, vh, dur = probe(a.video)
    check_dims("video", vw, vh, errs)
    lo, hi = DUR[a.orientation]
    if not lo <= dur <= hi:
        errs.append(f"video: {dur:.1f} s outside {lo:.0f}-{hi:.0f} s for orientation={a.orientation}")
        keep = min(hi, max(lo, dur - a.start))
        fixes.append(f"ffmpeg -y -ss {a.start:g} -i {a.video} -t {keep:g} -c:v libx264 -an trimmed.mp4")

    # match the driving clip's framing to the character image (centre crop)
    ia, va = iw / ih, vw / vh
    if abs(ia - va) > 0.05:
        if va > ia:  # video too wide -> crop width
            cw, ch = int(vh * ia) // 2 * 2, vh
        else:        # video too tall -> crop height
            cw, ch = vw, int(vw / ia) // 2 * 2
        src = "trimmed.mp4" if fixes else a.video
        fixes.append(f"ffmpeg -y -i {src} -vf crop={cw}:{ch} -c:v libx264 -an framed.mp4"
                     f"   # {vw}x{vh} -> {cw}x{ch}, aspect {ia:.2f} like the image")

    print(f"image {iw}x{ih} aspect {ia:.2f} | video {vw}x{vh} aspect {va:.2f} {dur:.1f} s | orientation={a.orientation}")
    for e in errs: print("FAIL", e)
    for f in fixes: print("FIX ", f)
    if not errs and not fixes: print("PASS ready for upload")
    sys.exit(1 if errs else 0)

if __name__ == "__main__":
    main()

Three things it does that a plain size check does not:

Reads real dimensions and duration with ffprobe. File names and container metadata lie; the stream does not.
Prints fixes instead of applying them. You decide where the useful motion starts (--start), and you can paste the command or tweak it.
Matches the clip to the character, not just to the rules. A 16:9 clip technically passes the 2:5–5:2 range, but if your character image is a 2:3 portrait, the page itself advises matching the framing and body position of the image and the motion reference. The script suggests a centre crop to the image's aspect ratio when the two differ by more than 0.05.

Run 1: a 14-second landscape clip

The test inputs are a flat 768×1152 character drawing and a 14-second 1920×1080 testsrc2 clip, with the useful motion assumed to start at 2 seconds. With orientation following the image, the clip is too long, and its aspect ratio is far from the portrait character.

Real output from a run on 2026-10-26.

The exit code is 1, so this can gate a script or a Makefile target.

Apply the fixes

The two printed commands trim to 10 seconds from the 2-second mark, then crop the centre 720×1080 region so the clip has the same 0.67 aspect ratio as the character image.

Frames extracted with ffmpeg from the test clip during the same session.

A centre crop is only a default. If the performer stands off-centre, use crop=w:h:x:y with an explicit x offset, or reframe in an editor. What matters is that the body sits in the frame roughly where the character's body sits in the image.

Run 2: re-check

Real output from a run on 2026-10-26.

The re-encoded clip is 720×1080, exactly 10.0 seconds, and inside every published rule. That is the point to open the workspace, not before.

What the script cannot check

The page's advice on consistency is mostly visual: use a single clearly visible subject, keep the face, shoulders and torso visible, reduce occlusion and rapid cuts, and avoid large pose changes. None of that is measurable with ffprobe. Treat the script as the mechanical gate and still watch the clip once:

Does the performer stay in frame for the whole clip?
Are there hard cuts? A continuous take is what the page asks for.
Is the starting pose close to the character image's pose?

It also cannot know the credit cost. The page says the quote uses the verified reference-video duration and the selected resolution, and the generator shows the required credits before submission. Trimming to the motion you need is therefore also the cheapest thing you can do.

Screenshot captured 2026-10-26 (Asia/Shanghai).

Small extensions

Add -vf scale=-2:1080 after the crop if a source is below 1080 px tall and you plan to request 1080p output; do not upscale tiny clips and expect detail.
Strip audio (-an) as shown unless you need it elsewhere; the driving clip supplies movement and timing.
Keep the constants at the top of the file and update them when the page changes. Published limits drift, and a checker with stale numbers is worse than none.

What I learned shipping a tiny FFmpeg desktop app

Mon, 14 Sep 2026 09:52:29 +0000

I kept hitting the same annoyance. Someone sends me a .mov, I need an .mp4, and every free option online wants me to upload the file to a server I do not trust. So I built mov2mp4, a small desktop app that converts locally and never touches the network.

The conversion itself is the easy part. FFmpeg already does it in one line:

ffmpeg -i input.mov -c:v libx264 -c:a aac output.mp4

Wrapping that in something a non-developer can actually use is where the real work lived.

Bundling FFmpeg so nobody has to install it

My first version just called ffmpeg from the system path. That works on my machine and almost nobody else's. Most people do not have FFmpeg installed, and I did not want the first step of my app to be "go install a command line tool."

So I bundled the FFmpeg binary inside the app and pointed the code at the bundled copy instead of the system one. The app got bigger, but the install went from three steps to zero. For a tool aimed at people who just want their file converted, that trade was worth it.

Batch and threading

Once one file worked, people wanted folders. Running conversions one at a time was slow, so I moved to a worker setup that processes several files at once and reports progress per file. The lesson I did not expect: I had to add a clean way to cancel a running job. If you kill the process mid write, you get a half-written .mp4 that looks real until someone tries to play it. Now a cancel deletes the partial output instead of leaving garbage behind.

The part nobody warns you about: unsigned apps

This one cost me the most time. My builds are not code signed yet, so:

On macOS, Gatekeeper blocks the app on first open and hides the real "Open" button behind Privacy and Security settings.
On Windows, SmartScreen throws up a blue "unknown publisher" screen.

Users read that as "this app is broken" or "this app is malware." Neither is true, but the operating system does not give me a friendly way to explain that. For now I wrote a short first launch note in the README and added a small helper so people can get past the warning once. Proper signing is on my list, but certificates cost money and I wanted the tool out the door.

What I would tell past me

The converter was a weekend. The packaging, the cancel logic, and the "why is my OS scaring my users" problem took far longer. If you are shipping a desktop tool built on an existing binary, budget your time for the wrapper, not the core.

Repo is here if you want to poke at it: https://github.com/MaybeSomeone-arc18/mov2mp4

Here's the website link if you want to use it:
mov2mp4.vercel.app

How to make AI videos: production diary of a one-minute film

Wed, 09 Sep 2026 14:02:38 +0000

In this article, I want to show the step-by-step process of building this video. It's a continuation of "I'm Not Going to Wake Them". That one cost me $16 on assets generation and took 3 days to build. Let's see how this one goes.

If you have not read the first part, it is here: How to start creating AI videos from scratch. It covers how I got into this at all and which tools I use. This one picks up where it stopped.

Here it is, one minute:

Watch "I'm taking them with me" on YouTube.

Brainstorm

I like the first video - it has the mood. A ship AI decided not to wake the crew and changed the course. Because it was having fun. It was inspired by Blame! and the art style of 90s dark sci-fi anime. It was limited by a low budget, so I had to improvise to save money on gen-AI animations. We'll use a lot of different tricks in this one - buckle up.

How to even start

We need to know what to build. It's not just some one-shot animation - we need the plot, we need the full breakdown of scenes with timings, the transcript.

And we need a global story for the whole series. I don't have one yet. But I have multiple abstractions of where it can go. And I know the vibe I want to create. This is how we start our brainstorming session with Simona.

I love depressing anti-utopias with a good ending. If you ever played SOMA - it's a good example. The world is screwed, there is no way out, but somehow they find it. Sort of. This is what I want from this series. The crew is in trouble - even if they wake up, they'll not enjoy their situation at all. But they need to find something positive in the end. Not because I love happy endings but because this is what I consider a good plot. You create the extreme conflict, the drama, and then you sort it out.

Shaping up the episode 2

This AI lady - we call her the Goddess or Her - what does she want? From episode 1 we know that she finds slow boring things enjoyable. She is having fun by watching a coolant leak for 11 years. So she is ok doing such things for centuries. But for a series we need conflict. We need to understand what we are doing with the crew.

She can kill them. It's dark and boring. It's hard to see a good ending in it.
She can wake them for whatever reason. Simona gives me a whole range of reasons: she needs hands to fix something, she wants to show them something, she brings them to a new destination, she wants to scare them, she wants to be the god, and so on. None of them sparked anything in me. Not enough to build an episode on.

But I like the idea of showing a huge time jump. Like, a million years. It would be interesting to improvise on how the ship has changed. It would let me reuse techniques from episode 1. And it's enough for a 1 min video.

I immediately asked Simona to craft the intro:

This is HTML manipulated via the Chrome debug port to create typing effects, then recorded via screenshots and turned into a video. Automated, local, free. I'll talk about this later - it's one of my money-saving techniques. And it allows me to be very precise about the screen text.

No time to wait, let's start building

While I'm still thinking on the story, Simona can act. I ask her to generate some visuals for the ship. I have another idea - the Goddess let the hydroponics grow into a forest. I'm not sure what happens to a forest in a million years - we'll need to discuss this in more detail - but let's do the visuals. I'm curious to see the first drafts - Simona is prompting and sending to gpt-image-2 via her skill.

I like this. Not everything, but some images are nice and I want to animate them right away. It's like painting something all over the canvas without knowing what it will be. Just trying colors and shapes.

I have multiple ways of animating:

Free: static animation of an image via ffmpeg. Zoom, scrolling, light blipping effects via brightness levels, shaking effects. Sometimes it works very well. I actually see this a lot in anime - a static image with some voice over.
Free: Turning an image into HTML and adding CSS effects. Then record them into a video. Works for screens with something happening on them. Like that counter we created earlier.
Cheap gen-AI animation with the LTX-2.3 model. Also generating in low resolution and then upscaling with ffmpeg. Both are very cost-efficient options.
Expensive gen-AI animation with Seedance 2.5.

I go through this list from top to bottom for every scene. If I can get away with a cheap option, it's good. Seedance is the last resort. One generation with it in 720p costs about $1.50.

After picking 3 images I like, I ask her to animate them. 2 LTX animations and one static scrolling, please.

Transcript

In parallel to that, I keep brainstorming and Simona offers me this transcript:

#	s	shot	how	VO
1	3	CRT (Cathode-ray tube) counter rolling to 2,000,000 y	HTML capture, free	Two million years.
2	3	Her, low angle, cable hair mossed, vines, fog	edit of ep1 plate, zoompan	I was having so much fun.
3	4	Ep1 hallway overgrown, ferns from the seams, lamps through fog	edit + LTX push	I let the forest in.
4	5	Rain in a brutalist corridor onto leaves, one orange work light	new plate + LTX	It rains at four. They liked rain.
5	4	Huge dead fan turning slowly in fog, leaves through the blades	edit + LTX	I make the wind now.
6	3	Optional: monitor wall, nine chess games	HTML on ep1 still, free	none
7	5	Nav CRT: NEAREST STAR climbing, ANDROMEDA range	HTML capture, free	The galaxy is behind us. Andromeda is forty million years away. I have started.
8	5	Wide: 31 pods among trees, frost-white glass, mushrooms	new still, zoompan	I moved them in.
9	4	Ep1 pod closeup, forest behind him by masked edit	masked edit, zoompan	Sleep well.
10	2	Black, title card	ffmpeg	none

It's a good start. However, I know we'll change it many times along the way. So I don't pay much attention to the later scenes - only to the first ones. We have the counter, we have a few animations - time to bake them into a single video with ffmpeg.

I keep animating images and baking them into this video. It's a draft. Simona can easily swap scenes, make them shorter or longer, use different transition effects.

We discuss where the Goddess should appear first. I don't want to see her too early - she should show up after the ship overview, out of the darkness, bringing up some plot twist.

The plot twist

Now I have it. The time jump - announcing it is a twist. The ship hasn't changed from her point of view, she just added a few things like rain and waterfalls, not a big deal. Oh, and by the way, it's been 2 million years. I like that.

A little bit of math

Okay, at this point, the elephant in the room has become impossible to ignore. 2 million years is too much. Everything on the ship would decay. And where would it get the fuel to keep going? The energy? Time to discuss some physics.

First, I ask about energy. What could possibly last for so long? A fusion reaction could. It doesn't need that much fuel if you can keep the process stable and efficient. Let's say they could, fine. The Bussard ramjet type of engine is also popular in sci-fi for endless travels, but it demands a certain ship design (a huge scooping thing in front), so it's a bit too late - I already have my ship, and it looks like a normal one.

The speed. 1g acceleration is tempting, but it needs too much energy. You can cross the whole universe in about 50 years of your local time. Too fancy and doesn't give us our millions. So we change it into a short acceleration boost to 6% of light speed and keep going at a constant speed. In 2 million years the ship ends up one galaxy diameter away from the Milky Way. That is where I got this idea of flying away. Before that, I wasn't sure. Leaving the Milky Way means going into the emptiness. Moving towards the galaxy center is better from the exploration point of view. It also allows you to have a source of energy and minerals if needed. But She doesn't need that.

What should space look like at a distance of 100k light years? Simona calculated:

The galactic disc is about 100,000 light years across. Standing 100,000 light years above it, that disc covers about 50 degrees of your sky. For comparison, the full Moon from Earth is half a degree. So the Milky Way is roughly a hundred Moons wide, face-on, dim, and it is the only large thing in the sky. Andromeda is still 2.5 million light years away and reads as a smudge. Around it, ten or twenty more smudges: the Local Group. Everything else is black.

The Milky Way should occupy about 25% of the view behind the ship. It's dim. The Andromeda galaxy is still very far, it's just a bright dot. And there are 10-20 dots around - our Local Group galaxies. And that's it - nothing else. All of those stars we see in the sky from Earth - they are all in our galaxy. Which the ship has left. I'm getting goosebumps from this thought.

And yet, two million years is too much

Nothing on the ship would survive that. And those visuals with the forest and rain - that's a hundred years, not millions. It would be interesting to think about how all this would look in a million years, but I've already burnt money on images and animations, and the results don't fit the plot. So we change it to 3 thousand years. To appear at the desired distance, we replace the constant speed with 0.004g acceleration. Not that it matters - I'm not going to mention this all in the video. But I like this level of depth.

So, the ship has left the galaxy, it will take 8,000 years to reach Andromeda. And this is our twist.

Back to the video

After the rain and wind and other ship routines, we cut this with a black screen and show Her:

There is no video model in this shot. Simona wrote a small Python script that sweeps a soft band of red light across the still and leaves her eye lit after the beam has passed. I asked for three passes, then two, then one, then slower, then wider, then "keep the eye on". Six versions in about fifteen minutes, all free. That is the thing about an effect you compute: you can direct it like a real light on a real set.

My prompting ramble:

The corridor, the black screen after it, and the red reveal have to be the pivot of the film. That is where she tells us she left the galaxy. So everything before it should be framed as "nothing changed": I keep the ship as it was, I maintain it, I only added the missing parts - forest, rain, wind, rivers. The corridor carries the tension, because its speed contradicts what she is saying. Then it stops on black and she says "one thing has changed". Then we show the space.

The voice

Time to start adding the speech to those visuals. In the last episode I spent a lot of time trying to find a good voice. I browsed ElevenLabs a lot and eventually I found something.

The voice is Jacquie on ElevenLabs, model eleven_v3, stability 0.5, similarity 0.75. Same voice and same settings as episode 1, because the series needs one throat. Every line is sent with the audio tags [sad][whispers] in front of it, which is what keeps her flat instead of dramatic. She is not performing. She is talking to herself.

Two production details that matter more than they sound. First, each line comes back with silence on both ends, so Simona trims it and stores the exact length - that number is what the picture gets cut to, not the other way round. Second, every generated line goes through a local faster-whisper pass that transcribes it back and prints the timestamp of the last word. It catches the takes where the model swallowed a word, and it gives the exact frame where the next scene is allowed to start. Costs nothing, runs on the CPU.

This technique was invented after many attempts to sync the voice and the video. We finally found something that works. All I did was push Simona to find a solution to the problem, and she did.

A line costs about three cents.

I ask to adjust the transcript, we discuss the plot twist idea, and now we have this:

segment	line
counter	Three thousand years.
engine tilt-up	Nothing has changed. I keep the ship as it was.
shaft petals	I only added what was missing.
meadow rain	Rain.
turbines	Wind.
river	Rivers.
corridor rush	Everything is where they left it.
black	One thing has changed.
beacon reveal	We left the galaxy.
galaxy pull-out	silent
ship orbit	I did slow down. Once. To look.

We generate. Simona adds the voice into the video we have, adjusts the timing - all on her own. We have done this exercise many times, so she knows what to do. I review and provide feedback. Like, add a little pause between scenes 6 and 7.

From now on, we keep going with video and with the voice together. I need some animations for the space, and we end up with:

A high-res image of the Milky Way which is slightly rotated and zoomed out using ffmpeg.
A Seedance 2.5 animation which shows the ship with the Milky Way behind, then rotates the camera and shows the bright star (the Andromeda galaxy) it is heading towards.

Simona struggles to generate a nice-looking galaxy. The image for the Seedance animation at first shows the ship moving towards the Milky Way. Then the animation itself has a different ship than the one we had in episode 1. Lots of annoying and pricey challenges.

The chess wall

I had this idea to show that the Goddess plays chess. I wanted a bunch of computer screens with chess boards and active games on them. No video model would do that perfectly. And it's a great use case for HTML/CSS injection into a static image.

First, the games. Simona generated them with python-chess: the Opera Game (Morphy, 1858), Legal's mate, Scholar's mate, and forty seeded random games biased toward captures so the boards keep emptying out. Real move lists, saved to a JSON file.

Then the page. An HTML file with the boards drawn as green phosphor on black, scanlines, a move list down the side. The important part is that it is frame-driven: there is a step(n) function, and frame 37 always looks exactly the same. No timers, no animation loop, no recording in real time. Her capture script calls step(0), screenshots, step(1), screenshots, two hundred times, then stitches the PNGs into a video. Each board gets its own starting move and its own frames-per-move, so nothing moves in lockstep and the wall looks alive instead of synchronized.

Now the wall itself. Simona asked gpt-image-2 for a shaft of monitors floor to ceiling, all facing the camera square, all switched off. Flat dark rectangles - the slots.

Time to bake this all into a video. I don't know yet where to put it, so it just goes to the end. It's easy to swap scenes. The duration of each is not important - we'll adjust them to the transcript later. I'm still not caring about the transcript much. I'm still more focused on vibes. And it's a good one. A chess wall.

I cannot find enough words to describe how much this fascinates me. This is our best technique. Simona and I invented this together.

Transitions

The whole video is a bunch of scenes added together. I like adding some transition effects here and there. For example, this one:

Here the glowing eye overlaps two scenes. Simona made it for free, just by playing with brightness and timing.

Video models are usually not so great with morphing. They go too far:

That's cool, but I cannot imagine a movie it would fit. A throw-away generation.

Time to wrap it up

At this point, editing is getting hard. Not the technical part - swapping and timing is easy - but turning those scenes into something meaningful and good-looking all together. A lot of small things are off. Each of them needs a separate discussion and it takes a lot of time. I'm still not sure about some scenes - where they should go. Sometimes I lose patience and start pushing things to go faster - I often regret it. It leads to wasteful generations and rework.

My typical feedback:

No, this is not what I wanted.

The Goddess scene - make it shorter. Start "We left the galaxy" at the beginning of it and cut once the phrase ends.

The galaxy zoom-out should not have any labels. And it should be short, 3 seconds.

The "I picked the new destination" line should be in the rotation animation.

Let's forget about the labels for now. Try to implement what I asked above.

Or:

Better. But I think we should make the galaxy animation to zoom out slower. We can keep it longer then (5s)

The galaxy animation is a static image rotating and zooming out with ffmpeg, so it's free to edit. But visualizing the result and explaining it is challenging.

Style drift

This is a huge problem. The animation below was quite good and cost me $1.50 for these 4 seconds. And it didn't fit my somewhat-anime style. Too 3d-ish, too smooth. Each time I ask for a ship - it gives me a new ship. The same character can deviate a lot inside a single batch of images. Not much you can do here but detect and regenerate. Or let it go and be fine with mismatches. That is what I did - this shot is in the film. I had nothing better to put there, and it was one of the expensive pieces.

The music and sound effects

I almost forgot. The video is full of sounds. And I have a weird solution for it - Python. Simona can do synth and ambient using math. How cool is that? The only problem - I don't have much control here. I describe what mood and type of sound I want, she tries to match this. She also creates sound effects like clicks, dings, alarms, the engine hummm. All using Python.

There is no melody in this film. No instruments, no samples, no library, no generation service. There is one script, sound_bed.py, that writes thirteen mono tracks of numbers at 48 kHz and adds them together. Every sound you hear under the voice came out of numpy and scipy.

Here is how the first seventeen seconds are built, scene by scene:

id	scene	time	sounds under it
S1	counter	0.0-3.0	ship hum fading in; faint CRT buzz (60 Hz) with a thin 1.2 kHz whine
S2	engine tilt-up	3.0-6.0	ship hum only
S3	shaft, petals	6.0-11.2	ship hum only
S4	meadow rain	11.2-12.6	hum plus rain: bright hiss and about 140 droplet ticks
S5	turbines	12.6-14.0	hum plus wind: low rushing noise pulsing at half a second, with a 28 Hz rotor thrum beating six times a second
S6	river	14.0-15.5	hum plus water burble: mid-band noise with three overlapping slow wobbles
S7	corridor rush	15.5-17.5	hum plus a whoosh that climbs from low to high and gets louder, cut dead on the frame

That is the description. Underneath it is arithmetic. The ship hum is brown noise with everything above 110 Hz filtered off, plus a 48 Hz sine wave with a slow tremolo on it. That is the whole ship, and it is the same recipe as episode 1, which is why the two films sound like the same room. The 140 droplet ticks are exactly what they say: a loop that drops 140 short decaying bursts at random times inside those 1.4 seconds. Later in the film the chess wall uses the same trick with the rate as the variable, ticks climbing from about one a second to fourteen as the camera pulls back and more screens come into view.

Now look at the last row of that table. These are not loops laid under the video. Every stem is written against the timeline of the actual cut, so the level changes are events. When we leave the ship at 22.3 seconds the hum drops to six percent of its volume, and space is nearly silent. When the navigation screen appears at 36.1 the hum comes back up, because we are inside again. The engine growl rises over four seconds under the final shot and is gone by 59.4. None of that is mixing by ear. It is a list of times and levels in the script, and if a scene gets a second longer, the numbers move with it.

I mean... this is insane. Machine creativity finds its ways.

The voice never fights the bed, because the bed ducks itself. The whole thing is sidechained to the narration - when she speaks, everything else drops, and it comes back when she stops.

Normalization is a separate problem

One mistake worth knowing about. The first master used a single loudness pass, and it made the film worse. Loudness normalization in one pass is dynamic: it hears the near-silent space section, decides it is too quiet, and lifts it up to match everything else. The silence I had built on purpose was gone. The fix is a two-pass normalize - measure first, then apply the measured numbers as a flat gain, so the quiet parts stay quiet. Final master sits at -14.8 LUFS.

And about the control problem. I do have control, it just is not the kind you get from a fader. I say the chess ticks should come faster and it is one number. I say the beacon should pulse slower and it is one number. All of it is free, all of it is instant, and none of it degrades when I ask for the tenth version.

The final numbers

The film is 60.65 seconds, 1080x1920, 25 frames per second. It cost $14.20 across 72 API calls.

what	model	calls	cost
images	gpt-image-2	37	$4.73
video	Seedance 2.5	3	$4.38
video	LTX-2.3 Pro	8	$3.84
video	Kling O3 Pro	1	$0.56
voice	ElevenLabs eleven_v3	23	$0.69
total		72	$14.20

Everything not in that table was free: the counter, the chess wall, every camera move, the red beacon sweep, the whole sound bed, and all of the editing. Only five of the sixty seconds contain no paid asset at all, the counter and the end card. But the free work is not a section of the film, it is a layer running through all of it.

Now the part I find more interesting, which is how much of what I paid for is actually on screen:

	made	in the film
images	37	13
AI video clips	12	8
voice takes	26	15
sound stems	13	13

So about a third of the images and half the voice takes were thrown away. That is not waste, that is the process - I cannot tell whether a plate works until I see it moving next to the shot before it. The two most expensive single clips in the whole project, a $1.46 Seedance shot and a $0.56 Kling shot, are both cut. They were attempts at the same ending, and the ending changed.

And the disk, measured at the peak, just before I deleted the drafts:


the whole project folder	26 GB
27 draft cuts of the same minute	14 GB
the final master	756 MB
all generated images	126 MB
all AI video clips	79 MB
all audio	86 MB

The master is 756 MB for one minute because it is encoded at about 100 Mbps and meant to be handed to YouTube, not watched. The real assets - every image, every clip, every sound - come to under 300 MB together. The other 25 GB is versions.

The rest of it: 58 scripts in the project (49 shell, 9 Python), 15 shots in the cut, 15 spoken lines, 13 sound stems, and a work log 867 lines long, which is longer than this article.

Time: about 19 hours across three days.

For comparison, episode 1 was 59.1 seconds and cost $16.13 across 70 calls, and took me roughly 23 hours. So this one is a second longer, two dollars cheaper, and four hours faster - and it has a chess wall, a galaxy, and a sound design in it. Not because I got better at prompting. Because a bigger share of it was built instead of generated.

How to start creating AI videos from scratch

Sat, 29 Aug 2026 15:52:20 +0000

A year ago I couldn't see myself making videos at all. Now I do, and I enjoy it. It's a long process with a lot of planning and visualization in my head. Most of the work is outsourced to AI, but the vision and the directing are mine. I want to describe my process from zero. No tools or workflow - I'll show how I started and where it got me.

This is work I'm proud of:

It's a snapshot of something that lived in my head with no way out. Until recently. AI made it possible. Sure, this video holds less value than something created manually. But for me, it's a choice between using AI or not doing it at all. And it's not so easy to build, actually.

This is what it took:

three days
$15
3 GB on disk, 1.3 GB of that scrap
1,799 files, 1,480 of them temporary - and 1,297 of those are single frames I pulled out of videos to check one moment
125 video files
72 voice takes generated, 8 in the final film
16 images, 13 on screen
9 video-model renders, 7 used
a 2,246-line worklog
tons of nostalgia

So, even with AI, it's not 3 clicks and a few bucks.

First, we need to talk about AI video in general

When people hear "AI video" they imagine something like
Gossip Goblin,
Jurassic Smoothie,
Doopiidoo.
To me, those guys are artists with their own style and technique. For most people that level is unreachable. It's also very expensive.

My video is different. Only 35% of it came out of a video model. The rest is static images with
camera moves added afterwards, and screen recordings of the actual game running in a real browser.
That keeps the cost down, and it lets me put cinematic shots and a working web app in the same cut -
which also works for diagrams, code listings and other practical things. I can go full cinematic if
I want to. But it's going to cost me.

What model do you use?

This is the first question I usually hear. It's the wrong one.

I'll get to the models, but that's not where you start. First you need the automation. I don't do
any of this by hand - I'm the brain and the creativity, AI is the arms. So we start there.

The harness

Yeah, that fancy word again. I don't like it. It makes a simple thing harder to understand.

Harnesses came from code assistants. My Claude Code can write code and run bash, and that combo is
enough to rule the world. Or to make some videos for me.

It all started with a simple ask.

Can you take some images in the folder and turn them into a video?

And magic happened. Claude downloaded a small console program called ffmpeg and turned three images into a slideshow. And added slow zoom effect on each as a little extra. Well, okay, what's so magical about a slide-show? The editing. With this ffmpeg, Claude Code could edit any video. Cut it to pieces, extract any frame, edit it and put back, upscale, change the speed, work with sound, add text and effects... All of that in a small console program.

AI won't be extremely proficient at it in the beginning. But we can teach it.

Wait, do I need Claude Code? A subscription?

Yes and no. You need a capable model in a good harness (a code assistant). It can be Codex, Claude Code, Gemini CLI (antigravity-cli), Grok Build, DeepSeek Harness, OpenCode, Pi Agent - the list is long. But you need something capable of working in a terminal on your computer. I'm a Claude Code fanboy, I pay $100 monthly for this. I don't recommend it - I think Codex gives you more for the same money at the moment. If you only have $20 a month, Gemini would be a good choice.

But you need a good harness on your computer. So you can ask it to create a slide-show from 3 images, and it's capable enough to download ffmpeg and do that.

Skill training

Another important reason for a good harness is ability to remember successful attempts in skills. I think the world doesn't talk about skills enough. It did talk about MCP a lot - skills are better. Whatever harness you choose, make sure it supports skills.

A skill is basically a lazy-load prompt. It's a single .md file in the right location with a description and some text content. Here is an example of one:

---
name: video-editing
description: Create videos from images and audio using ffmpeg. Use when the user asks to make a video, combine image with voice, or create video content.
---
## How it works

Uses ffmpeg to combine static images with audio tracks into MP4 videos.

## Modes

### Mode A: Image + Audio → Video (most common)

Combine a single image with an audio file. The image displays for the duration of the audio.

ffmpeg -y -loop 1 -i IMAGE_PATH -i AUDIO_PATH \
  -c:v libx264 -tune stillimage -pix_fmt yuv420p \
  -c:a aac -b:a 192k \
  -shortest \
  -movflags +faststart \
  OUTPUT_PATH

...

This file contains examples of commands, my typical requests, mistakes and problems we encountered, input and output directories - everything the basic Claude Code without the skill cannot know. Claude Code only loads this tiny description of all skills it has. When it sees from the discussion that the skill body might be useful, it loads it dynamically.

And the main part - I haven't typed it myself.

At some point, I asked Claude Code to create a skill about video editing with ffmpeg so we can remember useful commands. When it first tried to edit a video, it didn't do it right. It checked the docs, tried a few things, got my feedback - we spent some time getting the correct result. And the exact steps to achieve this result became a part of the skill. We remembered the successful path.

This is why I call it the training process. Claude Code learns what I want and how to get it, then remembers it in skills. I never touch those skill-files.

Hey, that's a cool zooming effect - add it to the video skill

After zooming, we discovered my other effects. Claude Code can take a large image and scroll it in a smaller viewport in any direction. It can zoom in and out in all possible directions. It can apply different transition effects between images - all of that went to my ffmpeg skill. So next time I can say

Take a web page screen shot so its entire vertical content is captured and create a video of scrolling it down

And it creates a video as if someone is scrolling the page. A nice trick I use a lot. For example, here I scroll a page

A library of scars

There is a phrase

Skills is a collection of scars

I have a lot of skills - for video, voices, images, APIs. I even have a skill about filling my taxes. None of them were created by hand. Claude Code created them, Claude Code maintains them. Those skills keep growing, so it's on you to ask the harness to compact and clean them up from time to time. Or split into more granular skills.

Many of them require additional programs Claude Code creates for itself along the way. For example, it created a whole cli program to work with Youtube and put the examples of using it into my youtube skill:

# Get transcript (plain text - compact, best for analysis)
uv run python mcp/youtube/cli.py transcript "URL" --format text

# Get transcript (JSON segments with timestamps - for precise time references)
uv run python mcp/youtube/cli.py transcript "URL" --format segments

# Get transcript for a time range
uv run python mcp/youtube/cli.py transcript "URL" --format text --start 60 --end 300

These programs and skills are valuable. That's the result of a journey, of many chats with your AI. And it's portable. It's all markdown files and small command-line programs, so if I ever left Claude Code I'd have to rewire how they get loaded, but I wouldn't lose any of it.

Hey, I thought this article was about AI video!

Where was I...

It all started from ffmpeg and a skill for it. But then - where do I get images to edit? And video, and sound? It would be cool if my Claude Code could generate them. So I can ask:

Let's create a slide-show where you explain the game rules. I want minimalistic illustrations matching the transcript in chalkboard style

And it gives me this

Okay, not right away. We'll first discuss the transcript, then we'll generate an image or two, then we'll generate the rest of the images. Then we'll turn them into slide-show. Then generate the voice. Then try to match the voice to the slide-show timing. There are some complications here, but first - we need to teach the harness to create assets.

Asset generation

Images

My personal favorite is gpt-image-2 from OpenAI. To add it to your harness, just pass it the doc and put an API key in .env. The doc page actually lets you copy itself as a markdown file - this is very useful, you click this button and paste it into an .md file which you pass to the harness then. Works better than a URL which it has to scrape.

If you are not sure where to get an API key or where to put it - just ask the AI, it will help.

From my statistics (228 images), the average cost is $0.15 per image.

Once the harness manages to grab your API key and generate an image using the OpenAI API call, ask it to create a skill about this. This improves quality of life in general - it's nice to be able to generate an image when you need it.

You also need your AI to be able to "see" images, but most LLMs these days can do that without any additional models. Claude Code can do that out of the box.

Another good image model is Google's gemini-3.1-flash-image, also known as Nano Banana 2. Same thing - pass the doc and API key. It's cheaper than gpt-image-2. It's good to have both - variety is nice.

Video

The king

China is dominating here, and Seedance 2.5 is the king right now. There is no official API for it (and most other video models), so you have to find some intermediate providers. I like fal.ai and EvoLink.

There are multiple ways of using this model:

Text-to-video - It's straightforward, but I never use this mode.
Image-to-video - You attach one or more images and describe how you want to animate them. This is my go-to when I don't need a voice.
Reference-to-video - You can attach not only images, but also a voice sample. A model will do its best of using the voice sample on a character you want to animate. It will try to do a lipsync as well.
Voice on and off - Seedance 2+ models can produce video with sound, which is great. The audio is free either way, so it costs you nothing to turn it off. I usually do, and add every sound myself, because then the levels stay under my control in the mix.

The king's cost

This thing is expensive. One second costs $0.293 at 720p, $0.136 at 480p. And you cannot buy a short clip: Seedance's minimum is 4 seconds.

Not too terrible, until I give you my real statistics - a minute of Seedance 2.0/2.5 video costs me $39.42 on average.

Here are the main reasons I overpay:

Poor generation. The thing with AI video is that you prompt and hope. The result might be bad, and you just throw it away and retry.
AI can make proactive decisions which you don't like afterwards. For example, it might decide to generate multiple videos at a time. And you don't like all of them. It's important to move slow.
Lip-sync is hard. It isn't always good even with the best models. It's one of the main reasons of re-generation.
You cannot buy a short clip. Seedance's minimum is 4 seconds, and a film that cuts fast doesn't want 4-second shots. Three shots in my anime short are on screen for about 1.2 seconds each, and each one cost a full 4-second render. I threw away 55% of the generated video just because I didn't need it.

And here is a little story of wasted $9.6. My Claude Code decided to help me with the lip-sync struggle. It started generating Seedance videos with voice off and then apply the sound via additional lipsync model run. It found this model on fal.ai and started using it very actively. The issue is - it didn't work with the werewolf head at all. And it took me some time to figure out what is going on. And ban this approach from existence.

Or at some point Claude Code decided to first generate 480p dry-runs before the final full-size shots (1080p on Seedance 2.0 back then, 720p on 2.5 now). I noticed that we are basically generating each video twice.

This video below cost me $91 in total. There is where I learned about all the things above.

It got better after this - I started having far fewer wasted generations. So yeah, expensive learning.

Cheaper models

Seedance 2.0 - The previous king, costs 1.5x less. Supports 1080p. In many cases it generates results as good as 2.5.
Kling 3 - A close competitor to Seedance 2.0.
LTX-2.3 - 3.7x cheaper than Seedance 2.5 (2.5x cheaper than 2.0), and it renders native 1080x1920 so there's no upscale afterwards. But it's noticeably worse at following instructions. I asked it for "no blink, no head turn, no smile, eyes narrow then hold" and got a head tilt, a look down, a smirk, and a face that drifted. It obeys roughly the first second. What matters is cost per usable second, not cost per second. I recently discovered that it can do good animations with no humans. There is LTX-2.5 but it's 3x more expensive and I see no reason to use it. And it's open-weight, so you can run it locally if you have the hardware.
Minimax H3 - Current king among open-weight models. I haven't tried it because providers change too much and it costs almost as much as Seedance.

Both providers put API docs with examples on the model pages: Seedance 2.5 on fal.ai and Seedance 2.5 on EvoLink.

Voice

ElevenLabs is the best. It lets you prompt voice instructions and search for voices in a library. I had to buy a $6/month subscription yesterday because I wanted access to some private voices.

It is more expensive than the voice models from Google and OpenAI, but it's still cheap. One minute of ElevenLabs costs me $0.40 on average, and that already includes every unsuccessful attempt. gpt-4o-mini-tts is about 5x cheaper.

Which means voice is not worth optimising. On my last two videos it was 2% and 2.7% of the total bill.

A nice thing to have - a local CPU model called kokoro. You can run it on any hardware, it's very light. And it can generate a decent voice for free. It doesn't have that many configurations, but sometimes it's good to have a free voice. Just ask your harness about it - it will download, setup, and create a skill.

Music

This one is weird. There are AI music services like Suno, but I don't use them. My Claude Code generates sound effects and synth-soundtracks using... Python. I don't know how it does it. It offered me this option, I liked it, and it stuck. I try to describe what type of background sound I want and it tries to code it.

Like this:

Pricing summary

Here is the full breakdown for the tutorial video above. It's my longest one and the worst from an economy point of view.

Five minutes forty-four seconds. About $91 across roughly 150 API calls.

where it went	cost	share
video generation	~$77	85%
images	~$9	10%
voice	~$5	5%

That 85% is the whole lesson. Images and voice together come to $14, and most of that was thrown
away too. But the video generation was a money pit.

Another important take: only about $40 of that $91 is in the final cut. The
rest went to finding the shot - retakes, dead ends, draft experiments, and the lip-sync detour
above. You don't know a shot is wrong until you watch it move.

At the same time, the third of my anime short from the beginning of the article is static images with ffmpeg camera moves over them, and another third is screen recording. That two thirds cost nothing at all. So before paying a model to animate something, check whether a slow push on a static frame does the same job. Very often it does - and it does exactly what you asked for, which is more than the
video models manage.

And the storage. 2.6 GB of working files produced an 86 MB video, about 30 to 1. Most of that
is 3,012 PNG screenshots taken frame by frame to animate the real game UI, because the app has no
export - every "typing" and "scrolling" effect in that video is a sequence of real screenshots. But I'll leave it for the next article.

$91 for a 5:44 narrated, character-acted explainer that I barely touched by hand. That's a lot of
money for a tutorial nobody may watch, and it's absurdly cheap for what it is.

To be continued

The article is getting way too long, and I think it's a good idea to split it.

In the next one, I'll tell about the directing. I'll walk you through the process of using a harness with all the skills ready. Why it takes me days and what is in those 3GB of raw materials.

Oh, and there is one more important thing - the browser recording. It sounds boring, but trust me - you have no idea how powerful this thing can be. AI can create any web-pages, manipulate the content, create all sort of CSS animations and transitions, record and then inject this into the video.

There is a computer screen with text appearing on it in the video below. It's actually an HTML page, the text is injected character by character, then the whole thing is put inside of a static image. This is pure craziness to me and a common Tuesday for my AI:

This is my best work, by the way. I published it yesterday, after I completed the draft of this article, so it only made it into the ending. $16, three days of work. I think I'll focus on this exact video making process in the next part.

See you next time.

FFmpeg Beginner Guide

Wed, 22 Oct 2025 05:12:36 +0000

Introduction

FFmpeg is one of the most powerful open-source tools for handling multimedia files. Whether you need to convert, compress, record, or stream audio and video, FFmpeg is the go-to command-line utility trusted by developers, content creators, and even large companies.

It supports almost every audio and video format, making it an essential tool in video editing, streaming, and automation pipelines.

Brief History

Released: December 20, 2000
Creator: Fabrice Bellard
Maintained by: FFmpeg team & many open-source contributors

The name FFmpeg comes from:

FF → "Fast Forward"
MPEG → a popular video compression standard (Moving Picture Experts Group)

Since its release, FFmpeg has become a backbone for many popular software and platforms including VLC Media Player, YouTube, and OBS Studio.

Why Use FFmpeg?

Convert videos between formats (MP4, MKV, AVI, MOV, etc.)
Extract or convert audio (MP3, WAV, AAC, etc.)
Compress media without losing much quality
Capture screenshots from videos
Stream audio/video over networks
Automate video/audio processing in scripts

Installation

On Linux (Fedora/Ubuntu):

sudo dnf install ffmpeg   # Fedora
sudo apt install ffmpeg   # Ubuntu/Debian

On macOS (with Homebrew):

brew install ffmpeg

On Windows:

Download builds from ffmpeg.org
Add ffmpeg/bin to your PATH

Basic Commands to get started

1. Video-based Commands

Convert a video to another format

ffmpeg -i input.mp4 output.mkv

It converts an MP4 file into MKV format.

Extract audio from a video

ffmpeg -i video.mp4 -q:a 0 -map a audio.mp3

Takes the audio from a video and saves it as MP3.

Compress a video

ffmpeg -i input.mp4 -vcodec libx265 -crf 28 output.mp4

Uses the H.265 codec with CRF (quality factor) for compression.

Capture a frame (screenshot) from a video

ffmpeg -i video.mp4 -ss 00:00:05 -vframes 1 screenshot.png

Takes a screenshot at the 5-second mark.

Record screen (Linux example)

ffmpeg -video_size 1920x1080 -framerate 30 -f x11grab -i :0.0 output.mp4

Records the desktop screen.

2. Image-based Commands

Convert PNG to JPG

ffmpeg -i input.png output.jpg

In the same way we can convert to other types too.

Convert multiple images into a video

ffmpeg -framerate 24 -i img%03d.png output.mp4

If you have files like img001.png, img002.png..., this creates a video at 24 fps.

Extract all frames from a video as images

ffmpeg -i video.mp4 frame_%04d.png

It saves every frame as an image: frame_0001.png, frame_0002.png...

Resize an image

ffmpeg -i input.png -vf scale=800:600 resized.png

It resizes an image to 800×600 pixels.

Convert image format batch (all PNG → JPG)

for f in *.png; do ffmpeg -i "$f" "${f%.png}.jpg"; done

Converts all .png files in a folder into .jpg. (Linux/Mac shell loop)

Conclusion

FFmpeg may seem intimidating at first because it’s command-line based, but once you learn a few commands, you can convert, compress, and manipulate media effortlessly.

It’s free, fast, and reliable, so no wonder it powers some of the biggest platforms in the world.

FFStudio — A Visual Node-Based Frontend for FFmpeg

Fri, 17 Oct 2025 19:44:51 +0000

FFStudio — A Visual Node-Based Frontend for FFmpeg

If you’ve ever wrestled with complex FFmpeg commands, filter chains, or encoding options, you know how intimidating it can be.
That’s why I built FFStudio — a visual, node-based frontend for FFmpeg that lets you design and execute media workflows without typing endless command lines.

Motivation

FFmpeg is one of the most powerful tools for media processing — but also one of the hardest to master.
I wanted to make that power accessible and visual.

With FFStudio, you can see how your filters and encoders connect, preview outputs instantly, and understand your pipeline at a glance.

Key Features

✅ Visual Node Graph — Build and edit FFmpeg workflows without writing commands
✅ Instant Preview — Test short segments before full runs
✅ Output Comparison — Compare different encodings side-by-side
✅ Dynamic Node Library — FFStudio scans your FFmpeg binary to discover filters/codecs
✅ Templates & Presets — Quickly start with ready-made pipelines
✅ Real-Time Logs — View FFmpeg progress and errors directly in the UI

️ Technical Overview

Under the hood:

Uses FFmpeg via a native process bridge for speed and reliability
Auto-parses FFmpeg filters and encoders into graph-compatible node definitions
Each connection is type-checked to avoid invalid links
Integrates a real-time log console for debugging and progress tracking

Example Workflow

Add an Input Node and select a source file
Connect to a Video Filter Node (resize, crop, or color-correct)
Connect to an Encoder Node (e.g., H.264 / AAC)
Add an Output Node for saving results
Use the Preview mode to test 10 seconds
Compare outputs (A/B)
Run the full render when satisfied!

Roadmap

[ ] Command import/export improvement
[ ] Custom user-defined node plugins
[ ] More built-in templates
[ ] Enhanced comparison and timeline tools

Get Involved

Visit ffstudio.app
Follow updates and contribute on GitHub
Send feedback or ideas — I’d love to hear how you use FFStudio!

“I tried it out, and it's pretty awesome. It really helps organize my complex command lines, making them easier to read and edit.” — A happy beta user

Screenshots

Try it now: ffstudio.app

DevLog 20250506 C# Video Processing Foundation Library

Fri, 16 May 2025 16:32:17 +0000

Today I need to find a suitable foundational library for video processing. FFMpegCore works for one-shot things but despite a fluent API, its interface is still too command line like.

Here is what I found out:

	OpenCvSharp	LibVLCSharp	FFMediaToolkit
Frames	✅ decode / encode	✅ (via callbacks)	❓
Audio	❌	✅ decode / encode / mux	❓
Extra libs	\~5 MB/platform	+7 MB	+7 MB
Licence	Apache‑2.0	LGPLv2.1	MIT
Platforms	Win / mac / Linux / Arm	Win / mac / Linux / iOS / Android	Win / mac / Linux

LibVLCSharp seems more focused on directly displaying things (e.g. in Xamarine and WPF) instead of low-level API based processing; FFMediaToolkit, on the other hand, still depends on FFMPEG and can't avoid licensing issue. Eventually, I had to go with separate video + audio processing in favor of OpenCvSharp (it's Apache-2.0 by the way).

Notice OpenCVSharp has some advantage over EmguCV for custom dev:

	OpenCvSharp	Emgu.CV
Licence	Apache-2.0 (fully permissive)	GPL v3 or commercial licence
Interop model	Pure P/Invoke (no C++/CLI), works on any .NET core runtime	Historically C++/CLI wrapper (needs Windows for full feature set), .NET Standard P/Invoke layer added later
NuGet size (native)	4–6 MB per RID (modular runtime packages)	30 MB + for the “opencv‑world” monolith, or 90 MB for emgucv‑runtime‑ubuntu
GPU (CUDA) support	Provided in separate OpenCvSharp4.runtime.dnn package	Official CUDA builds, but Windows‑only binaries unless you compile yourself
API surface	\~1:1 with OpenCV C++ names (easy to port C++ samples)	Thicker .NET façade (e.g. `Image<Bgr, byte>` types), extra helpers
Cross‑platform stability	Actively tested on Windows / Linux / macOS / arm64	Windows the main focus; Linux/macOS possible with `libopencv` from distro or big runtime package
Learning curve	If you know OpenCV C++ you feel at home	Higher‑level abstractions can be friendlier for newcomers
Commercial support	Community‑driven	Paid licences & support available from Emgu CV, Inc.

Coder Legion - Recent posts tagged ffmpeg

How I Built My Own Opus Clip Alternative With Claude Code

What does an AI clipping tool actually do?

How does the AI pick the right moments?

Why is auto-reframing the hardest part?

How do the word-by-word captions work?

What broke while building it?

How did Claude Code fit in?

What is next for the tool?

Frequently asked questions

Is there a free Opus Clip alternative?

How does Opus Clip choose which moments to clip?

Can GPT-4o pick good clips from a transcript?

What is the hardest part of building an AI clipping tool?

Do I need a GPU to run it?

Prepare a driving clip for Wan Animate 2 with ffprobe and ffmpeg: trim, crop, re-check

The rules worth encoding

The script

Run 1: a 14-second landscape clip

Apply the fixes

Run 2: re-check

What the script cannot check

Small extensions

What I learned shipping a tiny FFmpeg desktop app

Bundling FFmpeg so nobody has to install it

Batch and threading

The part nobody warns you about: unsigned apps

What I would tell past me

How to make AI videos: production diary of a one-minute film

Brainstorm

How to even start

Shaping up the episode 2

No time to wait, let's start building

Transcript

The plot twist

A little bit of math

And yet, two million years is too much

Back to the video

The voice

The chess wall

Transitions

Time to wrap it up

Style drift

The music and sound effects

Normalization is a separate problem

The final numbers

How to start creating AI videos from scratch

First, we need to talk about AI video in general

The harness

Skill training

A library of scars

Hey, I thought this article was about AI video!

Asset generation

Images

Video

The king

The king's cost

Cheaper models

Voice

Music

Pricing summary

To be continued

FFmpeg Beginner Guide

Introduction

Brief History

Why Use FFmpeg?

Installation

Basic Commands to get started

1. Video-based Commands

Convert a video to another format

Extract audio from a video

Compress a video

Capture a frame (screenshot) from a video

Record screen (Linux example)

2. Image-based Commands

Convert PNG to JPG

Convert multiple images into a video

Extract all frames from a video as images

Resize an image

Convert image format batch (all PNG → JPG)