I recently let go of my video editor.
Oof. Don’t feel great about it.
He cost around $1,000 a month. The last few videos on my YouTube channel have been edited by AI instead. One went to about 5,000 views, which is solid for my relatively small channel.
The work also gets done FAST (~1 hour) and costs basically nothing as I do it on my ChatGPT subscription
Three months ago I could not get this to work dependably. I had been trying for more than a year and it was a mess. Cut-off sentences. Audio out of sync. Weird joins. A lot of time spent producing something a human then had to rescue.
I gave it up as a bad job because I knew if I just waited a bit the AI would get better. And it did!
Now Claude Code/Codex can take my messy livestream, cut it into a proper video, check the render, create captions, produce three YouTube packages and hand me a private draft.
That is a BIG change.
But the viral demos you’ll see on Instagram are still talking shit.
You cannot chuck an hour-long video into Claude Code, type "make this cool and help me go viral" and wander off for lunch.
Well…you can. The result will probably be a bit crap. And it’ll have eaten all your tokens for its lunch.
Nah we need to do some proper work and planning. Sorry! The thing that works is a pipeline. Each tool gets one job and the system stops at the points where I still need to make a call.
My basic pipeline
First up (unsurprisingly) I shoot my video. Personally I do this via my livestream. You could just record offline. Same thing.
My raw video files stay on my Mac. Whisper turns the audio into a timestamped transcript. This is all local (no AI needed). Free.
A smart hosted model (Codex for me) reads the text and makes the editorial decisions. This part uses AI and my tokens.
FFmpeg cuts and renders the video locally. Again, no AI. Free.
Then the system checks the output, packages it and stops before publication. This part I bump back to Codex so it costs me tokens.
See the back and forth? We what we can locally and for free. Then bump the parts that genuinely needs intelligence to a model. Technically as/when local models get better we could also do those parts on our computer for free.
That split matters. Sending huge 4K files into a cloud model would be slow, expensive and daft. Text is cheap. AI is very good at reading it. FFmpeg is very good at making exact cuts.
Give each one the job it is actually good at please and thank you.
The transcript becomes the edit
The model never needs to "watch" the full video in the way people imagine.
Technically we have models that can “see” and they could watch frame by frame by frame. But good god that’s inefficient.
Instead we get a transcript. Basically what is being said, with timecodes.
This is what the AI works with.
It reads the transcript and decides where the useful video begins, which repeated explanations can go, where I wandered off into a side quest and where the livestream ends. It also decides when to show the clean face camera and when to keep the programme view with the slides.
That becomes an edit decision list. An EDL.
There was a slightly annoying technical catch here. Models can make the right editorial decision and still give you the wrong timestamp. We saw them drift by two to six seconds. Plenty of room to chop the end off a sentence. And very annoying when I was getting this up and running.
So the system now asks for the exact words around every cut. Local code finds those words in the transcript and places the blade. The model decides what to remove. Deterministic tools decide exactly where to cut. Works a treat.
That one change took the workflow from impressive demo to something I can use.
It is the same reason I keep banging on about building one shared AI vault. The value comes from keeping the rules, corrections and examples so the next run starts smarter.
Every time my editor runs there is a chance it’ll mess up and make an error.
We then FIX that error. And once it’s been fixed once that’s the last time it happens.
Render small. Check it. Then go big.
Super important if you are running this. Don’t work with full resolution video. Instead use “proxies”. Basically, lower resolution versions that are easier to handle and edit.
For me the system makes a 720p review version first. Rendering every experiment in 4K would take ages and set my Mac on fire.
And before we render back to full glorious 4K we also run all our quality checks. Boring but important stuff.
It decodes the full file. Re-transcribes the joins. Looks for long silences and black frames. Checks that the audio and video still end together. It even checks whether there is speech energy at the tail because Whisper sometimes decides the final words do not exist. Small tweaks that matter a lot unfortunately!
After all this I still watch the first minute, listen to every flagged join and approve the 720p review file. Only then does it render the 4K master and prepare the YouTube draft.
You still have to make something worth watching
AI can handle the transcript, rough cuts, rendering, captions and packaging.
It cannot rescue a useless idea.
If you give a model rubbish, it will edit the rubbish very neatly.
You still own the thesis. You still need to perform on camera. You still decide what gets published under your name.
Just because we can easily edit anything now doesn’t mean we should!
Still…it’s exciting. This is now VERY doable.
That has changed over the last three months. It’s not that the models suddenly developed video editing capabilities - nope. They just became good enough at long context, structured instructions, tool use and self-checks to run the system dependably.
I have turned the exact workflow into a step-by-step guide. It covers the local tools, the folder setup, the build prompt, the edit-decision format, the review gates and the stupid failures we hit so you do not have to repeat them.
Start with one camera and one talking-head video. Get the boring version working before asking it for swooshy animations eh?
To the Task,
Kyle