Creator Studio

My role

I directed AI (Claude Code, using an Opus reviewer and Sonnet fixer pattern) to build it, set the quality gates, tested real renders myself and reported bugs. AI wrote all the code; I did not. No other person built it.

What it is

Creator Studio is a private tool for making gameplay videos. You record or drop in a video, and it transcribes the speech, finds the funny or impressive moments, plans the story, cuts a rough edit, fine-cuts it like a professional editor, renders it and gives you a review screen to approve.

It also has a script mode: you write the beats of a video first, record, and when you press Stop the app reports which beats were captured and offers pickup recordings for any that are missing.

It is not finished. At the 2026-10-25 handoff no real recording had yet been rendered through the full pro editor, because I chose to test real renders myself instead of waiting on automated render gates. The app and its test suites are the substantive result so far.

How it was built

1. Define the pipeline stagesIngest, transcribe, understand (story map), rough cut, fine cut, render, review and package. The app/pipeline folder holds more than 100 .mjs modules, split into stages, story, cut, clips, review, upload, eval and script groups.
2. Single-decode ingestPE-3: one video decode feeds every signal at once (voice activity, prosody, laughs and shouts, motion, scene change, on-screen text and HUD, vision embeddings), then an index stage.
3. Understand and plan the storyPE-4: story map, stitch and reduce, atoms, selection by a CP-SAT solver with three lenses and a both-order judge, then a rough cut with a time-skip ladder and verifiers E-13 to E-22.
4. Fine cut like an editorPE-5: cut score, jump-cut hiders, J and L split edits (99.5% accuracy on real joins), retake and filler removal, ducking of crosstalk, a length governor and HUD-pinned punch-ins that never crop game counters.
5. Script spine and live Stop reportNB-1 and NB-2: write beats first, record, and get a live report from NeMo fastconformer plus a rolling faster-whisper base.en pass (py-ml/live_whisper.py) deciding which beats were captured, with pickup offers for missing ones.
6. Idea feed with evidenceNB-3: a corpus of videos, pattern tags and a backtest, then ideas that cite evidence links (pipeline/ideas/evidence.mjs).
7. Quality gates per milestoneEach milestone (PE-3, PE-4, PE-5, NB-1, NB-2, NB-3) has a gate file in eval/gates/*.json. They passed with waivers because I chose to test real renders myself.
8. Review screen and settingsA dark-theme app with Today, Script, Record, Import, Projects, Clips, Review, Package, Publish and Settings pages, a monthly API budget with a cap, and style presets for silences and speed-ups.
9. Multi-agent build loopWork was done in tickets by AI agents, with an Opus agent reviewing and a Sonnet agent fixing; progress lines go to research/pro-editor/build-progress.log and a 326-line BUILD-PLAN.md.
10. Test and packagenpm test (115 of 115 passing after the last fix) and a typecheck; npm run dist:dir rebuilds the installed Windows app. Credentials are kept out of the project repo.

Decisions and why

Ship evidence links for ideas, not predicted scores

A backtest of the scoring gave an AUC of 0.514, barely better than a coin flip, so showing a predicted score would mislead.

Never crop game counters when punching in

HUD numbers like health and kills matter to viewers; the jump-cut hider is allowed to fail on HUD-heavy footage rather than cover them.

Pass milestone gates with waivers and test real renders myself

I wanted to judge actual videos directly instead of waiting on automated blind judges and A/B gates.

Hard monthly spend cap on AI calls

New runs and re-plans are refused once the month's cap is reached, so a bug cannot run up a bill.

Scripted Shorts are cut from their beats

The clip hunter had blocked a Short where all beats were found but nothing looked funny; the beat plan should win.

Hardest problems

Only 50% of generated ideas cited real evidence

Added relevance checks, muted citations from other games and rejected format-only citations; strict citation rose to 16 of 20 (80%).

The Stop report offered pickups for beats that were fine

The main pickup button now appears only for beats the plan says are missing; false main offers fell from 12 of 20 to 3 of 11 on stored replays.

Test runs polluted the spend ledger

10,113 fake rows were quarantined in a reversible way so the monthly figure reflects real use.

A scripted Short was blocked with 'nothing funny found'

SMG-12: clip scanning never blocks a Short; it is cut from its beats and the review uses the beat plan.

A stale resume brief said work stopped at an earlier milestone

Checked the build log instead, which showed PE-3 to PE-5 already done; the lesson was to check logs before trusting a brief.

Numbers

Timeline

• 2026-10-21First project runs on bedwars and game recordings; first installer built that night.
• 2026-10-23PE-3 ingest and PE-4 story planning finished.
• 2026-10-24PE-5 fine cut, NB-1 script spine and most of NB-2 and NB-3 built.
• 2026-10-25Handoff and resume session: NB-2 gate passed at 92.7%, idea citation fixed to 80%, SMG-12 Short fix, app rebuilt.
• 2026-10-27Last progress log entry.

What I took from it

• Measuring gates (92.7% agreement, 80% citation) turned vague quality goals into pass or fail checks.
• A backtest that shows no predictive power is a reason to remove a feature, not ship it.
• Check build logs before trusting a handoff summary.
• AI can build a very large system quickly, but real end-to-end testing on real recordings still has to happen.