From a standalone Python frame extraction script to embedding it into the next-mdx-portfolio repository with an AI collaboration skill, ultimately forming a workflow where "drop a video path, AI automatically writes .mdx." This article documents the trade-offs along the way.
The hardest part of writing promotional video breakdowns isn't the writing itself—it's repeatedly scrubbing through the timeline in editing software to catch every detail. A 5-minute video, a 1500-word article, and you end up watching it back a dozen times.
So I built a little tool to watch videos for me.
Its job is simple: evenly slice a video into 20 frames, write a Markdown draft with empty slots, and leave the rest to the AI inside Cursor—read the frames, fill in the draft, and produce a finished .mdx following this repository's MDX conventions. The entire pipeline relies on zero online AI APIs; the critical model work uses Cursor's own vision capabilities.
This article traces its evolution: from a standalone Python script, to welding it entirely into this next-mdx-portfolio repository, to writing an AI workflow skill that trains both humans and models into the same assembly line.
We'll look at four things:
raw_frames.md splits "looking at images" from "writing" into two stepsThe earliest version was called Read video, a separate Python project. Its
core capability was just one thing: calling FFmpeg to extract frames.
The decision to avoid any third-party Python libraries was intentional. Fancy
video processing libraries like OpenCV, moviepy, and PyAV are all noticeably
slower than the FFmpeg CLI, and packaging them is a hassle. Going directly with
subprocess.run(["ffmpeg", ...]) is cleaner—as long as FFmpeg is installed and
in PATH, it just works.
Three frame extraction strategies were supported: fixed count (count), fixed
frame interval (frame), and fixed second interval (second). Early on I
thought the latter two would be more practical—after all, "one frame every 30
frames" sounds professional. But in practice, long videos would explode with
frames while short videos barely got any, requiring manual parameter tuning each
time. So the default was changed to "evenly extract N frames from the entire
video," with N defaulting to 20—just right for most 5- to 10-minute promotional
videos.
This version worked, but it was disconnected from my blog repository. Every time I wrote a breakdown, the flow went like this:
Extract frames in standalone project → Look at images → Write article manually → Copy to blog repo → Fix frontmatter → Pick a frame as cover → Compress video → Push to publicSeven or eight intermediate steps all relying on manual effort and copy-paste. By the time I finished writing, I had no energy left to double-check whether paths were correct.
The turning point came when I finished an article only to find the video path in
the final .mdx was /Projects/Liblib AI/... (with a space), but the actual
directory was /Projects/Liblib-AI/... (slugified). The frontend loaded fine,
but the video was just a silent blank rectangle.
Once something like that happens twice, it's time to eliminate it with code, not by trying to remember.
I moved Read video wholesale into tools/read-video/ as an embedded
repository tool. This decision had three direct benefits:
The tool can see the repository itself. The original version didn't know
about conventions like "project case directories must be slugified, blog
covers must be webp, videos go in video/ subdirectories"—because it was a
general-purpose tool. Once embedded, these rules could be written as Python
functions. The CLI just takes
--target projects --company "Liblib AI" --slug demo and computes the final
destination directly.
Intermediate artifacts can naturally be gitignored. All extracted frames
land in <repo>/articles/<slug>/, which was added to .gitignore early on.
No matter how many frames are generated, git stays clean. When the AI actually
needs to use a specific frame in an article, it manually copies it to
public/.../frames/.
AI collaboration becomes possible. Cursor can directly see this tool,
Read its code, Read the drafts it generates, and write .mdx following
the rules—provided the tool's behavior and output locations are deterministic
and predictable.
There are two output destinations in the repository: blog posts and project
cases. Each destination has both content (.mdx) and static assets
(public/...). Add slug naming conventions, and there were at least six or
seven places where mistakes could creep in. I centralized them all into one
layout.py:
def repo_root() -> Path:
return Path(__file__).resolve().parents[3]
def article_workspace_dir(slug: str) -> Path:
return repo_root() / "articles" / slug
def posts_mdx_path(slug: str) -> Path:
return repo_root() / "content" / "posts" / f"{slug}.mdx"
def projects_public_dir(company: str, slug: str) -> Path:
return repo_root() / "public" / "Projects" / slugify(company) / slugThe CLI never constructs paths directly—it only calls these functions. That one
line projects_public_dir is especially valuable—it encodes the repository
convention that "under content/projects/ we use Chinese directory names
(Liblib AI), but under public/Projects/ we use slugs (Liblib-AI)." No
matter how many times the AI calls it, it will never write a path with spaces
again.
slugify was also deliberately kept from full Anglicization. The repository
already has numerous Chinese filenames (e.g., 115-Move Object 功能.mdx), and
forcing kebab-case would conflict with the existing state. The rule was
simplified to: remove Windows-disallowed characters, replace spaces and Chinese
punctuation uniformly with hyphens, and trim leading/trailing hyphens. Both
Chinese and English characters are preserved.
Frame extraction is just grunt work. What truly determines the quality of the
resulting article is the intermediate raw_frames.md.
The earliest version treated it as a simple preview page: one image per frame, one timestamp, done. But then the AI would start making up stories from memory after looking at the images, forgetting by the second section what text actually appeared in the first section's frames.
The new draft added an "empty slot" mechanism. Below each frame, a structured empty table is pre-placed:
## #012 · 00:01:23.500
**Raw Information** (fill after looking at the image; keep `-` if not read)
- Subject: -
- Text/UI visible: -
- Key action/change: -
- Camera language (shot type/movement/transition): -
- Notes: -After the AI reads an image, it uses StrReplace to fill in the dashes with
factual information—only recording what text appears in the frame, who is doing
what, without evaluations or literary descriptions.
First lock down the visual facts, then start writing the article.
This trick had an unexpectedly significant effect. It split "looking at images" and "writing" into two phases: the first phase only converts visual signals into text; the second phase, when writing the article, no longer needs to look back at the images—it just reads this draft that it has already organized. Whenever uncertain about a detail, it references this textual fact table rather than the inherently ambiguous image.
The header of the draft also includes a lengthy usage guide, telling the AI that
this is a draft, not the final output, that empty frames can be skipped, and
where the final .mdx should be written. This guide is directly concatenated by
the CLI when generating the file. The first time the AI Reads this file, it
reads these instructions without needing any additional prompting.
Putting videos into public/ is a sweet trap. The advantage of the <MdxVideo>
component is that it can directly serve mp4 files from public/ in the
frontend. But promotional video footage often has bitrates of 15 to 25 Mbps. A
5-minute clip easily exceeds 60 MB. A few projects in and the repository
balloons past 100 MB.
I added a hard rule: videos going into public/ default to 4 Mbps total bitrate
(video + audio). This is the sweet spot where the difference is barely
noticeable to the naked eye but the repository pressure stays manageable.
The implementation uses H.264 two-pass encoding:
total_bps = _parse_bitrate(target_bitrate)
audio_bps = _parse_bitrate(audio_bitrate)
video_bps = max(200_000, total_bps - audio_bps)
maxrate_bps = int(video_bps * 1.15)
bufsize_bps = int(video_bps * 2)Audio is deducted first from the total bitrate (128 kbps), and the remainder
goes to video. maxrate at 1.15x and bufsize at 2x are common stabilization
values for H.264 ABR mode. The first pass only scans data without writing files,
dumping to NUL (Windows) or /dev/null, and the second pass does the actual
encoding.
There's a subtle safety detail here. Compression can fail partway through. If it
overwrites the target file directly, the original footage is lost. So the code
writes to a temporary file in the same directory first, then atomically replaces
with os.replace once fully complete. Same directory avoids the quirky behavior
of cross-drive renames on Windows:
tmp_fd, tmp_name = tempfile.mkstemp(
suffix=dst_path.suffix or ".mp4",
prefix=dst_path.stem + ".compress.",
dir=str(dst_path.parent),
)This means the compress subcommand's source and destination can be the same
path—"in-place compression" of an already imported video works without issues.
This capability later serves as a safety net in the skill: if a video is moved
into public/ before the compression step, the AI can still run
python -m read_video.compress <path> <path> --bitrate 4M to bring it back to
standard bitrate.
At this point, the tool was fully functional. But only by letting the AI call the tool autonomously could the "human remembers the rules" step be completely eliminated.
Cursor's skill system provided a clean entry point. I wrote a workflow document
in .cursor/skills/read-video/SKILL.md that is just under 400 lines, covering:
The most critical part was the first section—"Trigger Conditions." Cursor decides when to automatically load a skill based on its description paragraph. I made it very specific: when a user issues an absolute path to a video file in the repository, or drags a video into the chat and says "add it to project X," immediately enter this pipeline, do not ask about frame density, style, save location, or other parameters that already have defaults.
This "do not ask" rule was refined through iteration. Early on, the AI would always fire off three questions first: "How many frames would you like?" "Where should I save them?" "Do you need compression?" This behavior is disastrous for a tool-oriented workflow—the user has to hit enter every time to confirm defaults. The skill pins all default values firmly, and the AI just runs when it sees a video path. If it's wrong, it can be corrected afterward.
The skill also enforces an anti-laziness constraint. Another skill in this
repository called add-project is designed to quickly create a project page
with just a placeholder line and a video. But read-video explicitly states: when
a user drags in a video and says "add it to project X," treat it as a
publish-grade project case that must go through the full "extract frames →
fill draft → write four-section body → compress and import → generate cover"
pipeline, and must not just make do with add-project's minimal template.
This constraint is written firmly because if the AI has room to be lazy, it will
be lazy.
The tool worked, the pipeline flowed, but the last pitfall was the articles coming out with an unmistakable AI flavor.
So I put writing style requirements into the skill as well:
A blacklist of overused AI buzzwords was also compiled: "empower", "closed-loop", "anchor", "mindshare", "paradigm", "land", "grip", "feel", "immersive"—replace each with concrete descriptions whenever possible.
The effect of this section was gradual. Newer models would still occasionally
produce sentences like "this solution empowers content production...", but once
the skill was loaded together with reference articles
(content/projects/Liblib AI/libtv-720-panorama-feature.mdx), the output
stabilized into a consistent "methodology appreciation blog" tone.
Here's how the whole thing works now:
I only say one sentence: "Add it to Liblib AI's project cases." No more asking about frame density, save directories, or compression parameters.
The AI automatically loads the read-video skill and runs
python -m read_video "<path>" --target projects --company "Liblib
AI" --slug "<slug>" --copy-to-public
.
The tool completes frame extraction, compression, and video import, while
writing articles/<slug>/raw_frames.md, locking down all
paths and resource locations.
The AI reads 20 frames, fills in the draft, selects a key frame as the
cover, writes the .mdx following a four-section structure, and
finally updates CHANGELOG_AI.md.
The entire process, from my end, is: drag the video into the chat, type one sentence, wait a few minutes, skim the output for minor edits.
By now you might be thinking this is just an ordinary Python utility with a few markdown config files. And you'd be right. But it taught me one thing: when a workflow is deterministic enough, you should eliminate every piece of uncertainty—compute paths with functions, encode conventions in layout, constrain AI behavior with skills, backstop writing style with a blacklist, and handle compression failures with atomic rollback. The remaining uncertainty is what's truly worth your time.