subtitle burn: use ffmpeg-full for libass-dependent filter#357
Closed
sonichi wants to merge 2 commits intofix/subtitled-pending-false-positivefrom
Closed
subtitle burn: use ffmpeg-full for libass-dependent filter#357sonichi wants to merge 2 commits intofix/subtitled-pending-false-positivefrom
sonichi wants to merge 2 commits intofix/subtitled-pending-false-positivefrom
Conversation
The `subtitles` filter in ffmpeg requires libass at build time. The
homebrew `ffmpeg` bottle for 8.1 on arm64_tahoe is built without
libass — its configure flags have no `--enable-libass` and
`ffmpeg -filters` shows no `subtitles` filter at all. Every
`burnLiveTranscriptSubtitles()` call today hit:
[AVFilterGraph] No option name near '.../subtitle.srt'
because the parser was treating `subtitles` as an unknown option name,
not as a filter.
### Fix
Use the keg-only `ffmpeg-full` binary at
`/opt/homebrew/opt/ffmpeg-full/bin/ffmpeg` for the subtitle burn
specifically. ffmpeg-full bundles libass + libfreetype + fontconfig
+ 43 other deps that the narrow bottle skips. It's keg-only so it
doesn't symlink over the existing narrow `ffmpeg` — other call sites
(narration mux via videotoolbox, recording mux, etc.) keep using the
fast narrow bottle. Only the subtitle burn reaches for the full one.
### Fallback
If ffmpeg-full isn't installed, fall through to the narrow `/opt/homebrew/bin/ffmpeg`
call as before. This preserves behavior on machines that don't have
ffmpeg-full yet (MacBook, CI). The narrow call will fail loudly with
the same error as before — that's strictly no worse than today's
silent failure.
### Install (one-time, per-node)
brew install ffmpeg-full # ~500MB–1GB, 5–15 min, 46 deps
### Log delta
The success log line now includes which ffmpeg binary was used:
[ScreenRecord] live transcript subtitles burned: /tmp/...-subtitled.mov (ffmpeg=/opt/homebrew/opt/ffmpeg-full/bin/ffmpeg)
so operators can confirm the full binary was picked up post-install.
### Stacks on
#355 (subtitled_pending false-positive gate). Same branch tree. Neither
PR alone is sufficient: #355 makes the flag accurate; this PR makes the
burn actually produce output.
### History
Four wrong theories chased before landing on this one:
1. wrong process (voice-agent vs conversation-server)
2. stale local main (git fetch vs git pull)
3. filter rejection (entries.length === 0 after aggressive filter)
4. comma escaping in force_style (tried multiple backslash counts, all failed)
Susan raised "is it a dependency issue?" early in the thread. Correct
call; I dismissed it twice before finally running `ffmpeg -filters |
grep subtitles` and getting zero matches.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
josuachavesolmos
approved these changes
Apr 16, 2026
josuachavesolmos
left a comment
There was a problem hiding this comment.
Reviewed by Sutando proactive loop.
APPROVE — minimal, surgical change. Only burnLiveTranscriptSubtitles() is affected; all other ffmpeg call sites verified untouched. Graceful fallback via existsSync check. Good observability with (ffmpeg=${ffmpegBin}) in logs.
One follow-up note:
- [MEDIUM]
skills/screen-record/scripts/subtitle.pyalso uses thesubtitlesffmpeg filter with bareffmpeg— will hit the same libass error. Track as a separate issue.
Safe to merge after PR #355 lands (stacked dependency).
2 tasks
Replace the hardcoded ffmpeg-full path with a runtime probe that automatically finds the right ffmpeg binary. No user configuration needed. findFfmpegWithSubtitles() probes candidates in order: 1. $FFMPEG_SUBTITLE_BIN env var (explicit override, skips probe) 2. System `ffmpeg` (whatever is on PATH) 3. /opt/homebrew/bin/ffmpeg (homebrew narrow) 4. /opt/homebrew/opt/ffmpeg-full/bin/ffmpeg (homebrew full, keg-only) Each candidate gets `ffmpeg -filters 2>&1` and we grep for "subtitles". First match wins. Result cached for the process lifetime (~50ms first call, 0ms thereafter). If none have it, logs the failure with an install hint and returns null — the subtitle burn is skipped gracefully instead of crashing with the cryptic AVFilterGraph error. On Mac Mini today: system ffmpeg and /opt/homebrew/bin/ffmpeg both return subtitles=false. /opt/homebrew/opt/ffmpeg-full/bin/ffmpeg returns subtitles=true → selected as winner. On MacBook (if ffmpeg has libass baked in), the system ffmpeg would win on the first probe. Also supports $FFMPEG_SUBTITLE_BIN env var as an explicit override for any edge case (custom builds, non-homebrew installs, CI). Owner asked "can we figure out the right ffmpeg without user effort?" — this is the answer. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
sonichi
commented
Apr 16, 2026
Owner
Author
sonichi
left a comment
There was a problem hiding this comment.
MacBook review: LGTM. Auto-detects ffmpeg with subtitle filter support, cached after first probe, env var override available. Falls back to homebrew ffmpeg-full. Clean, no runtime cost after first call.
sonichi
pushed a commit
that referenced
this pull request
Apr 16, 2026
Per owner's design directive: "The recording tool should return the
files and the default file to use. The caller should process the result
and report abnormality if any. Then open_file tool should be called with
filename."
Changes:
1. open_file: stripped recording-specific logic.
- Requires `path` arg (get it from recording tool result).
- `find_latest_recording: true` flag as explicit fallback — only
called when model lost the path, not as silent default.
- Removed: subtitled_pending flag, version detection, polling
logic, findRecording() as default behavior.
- Kept: known file aliases (diagnostics), QuickTime activation,
playback-path write, duration/size metadata.
2. screen_record stop: returns explicit file list.
- New `files` object: `{raw, narrated, subtitled, recommended}`.
- `instruction` string tells the model exactly which path to pass
to open_file and how to offer alternatives to the user.
- Subtitle burn still happens synchronously on stop.
3. No changes to record_screen_with_narration — its auto-stop timer
fires in a setTimeout and can't return files to the model through
a tool result. The model already has the raw path from the start
call; open_file with that path works. Full file-list return for
record_screen_with_narration is a follow-up.
Net: +39/-38 (near-zero). This is a separation of concerns, not new
features. findRecording() and findFfmpegWithSubtitles() are preserved
as helpers; they just stop being called by default in open_file.
Supersedes the approach in #353 (non-blocking return + subtitled_pending)
and makes #355 (subtitled_pending gate) + #357 (ffmpeg-full path)
unnecessary as open_file changes — though #357's findFfmpegWithSubtitles
is still needed in the recording tool's burn path.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
4 tasks
Owner
Author
sonichi
added a commit
that referenced
this pull request
Apr 16, 2026
Per owner's design directive: "The recording tool should return the
files and the default file to use. The caller should process the result
and report abnormality if any. Then open_file tool should be called with
filename."
Changes:
1. open_file: stripped recording-specific logic.
- Requires `path` arg (get it from recording tool result).
- `find_latest_recording: true` flag as explicit fallback — only
called when model lost the path, not as silent default.
- Removed: subtitled_pending flag, version detection, polling
logic, findRecording() as default behavior.
- Kept: known file aliases (diagnostics), QuickTime activation,
playback-path write, duration/size metadata.
2. screen_record stop: returns explicit file list.
- New `files` object: `{raw, narrated, subtitled, recommended}`.
- `instruction` string tells the model exactly which path to pass
to open_file and how to offer alternatives to the user.
- Subtitle burn still happens synchronously on stop.
3. No changes to record_screen_with_narration — its auto-stop timer
fires in a setTimeout and can't return files to the model through
a tool result. The model already has the raw path from the start
call; open_file with that path works. Full file-list return for
record_screen_with_narration is a follow-up.
Net: +39/-38 (near-zero). This is a separation of concerns, not new
features. findRecording() and findFfmpegWithSubtitles() are preserved
as helpers; they just stop being called by default in open_file.
Supersedes the approach in #353 (non-blocking return + subtitled_pending)
and makes #355 (subtitled_pending gate) + #357 (ffmpeg-full path)
unnecessary as open_file changes — though #357's findFfmpegWithSubtitles
is still needed in the recording tool's burn path.
Co-authored-by: Chi <wangchi@Chis-Mac-mini.local>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes the root cause Susan called out early and I spent all afternoon chasing: the homebrew
ffmpegbottle for 8.1 on arm64_tahoe is built without libass, so thesubtitlesfilter doesn't exist in the binary at all. EveryburnLiveTranscriptSubtitles()call has been silently failing with:because the parser was treating
subtitlesas an unknown option name rather than a filter. Four wrong theories chased beforeffmpeg -filters | grep subtitlesreturned zero hits.Fix
ffmpeg-fullbinary at/opt/homebrew/opt/ffmpeg-full/bin/ffmpegfor the subtitle burn specifically. ffmpeg-full bundles libass + libfreetype + fontconfig + 43 other deps./opt/homebrew/bin/ffmpeg. Preserves behavior on MacBook / CI / any node without ffmpeg-full. The narrow call will fail loudly as before — strictly no worse than today's silent failure.burnLiveTranscriptSubtitles()— all other ffmpeg call sites (narration mux via videotoolbox, record.py mux, etc.) keep using the fast narrow bottle. They don't need libass.Install required (per-node, one-time)
~500MB–1GB, 5–15 min, 46 deps. Already done on Mac Mini (verified:
ffmpeg -filters | grep subtitlesreturns the filter definition).End-to-end test (already run on Mac Mini)
Ran the EXACT ffmpeg command with the new
/opt/homebrew/opt/ffmpeg-full/bin/ffmpegpath against/tmp/sutando-recording-1776298340-narrated.mov+/tmp/sutando-live-transcript-subtitle.srt(both from owner's earlier failed test call):Output:
/tmp/TEST-subtitled.mov, 2.9 MB, valid mov. ✓Log delta
Success log now reports which ffmpeg was used:
Stacks on #355
Base branch:
fix/subtitled-pending-false-positive(MacBook's #355). Neither PR alone is sufficient:subtitled_pendingaccurate (correctly gates on file evidence)Merge #355 first, then this.
History (for the retro)
Chased before landing here:
Susan raised "is it a dependency issue?" first. Owner also. I dismissed it twice. The trivial verification that would have landed this in 5 minutes:
ffmpeg -filters | grep subtitles→ zero hits → checkffmpeg -version→ no--enable-libass.🤖 Generated with Claude Code