spectrogram-videos-with-karaoke-style-subtitles
Made by JeffMakes MP4s with a scrolling spectrogram and word-by-word karaoke captions; finds phrases, supercuts, re-voices with TTS, writes SRT/VTT/ASS. Use when asked for spectrogram, karaoke or lyric videos.
From Jeffrey's Skills.md: professionally crafted skills for any AI agent, installed and kept in sync by the jsm CLI. Subscription only, $20/month.
Works With
Quality and Utility come from canonical skill metadata. Community ratings appear separately as stars when subscriber reviews exist.
Subscribe to download
Public previews are available, but downloads, CLI sync, and premium content require an active subscription.
Sign In to Subscribe- Full skill content with deterministic versioning
- CLI sync + local search index
- Access to Jeffrey’s premium library
Save to your account
Saved skills live in your cloud list. Run jsm sync on each machine to pull them locally.
What's New with this Skill
6Updates, enhancements, and release notes for this skill.
Updated: spectrogram-videos, Chinese and Japanese captions break between words
Chinese and Japanese caption lines, and the word groups of the pop caption style, now break between words instead of inside one. The word boundaries come from BudouX, the phrase model behind Chrome's auto-phrase line breaking. Text without Chinese or Japanese lays out exactly as before.
- Chinese and Japanese caption lines now break between words, found by the BudouX phrase model, instead of inside a word or a name
- On 26 hand-segmented sentences, line breaks inside a word went from 20 to 0, and pop groups starting inside one from 46 to 0
- Text without Chinese or Japanese lays out exactly as before: 9,000 random runs gave identical lines
Chinese and Japanese have no spaces to break at. v15 broke their lines by width, with rules only for small kana and punctuation, so a line could end inside a word or a name (友達とレ / ストランへ, 欧尔希 / 娜一起).
- Line breaks. A break that would fall inside a word moves to the nearer word boundary that BudouX finds. On 18 Japanese and 8 Chinese sentences segmented by hand, breaks inside a word at portrait width went from 15 to 0 in Japanese and from 5 to 0 in Chinese. A Chinese or Japanese word inside Latin text no longer splits either (
東/京).
Updated: spectrogram-videos, words kept at cuts, speaker labels read more carefully, and silence stays dark
At a cut or a supercut join, a word that keeps more than half its timed span is now trimmed and shown, with a CHECK naming it, instead of being dropped. Caption labels are read more carefully: role-first labels keep the person's name, numbered headings such as 'Step 3:' stay text, and a later 'Narrator:' no longer names the speech before it. Silent audio now stays at the palette's darkest colour, and --pip QA also checks the picture around scene changes.
- Words that keep more than half their timed span at a cut or supercut join are trimmed and shown, each with a CHECK
- Role-first labels keep the name, numbered headings stay text, and a late 'Narrator:' no longer names earlier speech
- Digital silence stays at the palette's darkest colour, and --pip QA checks the picture around up to eight scene changes
- Two new test suites: 119 caption tests and 32 tests for pitch repair, silent spectra and picture timing
Two new test suites ship with the skill: 119 caption tests and 32 tests for pitch repair, silent spectra and picture timing.
- Words at cuts. At a cut, or between
--keepand--supercutparts, a word that keeps more than half its timed span is trimmed and kept. Smaller fragments stay out. Every partly cut word gets a CHECK with the word, its source time and the percentage kept, because a timing percentage cannot tell you whether the remaining sound is still intelligible.
Updated: spectrogram-videos, the speaker's picture stays in step from any start, and no line break after 'Mr.'
With --pip, a video cut from anywhere but its start no longer shows the speaker's picture a frame or two early, and the picture changes on the cut at --keep and --supercut joins. A line no longer ends on 'Mr.' or 'Dr.' with the name on the next one.
- With --pip, a clip cut from anywhere but the start shows the picture in step with the sound instead of a frame or two early
- On 239 randomly cut test clips with numbered frames, every frame is right; v13 had 46,299 of 69,664 wrong
- Lines no longer break between 'Mr.' or 'Dr.' and the name: 3,237 fewer such breaks on 2,455 real caption files
Both changes were found while making a video of an eight-minute film scene, and measured on test videos and on public caption files.
- The picture with
--pip. v13 restarted each cut's clock at its first video frame, so anything cut from anywhere but the start (--start,--keep,--supercut,--slow) showed the picture one or two frames early, and at a join the next part's picture appeared a frame too soon. On a 23.976 fps film from--start 10, the QA check caught it and the run stopped with exit 7. The picture is now read in one pass and each frame placed at its time in the video
Updated: spectrogram-videos, --pitch leaves out bass notes, and a speaker swap in '>>' captions fixed
With --pitch, a note below 70 Hz is now left out instead of being drawn on one of its harmonics; a creaky voice loses its line where it creaks. A reply that repeats the question's words no longer swaps the speakers of '>>' captions after it.
- --pitch leaves out notes below 70 Hz instead of drawing them on a harmonic
- Frames drawn on a harmonic of a bass, tuba or throat-singing note: 189 on six recordings, against 476
- A reply that repeats the question's words no longer swaps the speakers of '>>' captions after it
Both changes were measured on public recordings and caption files.
--pitchand low notes. A note below 70 Hz is left out instead of being drawn on one of its harmonics, at two to five times its pitch. On six Wikimedia Commons recordings of walking bass, tuba and throat singing, such frames went from 476 to 189, and 82% of the in-range notes there are drawn, against 86% before. The cost is that a creaky voice loses its line where it creaks: on 33 synthetic voice recordings, 79.4% of the frames are drawn instead of 83.0%, most of the loss in one creaky voice.
Updated: spectrogram-videos, more bleeped-name forms masked in translations and SDH speaker tags read
A name you bleep is now masked in more of the forms a translation writes it in (declined, respelled, disguised or split), or the run stops and names the word to check. SDH tags such as '- [Name]' now name their lines, and a song clip that ends mid-line keeps the words sung before the cut.
- More declined, respelled, disguised and split forms of a bleeped name masked in translations, or the run stops
- SDH tags such as '- [Name]' name their lines
- Song clips cut mid-line keep their sung words: 5 dropped on 236 new clips, against 281 before
- --text reads a once-used label such as 'Holly:' as a label instead of a spoken word
Tested on inputs the changes were not tuned on: synthetic dialogues and songs, translations in 15 languages, and 912 caption files from Wikimedia Commons and archive.org. The audio bleeps themselves are unchanged.
- Bleeps in translations. More declined, respelled, disguised and split forms of a bleeped name are masked, or the run stops and names the word. In 52 new test translations the previous version showed the bleeped name in 3; this one showed it in none and stopped on 4 to ask about a word.
New skill: spectrogram-videos-with-karaoke-style-subtitles, a scrolling spectrogram with captions that fill word by word in each speaker's colour
An agent turns any audio or video file into an MP4 with a scrolling spectrogram and the transcript filling word by word in each speaker's colour. ASR word times are moved onto the sound, and pitch settles who said each sentence. The same script finds a phrase in a long recording and splices every place it is said, re-voices a recording in TTS voices, bleeps a name and listens to check that none of it is left, adds a translated second line, and writes SRT, WebVTT, ASS and LRC captions.
- Frames are drawn with numpy and Pillow and piped to ffmpeg as raw video, so the karaoke fill needs no drawtext or subtitles filter. The reference video still renders the same: 857 frames, 14 lines, 66 words.
- ASR word times are moved onto the sound, and pitch settles who spoke when two voices differ by 35% or more. Names written in a subtitle file are kept as written.
- --fix corrects a misheard name and --text puts the exact words, lyrics included, on the heard words' times. A text less than half heard stops the run.
- find searches a long recording after one transcription: in a 30-minute recording it found all four places a phrase was said. --supercut splices every one into one video, and --batch cuts many clips from one transcription.
- revoice re-speaks a recording with one TTS voice per speaker, cast by measured pitch, using FrankenTTS or macOS say, and transcribes every clip back to catch a garbled one.
- --bleep replaces a name with a tone and [bleep], transcribes its own result and stops before rendering if any of the name is still heard. A title or translation in a non-Latin script stops the run until a person has read it.
- --translation adds a translated line under each line, from a file the run writes with each line's text to translate. A translation made for another cut stops the run and names the line.
- --pitch draws the voice's pitch, 70 to 1000 Hz, in the speaker's colour. On 24 voices it drew 89.7% of the frames Praat calls voiced, and whispering draws nothing.
- Presets for landscape, 4K, square and portrait; --caption-style pop or sheet; --layout overlay on the picture or --pip beside it; --alpha for a video editor; --slow for learners, with pitch kept.
- Every run writes SRT, WebVTT with word timestamps, ASS karaoke, enhanced LRC and Markdown, and --plan prints every line first, with CHECK lines for what the QA checks cannot see.
scripts/karaoke_spectrogram.py make draws every frame with numpy and Pillow and pipes raw video to ffmpeg, so the word-by-word fill needs none of the text filters that many ffmpeg builds leave out. The picture is the render the skill started from: a magma, log-frequency spectrogram from 60 Hz to 8 kHz with 8.8 s in view, the half still to come dimmed, a strip coloured by whoever is speaking, the current line filling in that speaker's colour and the next line faint below it. Presets cover 1920×1080, 4K, square, portrait and a 960×540 preview. --pip puts the input's own picture beside the sp