HyperFrames Media Preprocessing#

Three CLI commands that produce assets for compositions: tts (speech), transcribe (timestamps), and remove-background (transparent video). Each downloads a model on first run and caches it under ~/.cache/hyperframes/.

Text-to-Speech (tts)#

Generate speech audio locally with Kokoro-82M. No API key required.

npx hyperframes tts "Text here" --voice af_nova --output narration.wav
npx hyperframes tts script.txt --voice bf_emma --output narration.wav
npx hyperframes tts --list                       # list all 54 voices

Voice Selection#

Multilingual#

Voice IDs encode language in the first letter:

• a = American English, b = British English, e = Spanish
• f = French, h = Hindi, i = Italian, j = Japanese
• p = Brazilian Portuguese, z = Mandarin

The CLI auto-detects the phonemizer locale from the prefix — no --lang needed when the voice matches the text.

npx hyperframes tts "La reunión empieza a las nueve" --voice ef_dora --output es.wav
npx hyperframes tts "今日はいい天気ですね" --voice jf_alpha --output ja.wav

Use --lang only to override auto-detection (stylized accents). Valid codes: en-us, en-gb, es, fr-fr, hi, it, pt-br, ja, zh.

Speed#

Long Scripts#

Write to a .txt file and pass the path. Inputs over ~5 minutes may benefit from splitting into segments.

Requirements#

Python 3.8+ with kokoro-onnx and soundfile (pip install kokoro-onnx soundfile). Model downloads on first use (~311 MB + ~27 MB voices, cached in ~/.cache/hyperframes/tts/).

Transcription (transcribe)#

Produce a normalized transcript.json with word-level timestamps.

npx hyperframes transcribe audio.mp3
npx hyperframes transcribe video.mp4 --model small --language es
npx hyperframes transcribe subtitles.srt          # import existing
npx hyperframes transcribe subtitles.vtt
npx hyperframes transcribe openai-response.json

Critical Language Rule#

Never use .en models unless the user explicitly states the audio is English. .en models (small.en, medium.en) translate non-English audio into English instead of transcribing it. This silently destroys the original language.

1. Language known and non-English → --model small --language <code> (no .en suffix)
2. Language known and English → --model small.en
3. Language unknown → --model small (no .en, no --language) — whisper auto-detects

Default model is small, not small.en.

Model Sizes#

Music with vocals: start at medium minimum.

Output Shape#

[
  { "id": "w0", "text": "Hello", "start": 0.0, "end": 0.5 },
  { "id": "w1", "text": "world.", "start": 0.6, "end": 1.2 }
]

Background Removal (remove-background)#

Remove the background from a video or image so the subject sits as a transparent overlay.

npx hyperframes remove-background subject.mp4 -o transparent.webm  # VP9 alpha WebM
npx hyperframes remove-background subject.mp4 -o transparent.mov   # ProRes 4444
npx hyperframes remove-background portrait.jpg -o cutout.png       # single-image cutout
npx hyperframes remove-background subject.mp4 -o subject.webm \
  --background-output plate.webm                                   # both layers
npx hyperframes remove-background --info                           # detected providers

Uses u2net_human_seg (MIT). First run downloads ~168 MB of weights.

Layer Separation (--background-output)#

Pass --background-output (or -b) to emit a second transparent video with the inverse alpha:

Both share the same quality preset and run from a single inference pass.

Output Format#

Quality Presets#

TTS -> Transcribe -> Captions Pipeline#

Generate voiceover, get word-level timestamps, and create captions:

npx hyperframes tts script.txt --voice af_heart --output narration.wav
npx hyperframes transcribe narration.wav   # -> transcript.json

Whisper extracts precise word boundaries from the generated audio, so caption timing matches delivery without hand-tuning.

Related Skills#