Skip to content

docs: clarify openai-whisper build-isolation fix and .npy requirement#147

Open
moduvoice wants to merge 1 commit into
bytedance:mainfrom
moduvoice:docs/setup-and-npy-error-clarity
Open

docs: clarify openai-whisper build-isolation fix and .npy requirement#147
moduvoice wants to merge 1 commit into
bytedance:mainfrom
moduvoice:docs/setup-and-npy-error-clarity

Conversation

@moduvoice

Copy link
Copy Markdown

Problem

Two separate rough edges found while following the README verbatim on a fresh environment (Tesla T4 16GB, driver 550.163.01 / CUDA 12.4, Python 3.10.12):

1. pip install -r requirements.txt fails to build openai-whisper==20240930:

error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
  ...
    File "<string>", line 5, in <module>
ModuleNotFoundError: No module named 'pkg_resources'
[end of output]
ERROR: Failed to build 'openai-whisper' when getting requirements to build wheel

This is a modern-pip/PEP517 build-isolation issue (the isolated build env doesn't include pkg_resources), not GPU/CUDA related — reproducible with a fresh venv + recent pip.

2. Using your own .wav (without a matching pre-extracted .npy) fails with a generic, unexplained error:

FileNotFoundError: [Errno 2] No such file or directory: '/path/to/my_own_voice.npy'

The README's "Model Download" section does explain that the WaveVAE encoder isn't released for security reasons and custom voices need a pre-extracted .npy requested via the linked Google Drive form — but that context is easy to miss, and the actual runtime error gives no hint that this is the reason. Someone naturally trying --input_wav my_own_recording.wav (a very natural first thing to try) hits a confusing crash with no clue why.

Fix

  • README — add a note next to the pip install -r requirements.txt step: if the openai-whisper build fails with a pkg_resources error, retry with pip install --no-build-isolation -r requirements.txt (after installing torch/torchaudio first to keep the pinned versions).
  • README — add a note near "Model Download" clarifying that --input_wav requires a matching .npy in the same directory, and that trying an arbitrary .wav without one will raise FileNotFoundError for the .npy — this is expected, not a bug, and is why the encoder-latent request process exists.

No code/behavior change — documentation only.

Verification (Tesla T4 16GB)

Working install (fixes issue 1):

python3 -m venv venv && source venv/bin/activate
pip install torch==2.6.0 torchaudio==2.6.0 --index-url https://download.pytorch.org/whl/cu124
pip install --no-build-isolation -r requirements.txt

Reproduced the documented CLI example twice using the repo's own bundled assets/Chinese_prompt.wav (+ its matching .npy):

PYTHONPATH=. CUDA_VISIBLE_DEVICES=0 python tts/infer_cli.py \
  --input_wav 'assets/Chinese_prompt.wav' --input_text "这是一个测试。" --output_dir ./gen
Metric Run 1 Run 2
Wall time (incl. model load) 66.4s 40.6s

Peak VRAM: 7.995 GB (fits Tesla T4's 16GB comfortably).

Also confirmed (not part of this PR, just verification notes): precision=torch.float16 is already correctly hardcoded for T4-class GPUs (tts/infer_cli.py:82), and all scaled_dot_product_attention call sites (tts/modules/ar_dur/commons/rot_transformer.py, tts/modules/aligner/whisper_small.py, tts/modules/llm_dit/transformer.py) don't force a specific attention backend, so they fall back correctly on Turing-class GPUs where the flash-attention kernel isn't available. No changes needed there.

Environment: Tesla T4 16GB · driver 550.163.01 (CUDA 12.4) · Python 3.10.12 · torch 2.6.0+cu124.

Raw metrics JSON
{"run1_wall_s": 66.406, "run2_wall_s": 40.594, "peak_vram_mib": 7995,
 "gpu": "Tesla T4", "driver": "550.163.01", "torch": "2.6.0+cu124"}

1. pip install -r requirements.txt can fail building openai-whisper
   with ModuleNotFoundError: No module named 'pkg_resources' on newer
   pip (PEP517 build isolation issue, not GPU-related). Add the
   --no-build-isolation workaround.
2. Using your own .wav without a matching pre-extracted .npy raises a
   generic FileNotFoundError for the .npy with no context. Add a note
   explaining this is expected (WaveVAE encoder isn't released).

No code/behavior change. Verified on Tesla T4 16GB (driver 550.163.01,
torch 2.6.0+cu124): documented CLI example reproduced 2x, peak VRAM
7.995GB.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant