docs: clarify openai-whisper build-isolation fix and .npy requirement#147
Open
moduvoice wants to merge 1 commit into
Open
docs: clarify openai-whisper build-isolation fix and .npy requirement#147moduvoice wants to merge 1 commit into
moduvoice wants to merge 1 commit into
Conversation
1. pip install -r requirements.txt can fail building openai-whisper with ModuleNotFoundError: No module named 'pkg_resources' on newer pip (PEP517 build isolation issue, not GPU-related). Add the --no-build-isolation workaround. 2. Using your own .wav without a matching pre-extracted .npy raises a generic FileNotFoundError for the .npy with no context. Add a note explaining this is expected (WaveVAE encoder isn't released). No code/behavior change. Verified on Tesla T4 16GB (driver 550.163.01, torch 2.6.0+cu124): documented CLI example reproduced 2x, peak VRAM 7.995GB.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Two separate rough edges found while following the README verbatim on a fresh environment (Tesla T4 16GB, driver 550.163.01 / CUDA 12.4, Python 3.10.12):
1.
pip install -r requirements.txtfails to buildopenai-whisper==20240930:This is a modern-pip/PEP517 build-isolation issue (the isolated build env doesn't include
pkg_resources), not GPU/CUDA related — reproducible with a fresh venv + recent pip.2. Using your own
.wav(without a matching pre-extracted.npy) fails with a generic, unexplained error:The README's "Model Download" section does explain that the WaveVAE encoder isn't released for security reasons and custom voices need a pre-extracted
.npyrequested via the linked Google Drive form — but that context is easy to miss, and the actual runtime error gives no hint that this is the reason. Someone naturally trying--input_wav my_own_recording.wav(a very natural first thing to try) hits a confusing crash with no clue why.Fix
pip install -r requirements.txtstep: if theopenai-whisperbuild fails with apkg_resourceserror, retry withpip install --no-build-isolation -r requirements.txt(after installingtorch/torchaudiofirst to keep the pinned versions).--input_wavrequires a matching.npyin the same directory, and that trying an arbitrary.wavwithout one will raiseFileNotFoundErrorfor the.npy— this is expected, not a bug, and is why the encoder-latent request process exists.No code/behavior change — documentation only.
Verification (Tesla T4 16GB)
Working install (fixes issue 1):
Reproduced the documented CLI example twice using the repo's own bundled
assets/Chinese_prompt.wav(+ its matching.npy):Peak VRAM: 7.995 GB (fits Tesla T4's 16GB comfortably).
Also confirmed (not part of this PR, just verification notes):
precision=torch.float16is already correctly hardcoded for T4-class GPUs (tts/infer_cli.py:82), and allscaled_dot_product_attentioncall sites (tts/modules/ar_dur/commons/rot_transformer.py,tts/modules/aligner/whisper_small.py,tts/modules/llm_dit/transformer.py) don't force a specific attention backend, so they fall back correctly on Turing-class GPUs where the flash-attention kernel isn't available. No changes needed there.Environment: Tesla T4 16GB · driver 550.163.01 (CUDA 12.4) · Python 3.10.12 · torch 2.6.0+cu124.
Raw metrics JSON
{"run1_wall_s": 66.406, "run2_wall_s": 40.594, "peak_vram_mib": 7995, "gpu": "Tesla T4", "driver": "550.163.01", "torch": "2.6.0+cu124"}