ComfyUI Custom Node for HeartMuLa AI Music Generation and Transcript Text
HeartMuLa official GITHUB https://github.com/HeartMuLa/heartlib
How To Use this In Basic: https://youtu.be/F9LFAeUbBIs
Let's make this project the true open source, anyone who is interested in making improvements feel free to let us know in Discussions : the https://github.com/benjiyaya/HeartMuLa_ComfyUI/discussions
We are not providing service here, we are group of hobbiests, developers who want to make something here. So don't take it for granted.
Update:
2026-01-21 (3)
- Integrates native progress bars, making it easy to implement real-time progress tracking in the user interface.
- Precise Temperature: Refined temperature step to 0.01 for more granular control over generation.
- Intuitive Audio Length: Renamed parameter to max_audio_length_seconds (Default: 240s) for better usability.
- Keep Model Loaded(Memory Settings): True: Keep model in VRAM for instant subsequent generations. False: Unload model after each task to free up memory.
- Offload Mode: Auto: Standard memory release for balanced performance. Aggressive: Full VRAM wipe + Garbage Collection.
2026-01-21 (2)
- Lazy Load Optimization , now able to load with 12GB VRAM.
- Path Configuration ,support custom model folder path in in the "extra_model_paths.yaml", Not limited by default ComfyUI/Models/ folder path only.
2026-01-21
- MEMORY CLEANUP and Pipeline changed for BF16 - Optimized for 16GB. dtype I don't recommand under bf16 for this model, audio quality will degrade too much.
Step 1
Go to ComfyUI\custom_nodes Command prompt:
git clone https://github.com/benjiyaya/HeartMuLa_ComfyUI
Step 2
cd /HeartMuLa_ComfyUI
Step 3
pip install -r requirements.txt
If no module name error pop up. some libraries might need to install Individually (For Windows users you need use Command Prompt as Administrator)
do this :
pip install soundfile
pip install torchtune
pip install torchao
Go to ComfyUI/models
Use HuggingFace Cli download model weights.
type :
hf download HeartMuLa/HeartMuLaGen --local-dir ./HeartMuLa
hf download HeartMuLa/HeartMuLa-oss-3B --local-dir ./HeartMuLa/HeartMuLa-oss-3B
hf download HeartMuLa/HeartCodec-oss --local-dir ./HeartMuLa/HeartCodec-oss
hf download HeartMuLa/HeartTranscriptor-oss --local-dir ./HeartMuLa/HeartTranscriptor-oss
Github Repo: https://github.com/HeartMuLa/heartlib
Paper: https://arxiv.org/abs/2601.10547
Demo: https://heartmula.github.io/
HeartMuLa-oss-3B: https://huggingface.co/HeartMuLa/HeartMuLa-oss-3B
HeartCodec-oss: https://huggingface.co/HeartMuLa/HeartCodec-oss
HeartTranscriptor-oss: https://huggingface.co/HeartMuLa/HeartTranscriptor-oss
HeartMuLa: https://huggingface.co/HeartMuLa/HeartMuLa-oss-3B