Skip to content

Support old and new MCore CUDA graph APIs#3546

Merged
yaoyu-33 merged 1 commit into
NVIDIA-NeMo:mainfrom
buptzyb:codex/cuda-graph-api-compat
Apr 28, 2026
Merged

Support old and new MCore CUDA graph APIs#3546
yaoyu-33 merged 1 commit into
NVIDIA-NeMo:mainfrom
buptzyb:codex/cuda-graph-api-compat

Conversation

@buptzyb
Copy link
Copy Markdown
Contributor

@buptzyb buptzyb commented Apr 28, 2026

Summary

Add a small CUDA graph compatibility layer so Megatron-Bridge can work with both the old MCore cuda_graph_scope API and the newer split API using cuda_graph_modules plus cuda_graph_impl="full_iteration". (MCore PR NVIDIA/Megatron-LM#4293)

This updates Bridge call sites that inspect CUDA graph scope directly to go through the compatibility helpers, including full-iteration training/eval paths, local CUDA graph manager creation, delayed wgrad validation, and Qwen3-VL vision CUDA graph config propagation.

Motivation

Megatron-LM is refactoring CUDA graph configuration away from overloaded CudaGraphScope values. Bridge still needs to support existing MCore releases while also remaining compatible with the new API shape.

Validation

  • git diff --check
  • python -m py_compile src/megatron/bridge/models/qwen_vl/modelling_qwen3_vl/transformer_config.py tests/unit_tests/models/qwen_vl/modelling_qwen3_vl/test_qwen3_vl_transformer_config.py

Full pytest was not run locally because dependency setup fails while building nvidia-resiliency-ext without a CUDA installation / CUDA_PATH in this environment.

@copy-pr-bot
Copy link
Copy Markdown

copy-pr-bot Bot commented Apr 28, 2026

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@buptzyb buptzyb changed the title [codex] Support old and new MCore CUDA graph APIs Support old and new MCore CUDA graph APIs Apr 28, 2026
@buptzyb buptzyb force-pushed the codex/cuda-graph-api-compat branch from 7857f39 to 440c0b3 Compare April 28, 2026 03:10
@wplf
Copy link
Copy Markdown
Contributor

wplf commented Apr 28, 2026

/ok to test 440c0b3

Signed-off-by: Robin Zhang <robinz@nvidia.com>
@buptzyb buptzyb force-pushed the codex/cuda-graph-api-compat branch from 440c0b3 to 6815c35 Compare April 28, 2026 03:46
@wplf
Copy link
Copy Markdown
Contributor

wplf commented Apr 28, 2026

/ok to test 6815c35

@buptzyb buptzyb marked this pull request as ready for review April 28, 2026 05:11
@wplf wplf requested a review from yaoyu-33 April 28, 2026 05:27
@yaoyu-33 yaoyu-33 merged commit 56e56d7 into NVIDIA-NeMo:main Apr 28, 2026
86 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants