Register GLM-4.7-Flash bridge and bump megatron-bridge#1214
Register GLM-4.7-Flash bridge and bump megatron-bridge#1214tyler-griggs wants to merge 1 commit intomainfrom
Conversation
Register Glm4MoeLiteForCausalLM as a trivial DeepSeekV3Bridge subclass. GLM-4.7-Flash (zai-org/GLM-4.7-Flash) uses the identical architecture as DeepSeek-V3 (MLA + MoE), so the same bridge handles weight conversion. Even the latest upstream Megatron-Bridge does not register this model. Bump megatron-bridge from 04e370ee (Jan 14) to b058b662 (HEAD, +252 commits). Key fixes included: - DeepSeek-V3 H100 large-scale config fix - DeepSeek num_query_groups mapping correction - MoE FlexDispatcher backend fix - Memory savings for MoE param_l2_norm Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
tyler-griggs
force-pushed
the
tgriggs/glm47-bridge-registration
branch
from
04bd877 to
d4e5ee9
Compare
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request enables support for the GLM-4.7-Flash model by registering its architecture with AutoBridge and updating the megatron-bridge dependency to a more recent commit. The changes are clear and well-justified. The registration of Glm4MoeLiteForCausalLM by subclassing DeepSeekV3Bridge is a clean approach, leveraging the architectural similarities. The dependency bump incorporates numerous fixes from upstream. I have one suggestion regarding long-term maintainability of the model registration.
| try: | ||
| from megatron.bridge.models.conversion.model_bridge import MegatronModelBridge | ||
| from megatron.bridge.models.deepseek.deepseek_v3_bridge import DeepSeekV3Bridge | ||
| from megatron.bridge.models.mla_provider import MLAModelProvider | ||
| from megatron.core.models.gpt.gpt_model import GPTModel | ||
|
|
||
| @MegatronModelBridge.register_bridge( | ||
| source="Glm4MoeLiteForCausalLM", | ||
| target=GPTModel, | ||
| provider=MLAModelProvider, | ||
| model_type="glm4_moe_lite", | ||
| ) | ||
| class _GLM47FlashBridge(DeepSeekV3Bridge): | ||
| pass | ||
|
|
||
| except ImportError: | ||
| pass # megatron-bridge not installed (e.g. CPU-only environment) | ||
|
|
There was a problem hiding this comment.
This runtime registration is a good way to add support for the new model. To improve long-term maintainability and ensure this registration is available to the wider community, consider contributing this change upstream to the NVIDIA-NeMo/Megatron-Bridge repository. This would remove the need for this local patch in the future.
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
1 participant
Summary
Enables Megatron backend support for GLM-4.7-Flash (
zai-org/GLM-4.7-Flash) by registering its architecture with AutoBridge and bumping megatron-bridge to pick up 252 commits of fixes.Bridge registration
Glm4MoeLiteForCausalLMas a trivialDeepSeekV3Bridgesubclassmegatron-bridge bump
04e370ee(Jan 14 2026) →b058b662(HEAD, +252 commits)num_query_groupsmapping, MoE FlexDispatcher backend, memory savings for MoEparam_l2_norm