feat: add hybridep

[hemildesai]

Wandb - https://wandb.ai/Nemo-automodel/automodel-moe-dispatcher

This PR adds HybridEP support for MoE token dispatch and updates dependency/docker wiring needed to run it reliably.

Changelog

• Add HybridEP backend support to MoE flex token dispatch:
  • Introduce _HybridEPManager in nemo_automodel/components/moe/megatron/token_dispatcher.py.
  • Add moe_flex_dispatcher_backend ("deepep" or "hybridep"), plus backend-specific SM settings.
  • Add HybridEP preprocessing path that converts top-k indices to multihot routing metadata.
• Extend fused all-to-all utilities in nemo_automodel/components/moe/megatron/fused_a2a.py:
  • Add set_deepep_num_sms.
  • Add HybridEP dispatch/combine autograd wrappers and buffer initialization/reset helpers.
• Extend backend config and MoE wiring:
  • BackendConfig.dispatcher now accepts "hybridep".
  • Add dispatcher_num_sms.
  • Treat "hybridep" as valid with te/gmm experts.
  • Pass dispatcher backend + SM settings through MoE, GroupedExpertsDeepEP, and GroupedExpertsTE.
• Add unit tests for HybridEP paths:
  • tests/unit_tests/moe/test_backend_config.py
  • tests/unit_tests/moe/test_experts.py
  • tests/unit_tests/moe/test_layers.py
• Update dependencies:
  • Bump deep_ep to 7febc6e25660af0f54d95dd781ecdcd62265ecca (v1.2.1+7febc6e metadata).
  • Add Linux-only override: nvidia-cudnn-cu12==9.19.0.56; sys_platform == 'linux'.
  • Update uv.lock accordingly.
• Fix Docker update behavior:
  • docker/common/update_pyproject_pytorch.sh now removes existing override-dependencies and reinserts docker/common/uv-pytorch.toml under
    [tool.uv] to avoid duplicate/incorrect override blocks.

Additional Information

• Closes Support HybridEP #891

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[hemildesai]

/ok to test 8d0be12

[hemildesai]

/ok to test 6233747