[中文版|English]
Important
2026/03 UPDATE
v1.0.0 is now officially released as the first stable version.
The codebase has been significantly refactored since v1.0.0-alpha.0.
The hardware-specific (multi-chip) support has been moved into plugin repositories such as
TransformerEngine-FL and
vllm-plugin-FL.
These plugins build on top of FlagOS, a unified open-source AI system software stack.
If you are using or upgrading from a version earlier than v1.0.0-alpha.0, please use the
main-legacy branch.
It will continue to receive critical bug fixes and minor updates for a period of time.
FlagScale is a core component of FlagOS — a unified, open-source AI system software stack that fosters an open technology ecosystem by seamlessly integrating various models, systems, and chips. Following the principle of "develop once, migrate across various chips", FlagOS aims to unlock the full computational potential of hardware, break down barriers between different chip software stacks, and effectively reduce migration costs.To get started, see the Getting Started Guide.
As the central toolkit of this ecosystem, FlagScale provides a unified interface covering the complete lifecycle of large language models, multimodal models, and embodied AI models. It integrates multiple open-source backend engines under a single configuration and CLI interface, supporting key workflows including model training, reinforcement learning, and inference — with consistent operation across diverse chip vendors.
Within the FlagOS ecosystem, FlagScale works together with several other components:
- FlagOS Plugins – hardware-adapted integrations of upstream AI frameworks
- FlagCX – a scalable and adaptive cross-chip communication library
- FlagOS-Robo related – infrastructure for embodied AI workloads
FlagOS plugin projects are built on top of widely used upstream open-source frameworks and extend them to support multiple AI chips. These plugins provide hardware compatibility and runtime integration for training, reinforcement learning, and inference.
The following table lists the mapping between FlagOS plugins and their corresponding upstream projects.
| Task | FlagOS Plugin Projects | Upstream Projects |
|---|---|---|
| Training | Megatron-LM-FL TransformerEngine-FL |
Megatron-LM TransformerEngine |
| Reinforcement Learning | VeRL-FL | veRL |
| Serve / Inference | vllm-plugin-FL | vllm |
| Model | Example config File |
|---|---|
| DeepSeek-V3 | 16b_a3b.yaml |
| Qwen2/2.5/3 | 235b_a22b.yaml |
| Qwen2.5-VL | 7b.yaml |
| QwQ | 32b.yaml |
| LLaMA2 | 7b.yaml |
| LLaMA3/3.1 | 70b.yaml |
| LLaVA-OneVision | 7b.yaml |
| LLaVA1.5 | 7b.yaml |
| Mixtral | 8x7b.yaml |
| RWKV | 7b.yaml |
| Aquila | 7b.yaml |
| ... | ... |
| Model | Example config File |
|---|---|
| DeepSeek-V3 | 671b.yaml |
| DeepSeek-R1 | 671b.yaml |
| Qwen2.5 | 72b.yaml |
| Qwen3 | 8b.yaml |
| Qwen2.5-VL | 32b_instruct.yaml |
| Qwen3-Omni | 30b.yaml |
| QwQ | 32b.yaml |
| Grok2 | 270b.yaml |
| Kimi-K2 | 1t.yaml |
| ... | ... |
join our WeChat Group
This project is licensed under the Apache License (Version 2.0). This project also contains other third-party components under other open-source licenses.