Bump vLLM to 0.16.0 and transformers to >=5.0.0 for GLM-4.7-Flash

[tyler-griggs]

Summary

Upgrades vLLM and transformers to support GLM-4.7-Flash (zai-org/GLM-4.7-Flash), which requires transformers>=5.0 for the glm4_moe_lite model type.

Version bumps

Package	Before	After
transformers	>=4.56.1,<5	>=5.0.0
vllm	0.13.0	0.16.0
torch	2.9.0	2.9.0 (unchanged)
flash-attn	2.8.x	2.8.x (unchanged)

PyTorch stays at 2.9.x, so flash-attn prebuilt wheels continue to work. No flash-attn stub or rebuild needed.

vLLM API migration (0.13 -> 0.15+)

The vllm.entrypoints.openai module was restructured from flat files into subdirectories:

• serving_chat -> chat_completion.serving
• serving_completion -> completion.serving
• serving_models -> models.serving
• protocol split into chat_completion.protocol, completion.protocol, engine.protocol

transformers 5.x API migration

apply_chat_template now returns BatchEncoding by default. Added return_dict=False to all 15 call sites that expect list[int]. 4 call sites that already used return_dict=True (for dict-based access to input_ids/assistant_masks) were left unchanged.

Tested on 4xL4 (vllm==0.15.1 + transformers==5.2.0)

• GLM-4.7-Flash config loads (model_type=glm4_moe_lite, 64 experts)
• All vLLM import paths resolve correctly
• apply_chat_template returns list with return_dict=False

Note on vLLM 0.16.0

vLLM 0.16.0 is tagged on GitHub (includes "Transformers v5 compatibility fixes") but not yet on PyPI. The pin targets 0.16.0; for local dev use 0.15.1 with force-installed transformers 5.x. The 0.16.0 pin will resolve once the package is published.

---

[gemini-code-assist]

Code Review

This pull request upgrades vLLM to version 0.16.0 and transformers to version 5.0.0 or higher. The changes primarily consist of necessary API migrations. Import paths in vllm.entrypoints.openai have been updated to reflect the new module structure. Additionally, return_dict=False has been added to all relevant calls of tokenizer.apply_chat_template to maintain the expected list[int] return type, which is a change in transformers v5. The dependency updates in the pyproject.toml files are also correct. The changes appear to be thorough and correct for the library upgrades.

[devin-ai-integration]

Devin Review found 2 potential issues.

View 5 additional findings in Devin Review.

[devin-ai-integration]

🔴 vLLM 0.16.0 pin in skyrl-train breaks old import paths in skyrl-train's own vllm_engine.py

The skyrl-train/pyproject.toml bumps vllm from 0.13.0 to 0.16.0 in both the vllm and mcore extras, but the corresponding skyrl-train/skyrl_train/inference_engines/vllm/vllm_engine.py was not updated with the new import paths. The PR description notes that vLLM 0.15+ restructured vllm.entrypoints.openai from flat files into subdirectories, and the skyrl/ copy of vllm_engine.py was correctly updated — but the skyrl-train/ copy was not.

Root Cause and Impact

The skyrl-train package at skyrl-train/skyrl_train/inference_engines/vllm/vllm_engine.py:14-23 still uses the old flat import paths:

from vllm.entrypoints.openai.serving_chat import OpenAIServingChat
from vllm.entrypoints.openai.serving_completion import OpenAIServingCompletion
from vllm.entrypoints.openai.serving_models import BaseModelPath, OpenAIServingModels
from vllm.entrypoints.openai.protocol import (
    ChatCompletionRequest, ChatCompletionResponse, ErrorResponse, ...
)

These modules no longer exist in vLLM 0.15+/0.16.0 — they moved to chat_completion.serving, completion.serving, models.serving, etc. Installing skyrl-train[vllm] or skyrl-train[mcore] will pull vllm==0.16.0 and immediately fail with ImportError when the engine is loaded.

Impact: Any user of skyrl-train[vllm] or skyrl-train[mcore] will hit an ImportError at import time, completely blocking vLLM inference.

Prompt for agents

Update skyrl-train/skyrl_train/inference_engines/vllm/vllm_engine.py to use the new vLLM 0.15+/0.16.0 import paths, matching the changes already made in skyrl/backends/skyrl_train/inference_engines/vllm/vllm_engine.py. Specifically, replace:

  from vllm.entrypoints.openai.serving_chat import OpenAIServingChat
  from vllm.entrypoints.openai.serving_completion import OpenAIServingCompletion
  from vllm.entrypoints.openai.serving_models import BaseModelPath, OpenAIServingModels
  from vllm.entrypoints.openai.protocol import (ChatCompletionRequest, ChatCompletionResponse, ErrorResponse, CompletionRequest, CompletionResponse)

With:

  from vllm.entrypoints.openai.chat_completion.serving import OpenAIServingChat
  from vllm.entrypoints.openai.completion.serving import OpenAIServingCompletion
  from vllm.entrypoints.openai.models.serving import BaseModelPath, OpenAIServingModels
  from vllm.entrypoints.openai.chat_completion.protocol import (ChatCompletionRequest, ChatCompletionResponse)
  from vllm.entrypoints.openai.completion.protocol import (CompletionRequest, CompletionResponse)
  from vllm.entrypoints.openai.engine.protocol import ErrorResponse

Also update any lazy imports of ErrorInfo inside the same file from vllm.entrypoints.openai.protocol to vllm.entrypoints.openai.engine.protocol.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[devin-ai-integration]

🔴 skyrl-train's apply_chat_template calls missing return_dict=False, will break with transformers 5.x

The skyrl-train/pyproject.toml bumps vllm to 0.16.0, and skyrl-tx/pyproject.toml bumps transformers to >=5.0.0. Since skyrl-tx depends on skyrl-train[vllm], transformers 5.x will be installed. In transformers 5.x, apply_chat_template returns BatchEncoding by default instead of list[int]. The skyrl/ copy of the files were updated with return_dict=False, but the skyrl-train/ copies were not.

Affected call sites in skyrl-train

The following files in skyrl-train/ still call apply_chat_template without return_dict=False while expecting list[int]:

1. skyrl-train/skyrl_train/dataset/dataset.py:61 — calls len() on the result, expecting a list
2. skyrl-train/skyrl_train/generators/skyrl_gym_generator.py:144 — assigns to self.base_conversation_token_ids, used with list operations like .index() and slicing
3. skyrl-train/skyrl_train/generators/skyrl_gym_generator.py:241 — assigns to initial_input_ids, uses len() on it
4. skyrl-train/skyrl_train/generators/skyrl_gym_generator.py:293 — assigns to agent_loop_state.input_ids
5. skyrl-train/skyrl_train/generators/skyrl_gym_generator.py:531 — slices the result with [len(...):]
6. skyrl-train/skyrl_train/generators/skyrl_gym_generator.py:611 — assigns to prompt_token_ids
7. skyrl-train/skyrl_train/generators/utils.py:158-162 — get_generation_prompt_ids slices result
8. skyrl-train/skyrl_train/generators/utils.py:446-459 — encode_messages_subset slices result

With transformers 5.x, these will receive BatchEncoding objects instead of list[int], causing TypeError or incorrect behavior when list operations are performed.

Impact: All non-batched multi-turn generation and dataset filtering in skyrl-train will break when transformers 5.x is installed (which happens via skyrl-tx[fsdp]).

Prompt for agents

Add return_dict=False to all apply_chat_template calls in skyrl-train that expect list[int] return type, mirroring the changes already made in the skyrl/ package. The affected files are:

1. skyrl-train/skyrl_train/dataset/dataset.py line 61: add return_dict=False to the apply_chat_template call
2. skyrl-train/skyrl_train/generators/skyrl_gym_generator.py lines 144, 241, 293, 531, 611: add return_dict=False to each apply_chat_template call (but NOT line 413 which correctly uses return_dict=True)
3. skyrl-train/skyrl_train/generators/utils.py lines 158, 161, 446, 454: add return_dict=False to each apply_chat_template call

Do NOT change calls that already use return_dict=True (like line 413 and inference_engine_client.py line 96 which access dict keys).

---

Was this helpful? React with 👍 or 👎 to provide feedback.