Skip to content

fix(glm4v): version-aware hidden_size/vocab_size for Glm4vConfig#1214

Open
1fanwang wants to merge 1 commit intolinkedin:mainfrom
1fanwang:fix/glm4v-config-attribute-access-1152
Open

fix(glm4v): version-aware hidden_size/vocab_size for Glm4vConfig#1214
1fanwang wants to merge 1 commit intolinkedin:mainfrom
1fanwang:fix/glm4v-config-attribute-access-1152

Conversation

@1fanwang
Copy link
Copy Markdown
Member

@1fanwang 1fanwang commented May 5, 2026

Reading config.hidden_size and config.vocab_size directly in glm4v.py and glm4v_moe.py fails on transformers v5+, where Glm4vConfig and Glm4vMoeConfig are composite (text_config + vision_config) and the top-level fields are no longer present. Reported in #1152:

File ".../liger_kernel/transformers/model/glm4v.py", line 130, in lce_forward
    hidden_size=self.config.hidden_size,
AttributeError: 'Glm4vConfig' object has no attribute 'hidden_size'

Same shape PR #1062 already fixed for Qwen2-VL / Qwen2.5-VL — mirror that helper (_get_hidden_size, _get_vocab_size) gated on transformers >= 5.0.0, applied to both glm4v.py and glm4v_moe.py. Pre-v5 still works because the helpers fall back to the top-level attribute. Helper signatures are typed (config: PretrainedConfig -> int) and the version flag is annotated bool.

Reproducer (transformers 5.7.0)

from transformers import Glm4vConfig
cfg = Glm4vConfig(text_config={"hidden_size": 1024, "vocab_size": 32000})
cfg.text_config.hidden_size   # 1024
cfg.hidden_size               # AttributeError: 'Glm4vConfig' object has no attribute 'hidden_size'

Same shape reproduces for Glm4vMoeConfig. Identical to the trace in #1152.

After the helper change, both _get_hidden_size(cfg) and _get_vocab_size(cfg) route through text_config on v5+ and fall back to the top-level attribute pre-v5:

transformers=5.7.0  v5_or_later=True
_get_hidden_size(Glm4vConfig)    = 1024
_get_vocab_size(Glm4vConfig)     = 32000
_get_hidden_size(Glm4vMoeConfig) = 1024
_get_vocab_size(Glm4vMoeConfig)  = 32000

Tests

End-to-end coverage already in place in test/convergence/bf16/test_mini_models.py and test/convergence/fp32/test_mini_models.py — both mini_glm4v and mini_glm4v_moe configs construct via the text_config sub-dict (lines 1245–1308 / 1311–1368), so CI exercises the fixed path on v5+ runners. No new test file added — matches the precedent set by PR #1062 (qwen2_vl fix), which also relied on the existing convergence tests rather than introducing a helper-level test layer.

Closes #1152.

@1fanwang 1fanwang force-pushed the fix/glm4v-config-attribute-access-1152 branch from cd91744 to 3d2266b Compare May 5, 2026 08:44
…kedin#1152)

Reading config.hidden_size and config.vocab_size directly fails on
transformers v5+, where Glm4vConfig is a composite (text_config +
vision_config) and the top-level fields are removed. Mirror the helper
pattern introduced by linkedin#1062 for Qwen2-VL/Qwen2.5-VL.

Closes linkedin#1152.
@1fanwang 1fanwang force-pushed the fix/glm4v-config-attribute-access-1152 branch from 3d2266b to d7915ae Compare May 5, 2026 08:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Glm4vConfig object has no attribute 'hidden_size'

1 participant