Skip to content

Fix: update app for mlx-swift-lm 3.x compatibility#78

Open
tmorales2000 wants to merge 4 commits into
apple:mainfrom
tmorales2000:fix/mlx-swift-lm-3x
Open

Fix: update app for mlx-swift-lm 3.x compatibility#78
tmorales2000 wants to merge 4 commits into
apple:mainfrom
tmorales2000:fix/mlx-swift-lm-3x

Conversation

@tmorales2000

Copy link
Copy Markdown

Fixes #77

The FastVLM app fails to build against current package versions due to breaking
changes in mlx-swift-lm 3.x and the migration of MLXLMCommon/MLXVLM out of
mlx-swift-examples into a new repository.

Changes

Package dependencies:

  • Replace mlx-swift-examples with mlx-swift-lm 3.x
  • Add swift-huggingface 0.9.x
  • Update swift-transformers to 1.3.x (exposes Tokenizers as standalone product)

FastVLM.swift:

  • Update attention mask type to ScaledDotProductAttentionMaskMode
  • Fix createAttentionMask signature (windowSize parameter, single cache)
  • Fix LoRAModel protocol conformance (loraLayers, loraDefaultKeys)
  • Fix UserInput.Prompt handling (asMessages() removed in 3.x)
  • Fix LMInput.ProcessedImage (imageGridThw -> frames)
  • Fix ModelConfiguration loading API (url -> data in registry closures)
  • Fix register() actor isolation (async, await)
  • Qualify Tokenizer as MLXLMCommon.Tokenizer to resolve ambiguity

FastVLMModel.swift:

  • Fix MLX.GPU.set(cacheLimit:) -> MLX.Memory.cacheLimit
  • Fix loadContainer API (local directory + huggingFaceTokenizerLoader)
  • Fix generate() closure (single Int token, accumulator pattern)
  • Fix decode() label (tokens: -> tokenIds:)
  • Fix GenerateCompletionInfo (output property removed)

Scripts:

  • Replace wget with curl in get_models.sh and app/get_pretrained_mlx_model.sh

Testing

Tested on macOS with M1 Pro and M4 Pro:

  • mlx-swift 0.25.6
  • mlx-swift-lm 3.31.3
  • swift-transformers 1.3.3
  • swift-huggingface 0.9.0

App builds and runs successfully with 0.5B (FP16) and 1.5B (INT8) models.

The mlx-swift-lm libraries (MLXLMCommon, MLXVLM) moved from
mlx-swift-examples to a new repository in late 2025, breaking
the app against current package versions.

Package changes:
- Replace mlx-swift-examples with mlx-swift-lm 3.x
- Add swift-huggingface 0.9.x
- Update swift-transformers to 1.3.x (exposes Tokenizers as standalone product)

FastVLM.swift:
- Update attention mask type to ScaledDotProductAttentionMaskMode
- Fix createAttentionMask signature (windowSize parameter, single cache)
- Fix LoRAModel protocol conformance (loraLayers, loraDefaultKeys)
- Fix UserInput.Prompt handling (asMessages() removed in 3.x)
- Fix LMInput.ProcessedImage (imageGridThw -> frames)
- Fix ModelConfiguration loading API (url -> data in registry closures)
- Fix register() actor isolation (async, await)
- Add @preconcurrency import Tokenizers
- Qualify Tokenizer as MLXLMCommon.Tokenizer to resolve ambiguity

FastVLMModel.swift:
- Fix MLX.GPU.set(cacheLimit:) -> MLX.Memory.cacheLimit
- Fix loadContainer API (local directory + huggingFaceTokenizerLoader)
- Fix generate() closure (single Int token, accumulator pattern)
- Fix decode() label (tokens: -> tokenIds:)
- Fix GenerateCompletionInfo (output property removed)

Tested on macOS with M1 Pro, mlx-swift 0.25.6, mlx-swift-lm 3.31.3,
swift-transformers 1.3.3, swift-huggingface 0.9.0.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

App fails to build against current mlx-swift-lm packages (3.x compatibility)

1 participant