Skip to content

LFM2 parser #4085

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
przepeck wants to merge 21 commits into
base: main
Choose a base branch
from
+1,259 −0
Open

LFM2 parser #4085

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
d10ef38
save
przepeck Mar 10, 2026
cd90b91
save
przepeck Mar 11, 2026
087d054
save
przepeck Mar 12, 2026
2f6d306
save
przepeck Mar 12, 2026
f6e900d
save
przepeck Mar 13, 2026
5cdcd2e
build fix
przepeck Mar 18, 2026
d835022
save
przepeck Mar 19, 2026
7af29b0
save
przepeck Mar 20, 2026
53cc07c
save
przepeck Mar 20, 2026
c2bda14
save
przepeck Mar 23, 2026
70b063b
streaming
przepeck Mar 24, 2026
1782e6a
fix and test
przepeck Mar 24, 2026
e7d8581
save
przepeck Mar 24, 2026
f69fed0
save
przepeck Mar 25, 2026
bb6366d
improvements
przepeck Mar 25, 2026
daf83e8
Style
przepeck Mar 25, 2026
d481a5a
Apply suggestions from code review
przepeck Mar 25, 2026
7076415
fixes
przepeck Mar 25, 2026
93806de
clang-format
przepeck Mar 25, 2026
3f32e01
cpplint
przepeck Mar 25, 2026
15d8c5f
adding model to script, enabling tests
przepeck Mar 25, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions prepare_llm_models.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ PHI4_MODEL="microsoft/Phi-4-mini-instruct"
MISTRAL_MODEL="mistralai/Mistral-7B-Instruct-v0.3"
GPT_OSS="openai/gpt-oss-20b"
DEVSTRAL_MODEL="unsloth/Devstral-Small-2507"
LFM2_MODEL="LiquidAI/LFM2-2.6B"

if [ "$(python3 -c 'import sys; print(sys.version_info[1])')" -le "8" ]; then echo "Prepare models with python > 3.8."; exit 1 ; fi

Expand Down Expand Up @@ -216,3 +217,14 @@ if [ ! -f "$1/$DEVSTRAL_MODEL/$TOKENIZER_FILE" ]; then
echo "[ERROR] Models file $1/$DEVSTRAL_MODEL/$TOKENIZER_FILE does not exist."
exit 1
fi

if [ -f "$1/$LFM2_MODEL/$TOKENIZER_FILE" ]; then
echo "Models file $1/$LFM2_MODEL/$TOKENIZER_FILE exists. Skipping downloading models."
else
mkdir -p $1/$LFM2_MODEL
convert_tokenizer $LFM2_MODEL --with_detokenizer -o $1/$LFM2_MODEL
fi
if [ ! -f "$1/$LFM2_MODEL/$TOKENIZER_FILE" ]; then
echo "[ERROR] Models file $1/$LFM2_MODEL/$TOKENIZER_FILE does not exist."
exit 1
fi
19 changes: 19 additions & 0 deletions src/llm/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,24 @@ ovms_cc_library(
],
visibility = ["//visibility:public"],
)

ovms_cc_library(
name = "io_processing_lfm2_tool_parser",
hdrs = ["io_processing/lfm2/tool_parser.hpp"],
srcs = ["io_processing/lfm2/tool_parser.cpp"],
deps = [
"@com_github_tencent_rapidjson//:rapidjson",
"//src:libovmslogging",
"//src:libovmsstatus",
"//src/utils:rapidjson_utils",
":io_processing_utils",
":io_processing_base_output_parser",
":apis_tool_schema_wrapper",
Comment on lines +142 to +146
Copy link

Copilot AI Mar 25, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Bazel target io_processing_lfm2_tool_parser lists several deps that aren't used by this parser (//src:libovmsstatus, //src/utils:rapidjson_utils, :apis_tool_schema_wrapper). Keeping unused deps increases build times and dependency surface; please remove any deps that aren't required for compilation/linking.

Suggested change
"//src:libovmsstatus",
"//src/utils:rapidjson_utils",
":io_processing_utils",
":io_processing_base_output_parser",
":apis_tool_schema_wrapper",
":io_processing_utils",
":io_processing_base_output_parser",

Copilot uses AI. Check for mistakes.
"//third_party:genai",
],
visibility = ["//visibility:public"],
)

ovms_cc_library( # TODO split further so we don't have to recompile everything when changing one parser ...
name = "output_parsers",
hdrs = [
Expand Down Expand Up @@ -166,6 +184,7 @@ ovms_cc_library( # TODO split further so we don't have to recompile everything w
":partial_json_builder",
":io_processing_base_output_parser",
":io_processing_qwen3coder_tool_parser",
":io_processing_lfm2_tool_parser",
":io_processing_utils",
":apis_tool_schema_wrapper",
],
Expand Down
Loading