chore(deps): update bundled llama.cpp engine to v9946#310
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR contains the following updates:
b9860→b9946b9982(+29)Automated engine bump. The Engine Regression Gate is the hard check on this PR. If it reports a dylib closure change, check out this branch, run
bun run engine:ensure(it prints the exact frameworks diff), updatesrc-tauri/tauri.conf.jsonto match, and push.Release Notes
ggml-org/llama.cpp (ggml-org/llama.cpp)
vb9946Compare Source
Details
hexagon: tiling, tracing and optimizations for unary ops (#25474)
hexagon: tile wide rows in pointwise unary ops to avoid VTCM overflow
unary: reject permuted tensors for now (not used by models)
hex-unary: replace divs with fastdiv
hex-unary: add vtcm layout and host computed kernel params
hex-unary: move fastdiv init into kernel params
hex-unary: add specialized thread functions to improve generated code
hex-unary: tracing instrumentation for unary ops
hex-unary: factor out hvx kernels, streamline and remove more duplication
ggml-hexagon: fix std::min collision with Windows min macro
hex-cmake: make lto build happy
Co-authored-by: Max Krasnyansky maxk@qti.qualcomm.com
macOS/iOS:
Linux:
Android:
Windows:
openEuler:
UI:
vb9945Compare Source
Details
server : move chat-template thinking probe inside the init try/catch (#24093)
A model whose chat template parses at init but fails parser generation
at apply time (e.g. uses {% call %}) throws std::invalid_argument from
common_chat_templates_support_enable_thinking(), which ran outside the
try/catch guarding common_chat_templates_init(). The throw was uncaught
and llama-cli aborted (SIGABRT) instead of failing to load. Moved the
probe inside that try/catch so an apply-time error fails load the same
way an init parse error does.
Signed-off-by: Jesse LaRose jesse@taey.ai
macOS/iOS:
Linux:
Android:
Windows:
openEuler:
UI:
vb9941Compare Source
Details
Only index by compile times + always multiply/add (#25445)
The first one avoids relying on compile to optimize local memory away,
and the second is cheaper than issuing control flow statements
macOS/iOS:
Linux:
Android:
Windows:
openEuler:
UI:
vb9940Compare Source
Details
llama-bench : init params.offline (#25476)
Signed-off-by: Adrien Gallouët angt@huggingface.co
macOS/iOS:
Linux:
Android:
Windows:
openEuler:
UI:
vb9939Compare Source
Details
metal : add CONV_2D_DW (depthwise convolution) support (#21565)
metal : add CONV_2D_DW (depthwise 2D convolution) support
test : add perf cases for CONV_2D_DW
metal : use 3D dispatch for CONV_2D_DW kernel
metal : add channel-tiled CONV_2D_DW kernel for non-contiguous layouts
metal : simplify CONV_2D_DW dispatch and trim comments
metal : merge duplicate CONV_2D_DW pipeline getters
tests : add F16 CONV2D_DW tests
cpu : fix F16 kernel support for CONV_2D_DW
tests : remove commented-out CONV_2D_DW test block
Co-authored-by: Georgi Gerganov ggerganov@gmail.com
macOS/iOS:
Linux:
Android:
Windows:
openEuler:
UI:
vb9938Compare Source
Details
ggml-hip: enable -funsafe-math-optimizations (#24668)
CUDA is compiled with fast math and AMD/HIP is not — this flag lets AMD use fast math too.
We can't use -ffast-math: it implies -ffinite-math-only, which won't compile (ggml uses INFINITY for masking) and produces NaNs. -funsafe-math-optimizations gives the speedup without the NaN problems.
Co-authored-by: Mark Caldwell mark@cloudhands.ai
macOS/iOS:
Linux:
Android:
Windows:
openEuler:
UI:
vb9937Compare Source
Details
cuda: align snake fusion matcher with the other backends (#25460)
The matcher required a->type == x->type while launch_snake reads both
as const float *, matching the CPU and Metal contract where a and inv_b
stay F32. F16/BF16 chains never fused and fell back to the naive path,
and a hypothetical all F16 chain would have read F16 bits as float.
Aligns the predicate and the comment with ggml-cpu.c
The kernel reads x[idx] and a[c] / inv_b[c] linearly, so a
non-contiguous view passing the matcher would silently read wrong data.
Mirror the contiguity guard already present in the CPU, Vulkan and
Metal matchers.
macOS/iOS:
Linux:
Android:
Windows:
openEuler:
UI:
vb9936Compare Source
Details
server : respect min-step when splitting prompt batches (#25420)
macOS/iOS:
Linux:
Android:
Windows:
openEuler:
UI:
vb9935Compare Source
Details
hexagon: add VISION RoPE support (#25216)
hexagon: add VISION RoPE support
hexagon: support RoPE on strided half-dim views for all modes
hex-rope: decouple src0 DMA copy size from row stride
hex-rope: support non-contiguous dst for RoPE
hex-rope: fix dst spad pitch for non-contiguous dst
macOS/iOS:
Linux:
Android:
Windows:
openEuler:
UI:
vb9934Compare Source
Details
ggml-webgpu: tune subgroup split (d_split) in flash_attn_vec (#25418)
macOS/iOS:
Linux:
Android:
Windows:
openEuler:
UI:
vb9933Compare Source
Details
opencl: Q6_K GEMM/GEMV fix for ne01 of weights that are not multiples of 128. (#25464)
Observed with granite-3.1-3b-a800m-instruct, whose vocab is an odd number.
Route Q6_K dense mul_mat with ne01 % 128 != 0 off the noshuffle path:
decode (ne1==1) uses the correct flat GEMV and the matching GEMM (ne1>1)
falls back to CPU (the flat convert has no verified small-batch GEMM kernel
for these shapes). All standard hidden/FFN/vocab dims are multiples of 128
and keep the noshuffle path.
set_tensor carves quantized weights into per-component subbuffers (d/q,
ql/qh/s/d, ...) whose origins are each rounded up to the device base
address alignment. When a component's size is not a multiple of the
alignment, the carve extends past ggml_nbytes(tensor) and the last
subbuffer overlaps the next tensor in the pool -- e.g. q6_K [1536, 49155]:
size_s = 49155*96 ends 32 bytes past a 128-byte boundary, so the d
subbuffer ends 96 bytes past the tensor's allocation, and whichever of the
two neighboring tensors is uploaded last silently corrupts the other (here:
the last vocab rows' block scales). This affects any quant type whose
component sizes can be misaligned, on any shape with ne01 not a multiple of
the alignment granularity; standard power-of-two dims are unaffected.
Implement get_alloc_size for the OpenCL buffer type and reserve the
worst-case carve slack (4 aligned gaps; 5 components max, q5_K) for
quantized tensors. Costs at most 512 bytes per quantized tensor at the
observed 128-byte alignment.
Co-authored-by: Li He lih@qti.qualcomm.com
macOS/iOS:
Linux:
Android:
Windows:
openEuler:
UI:
vb9932Compare Source
Details
vulkan: disable FA mask_opt on GCN to improve performance (#24362)
vulkan: disable FA mask_opt on GCN to improve performance
reenable mask opt over attention head size 256
macOS/iOS:
Linux:
Android:
Windows:
openEuler:
UI:
vb9931Compare Source
Details
opencl: ragged-tile MoE prefill FP16 GEMM optimization (skip padded expert tiles) (#25433)
The MoE prefill GEMM groups tokens into TILESIZE_N=32 per-expert tiles; at low
tokens-per-expert most tiles are mostly padding. When a tile's upper 16 slots
are all padding (router index 0xFFFFFFFF), skip the second dotx16_reduce8 half.
Numerically identical (skipped lanes are padding). Applied to all eight *_f32_ns
MoE GEMMs; default on, opt out with GGML_OPENCL_MOE_RAGGED_FP16=0.
Replace the two half-tile dotx16_reduce8 calls in the 8 *_f32_ns MoE GEMMs with
four dotx8_reduce4 (8-column) calls, skipping each empty trailing skip-group
independently. Padding is always trailing, so the kernel rounds the valid count
up to the skip granularity and skips fully-padding groups. Byte-identical to the
non-skipped path. New env GGML_OPENCL_MOE_RAGGED_GRAN={8,16,32} (quarter/half/
off); default quarter.
Co-authored-by: Li He lih@qti.qualcomm.com
macOS/iOS:
Linux:
Configuration
📅 Schedule: (UTC)
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR is behind base branch, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.
This PR was generated by Mend Renovate. View the repository job log.