Hexagon: Add support for CONCAT operation#58
Conversation
This adds the CONCAT operator to the Hexagon backend. It supports F32 and F16 and uses DMA to process blocks of memory within double-buffered VTCM. It handles generalized strides and transposed tensors via block splitting. Co-authored-by: max-krasnyansky <1380796+max-krasnyansky@users.noreply.github.com>
|
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with New to Jules? Learn more at jules.google/docs. For security, I will only act on instructions from the user who triggered this task. |
This PR adds support for the CONCAT operator on the Hexagon backend. It properly pipes the operation from ggml-hexagon.cpp down to the HTP context. `concat-ops.c` utilizes DMA and double-buffering ping-pong logic inside VTCM for maximum efficiency when dealing with non-contiguous and transposed matrices. The VTCM buffer fallback allocates based on available size. Co-authored-by: max-krasnyansky <1380796+max-krasnyansky@users.noreply.github.com>
This PR adds support for the CONCAT operator on the Hexagon backend. It properly pipes the operation from ggml-hexagon.cpp down to the HTP context. `concat-ops.c` utilizes DMA and double-buffering ping-pong logic inside VTCM for maximum efficiency when dealing with non-contiguous and transposed matrices. The VTCM buffer fallback allocates based on available size. Co-authored-by: max-krasnyansky <1380796+max-krasnyansky@users.noreply.github.com>
This PR adds support for the CONCAT operator on the Hexagon backend. It properly pipes the operation from ggml-hexagon.cpp down to the HTP context. `concat-ops.c` utilizes DMA and double-buffering ping-pong logic inside VTCM for maximum efficiency when dealing with non-contiguous and transposed matrices. The inner loops correctly use precomputed `fastdiv_values` and `fastmodulo` to calculate dimensions without expensive scalar divisions. It verifies the VTCM layout to ensure allocations do not overflow limits. Co-authored-by: max-krasnyansky <1380796+max-krasnyansky@users.noreply.github.com>
1 participant
This PR adds support for the
CONCAToperation to the Hexagon DSP backend in ggml. It properly maps the operation from CPU down to the Hexagon graph format, implements support checks forF32andF16typed tensors, and performs the actual concatenation in VTCM memory using Hexagon's DMA queues.The implementation handles transposed and arbitrarily strided tensors efficiently by decomposing the operation into chunks and executing
dma_queue_push_single_2dto extract exact memory slices. The operations are threaded utilizing the backend's worker pool logic.PR created automatically by Jules for task 9193647139875318755 started by @max-krasnyansky