Rework runtime and add Mppa target by ElectrikSpace · Pull Request #47 · xtc-tools/xtc

ElectrikSpace · 2026-02-12T17:30:30Z

This PR rework the runtime to facilitate the support of new targets, especially accelerators.

It also proposes an integration of the Kalray Mppa target.

ElectrikSpace · 2026-02-13T13:54:32Z

PR #49 is required to run the CI on MacOS.

ElectrikSpace · 2026-02-16T12:44:02Z

Update SDist requirements

guillon · 2026-03-02T18:32:01Z

@ElectrikSpace I do not understand the status of this review. The description seems to still depend on a review which was merged and specify to look only at the last commit. Though it seems that you rebased it already. But there are still 5 commits.
Can you rebase and update the description?

ElectrikSpace · 2026-03-03T09:46:13Z

I didn't update the description because the status of the dependent PR has been set to merged automatically, but I can remove it

guillon

Thanks for the proposal, it's very cool.

In addition to inline comments:

you may add to docs/develop/optional_backends.md in the sdist section possibly the way to test for mppa as we do not have automation there yet, I can verify if I can run it on our local machine.
also for the nvgpu, it seems that there are tests missing, can you add there also a section for this target and how to run it? I may try to run it on a grid 5000 machine with GPU.

src/xtc/itf/runtime/common.py

src/xtc/itf/runtime/embedded.py

src/xtc/runtimes/host/__init__.py

src/xtc/targets/accelerator/gpu/GPUModule.py

src/xtc/utils/numpy.py

tests/filecheck/backends/target_mppa/test_matmul_mlir_offload_tensor.py

Makefile

This goal of the work on the runtime is to facilitate the future integration of non-host targets. New features are also added, especially for accelerators. Create common interfaces that runtimes will need to implement. This patch introduces a runtime base named CommonRuntimeInterface which is common among three derives classes of runtimes: - Host (no derived class for now) - AcceleratorDevice for accelerators. Instance of this class are called devices. - EmbeddedDevice for external embedded processors. Instances of AcceleratorDevice and EmbeddedDevice are called devices. Unlike the current lazy runtime resolution, the concept of devices will allow to handle multiples accelerator of the same class.

Apply the new runtime interfaces on the two existing runtimes: - Create HostRuntime singleton class that derives from CommonRuntimeInterface - Create GPUDevice singleton (for now) class that derives from AcceleratorDevice to implement the GPU runtime. Some method implementations are shared accross the two, but adding a specific implemention for a particular runtime is now easier. The GPU target has been completely split from the Host target. To prevent code duplication due to a clear split between Host and GPU runtimes, common code portions have been factorized in utils. With this rework of the runtimes, the call path has been simplified, confusing classes like Executor/Evaluator and functions like load_and_evaluate have been removed.

In the context on computation offloaded on a accelerator device, the user can specify where input/output tensors live when the evaluation begins. This allows to simulate weights tensors to be transfered ahead of time. This feature is only supported for the MLIR backend, but setting a "memref.on_device" attribute.

ElectrikSpace · 2026-03-11T13:36:54Z

you may add to docs/develop/optional_backends.md in the sdist section possibly the way to test for mppa as we do not have automation there yet, I can verify if I can run it on our local machine.

Done

also for the nvgpu, it seems that there are tests missing, can you add there also a section for this target and how to run it? I may try to run it on a grid 5000 machine with GPU.

There are not special primitives used only on GPUs (not yet), so GPU tests files are exactly the same as host ones for now. But it would be nice to run GPU tests in CI (or maybe wait until we get real performances on GPUs)

Create a new Mppa compilation target for MLIR. This target is the first one to implement the ahead of time offloading of tensors on device. Create a new Mppa runtime derived from AcceleratorDevice. The runtime can be configured on various aspect (check MppaConfig class). Execution is supported on ISS, Qemu, and Hardware. Note: In order to use the Mppa target, the Kalray Core Toolchain must be installed. mlir_sdist and mlir_mppa must also be installed.

Rely on the kvxuks-catch pass to catch micro-kernels in replacement of the transform dialect based vectorization.

guillon

This review was generated by Father, Guils' AI assistant.

Review: #47

Summary

This PR adds a substantial MPPA target/runtime path and refactors evaluation/runtime plumbing to share more code across host and accelerator backends. The overall direction makes sense, but I found two correctness regressions that should be addressed before merge.

Findings

[high] Device-aware NDArray creation breaks existing GPU evaluation path

Where

src/xtc/utils/evaluation.py:89-107
src/xtc/runtimes/types/ndarray.py:84-105,159-164
src/xtc/runtimes/accelerator/gpu/GPUDevice.py:126-149

Why this is a problem
ensure_ndarray_parameters() now propagates spec["device"] into NDArray(...). For accelerator-backed tensors, NDArray.__init__() immediately calls _to_device(), and later NDArray.data / NDArray.numpy() use the accelerator memory APIs.

That is fine for MPPA because MppaDevice implements memory_allocate, memory_copy_to, memory_copy_from, and memory_data_pointer. But it regresses the pre-existing GPU path: GPUDevice still raises NotImplementedError for all of those methods.

So any graph/spec that now carries device=gpu will fail before execution even starts. In practice, this change converts what used to be host-backed + registered buffers in GPUEvaluator into true device-backed NDArrays, but the GPU runtime was not updated to support that contract.

Impact
Existing GPU evaluation/offload flows become unusable as soon as tensor specs propagate a GPU device through the new generic evaluation helpers.

Actionable fix
Pick one model and make it consistent:

Short-term safe fix: keep GPU on the old host-buffer model in ensure_ndarray_parameters() (only create device-backed NDArrays for accelerators that fully implement the memory API, e.g. MPPA).
Long-term fix: implement GPUDevice.memory_allocate/free/copy_to/copy_from/memory_data_pointer and then remove the manual host-buffer registration path from GPUEvaluator.

Until one of those is done, the generic device propagation introduced here is incomplete.

[medium] MPPA codegen builds intermediates in the current working directory instead of the chosen temp/output directory

Where

src/xtc/backends/mlir/MlirTarget/MlirMppaTarget.py:78-85,94-145

Why this is a problem
The method computes dump_tmp_file = f"{dump_tmp_dir}/{dump_base}", but the actual intermediate filenames are built as bare relative names:

f"{dump_base}.before_mppa.mlir"
f"{dump_base}.after_mppa.mlir"
f"{dump_base}.host.c"
f"{dump_base}.accelerator.c"
etc.

Those files are then opened, compiled, and deleted via relative paths. That means they land in the process CWD rather than dump_tmp_dir, unlike the other MLIR targets in this tree which use dump_tmp_file for intermediates.

Impact

unexpected file leakage into the caller’s CWD
collisions between concurrent builds using the same payload name
failures when the CWD is not writable
tempfile.mkdtemp() is effectively ignored for most intermediates

Actionable fix
Construct the intermediate paths from dump_tmp_file, e.g.:

mlir_bmppa_dump_file = f"{dump_tmp_file}.before_mppa.mlir"
mlir_amppa_dump_file = f"{dump_tmp_file}.after_mppa.mlir"
c_host_dump_file = f"{dump_tmp_file}.host.c"
etc.

That would align MPPA with the path handling already used by the C/LLVM/NVGPU targets.

Notes

I also noticed some unsafe shell/path handling in the new MPPA config/runtime (os.system("rm -r " + path) style cleanup/build commands), but I’m treating that as follow-up hardening rather than a merge blocker for this PR.
I did not modify repository code; this was a read-only review.

guillon · 2026-03-12T11:27:51Z

src/xtc/utils/evaluation.py

+                    out,
+                    runtime=spec["device"] if "device" in spec else HostRuntime.get(),
+                )
+                for out, spec in zip(outputs, outputs_spec)


[Father] [high] Device-aware NDArray creation breaks existing GPU evaluation path — ensure_ndarray_parameters() now propagates spec["device"] into NDArray(...).

See the main review comment for full details and recommended fixes.

guillon · 2026-03-12T11:27:52Z

src/xtc/runtimes/types/ndarray.py

+        if self.is_on_device():
+            assert isinstance(self.runtime, AcceleratorDevice)
+            return self.runtime.memory_data_pointer(self.device_handle)
        return self.handle.contents.dl_tensor.data


[Father] [high] Device-aware NDArray creation breaks existing GPU evaluation path — ensure_ndarray_parameters() now propagates spec["device"] into NDArray(...).

See the main review comment for full details and recommended fixes.

guillon · 2026-03-12T11:27:53Z

src/xtc/runtimes/accelerator/gpu/GPUDevice.py

+    def memory_data_pointer(self, acc_handle: Any) -> ctypes.c_void_p:
+        raise NotImplementedError(
+            "memory_data_pointer is not implemented for GPU device"
+        )


[Father] [high] Device-aware NDArray creation breaks existing GPU evaluation path — ensure_ndarray_parameters() now propagates spec["device"] into NDArray(...).

See the main review comment for full details and recommended fixes.

guillon · 2026-03-12T11:27:54Z

src/xtc/backends/mlir/MlirTarget/MlirMppaTarget.py

+            os.remove(mlir_bmppa_dump_file)
+            os.remove(mlir_amppa_dump_file)
+            os.remove(c_host_dump_file)
+            os.remove(c_accelerator_dump_file)


[Father] [medium] MPPA codegen builds intermediates in the current working directory instead of the chosen temp/output directory — The method computes dump_tmp_file = f"{dump_tmp_dir}/{dump_base}", but the actual intermediate filenames are built as bare relative names: - f"{dump_base}.before_mppa.mlir" - f"{dump_base}.after_mppa.mlir" - f"{dump_base}.host.c" -

See the main review comment for full details and recommended fixes.

Indeed seems to not be generated in tmp_dir when dump_file is not none.

guillon

Seems to still have the issue with tmp files not in a tmpdir when: dump_file is not None and save_temps is False.

guillon · 2026-03-12T11:36:57Z

src/xtc/backends/mlir/MlirTarget/MlirMppaTarget.py

+            os.remove(mlir_bmppa_dump_file)
+            os.remove(mlir_amppa_dump_file)
+            os.remove(c_host_dump_file)
+            os.remove(c_accelerator_dump_file)


Indeed seems to not be generated in tmp_dir when dump_file is not none.

ElectrikSpace force-pushed the dev/snoiry/mppa_target branch 2 times, most recently from 8718fbd to bfb8226 Compare February 13, 2026 13:33

ElectrikSpace marked this pull request as ready for review February 13, 2026 13:34

ElectrikSpace force-pushed the dev/snoiry/mppa_target branch from bfb8226 to 0910617 Compare February 16, 2026 12:43

ElectrikSpace force-pushed the dev/snoiry/mppa_target branch 2 times, most recently from 6f1ffe9 to eaac320 Compare February 19, 2026 13:34

ElectrikSpace mentioned this pull request Feb 19, 2026

[runtime/mppa] Add Support for Mppa PMU counters #53

Open

ElectrikSpace self-assigned this Feb 19, 2026

ElectrikSpace force-pushed the dev/snoiry/mppa_target branch from eaac320 to 4605de2 Compare February 19, 2026 14:24

ElectrikSpace requested a review from guillon February 19, 2026 14:31

ElectrikSpace force-pushed the dev/snoiry/mppa_target branch from 4605de2 to 54ab8a3 Compare February 20, 2026 11:27

This was referenced Feb 24, 2026

Add Amd AIE target #56

Draft

[gpu] Support AOT offloading of tensors #57

Open

ElectrikSpace force-pushed the dev/snoiry/mppa_target branch 2 times, most recently from 759afa8 to 74f3b3b Compare February 26, 2026 16:58

guillon added the enhancement New feature or request label Feb 27, 2026

ElectrikSpace force-pushed the dev/snoiry/mppa_target branch from 74f3b3b to 43d5d1f Compare February 27, 2026 12:54

guillon assigned guillon and unassigned ElectrikSpace Mar 3, 2026

guillon requested changes Mar 3, 2026

View reviewed changes

guillon assigned ElectrikSpace and unassigned guillon Mar 3, 2026

ElectrikSpace force-pushed the dev/snoiry/mppa_target branch from 43d5d1f to 1566ce6 Compare March 10, 2026 16:18

ElectrikSpace added 3 commits March 11, 2026 12:18

ElectrikSpace force-pushed the dev/snoiry/mppa_target branch from 1566ce6 to bad01b8 Compare March 11, 2026 13:30

ElectrikSpace requested a review from guillon March 11, 2026 13:37

ElectrikSpace added 2 commits March 11, 2026 15:04

[Mppa] Delay vectorization to the lowering pipeline

c419ac4

Rely on the kvxuks-catch pass to catch micro-kernels in replacement of the transform dialect based vectorization.

ElectrikSpace force-pushed the dev/snoiry/mppa_target branch from bad01b8 to c419ac4 Compare March 11, 2026 14:04

guillon assigned guillon and unassigned ElectrikSpace Mar 12, 2026

This comment was marked as duplicate.

Sign in to view

guillon reviewed Mar 12, 2026

View reviewed changes

guillon requested changes Mar 12, 2026

View reviewed changes

guillon assigned ElectrikSpace and unassigned guillon Mar 12, 2026

Conversation

ElectrikSpace commented Feb 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ElectrikSpace commented Feb 13, 2026

Uh oh!

ElectrikSpace commented Feb 16, 2026

Uh oh!

guillon commented Mar 2, 2026

Uh oh!

ElectrikSpace commented Mar 3, 2026

Uh oh!

guillon left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ElectrikSpace commented Mar 11, 2026

Uh oh!

This comment was marked as duplicate.

Uh oh!

This comment was marked as duplicate.

Uh oh!

guillon left a comment

Choose a reason for hiding this comment

Review: #47

Summary

Findings

[high] Device-aware NDArray creation breaks existing GPU evaluation path

[medium] MPPA codegen builds intermediates in the current working directory instead of the chosen temp/output directory

Notes

Uh oh!

guillon Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

guillon Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

guillon Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

guillon Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

guillon Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

guillon left a comment

Choose a reason for hiding this comment

Uh oh!

guillon Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ElectrikSpace commented Feb 12, 2026 •

edited

Loading