Skip to content

improve tensor error handling #82

@xmnlab

Description

@xmnlab

Tensor backing is correct, but the generated LLVM currently treats several Tensor runtime calls as if they cannot fail.

Examples in packages/irx/src/irx/builder/lowering/tensor.py:

  • _build_arrow_tensor_from_values() calls:

    • irx_arrow_tensor_builder_new
    • irx_arrow_tensor_builder_append_int/uint/double
    • irx_arrow_tensor_builder_finish

    These native functions return an int status, but the lowering ignores it and immediately loads/uses the output handle.

  • _wrap_arrow_tensor_handle_as_tensor() calls:

    • irx_arrow_tensor_borrow_buffer_view
    • irx_buffer_owner_external_new

    Those also return status values, but the lowering ignores them before reading the borrowed view / owner handle.

Why that matters:

  • If native Arrow tensor construction fails, out_tensor may remain null.
  • If irx_arrow_tensor_borrow_buffer_view fails, the buffer-view alloca may contain undefined/uninitialized data.
  • If owner creation fails, the lowered Tensor may not properly retain/release the Arrow tensor handle.
  • Runtime failures would likely turn into confusing crashes or bad loads instead of a clear diagnostic.

By contrast, DataFrame lowering has explicit status checking in:

packages/irx/src/irx/builder/lowering/dataframe.py

via _check_arrow_status(...), which branches to the assertion runtime and reports irx_arrow_last_error().

So “more robust” means Tensor lowering should probably mirror DataFrame lowering: check every irx_arrow_tensor_* / owner-creation status, fail through the
assertion runtime with the Arrow error message, and avoid reading output slots unless the call succeeded. This is mostly about defensive failure behavior;
successful-path Tensor lowering is still Arrow-backed.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions