Skip to content

Emit encoded arrays in the nd scan + declare schema/ordering #279

@robinskil

Description

@robinskil

Part of #276 (Phase 1). Depends on the classifier + constructor sub-issues.

Goal

Wire the encoded-array path into the nd scan so batches actually carry REE/Dict columns, and declare it.

Tasks

  • Use the classifier + constructor in beacon-nd-array/src/arrow/batch.rs (chunked + ragged paths).
  • Declare the encoded column types on the scan's output schema (the FileScanConfig / source schema).
  • Set output_ordering for monotonic broadcast coordinates (enables later sort elimination).

Acceptance

  • Queries over netCDF/Zarr return correct results with encoded batches end-to-end.
  • Scan output schema reflects the encoding; ordering is advertised for monotonic coords.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestrustPull requests that update rust code

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions