Skip to content

(Stretch) dedicated NdScanExec + BroadcastExec #283

@robinskil

Description

@robinskil

Part of #276 (Stretch / Phase 3+). Only pursue if profiling justifies it.

Goal

Promote broadcast into first-class physical operators to unlock optimizer-level wins.

Design

  • NdScanExec emits ND-struct columns (Struct{ values: List<T>, dim_names, dim_sizes }), keeps coordinate pushdown.
  • BroadcastExec: ND-struct schema in → encoded tabular schema out; broadcasts + REE/Dict-encodes; advertises EquivalenceProperties ordering for monotonic coordinates (→ sort elimination, streaming group-by).
  • TableProvider::scan() wires BroadcastExec(NdScanExec(..)) and advertises the flat logical schema. Not inserted by an analyzer rule.
  • Optional follow-up: optimizer rule rewriting aggregate-over-dimension into an ND-form reduction (no flatten).

Acceptance

  • Plans show ordering propagated from broadcast; downstream sorts eliminated where valid.
  • Results identical to the in-place path; bench shows benefit for ordering-sensitive queries.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestrustPull requests that update rust code

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions