Skip to content

EnforceColumnEncoding physical optimizer rule #281

@robinskil

Description

@robinskil

Part of #276 (Phase 2). Builds on the flat↔REE SchemaAdapter.

Goal

An EnforceColumnEncoding PhysicalOptimizerRule (in beacon-datafusion-ext, registered at beacon-core/src/runtime.rs:154) that lets REE survive through the plan and reconciles it at boundaries.

Behavior

  • Walk the plan; at every schema-combining node (UnionExec/InterleaveExec/join) and every flat-only operator, insert a ProjectionExec cast on the branch(es) that disagree.
  • Choose target encoding cost-driven (toward dominant branch / stats cache in beacon-datafusion-ext/src/stats_cache.rs).
  • Push the materializing cast as late as possible so REE survives projections/simple filters.
  • Model per-operator "required input encoding" (analogous to required ordering/distribution); custom ops accept REE, built-in DF ops generally require flat.

Acceptance

  • Union/join of tables with differing encodings executes correctly.
  • Plan inspection shows casts inserted only at necessary boundaries; flat-only ops get materialized inputs.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestrustPull requests that update rust code

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions