Skip to content

[FLINK-40120][table] Persist RowData field names in RowDataSerializerSnapshot#28717

Open
weiqingy wants to merge 1 commit into
apache:masterfrom
weiqingy:FLINK-40120-impl
Open

[FLINK-40120][table] Persist RowData field names in RowDataSerializerSnapshot#28717
weiqingy wants to merge 1 commit into
apache:masterfrom
weiqingy:FLINK-40120-impl

Conversation

@weiqingy

@weiqingy weiqingy commented Jul 11, 2026

Copy link
Copy Markdown

What is the purpose of the change

This pull request persists the top-level field names of a RowData state serializer in RowDataSerializerSnapshot, so they survive a save/restore round-trip. Today the snapshot stores the field types but not the names.

It is a self-contained, shared building block: the names are needed by schema-aware consumers — the State Catalog (FLIP-599) to expose a readable schema for keyed state, and name-based schema evolution (FLIP-527). They are written unconditionally and decoupled from any evolution logic, so whichever effort lands first establishes a single snapshot format.

Brief change log

  • Add a nullable top-level String[] fieldNames to RowDataSerializer, threaded through duplicate() and the snapshot round-trip; equals/hashCode are unchanged (names derive from the types).
  • Bump RowDataSerializerSnapshot from V3 to V4, appending the names block after the existing types / nested-serializer section. readSnapshot reads the names only for readVersion >= 4, so V4 readers still restore V3 snapshots (names absent -> null).
  • InternalSerializers builds the ROW serializer with its top-level field names (split from the structured-type case, which stays name-less).

Verifying this change

This change added tests and can be verified as follows:

  • V4 snapshot round-trip preserves the top-level names, and nested ROW field names remain recoverable from the child logical types.
  • Genuine V3 -> V4 backward compatibility: a hand-crafted V3-layout snapshot restores with fieldNames == null and the serializer still round-trips.
  • InternalSerializers.create(<a RowType>) yields a serializer whose snapshot carries the names unconditionally.
  • duplicate() preserves the names.
  • resolveSchemaCompatibility is unchanged: a field-type-array difference still returns incompatible() regardless of names.

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): no
  • The public API, i.e., is any changed class annotated with @Public(Evolving): no
  • The serializers: yes — RowDataSerializer / RowDataSerializerSnapshot, snapshot format V3 -> V4
  • The runtime per-record code paths (performance sensitive): no
  • Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: yes — the RowData state snapshot format moves V3 -> V4. Backward compatible: V4 readers restore V3 snapshots (names absent -> null); no runtime behavior change.
  • The S3 file system connector: no

Documentation

  • Does this pull request introduce a new feature? no — it persists metadata not yet consumed by any decision; no user-facing behavior change.
  • If yes, how is the feature documented? not applicable
Was generative AI tooling used to co-author this PR?
  • Yes (Claude Code, Claude Opus 4.8)

…Snapshot

Add the top-level field names to RowDataSerializerSnapshot so they survive a
save/restore round-trip, as shared infrastructure for schema-aware consumers.

The names are written unconditionally and the change is behavior-neutral: no
compatibility or evolution logic is added. The snapshot format moves V3 to V4;
V4 readers still read V3 snapshots (names absent -> null), so existing savepoints
continue to restore.

Generated-by: Claude Code (Claude Opus 4.8)
@weiqingy weiqingy marked this pull request as ready for review July 11, 2026 05:25
@flinkbot

flinkbot commented Jul 11, 2026

Copy link
Copy Markdown
Collaborator

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

@spuru9 spuru9 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions Bot added the community-reviewed PR has been reviewed by the community. label Jul 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

community-reviewed PR has been reviewed by the community.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants