[FLINK-39964][checkpoint] Fix unrestorable checkpoint after CLAIM restore from native savepoint

[Savonitar]

What is the purpose of the change

The first incremental RocksDB checkpoint taken after a CLAIM-mode restore from a NATIVE savepoint reuses the savepoint's SST files. The savepoint writes those files as RelativeFileStateHandles, which are carried into the new checkpoint's shared state. The metadata serializer records only their relative path, and on restore they are re-anchored to the new checkpoint's exclusive directory. The referenced state file is therefore looked up under / while the bytes still live in /, so restoring from that checkpoint fails with a missing-file error.
The fix pushes the exclusive directory of the checkpoint being written into metadata serialization and decides, per state handle, whether the compact relative encoding is still safe: a relative handle whose file actually lives in that directory keeps it (so savepoints stay relocatable), and a foreign one (a reused savepoint SST) is persisted with its absolute path using the pre-existing FileStateHandle encoding.

Brief change log

• Metadata serialization now knows the exclusive directory of the checkpoint being written: CheckpointMetadataOutputStream#getExclusiveCheckpointDir() (new, default null, overridden by FsCheckpointMetadataOutputStream) is used through a new Checkpoints.storeCheckpointMetadata(CheckpointMetadata, CheckpointMetadataOutputStream) overload, which PendingCheckpoint#finalizeCheckpoint picks up without a call-site change.
• MetadataV2V3SerializerBase#canKeepRelativeEncoding(...) decides per handle: a RelativeFileStateHandle whose file lives in that directory keeps the compact relative encoding a foreign one (a reused savepoint SST) is written with its absolute path via the pre-existing FileStateHandle encoding.
• Separate one-line [hotfix][tests] commit: SavepointDeepCopyTest had compared savepoint1's file listing with itself (which is incorrect) and it now reads savepoint2's directory.

Verifying this change

• Serializer level: MetadataV2V3SerializerBaseTest (foreign relative handles round-trip as absolute, own handles stay relative and relocatable, null directory keeps the legacy encoding byte-for-byte) and MetadataSerializerEntryPointsTest (both entry points across all metadata versions).
• End to end: ResumeCheckpointAfterClaimedNativeSavepointITCase on a real MiniCluster -> stop-with-savepoint (NATIVE) → CLAIM restore → first incremental checkpoint (asserting reused savepoint files are referenced by absolute location) → restore from that checkpoint. Plus the backend-level reproducer RocksDBClaimSavepointThenCheckpointRestoreTest.
• State Processor API guard: SavepointOutputFormatSelfContainedTest and SavepointDeepCopyTest (source savepoint deleted before restoring from the copy) prove deep-copied savepoints keep no references into the source.

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): (no)
• The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
• The serializers: yes (the checkpoint metadata serializer's write-side encoding decision changed, the wire format and metadata version are unchanged, foreign relative handles now use the pre-existing absolute FileStateHandle encoding, so the read side and older readers are unaffected)
• The runtime per-record code paths (performance sensitive): (no)
• Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: yes (that is the point of the change: the first incremental checkpoint taken after a CLAIM-mode restore from a NATIVE savepoint becomes restorable)
• The S3 file system connector: no

Documentation

• Does this pull request introduce a new feature? (no)
• If yes, how is the feature documented? (not applicable)

---

Was generative AI tooling used to co-author this PR?

• Yes (please specify the tool below)
  Generated-by: Claude Code (Anthropic), Opus 4.8 / Fable.

[flinkbot]

CI report:

• 02aeaf4 Azure: SUCCESS

Bot commands

The @flinkbot bot supports the following commands:

• @flinkbot run azure re-run the last Azure build