[FLINK-38783][runtime] Improved TieredStorageResourceRegistry Thread-Safety/Concurrency

[rionmonster]

What is the purpose of the change

This pull request addresses the issue detailed in FLINK-38783 which detailed how the registration process within TieredStorageResourceRegistry was not properly handling concurrent operations and as such could throw a ConcurrentModificationException under concurrent load.

Brief change log

• Replaced the previous registeredResources general-purpose hash map (Map<..., List<...>>) with corresponding thread-safe structures (ConcurrentHashMap<..., CopyOnWriteArrayList<...>>) to better handle concurrent operations.

Verifying this change

This change added a series of tests in TieredStorageResourceRegistryTest to originally reproduce the issue and later confirm the fix worked as expected including:

• testConcurrentRegisterResource to test concurrent resource registration across separate threads (10 total with same owner/identifier)
• testConcurrentRegisterResourceWithDifferentOwners to test concurrent registration across separate threads (10 total with different owners/identifiers)
• testConcurrentRegisterAndClear to test concurrent registration and clearing across separate threads

Example Tests

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): no
• The public API, i.e., is any changed class annotated with @Public(Evolving): no
• The serializers: no
• The runtime per-record code paths (performance sensitive): no
• Anything that affects deployment or recovery: no/don't know
• The S3 file system connector: no

Documentation

• Does this pull request introduce a new feature? no
• If yes, how is the feature documented? not applicable

[rionmonster]

CC: @TanYuxin-tyx for review (please feel free to tag someone else). You seemed to be the most appropriate given the history.

[flinkbot]

CI report:

• Unknown: CANCELED
• 448adde Azure: PENDING

Bot commands

The @flinkbot bot supports the following commands:

• @flinkbot run azure re-run the last Azure build

[rionmonster]

@flinkbot run azure