Skip to content

Conversation

@NikitaMatskevich
Copy link

@NikitaMatskevich NikitaMatskevich commented Feb 2, 2026

Which issue does this PR close?

Subtask of ##4321. This PR also suggests a certain design for currently non-existing undelete API (massively inspired by existing API for delete though).

Rationale for this change

I'll add my 2 cents here. Iceberg Rust FileIO is powered by OpenDAL, which makes OpenDAL an important piece of the whole Iceberg infrastructure. There are known bugs in Iceberg causing involuntary file deletion. In order to secure our pipelines, currently our team has to maintain implementations of this logic on several languages. Ideally we would like to have a file restore (and later also full table restore) functionality available downstream.

What changes are included in this PR?

Core Changes:

  1. Added undelete capability to Capability struct (core/src/types/capability.rs)
  2. Created operation structs OpUndelete and RpUndelete (core/src/raw/ops.rs, core/src/raw/rps.rs)
  3. Added Operation::Undelete variant (core/src/raw/operation.rs)
  4. Extended Access trait with undelete method and added to all trait forwarding (core/src/raw/accessor.rs, core/src/raw/layer.rs)
  5. Added public API Operator::undelete() method (core/src/types/operator/operator.rs)

Layer APIs:

  1. RetryLayer - Added undelete with retry logic (core/layers/retry/src/lib.rs)
  2. TimeoutLayer - Added undelete with timeout handling (core/layers/timeout/src/lib.rs)
  3. LoggingLayer - Added undelete with logging (core/layers/logging/src/lib.rs)
  4. ConcurrentLimitLayer - Added undelete with semaphore.acquire (core/layers/concurrent-limit/src/lib.rs)

NB: I checked all 20 layers that implement LayeredAccess. The others don't need explicit implementations because:

  • Default forwarding works: They rely on the default LayeredAccess::undelete which forwards to self.inner().undelete()
  • No special behavior: They either:
    • Don't modify simple operations (like throttle, which only affects read/write streaming)
    • Only affect specific operations (like chaos, which only adds errors to reads)
    • Are purely observability layers that use default forwarding

Concrete Azure Blob Storage Implementation:

  1. Implemented azblob_undelete_blob in core/services/azblob/src/core.rs using Azure's PUT ?comp=undelete REST API
  2. Added undelete implementation to AzblobBackend in core/services/azblob/src/backend.rs
  3. Enabled capability when enable_soft_deletes=true in azblob config

Testing:

  1. Added test_undelete_file behavior test in core/tests/behavior/async_delete.rs that:
    - Creates a file
    - Deletes it (soft delete)
    - Restores it using undelete
    - Verifies the content is intact
    - Exits early for filesystems that don't support undelete (Capability check)

Are there any user-facing changes?

New feature: op.undelete(&file_path) for Azblob when soft deletes are enabled.

AI Usage Statement

Claude Code with its default model (Claude Sonet)

@NikitaMatskevich NikitaMatskevich changed the title Implement Azblob undelete [ADP Lakehouse] Implement Azblob undelete Feb 2, 2026
@NikitaMatskevich NikitaMatskevich changed the title [ADP Lakehouse] Implement Azblob undelete feat(services/azblob): implement "undelete" operation in azblob Feb 3, 2026
@NikitaMatskevich NikitaMatskevich force-pushed the nmh/add-azblob-undelete branch 2 times, most recently from e24f695 to 0e9b4fc Compare February 3, 2026 20:13
@NikitaMatskevich NikitaMatskevich marked this pull request as ready for review February 3, 2026 20:26
@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. releases-note/feat The PR implements a new feature or has a title that begins with "feat" labels Feb 3, 2026
@Xuanwo
Copy link
Member

Xuanwo commented Feb 4, 2026

Hi, please starting a RFC first to add new public API.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

releases-note/feat The PR implements a new feature or has a title that begins with "feat" size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants