Skip to content

Optimize sparse delta apply path#83

Open
peters wants to merge 2 commits intomainfrom
fix/optimize-sparse-delta-apply
Open

Optimize sparse delta apply path#83
peters wants to merge 2 commits intomainfrom
fix/optimize-sparse-delta-apply

Conversation

@peters
Copy link
Copy Markdown
Contributor

@peters peters commented Apr 11, 2026

Summary

  • apply sparse-file-ops deltas directly to a staged filesystem tree restored from the pristine base package instead of round-tripping through an intermediate rebuilt archive during update materialization
  • keep the rebuilt latest full archive warm in the local artifact cache after delta application
  • add regression coverage for direct sparse delta directory application, persistent-asset divergence, and preserved directory modes

Why

A recent Linux x64 update spent most of its time preparing a tiny sparse delta against a cached full package.

The hot path was not network transfer or app shutdown. It was local archive work:

  1. load the cached full archive
  2. expand it to a temporary tree
  3. apply sparse file ops
  4. repack a new full archive
  5. extract that rebuilt archive again into the staged app tree

That makes sparse deltas pay full-archive costs even when the delta payload itself is very small.

Root Cause

UpdateManager treated sparse file deltas like archive deltas all the way through staging. The updater rebuilt a full archive in memory and then extracted it, even though the sparse delta format already describes filesystem operations.

What Changed

  • added a direct sparse apply path in UpdateManager that:
    • restores the pristine base full archive into the staged tree
    • applies decoded sparse file ops directly to that staged tree
    • repacks once at the end for integrity verification and cache warming
  • preserved the existing archive-based path for non-sparse delta formats
  • retained the installed-app fallback archive reconstruction path only for cases where no base full archive can be restored
  • cached the rebuilt latest full archive after delta application so the next update can reuse it
  • added regressions covering:
    • persistent assets that intentionally diverge from packaged contents
    • unchanged non-default directory modes during sparse apply

Impact

For sparse deltas, this removes an unnecessary archive round-trip from the apply path while preserving the packaged archive basis required for persistent assets and hash-stable directory metadata.

Validation

  • ./scripts/sync-surge-core-vendor.sh --check
  • ./scripts/check-version-sync.sh
  • cargo fmt --all -- --check
  • RUSTFLAGS='-D warnings' cargo test --workspace
  • cargo clippy --all-targets --all-features -- -D warnings
  • cargo clippy --workspace --lib --bins --examples -- -D warnings -D clippy::unwrap_used -D clippy::expect_used
  • cargo clippy --workspace --all-targets --all-features -- -D warnings -W clippy::pedantic
  • dotnet format dotnet/Surge.slnx --verify-no-changes
  • dotnet test dotnet/Surge.slnx --configuration Release

@peters peters marked this pull request as ready for review April 11, 2026 22:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant