Add upgrade, DR, backup-integrity and canary deployment testing (#557–#560)#606
Merged
fejilaup-cloud merged 1 commit intoMay 30, 2026
Conversation
|
@Nexha-dev Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits. You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀 |
…icIP#557–AtomicIP#560) Implements testing for four reliability/ops issues: AtomicIP#557 Contract Upgrade Testing - New contracts/ip_registry/src/upgrade_tests.rs covering upgrade compatibility validation (accept/reject), state preservation across the validation check, and the admin-authorization guard on upgrade(). - docs/contract-upgrade-testing.md. AtomicIP#558 Disaster Recovery Testing - scripts/test-disaster-recovery.sh runs the full backup -> verify -> restore -> service-verify chain hermetically (external services mocked). - docs/disaster-recovery-testing.md. AtomicIP#559 Backup Integrity Verification - verify-backup-integrity.sh now verifies a SHA-256 sidecar when present; backup-contract-state.sh writes and uploads that sidecar (backward compatible). - scripts/test-backup-integrity.sh covers valid/corrupt/missing/malformed/ checksum-mismatch backups. - docs/backup-integrity-verification.md. AtomicIP#560 Canary Deployment Testing - scripts/canary-deploy.sh: health-gated canary rollout with automatic rollback; side-effecting steps are pluggable hooks for testability. - scripts/test-canary-deployment.sh exercises promote/rollback branches. - docs/canary-deployment.md. Shared: - scripts/tests/lib.sh: hermetic shell-test harness (mock bins, assertions, fixtures). - scripts/run-ops-tests.sh + .github/workflows/ops-tests.yml run the ops suites in CI. - README documentation index updated. Test status: shell suites all pass (46 checks). The Rust upgrade_tests module compiles cleanly and is purely additive (identical error count with the module enabled or disabled), but cannot be executed locally because the ip_registry crate does not currently build on main — pre-existing and unrelated: a duplicate batch_verify_commitments definition (lib.rs:2895 and lib.rs:3710, E0592) plus test code referencing absent DataKey/ContractError variants.
Nexha-dev
force-pushed
the
feature/557-560-upgrade-dr-backup-canary-testing
branch
from
d02eb30 to
bbc2a90
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements testing for four reliability / operations issues:
contracts/ip_registry/src/upgrade_tests.rs(7 tests): upgrade-compatibility validation (accept valid hashes, reject the zero hash), state preservation across the compatibility check, idempotency, and the admin-authorization guard onupgrade(). Docs:docs/contract-upgrade-testing.md.scripts/test-disaster-recovery.shruns the full backup → verify → restore → service-verify chain hermetically, mocking Stellar/AWS/API/Postgres. Docs:docs/disaster-recovery-testing.md.verify-backup-integrity.shnow verifies a SHA-256 sidecar when present;backup-contract-state.shwrites and uploads that sidecar (backward compatible).scripts/test-backup-integrity.shcovers valid / corrupt / missing-file / malformed-JSON / checksum-mismatch cases. Docs:docs/backup-integrity-verification.md.scripts/canary-deploy.sh: health-gated canary rollout with automatic rollback; side-effecting steps are pluggable hooks so it runs against real infra in prod and mocks in tests.scripts/test-canary-deployment.shexercises every promote/rollback branch. Docs:docs/canary-deployment.md.Shared infrastructure
scripts/tests/lib.sh— hermetic shell-test harness (mock binaries, assertions, fixtures).scripts/run-ops-tests.sh+.github/workflows/ops-tests.yml— run the ops suites in CI (no secrets/infra needed).Test results
test-disaster-recovery.sh(#558)test-backup-integrity.sh(#559)test-canary-deployment.sh(#560)upgrade_tests(#557)mainbuild breakage (not introduced here)The
ip_registrycrate does not currently compile onmain, socargo test -p ip_registrycannot be executed locally. This is pre-existing and unrelated to this PR:pub fn batch_verify_commitmentsdefinitions incontracts/ip_registry/src/lib.rs(lines 2895 and 3710) — hardE0592/E0428errors from two separately-merged PRs.DataKey/ContractErrorvariants that don't exist andstdinside the#![no_std]crate.The new
upgrade_testsmodule is purely additive: the crate's error count is identical (64) whethermod upgrade_tests;is enabled or disabled, confirming this PR adds no new errors. The module mirrors the existing (passing)test.rsidioms and will run green once the duplicate-definition bug onmainis fixed. Happy to address that separately if useful.Closes #557
Closes #559
Closes #558
Closes #560