Improve pg_restore errors; auto-detect archives; gzip/ZIP input support#70
Merged
Conversation
cursor
Bot
changed the title
- Add pg_restore_decode module: preflight when --pg-restore-path is a concrete path, improved spawn/exit errors with install hint, capture pg_restore stderr (tee to stderr) and include on non-zero exit - Reap pg_restore in scaffold-config when discovery fails; only remove archive after full success - Remove unused Context import from main Co-authored-by: Andy Babic <ababic@users.noreply.github.com>
- Add compressed_input: sniff gzip (1f 8b) and ZIP (PK), decompress/extract to temp files with Drop cleanup; combined gzip-then-ZIP supported - ZIP: single member wins; else exactly one .sql file among multiple files - Fail --in-place when input used gzip/ZIP wrappers (unsafe path semantics) - pg_restore archive removal skips paths under temp dir (avoid double-delete) - Dependencies: flate2, zip (deflate only) - scaffold-config mirrors main pipeline; changelog [Unreleased] Co-authored-by: Andy Babic <ababic@users.noreply.github.com>
--keep-original combined with --in-place now fails fast with a clear error, matching gzip/ZIP and pg_restore archive restrictions. Updated scaffold path, ResolvedConfig test helpers, and changelog for unified keep-original/pg_restore settings. Co-authored-by: Andy Babic <ababic@users.noreply.github.com>
Gzip-wrapped plain SQL is decompressed with flate2 and fed into the pipeline without a temp file. PGDMP, nested gzip->ZIP, and ZIP extraction still use temp paths registered on CompressionCleanup. --in-place now rejects only inputs that required a materialized temp file from compression handling. Refactors MSSQL classification with classify_mssql_prefix for sniff buffers. Co-authored-by: Andy Babic <ababic@users.noreply.github.com>
…ginal Replace deprecated --dump-decode guidance with auto-detection, streaming gzip, temp ZIP/PGDMP behavior, [pg_restore] TOML, --keep-original vs --in-place, and --check with archives. Update README and mdBook configuration/getting-started/index. Co-authored-by: Andy Babic <ababic@users.noreply.github.com>
ababic
force-pushed
the
cursor/pg-restore-detect-bad8d
branch
from
39e94cd to
f31d6de
Compare
Co-authored-by: Andy Babic <ababic@users.noreply.github.com>
Co-authored-by: Andy Babic <ababic@users.noreply.github.com>
Extract resolve_dump_input_from_path into dump_input_resolve.rs so scaffold-config uses the same gzip/ZIP, MSSQL classification, pg_restore, and archive-removal logic as run_anonymize. Document new modules in AGENTS.md. Co-authored-by: Andy Babic <ababic@users.noreply.github.com>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
Add log_sanitize::path_basename_for_log and use it for pg_restore progress lines, gzip/ZIP notices, archive removal, scaffold write/remove messages, and plaintext salt warnings — avoids emitting full filesystem paths that may contain sensitive directory names. Context/full paths remain in anyhow messages where appropriate. Co-authored-by: Andy Babic <ababic@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Expands input handling for Dumpling’s default
--inputpath:gzip / ZIP auto-handling
0x1f 0x8b): decompress to a temp file (streaming decompress viaflate2), then continue — supports e.g..sql.gz, gzippedPGDMPcustom-format archives beforepg_restore.PK\x03\x04): extract one member — either the only file in the archive, or the only.sqlwhen multiple files exist. Inner extension preserved (.dump,.sql, …).Combinations gzip → ZIP → inner file work (e.g.
.zip.gz).Safety / ergonomics
--in-place: rejected when gzip or ZIP wrappers were applied (must use--output/stdout; avoids ambiguous overwrite of the compressed path vs temp decoded content).pg_restorearchive deletion: only removes the original archive path when appropriate; skips removal when the archive lived under system temp (already cleaned byCompressionCleanup).process::exit: temp cleanup runs (drop(compression_cleanup)) before exit codes 1–3.Earlier PR behavior (still included)
PGDMP) / directory (toc.dat) auto-detect, MSSQL binary/ZIP UTF-16 hints, friendlierpg_restoreerrors.Dependencies
flate2,zip(deflate only).Tests / CI
cargo fmt,cargo clippy --all-targets --all-features,cargo test --all-targets --all-featuresChangelog
[Unreleased]entry for gzip/ZIP.Slack Thread