docs: Getting started in mdBook; remove table filter flags (0.7.0)

CHANGELOG.md

@@ -7,11 +7,21 @@ and this project follows [Semantic Versioning](https://semver.org/spec/v2.0.0.ht
 
 ## [Unreleased]
 
+## [0.7.0] - 2026-05-04
+
+### Removed
+
+- **`--include-table` / `--exclude-table`**: these CLI flags and the associated per-table skip logic in the SQL stream processor are removed. Anonymize the full dump, or split/filter dumps outside Dumpling if you need a smaller input.
+
+### Changed
+
+- **Dump seal (`v=3`):** the fingerprint JSON no longer includes table-filter fields. Seals produced by Dumpling **0.6.x** (`v=2`) will not match **0.7.x**; the first line is treated as stale and the dump is re-processed.
+
 ## [0.6.0] - 2026-05-03
 
 ### Added
 
-- **Dump seal** (leading `-- dumpling-seal:` SQL comment): records Dumpling version, security profile, a SHA-256 fingerprint of the resolved policy, and runtime CLI options that affect transforms (`--format`, sorted `--include-table` / `--exclude-table`, effective PRNG seed in standard profile). When the input already begins with a **matching** seal, the remainder is copied through unchanged; stale or unknown seal lines are stripped and the dump is re-processed. See README for full semantics ([#58](https://github.com/ababic/dumpling/pull/58)).
+- **Dump seal** (leading `-- dumpling-seal:` SQL comment): records Dumpling version, security profile, a SHA-256 fingerprint of the resolved policy, and runtime CLI options that affect transforms (`--format` and the effective PRNG seed in standard profile; `null` in hardened, where seeds are ignored). When the input already begins with a **matching** seal, the remainder is copied through unchanged; stale or unknown seal lines are stripped and the dump is re-processed. See README for full semantics ([#58](https://github.com/ababic/dumpling/pull/58)).
 - **`--stats`**: prints `wall_ms` plus `domain_cache_hits` and `domain_cache_misses` for quick profiling of large runs ([#59](https://github.com/ababic/dumpling/pull/59)).
 - **`CONTRIBUTORS.md`** ([#59](https://github.com/ababic/dumpling/pull/59)).
 

Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "dumpling"
-version = "0.6.0"
+version = "0.7.0"
 edition = "2021"
 
 [dependencies]

README.md

@@ -87,6 +87,8 @@ Follow these steps once; you will have a working path from “raw dump” to “
 
 **Draft policy generation (planned)** — A future command will stream a dump and emit a **draft** starter TOML so you spend less time hunting table and column names and basic DDL hints (for example `varchar(N)` lengths). Output will be explicitly **draft**: always review and edit before production or compliance workflows; it is a time-saver, not a full policy.
 
+The same flow is spelled out in the docs: [Getting started](https://ababic.github.io/dumpling/getting-started.html).
+
 ---
 
 ## Usage
@@ -102,8 +104,6 @@ dumpling --report report.json -i dump.sql       # write detailed JSON report of
 dumpling --strict-coverage --report report.json -i dump.sql --check  # fail on uncovered sensitive columns
 dumpling --scan-output --report report.json -i dump.sql               # scan transformed output for residual PII-like patterns
 dumpling --scan-output --fail-on-findings --report report.json -i dump.sql --check  # fail if scan thresholds are exceeded
-dumpling --include-table '^public\\.' -i dump.sql -o out.sql
-dumpling --exclude-table '^audit\\.' -i dump.sql -o out.sql
 dumpling --allow-ext dmp -i data.dmp            # restrict processing to specific extensions
 dumpling --allow-noop -i dump.sql -o out.sql    # explicitly allow no-op when config is missing
 dumpling --format sqlite -i data.db.sql -o out.sql  # process a SQLite .dump file

docs/src/configuration.md

@@ -59,9 +59,9 @@ run, pass `--allow-noop`.
 
 Every successful run that writes output prefixes the stream with a single-line SQL comment:
 
-`-- dumpling-seal: v=2 version=<semver> profile=<standard|hardened> sha256=<64 hex chars>`
+`-- dumpling-seal: v=3 version=<semver> profile=<standard|hardened> sha256=<64 hex chars>`
 
-The `sha256` is over canonical JSON that includes the Dumpling version, the active security profile, a stable encoding of the resolved policy (rules, row filters, column cases, sensitive columns, output scan, global salt), and **runtime options** that affect transforms: `--format`, sorted `--include-table` / `--exclude-table` patterns, and the effective `--seed` / `DUMPLING_SEED` value in standard profile (`null` in hardened, where seeds are ignored).
+The `sha256` is over canonical JSON that includes the Dumpling version, the active security profile, a stable encoding of the resolved policy (rules, row filters, column cases, sensitive columns, output scan, global salt), and **runtime options** that affect transforms: `--format` and the effective `--seed` / `DUMPLING_SEED` value in standard profile (`null` in hardened, where seeds are ignored).
 
 If the **input** already begins with a seal line and it **matches** the current run, Dumpling copies the rest of the file through unchanged. If the line looks like a seal but does **not** match (stale policy, different flags, or older `v=`), that line is **dropped** and the dump is re-processed so you do not end up with two seal lines. `--strict-coverage` cannot be combined with a matching seal (table definitions are not scanned in passthrough mode). `--check` writes no output and therefore emits no seal line.
 

docs/src/getting-started.md

@@ -1,5 +1,7 @@
 # Getting started
 
+This page is the **shortest path** from zero to a first successful run. For strategy details, row filters, dump seals, and CI patterns, continue with the [configuration guide](configuration.md) and the repository `README.md`.
+
 ## Prerequisites
 
 - Rust **stable** toolchain (`rustup` recommended). The repo includes `rust-toolchain.toml` (stable + `rustfmt` + `clippy`) so CI and local `cargo` stay aligned.
@@ -14,20 +16,30 @@ cargo build --release
 ./target/release/dumpling --help
 ```
 
-## Test locally
+### Python / pip (`dumpling-cli`)
 
 ```bash
-cargo fmt --all -- --check
-cargo clippy --all-targets --all-features
-cargo test --all-targets --all-features
+pip install dumpling-cli
+dumpling --help
 ```
 
-## Run against a dump
+## First anonymization (four steps)
 
-```bash
-dumpling -i dump.sql -o sanitized.sql
-```
+1. **Start from the example policy** — In the repository, copy [`.dumplingconf.example`](https://github.com/ababic/dumpling/blob/main/.dumplingconf.example) to `.dumplingconf` in your project root (or merge the same keys under `[tool.dumpling]` in `pyproject.toml`). Set environment variables for `salt` and any `${…}` references so Dumpling can resolve secrets at startup.
+2. **Name your tables and columns** — Open your dump next to the config. `CREATE TABLE`, `COPY … (…)` and `INSERT INTO … (…)` lines list the identifiers you need for `[rules."table"]` or `[rules."schema.table"]`. Trim the example rules to the tables you care about first, then add columns and strategies as you go.
+3. **Run Dumpling** — `dumpling -i dump.sql -o sanitized.sql` (add `-c path` if the config is not in the default search path). Use `dumpling --check -i dump.sql` when you only want to know whether anything would change.
+4. **Tighten the policy** — Run `dumpling lint-policy` on your config. When you are ready for stricter gates, add `[sensitive_columns]` and use `--strict-coverage`, `--report`, and `--scan-output` as described in the [configuration guide](configuration.md) and the repository `README.md`.
+
+**Draft policy generation (planned)** — A future command will stream a dump and emit a **draft** starter TOML (table/column names and basic DDL hints) so you spend less time spelunking large dumps by hand. Treat that output as **draft** only: review and edit before any production or compliance workflow.
+
+## PostgreSQL custom-format archives
 
 If your input is a PostgreSQL **custom-format** file (not plain SQL), decode and anonymize in one step with **`--dump-decode`** (needs `pg_restore` from PostgreSQL client tools). See [PostgreSQL custom-format archives](configuration.md#postgresql-custom-format-archives---dump-decode) in the configuration guide.
 
-For full command examples and strategy options, see the repository `README.md`.
+## Test locally (contributors)
+
+```bash
+cargo fmt --all -- --check
+cargo clippy --all-targets --all-features
+cargo test --all-targets --all-features
+```

docs/src/index.md

@@ -2,6 +2,8 @@
 
 Dumpling is a streaming anonymizer for plain SQL dumps. It supports PostgreSQL (`pg_dump` plain format), SQLite (`.dump`), and SQL Server / MSSQL (SSMS / mssql-scripter plain SQL output). For PostgreSQL **custom-format** archives (e.g. Heroku `pg:backups:download`), use **`--dump-decode`** so Dumpling invokes `pg_restore` and streams plain SQL—see [Dump format](configuration.html#postgresql-custom-format-archives---dump-decode) in the configuration guide.
 
+**New here?** Start with [**Getting started**](getting-started.html): copy the example config, align rules with your dump, run Dumpling, then tighten with `lint-policy` and optional CI flags.
+
 This documentation covers the operating model for day-to-day use:
 
 - how to build and run Dumpling locally,

pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "maturin"
 
 [project]
 name = "dumpling-cli"
-version = "0.6.0"
+version = "0.7.0"
 description = "Static anonymizer for plain SQL dumps (PostgreSQL, SQLite, SQL Server)."
 readme = "README.md"
 requires-python = ">=3.8"

src/main.rs

@@ -21,7 +21,6 @@ mod sql;
 mod transform;
 
 use anyhow::Context;
-use regex::Regex;
 use report::Reporter;
 use scan::{OutputScanner, ScanningWriter};
 use seal::{
@@ -91,14 +90,6 @@ struct Cli {
     #[arg(long = "fail-on-findings", action = ArgAction::SetTrue)]
     fail_on_findings: bool,
 
-    /// Include only tables matching these regex patterns (repeatable). If none provided, include all.
-    #[arg(long = "include-table")]
-    include_table: Vec<String>,
-
-    /// Exclude tables matching these regex patterns (repeatable).
-    #[arg(long = "exclude-table")]
-    exclude_table: Vec<String>,
-
     /// Only process input files with these extensions (repeatable), e.g. --allow-ext sql --allow-ext dmp
     /// Case-insensitive; leading dot optional. Ignored when reading from stdin.
     #[arg(long = "allow-ext")]
@@ -296,16 +287,7 @@ fn run_anonymize(cli: Cli) -> anyhow::Result<()> {
         );
     }
 
-    // Compile table include/exclude regex patterns
-    let include_res = compile_patterns(&cli.include_table)?;
-    let exclude_res = compile_patterns(&cli.exclude_table)?;
-
-    let seal_runtime = SealRuntimeParams::new(
-        dump_format,
-        &cli.include_table,
-        &cli.exclude_table,
-        prng_seed_override_for_fingerprint(),
-    );
+    let seal_runtime = SealRuntimeParams::new(dump_format, prng_seed_override_for_fingerprint());
 
     // Determine IO (optional pg_restore child when --dump-decode)
     let mut pg_restore_child: Option<std::process::Child> = None;
@@ -449,8 +431,6 @@ fn run_anonymize(cli: Cli) -> anyhow::Result<()> {
     let mut processor = SqlStreamProcessor::new(
         anonymizers,
         resolved_config,
-        include_res,
-        exclude_res,
         Some(&mut reporter),
         dump_format,
     );
@@ -640,14 +620,6 @@ fn remove_pg_archive(path: &Path) -> std::io::Result<()> {
     }
 }
 
-fn compile_patterns(patterns: &[String]) -> anyhow::Result<Vec<Regex>> {
-    let mut out = Vec::new();
-    for p in patterns {
-        out.push(Regex::new(p)?);
-    }
-    Ok(out)
-}
-
 fn has_allowed_extension(path: &Path, allow_exts: &[String]) -> bool {
     if allow_exts.is_empty() {
         return true;

src/seal.rs

@@ -13,38 +13,25 @@ use std::io::{self, BufRead, Read};
 pub const SEAL_LINE_PREFIX: &str = "-- dumpling-seal:";
 
 /// Bump when the JSON payload shape changes (parsers accept the `v=` field on the comment line).
-pub const SEAL_PAYLOAD_VERSION: u32 = 2;
+pub const SEAL_PAYLOAD_VERSION: u32 = 3;
 
-#[derive(Debug, Serialize)]
+#[derive(Debug, Clone, Serialize)]
 pub struct SealRuntimeParams {
     pub dump_format: String,
-    pub include_table: Vec<String>,
-    pub exclude_table: Vec<String>,
     /// `Some` when standard profile and `--seed` / `DUMPLING_SEED` is set; `None` otherwise.
     pub prng_seed: Option<u64>,
 }
 
 impl SealRuntimeParams {
-    pub fn new(
-        dump_format: DumpFormat,
-        include_table: &[String],
-        exclude_table: &[String],
-        prng_seed: Option<u64>,
-    ) -> Self {
+    pub fn new(dump_format: DumpFormat, prng_seed: Option<u64>) -> Self {
         let dump_format = match dump_format {
             DumpFormat::Postgres => "postgres",
             DumpFormat::Sqlite => "sqlite",
             DumpFormat::MsSql => "mssql",
         }
         .to_string();
-        let mut include_table: Vec<String> = include_table.to_vec();
-        include_table.sort();
-        let mut exclude_table: Vec<String> = exclude_table.to_vec();
-        exclude_table.sort();
         Self {
             dump_format,
-            include_table,
-            exclude_table,
             prng_seed,
         }
     }
@@ -183,12 +170,7 @@ pub fn compute_seal_digest(
         dumpling_version: env!("CARGO_PKG_VERSION"),
         security_profile: security_profile.to_string(),
         policy: resolved_to_raw_for_fingerprint(cfg),
-        runtime: SealRuntimeParams {
-            dump_format: runtime.dump_format.clone(),
-            include_table: runtime.include_table.clone(),
-            exclude_table: runtime.exclude_table.clone(),
-            prng_seed: runtime.prng_seed,
-        },
+        runtime: runtime.clone(),
     };
     let mut val = serde_json::to_value(&payload)?;
     sort_json_value(&mut val);
@@ -388,7 +370,7 @@ mod tests {
     }
 
     fn default_runtime() -> SealRuntimeParams {
-        SealRuntimeParams::new(DumpFormat::Postgres, &[], &[], None)
+        SealRuntimeParams::new(DumpFormat::Postgres, None)
     }
 
     #[test]
@@ -418,8 +400,8 @@ mod tests {
     #[test]
     fn runtime_options_change_digest() {
         let cfg = minimal_cfg();
-        let rt1 = SealRuntimeParams::new(DumpFormat::Postgres, &[], &[], Some(42));
-        let rt2 = SealRuntimeParams::new(DumpFormat::Postgres, &[], &[], Some(43));
+        let rt1 = SealRuntimeParams::new(DumpFormat::Postgres, Some(42));
+        let rt2 = SealRuntimeParams::new(DumpFormat::Postgres, Some(43));
         assert_ne!(
             compute_seal_digest(&cfg, "standard", &rt1).unwrap(),
             compute_seal_digest(&cfg, "standard", &rt2).unwrap()