Performance + first-class email/name/phone strategies#55
Merged
Conversation
- Use 256 KiB BufReader/BufWriter for file and pipe I/O to cut syscalls. - Avoid full-string to_uppercase for INSERT detection and VALUES search; add ASCII case-insensitive find and prefix checks. - statement_complete: iterate chars without collecting Vec<char>. - parse_parenthesized_values: char_indices iterator instead of Vec<char>. - Row filters and column_cases: pass &[Option<&str>] to avoid per-row String clones for COPY and INSERT; only allocate when JSON paths need owned strings. - INSERT output: pre-size String, manual comma join; skip stmt.to_string() for disabled tables. Co-authored-by: Andy Babic <ababic@users.noreply.github.com>
…ker path - Add email, name, first_name, last_name to KNOWN_STRATEGIES; stop rewriting them to faker at resolve time; lowercase strategy names in resolve(). - Transform: dedicated match arms calling shared pii_* helpers in faker_dispatch; deterministic path unchanged except locale via resolved_locale_key. - AnonymizerRegistry: RefCell<StdRng> reused for random-path faker, phone, and PII strategies (no per-cell StdRng::from_seed). - faker_dispatch: resolved_locale_key (no String alloc for default/en); faker_string_with_rng uses it; add pii_* + pii_phone_number exports. - sql infer_auto_strategy uses built-in strategies; drop unused faker_spec. - Docs: README table and examples; CHANGELOG Unreleased; .dumplingconf.example. Co-authored-by: Andy Babic <ababic@users.noreply.github.com>
cursor
Bot
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Performance work on the SQL hot path (previous revision) plus first-class PII strategies restored in config and runtime, with faker/phone random-path optimizations.
Performance (SQL / I/O)
to_uppercase; lighter INSERT output building.Strategies (PII)
email,name,first_name,last_nameare again realKNOWN_STRATEGIESentries. Config is no longer rewritten tostrategy = "faker"at resolve time;fakeris only required when using the open-ended faker allowlist.phonewas already first-class; validation now allowslocaleon the name/email strategies too.resolve()lowercasesstrategystrings on load soEmail/EMAILbehave likeemail.Faker / RNG optimizations
AnonymizerRegistryholds aRefCell<StdRng>seeded once (make_random_rngat registry construction) and reuses it for random-pathfaker,phone, and built-in PII strategies (no per-cellStdRng::from_seed).resolved_locale_keyinfaker_dispatchavoids allocating aStringfor locale on every faker call when defaulting toenor when the value is already a known key.Docs / examples
.dumplingconf.example; CHANGELOG [Unreleased]; settings field docs.Testing
cargo test --all-targets --all-featurescargo clippy --all-targets --all-featurescargo fmt --all -- --checkSlack Thread