test(eql-types): SQL-backed property tests for int4 v3 domains#290
test(eql-types): SQL-backed property tests for int4 v3 domains#290coderdan wants to merge 1272 commits into
Conversation
eql_v2_encrypted is a composite type and eql_v2.encrypted_operator_class is its DEFAULT btree opclass, so PostgreSQL invokes the opclass FUNCTION 1 comparator during ANALYZE (and for sort-based GROUP BY / DISTINCT). FUNCTION 1 was eql_v2.compare, which #211 made strict — it raises without a Block-ORE `ob` term (U-005). So ANALYZE raised on every hm-only encrypted column, and autovacuum runs ANALYZE routinely; this broke silently across managed installs. Give the opclass its own FUNCTION 1, eql_v2.encrypted_btree_compare: a total, non-raising 3-way comparator — Block ORE when present, else a total order on the hm term, else a deterministic payload tie-break. eql_v2.compare stays strict for the < / > range-operator path. This also fixes GROUP BY / DISTINCT on a bare encrypted column: with a raising opclass comparator the planner fell through to PostgreSQL's built-in record comparison (the composite type's implicit record_ops), which compares raw ciphertext — so two encryptions of the same plaintext (same hm, different c) failed to deduplicate. - src/operators/operator_class.sql: add encrypted_btree_compare, repoint FUNCTION 1; the strict eql_v2.compare is untouched. - operator_class_tests.rs: un-ignore index_behavior_with_different_data_types (broken by the strict comparator); add analyze_on_hmac_only_column_does_not_raise. - CHANGELOG: Fixed entry.
…ted-columns fix: non-raising btree comparator for eql_v2_encrypted (ANALYZE fix)
Promote the [Unreleased] section to [2.3.0] — 2026-05-20, add a fresh empty [Unreleased], and update the link references.
chore(changelog): cut 2.3.0
The Upgrade notes paragraph said "Six numbered notes ... (U-006)", but v2.3 has eight — U-007 (typed `->` selector lookup) and U-008 (typed `stevec_query` containment), both added by #223, were already referenced by the section's own Added entries. Also drop the mention of the removed fused `eql_v2.hmac_256(col, '<selector>')` recipe from the U-004 summary.
docs(changelog): correct the 2.3.0 upgrade-notes summary
Surfaces query-only medians across four row-count tiers (10k–10M) in the README, with a pointer to the cipherstash/benches repository for full methodology, per-scenario SQL, and EXPLAIN plans.
docs: add Performance section to README
Fixes #232. Under the v2.3 payload schema an `eql_v2_encrypted` column admits two mutually exclusive top-level shapes: `EncryptedPayload` (carries `c` at the root) and `SteVecPayload` (carries `sv` at the root with the root document ciphertext at `sv[0].c`). `_encrypted_check_c` previously required `val ? 'c'`, which rejected every insert of a populated SteVec payload — blocking the entire JSON-encryption path for callers on `@cipherstash/protect-ffi >= 0.22.0` (where `mode: standard` is the new default). The check now passes when either `c` or `sv` is present at the root; per-element ciphertext validity on `sv` entries continues to be enforced by the `eql_v2.ste_vec_entry` DOMAIN. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Addresses review feedback from @coderdan on #233. The pre-existing `ste_vec` and `ste_vec_vast` fixture migrations already emit real v2.3 SteVec-shape payloads (root `sv`, no root `c`), but they created their tables without applying `eql_v2.add_encrypted_constraint` — so the migrations passed even under the pre-fix `_encrypted_check_c` that rejected every populated SteVec payload. Apply the production CHECK constraint at fixture-load time so the existing fixture data is the regression guard: with the pre-fix function, all 10 inserts in `003_install_ste_vec_data.sql` (and all 500 in `005_install_ste_vec_vast_data.sql`) raise `Encrypted column missing ciphertext (c) field`; with this PR's fix they load cleanly. Verified empirically — temporarily reverted the `_encrypted_check_c` body, re-ran 003 against the resulting bundle, and confirmed it raises on the first insert. Also addresses two CodeRabbit nits: - `CHANGELOG.md`: change the parenthetical link from #232 (the issue) to #233 (this PR), per the repo's changelog convention. - `tests/sqlx/tests/constraint_tests.rs`: switch the `check_encrypted_accepts_stevec_payload` insert from `format!`-built SQL to a parameterized `$1` bind, matching the rest of the file's pattern around dynamic payload values. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Addresses follow-up review feedback from @coderdan on #233 ("I recommend adding the check for all fixture tables to prevent any regressions"). Extends the regression guard added in cf3d043 to the other fixture tables whose seed data is the production EQL payload shape: - `bench` (`tests/sqlx/migrations/007_install_bench_data.sql`) — loaded via `create_encrypted_json()` from `bench_data.sql`, every row carries the real envelope (`c`, `i`, `v=2`); constraint applied to all three encrypted columns. Verified by loading the 10K-row seed against the constraint locally. - `encrypted` in `tests/sqlx/fixtures/like_data.sql` — also seeded from `create_encrypted_json()`. The remaining fixture tables use intentionally-minimal synthetic payloads that lack required EQL envelope fields and therefore can't carry the production CHECK without rewriting the fixture data: - `ore` / `ore_text` — `{"ob": [...]}` only (no `v`, `i`, `c`), testing ORE block operator logic in isolation. - `agg_test` — v=1 payloads from the legacy aggregate-test SQL file. - `encrypted` in `encrypted_json.sql` / `array_data.sql` / `order_by_null_data.sql` — used by tests that add/remove the constraint dynamically or copy minimal-shape rows from `ore`. - `constrained` (`constraint_tables.sql`) — exists specifically to exercise Postgres-side UNIQUE/NOT-NULL/CHECK behaviour, including NULL inserts that would also collide with a production EQL CHECK. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ncrypted fix(constraints): accept SteVec payloads in _encrypted_check_c
The eql-2.3.1 tag and GitHub release were cut, but CHANGELOG.md was not promoted — the #233 SteVec CHECK fix was still under [Unreleased]. Rename [Unreleased] -> [2.3.1] — 2026-05-21, add a fresh empty [Unreleased], and update the link references (new [Unreleased] compare URL from eql-2.3.1, new [2.3.1] tag URL).
Add a verify-changelog job to the Release EQL workflow that fails the release run when CHANGELOG.md (at the tagged commit) has no `## [<version>]` section — i.e. the release was cut without first promoting [Unreleased]. This is what happened with eql-2.3.1. Skips pre-releases, which keep entries under [Unreleased] until the final cut. It's an independent job: artifact build/publish still proceed (the release is already published by then), but the workflow goes red so the omission is visible. Also fix the README Release EQL badge: it tracked `?branch=main`, but the workflow effectively never runs on main (it runs on `release` tag events). The last main run was a cancelled workflow_dispatch from 2025-11-26, so the badge showed "failing" despite every release succeeding. Switch to `?event=release`.
chore(changelog): promote [Unreleased] to [2.3.1]
Adds the encrypted-int4 fixture generator and benchmark dataset generation tooling: tasks/fixtures/ and tasks/bench.toml, the tests/benchmarks/ harness, the CipherStash Proxy docker-compose used to encrypt fixture data, the int4 fixture install migration, and bench-data test coverage. Registers the new task files in mise.toml and ignores local mise credential overrides.
Generator no longer shells out to a CipherStash Proxy container to encrypt fixture values. `cipherstash-client` 0.35 is now a direct dependency of the SQLx test crate and the new `fixtures::cipherstash` module owns ZeroKMS bootstrap (cached in a OnceCell), the index-name -> ColumnConfig mapping, and the per-value encrypt helper. The Rust fixture owns configuration and encryption; the DB only owns table structure. Removes: `tests/docker-compose.proxy.yml`, the `proxy:up`/`proxy:logs`/ `proxy:down` mise tasks, the `mise run proxy:up` step from `test:sqlx`, and the `restart_proxy_and_wait` / `insert_through_proxy` / `PROXY_PORT` plumbing in `driver.rs`. The working-table `payload` column is now plain `jsonb` (no `eql_v2_encrypted` composite) and no `add_search_config` rows are written to `eql_v2_configuration` during generation. `mise run test:sqlx` now needs `CS_CLIENT_ACCESS_KEY` (or `CS_CLIENT_ID` + `CS_CLIENT_KEY`) and `CS_WORKSPACE_CRN` in the process env so `AutoStrategy::detect()` / `EnvKeyProvider` can pick them up.
`cipherstash::encrypt_store` now takes a slice and returns a `Vec`, so a fixture run issues one `encrypt_eql` call regardless of value count instead of one per value. The 14-row `eql_v2_int4` fixture drops from 14 ZeroKMS round trips to one; the planned bench fixture (~10k rows) gets the same treatment. Empty input short-circuits before `cipher()` so a caller with nothing to encrypt does not pay the bootstrap cost. `driver::insert_direct` now does one batched encrypt then a per-row INSERT loop — the INSERT is invariant across rows so the SQL string is lifted out of the loop. The working table is local Postgres and the per-row execute cost is in microseconds, so batching the INSERTs is not worth the dynamic-SQL complexity. Test coverage closes the gap flagged in the PR review: - Cheap unit tests (no live ZeroKMS): `index_type_for` mapping for unique/ore/match plus the unknown-name error path; an empty-batch short-circuit test that proves `cipher()` is not reached when there is nothing to encrypt (visible because the test runs without CS_* env vars). - Live tests gated by `fixture-gen` + `#[ignore]`: single-value round trip, batch length + per-payload identifier check, and a distinct-plaintexts → distinct `hm` assertion that mirrors the fixture-tests' equality term check at the unit-test layer.
`test:sqlx` copies `release/cipherstash-encrypt.sql` into `tests/sqlx/migrations/001_install_eql.sql` so the EQL extension is applied to each per-test database. Without an explicit build dep a stale release artifact silently ships an old EQL extension — visible when regression-guard migrations (e.g. 003_install_ste_vec_data.sql) fail on a `_encrypted_check_c` shape the current source has already fixed. Declaring `depends = ["build"]` makes the release artifact fresh on every direct invocation of `test:sqlx`; the top-level `test.sh` already builds, so CI behaviour is unchanged.
The eql_v2_int4 fixture generator encrypts via cipherstash-client, whose EnvKeyProvider requires CS_CLIENT_ID + CS_CLIENT_KEY to load the client key. test:sqlx regenerates fixtures every run, so CI needs that pair — but f8dfd92 only wired CS_CLIENT_ACCESS_KEY + CS_WORKSPACE_CRN (ZeroKMS auth), leaving the generator failing with "CS_CLIENT_ID environment variable not set". Add the client-key pair to the test job env. Also correct comments in mise.toml, tasks/fixtures.toml, Cargo.toml, and FIXTURE_SCHEMA.md that framed the two credential pairs as alternatives: auth (access key + workspace CRN) and key material (client id + key) are distinct roles and both are required.
feat(fixtures): plug-in framework + eql_v2_int4 reference fixture
Render eql_v2_<T> jsonb-backed domain families (types, functions, operators, aggregates) from a minimal TOML manifest. Term capabilities are fixed in terms.py (hm -> equality, ore -> equality+ordering); output is byte-identical and headed AUTO-GENERATED — DO NOT EDIT. Hardening: _sql_str() quote-doubler at every SQL interpolation boundary, distinct-fixture-value validation in spec.py, SQL-identifier validation of manifest domain names, and codegen:domain / codegen:domain:all mise tasks. mise run build regenerates all domains every build. Part of PR #239 (eql_v2_int4 variant family).
Four jsonb-backed domains for encrypted int4: storage-only eql_v2_int4 (every operator blocked), eql_v2_int4_eq (hm; =, <>), and the ordered pair eql_v2_int4_ord / eql_v2_int4_ord_ore (ore; = <> < <= > >=). Domain CHECK pins the envelope version (VALUE->>'v' = '2'). Unsupported operators are error-throwing placeholders carrying an explanatory comment. Uniform extractors eql_v2.eq_term/ord_term keep functional indexes engaging without ::jsonb casts. Committed reference baselines guard byte parity. Part of PR #239.
MIN/MAX aggregates on ord-capable int4 domains route comparison through the domain's ORE operators — no decryption. min/max are associative, so the state function doubles as combinefunc; with PARALLEL SAFE sfuncs and parallel = safe, PostgreSQL can use partial/parallel aggregation on the large GROUP BY workloads these aggregates exist to serve. Part of PR #239.
Add blocker_language, blocker_strict, domain_over_domain, and domain_opclass structural lints enforcing the encrypted-domain footguns (blockers must be plpgsql and non-STRICT; no domain-over-domain; no opclass on a domain). pin_search_path recognises the converged extractor/wrapper names intrinsically. Part of PR #239.
feat: eql-types — JSON Schemas (stacked on #268)
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
b1abeb6 to
4f6e131
Compare
4f6e131 to
eb4a57d
Compare
|
Reworked the approach per the feedback (force-pushed Per-function doubles mirroring the domain "trait". Instead of one generic
Each domain's test module then has a property per function: for Inline comments all addressed (replies on each thread): use Verified locally: builds (default + |
eb4a57d to
d5e7994
Compare
d6f1a0b to
2946b68
Compare
|
I am rolling this into the core property test work in PR 275 as additional property test coverage. Proptest seems to now be the preferred option over quickcheck, but happy to revisit or use both if they provide orthogonal coverage. |
2946b68 to
bf8b9de
Compare
bf8b9de to
a3ea6f2
Compare
Checks the eql-types Rust wire types against the real `eql_v3.*` SQL functions (issue #237) via the "function double" pattern, modeled on the cllw-ore crate: encrypt with cipherstash-client, run the generated SQL function, assert it agrees with the plaintext oracle. Shared harness in `proptest_support` (feature `proptest`): a `Runner` of 1:1 SQL-function doubles gated by per-capability traits (`EqTerm`/`OrdTerm`, `Comparable`/`Ordered`); operand-typed comparison overloads (`&x` casts to the domain, `Jsonb(&x)` stays raw jsonb); `ProptestScalar` (i32/i64) for the plaintext→`Plaintext` mapping; `eq_comparisons_match`/`ordered_comparisons_match` for the oracle; and `impl_eq_domain!`/`impl_ord_domain!` macros that stamp the per-domain v2→v3 conversion + `EncryptableScalar` + `Arbitrary` + capability markers. The jsonb `Type`/`Encode`/`Decode` macro moves to `sqlx_support` (feature `sqlx`). With that, a scalar family is ~3 macro invocations + the readable per-function properties. - int4 (plaintext i32) and int8 (plaintext i64) families covered: `eq_term`/ `ord_term` extractors + `eq`/`neq`/`lt`/`lte`/`gt`/`gte` across all three generated overloads, co-located per domain (`int4.rs`, `int8.rs`). - cipherstash-client only emits the general EQL v2 `EncryptedPayload`; the conversions read its typed fields to build the narrow v3 structs. - Auth: auto mode (`CS_*` env or `stash auth login` profile) + a FallbackKeyProvider for the client key; the tests panic (fail) without it. - mise `test:proptest` (pins `QUICKCHECK_TESTS=32`); CI `proptest` job. Default build stays lean (all new deps optional/feature-gated); `test:crates` unchanged. Full DB run needs Postgres + ZeroKMS auth (CI); verified locally via build / `--no-run` / clippy / fmt. Follow-up filed: #292 (SQL-level mutation testing / fuzzing for these functions).
a3ea6f2 to
b2ba1e9
Compare
We've used quickcheck everywhere else (other repos) and its a lot simpler but we can shift to proptest if you think its better. TBH I like the simplicity of the Also, I feel pretty strongly about keeping these tests right next to the types themselves. Its super hard to reason about what unit tests are present if they're in a different crate. |
Adds SQL-backed property tests for the
eql-typesv3 scalar domains — first cut (int4), per issue #237.Why
The pure-Rust tests prove the types match the catalog and round-trip through serde, but nothing checks them against the actual
eql_v3.*SQL functions — the structs and the SQL domains are defined independently, so drift can pass silently.Approach — a double + property per SQL function (the domain's "trait")
Each
eql_v3scalar domain has a set of generated SQL functions; the doubles mirror that surface:term_compareexercises the extractor (eq_term/ord_term) directly, composed with the term type's own operator (the generated wrapper body).call_boolcovers each comparison operator across all three generated overloads —(domain,domain),(domain,jsonb),(jsonb,domain).Each domain's test module wires a property per function against the plaintext oracle:
int4_eq→eq_term/eq/neq;int4_ord&int4_ord_ore→ord_term+eq/neq/lt/lte/gt/gte(all overloads).What's here
sqlxfeature — jsonbType/Encode/Decode<Postgres>for the domain types, so a payload binds as a function-double arg and casts to its domain in SQL.proptestfeature (depends onsqlx) — shared harness insrc/v3/proptest_support.rs: one Tokio runtime + sqlx pool + cipherstash-client cipher (auto mode), a quickcheckArbitrary(Sample<T>) that encrypts a randomi32, and thecall_bool/term_comparedoubles.src/v3/int4.rs.test:proptest:prep(build + install standalone v3 SQL into a scratch DB) andtest:proptest. CI: a creds-bearingproptestjob wired intoci-required.cipherstash-client emits EQL v2 — so we convert
cipherstash-client only encrypts to the general EQL v2 payload (
cipherstash_client::eql::EncryptedPayload—v/i/c+ optionalhm/bf/ob). Each domain's test module owns the conversion into its specific v3 struct (From<&EncryptedPayload>), reading the client type's typed fields. The v3 domain CHECK accepts extra keys (it only tests presence) — flag separately if we want it tightened.Auth — no skipping
The cipher is built in auto mode (
ZeroKMSBuilder::auto+EnvKeyProvider), exactly like the SQLx fixture generator:AutoStrategyresolves credentials from env (CI:CS_*) or astash auth loginprofile. If auth is unavailable the build panics (the test fails) — the tests never silently skip.Lean build is unaffected
All new deps (
sqlx,tokio,quickcheck,cipherstash-client) are optional / feature-gated; the default build andmise run test:cratesare unchanged.Verification
cargo build(default +--features sqlx/proptest),cargo test --features proptest --no-run(compiles all 7 property tests), clippy + fmt clean,mise run test:cratesgreen.proptestjob, or a dev box with Postgres + ZeroKMS auth):mise run test:proptest.Scope / follow-ups
int4 first (issue #237's scope). The harness generalizes: other scalar families (int2/int8/date/timestamptz/text) and bloom-
matchare follow-ups, each adding its ownFrom<&EncryptedPayload>conversions + per-function properties.Addresses #237.