Skip to content

feat(kms-app): ZeroKMS vs AWS KMS load-test harness#18

Open
coderdan wants to merge 2 commits into
mainfrom
feat/kms-comparison-harness
Open

feat(kms-app): ZeroKMS vs AWS KMS load-test harness#18
coderdan wants to merge 2 commits into
mainfrom
feat/kms-comparison-harness

Conversation

@coderdan

@coderdan coderdan commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

Scaffolds a self-contained Next.js + Artillery subproject under kms-app/ for comparing field-level encryption backends under HTTP load. Replaces the older comparison app (which appears lost) and gives it a home next to the EQL/PostgreSQL benchmarks.

What it does

The same thin CRUD app, same Postgres, same load profile — only the encryption backend changes, selected per server process by ENCRYPTION_BACKEND:

Backend Pattern
zerokms CipherStash Encryption SDK (@cipherstash/stack), unique key per record
aws-kms Naive direct KMS Encrypt/Decrypt per value
aws-kms-envelope Production AWS pattern: KMS-wrapped AES-256 data key + local AES-256-GCM, with data-key caching (ENVELOPE_DATA_KEY_MAX_USES)

All three implement a common EncryptionBackend interface (lib/encryption/). API: POST /api/users (encrypt+store), GET /api/users/:id (read+decrypt), GET /api/health (readiness). Artillery drives a create-then-read flow; scripts/summarize.mjs turns the JSON outputs into a side-by-side latency/throughput table (skips any backend you didn't run).

Why three AWS variants matter

aws-kms (direct) is 4 KB-limited and region-rate-limited — a saturation run partly measures KMS throttling. aws-kms-envelope is how AWS recommends encrypting bulk application data and, with data-key caching, is the fair high-throughput comparison (set ENVELOPE_DATA_KEY_MAX_USES=1 for the no-caching worst case). Report write and read paths separately. Full fairness notes in the README.

Conventions

  • Benches native Postgres cluster (:5400); sql/schema.sql
  • Outputs to results/ (gitignored, dir kept)
  • Self-contained package.json scripts — does not touch the root mise.toml

Status

  • File syntax (JS/JSON/YAML), the summarizer, and the envelope's AES-256-GCM round-trip + auth-tag tamper rejection are validated standalone.
  • Not yet npm install-ed or built — needs network + backend credentials. The @cipherstash/stack API surface is modeled on the published docs examples; confirm on install.
  • Follow-ups: reconcile load/users.yml against the original Artillery Cloud scenario (share sh_75edb…); add a committable Markdown report:build. This harness is the intended source for the docs' currently-unsourced "14× faster than AWS KMS" claim (Finish ZeroKMS vs HSM comparison + add comparison diagrams docs#32).

coderdan added 2 commits June 23, 2026 20:29
A self-contained Next.js + Artillery subproject for comparing field-level
encryption backends under HTTP load, replacing the older lost harness.

- Swappable backend via ENCRYPTION_BACKEND (zerokms via @cipherstash/stack,
  aws-kms via @aws-sdk/client-kms) behind a common EncryptionBackend interface
- CRUD API: POST /api/users (encrypt+store), GET /api/users/:id (read+decrypt),
  GET /api/health (readiness)
- Artillery profile (create-then-read flow) + results summarizer
- Uses the benches native Postgres cluster (:5400); sql/schema.sql
- README documents fairness caveats (direct-KMS vs envelope encryption) and
  the run/compare workflow

Self-contained npm scripts; does not touch the root mise.toml. Not yet
installed/built (needs network + backend credentials).
KMS protects a local AES-256 data key; field values are encrypted locally
with AES-256-GCM. Removes the 4KB direct-KMS limit and, via data-key caching
(ENVELOPE_DATA_KEY_MAX_USES), avoids one KMS call per value — the fairer
high-throughput comparison against ZeroKMS.

- new backend behind the existing EncryptionBackend interface; selectable via
  ENCRYPTION_BACKEND=aws-kms-envelope
- write-side DEK reuse + bounded read-side plaintext-DEK cache
- serve/load npm scripts; report now spans all three backends (skips any not run)
- README fairness section + .env.example updated; naive aws-kms TODO resolved

Local AES-GCM round-trip + auth-tag tamper rejection verified standalone.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant