diff --git a/.changeset/more-usage-patterns.md b/.changeset/more-usage-patterns.md new file mode 100644 index 000000000..f6f1d9784 --- /dev/null +++ b/.changeset/more-usage-patterns.md @@ -0,0 +1,5 @@ +--- +"counterfact": patch +--- + +add two new usage pattern docs: Automated Integration Tests (programmatic API in test suites) and Custom Middleware (_.middleware.ts for cross-cutting concerns); propose three future patterns as GitHub issues: Record and Replay, Webhook Simulation, and Persistent State diff --git a/.github/issue-proposals/docs-pattern-persistent-state.md b/.github/issue-proposals/docs-pattern-persistent-state.md new file mode 100644 index 000000000..98d2869b2 --- /dev/null +++ b/.github/issue-proposals/docs-pattern-persistent-state.md @@ -0,0 +1,43 @@ +--- +title: "docs: add Persistent State usage pattern" +parentIssue: 1805 +labels: + - documentation + - enhancement +assignees: [] +milestone: +--- + +Add a usage pattern for persisting Counterfact context state to disk so that it survives server restarts. + +## Context + +Counterfact's in-memory state resets to its initial values every time the server restarts. This is intentional and usually desirable during development — it provides a clean slate. However, some workflows need state to survive restarts: + +- **Shared demo environments**: a team seeds realistic data and then shares the mock across a demo session that spans multiple days. Restarting the server to pick up a new version of a handler file should not wipe the demo data. +- **Long-running integration environments**: a staging or QA environment runs the mock for days or weeks; testers build up test data over multiple sessions. +- **Seeded fixtures**: a project ships a `seed.json` file with canonical test data that is loaded at startup rather than recreated by hand after every restart. + +Counterfact currently has no built-in persistence mechanism. Workarounds involve writing a startup script that calls context methods to re-seed data, or serializing context state to a file manually and loading it back in a custom startup hook. + +## Proposed feature + +Add a `--persist-state ` CLI flag (or a `persist` option in `counterfact.yaml`) that: +1. At shutdown, serializes the context's state to a JSON file at the specified path. +2. At startup, loads the JSON file and hydrates the context before the server begins accepting requests. + +Context classes would opt in by implementing `toJSON()` and `fromJSON(data)` methods (or a similar convention). This keeps the persistence mechanism decoupled from the context class interface. + +A `Persistent State` pattern document would describe: +- When to use it: shared demo environments, long-running QA mocks, or fixture-based seeding +- How to enable persistence with `--persist-state` +- How to implement `toJSON` / `fromJSON` in a context class +- Consequences: state files can drift from the spec; schema migrations are the author's responsibility; not suitable for high-concurrency scenarios + +## Acceptance criteria + +- [ ] `--persist-state ` CLI flag (or equivalent) is implemented +- [ ] Context classes can opt in to persistence by implementing a documented interface +- [ ] `docs/patterns/persistent-state.md` is added following the established pattern format +- [ ] The new pattern is linked in `docs/usage-patterns.md` +- [ ] The reference doc is updated to describe the new CLI flag and context interface diff --git a/.github/issue-proposals/docs-pattern-record-and-replay.md b/.github/issue-proposals/docs-pattern-record-and-replay.md new file mode 100644 index 000000000..1021a0375 --- /dev/null +++ b/.github/issue-proposals/docs-pattern-record-and-replay.md @@ -0,0 +1,34 @@ +--- +title: "docs: add Record and Replay usage pattern" +parentIssue: 1805 +labels: + - documentation + - enhancement +assignees: [] +milestone: +--- + +Add a usage pattern documenting a record-and-replay workflow where real API traffic is captured and replayed by Counterfact as a mock. + +## Context + +Record-and-replay is a common mock strategy supported by tools like WireMock, VCR (Ruby), Polly.js, and others. The idea is: point a proxy at the real API once, record all the responses, then replay those exact responses from the mock without hitting the real API again. + +Counterfact currently has no built-in record-and-replay feature. The proxy (`--proxy-url`) forwards requests to the real backend but does not persist the responses. Users who want to bootstrap a mock from real traffic must manually inspect responses and write handlers by hand. + +## Proposed feature + +Add a `--record` CLI flag (or similar) that, when combined with `--proxy-url`, captures every request-response pair and writes it to the handler files under the output directory. Each captured response is stored as a named example in the handler, so subsequent requests are served from the recorded data without proxying. + +A `Record and Replay` usage pattern document would describe: +- When to use it: you want realistic responses without writing handlers by hand, and the real API is available at least once +- How to record: start Counterfact with `--proxy-url` and `--record`, then exercise the relevant API paths +- How to replay: restart without `--proxy-url`; all recorded responses are served from the generated handlers +- Consequences: recorded responses go stale when the real API changes; good for bootstrapping, not for long-lived mocks + +## Acceptance criteria + +- [ ] A `--record` (or equivalent) CLI flag is implemented that writes captured responses to handler files +- [ ] `docs/patterns/record-and-replay.md` is added following the established pattern format (Context, Problem, Solution, Example, Consequences, Related Patterns) +- [ ] The new pattern is linked in `docs/usage-patterns.md` +- [ ] The reference doc is updated to describe the new CLI flag diff --git a/.github/issue-proposals/docs-pattern-webhook-simulation.md b/.github/issue-proposals/docs-pattern-webhook-simulation.md new file mode 100644 index 000000000..e35744fc2 --- /dev/null +++ b/.github/issue-proposals/docs-pattern-webhook-simulation.md @@ -0,0 +1,38 @@ +--- +title: "docs: add Webhook Simulation usage pattern" +parentIssue: 1805 +labels: + - documentation + - enhancement +assignees: [] +milestone: +--- + +Add a usage pattern documenting how to simulate webhooks and push events from a Counterfact mock to a client application. + +## Context + +Many modern APIs are not purely request/response. They also push data to clients via webhooks (HTTP callbacks), Server-Sent Events (SSE), or WebSocket messages. Examples include Stripe's payment webhooks, GitHub's repository event hooks, and Slack's event subscriptions. + +Counterfact currently has no built-in mechanism to initiate outbound HTTP calls or push events to a client. Developers building or testing webhook consumers have to either hit the real API or write a separate one-off script to send fake webhook payloads. + +## Proposed feature + +Add a way to trigger outbound HTTP calls from a Counterfact mock — for example, a `$.webhook(url, payload)` helper available in handlers or via the REPL. Combined with the existing REPL, this would allow developers to simulate the full webhook lifecycle interactively: + +1. Client subscribes to a webhook by calling `POST /webhooks` on the mock +2. Developer triggers the event from the REPL: `client.post("/webhooks/trigger", { event: "payment.succeeded", ... })` +3. Counterfact sends an HTTP POST to the registered callback URL with the event payload + +A `Webhook Simulation` pattern document would describe: +- When to use it: you are building a webhook consumer and need to test it locally without a real event source +- How to register a callback URL in the mock's context on subscription +- How to trigger an outbound call from the REPL or a handler +- Consequences and limitations: the callback URL must be reachable from the mock's process; production webhook signatures are not simulated unless explicitly added to the handler + +## Acceptance criteria + +- [ ] An outbound-call mechanism (e.g., `$.webhook()` or `$.emit()`) is implemented and documented +- [ ] `docs/patterns/webhook-simulation.md` is added following the established pattern format +- [ ] The new pattern is linked in `docs/usage-patterns.md` +- [ ] The reference doc is updated to describe the new API diff --git a/docs/patterns/automated-integration-tests.md b/docs/patterns/automated-integration-tests.md new file mode 100644 index 000000000..be6014c39 --- /dev/null +++ b/docs/patterns/automated-integration-tests.md @@ -0,0 +1,131 @@ +# Automated Integration Tests + +You have a client application or SDK and want to write automated integration tests that make real HTTP requests against a controlled mock server. + +## Problem + +Unit tests that mock HTTP calls at the library boundary (using `jest.mock()`, `vi.mock()`, or a fetch interceptor) do not exercise real HTTP behavior — headers, routing, content negotiation, middleware, or status code handling. Running tests against a real backend is slower, requires infrastructure, and produces non-deterministic results. + +## Solution + +Use Counterfact's programmatic API to embed the mock server directly in your test suite. Start it in a `beforeAll` hook, run every test against it, and stop it in `afterAll`. The server is entirely local, so tests are fast and deterministic. Handler files control exactly what the mock returns for each test scenario. + +## Example + +Install Counterfact as a dev dependency and generate route files from your spec: + +```sh +npm install --save-dev counterfact +npx counterfact openapi.yaml api --generate +``` + +Start and stop the server around your test suite: + +```ts +import { counterfact } from "counterfact"; + +const port = 4001; +let stop: () => Promise; + +const config = { + openApiPath: "./openapi.yaml", + basePath: "./api", + port, + startServer: true, + generate: { routes: false, types: false }, + watch: { routes: false, types: false }, +}; + +beforeAll(async () => { + const app = await counterfact(config); + ({ stop } = await app.start(config)); +}); + +afterAll(async () => { + await stop(); +}); +``` + +Write tests that send real HTTP requests to the running server: + +```ts +it("returns 200 and a pet when the pet exists", async () => { + const response = await fetch(`http://localhost:${port}/pet/1`); + expect(response.status).toBe(200); + const body = await response.json(); + expect(body).toHaveProperty("id"); +}); + +it("returns 404 when the pet does not exist", async () => { + const response = await fetch(`http://localhost:${port}/pet/99999`); + expect(response.status).toBe(404); +}); +``` + +To control exactly what a handler returns for a specific test, use a context flag: + +```ts +// api/routes/_.context.ts +export class Context { + simulatePetNotFound = false; +} +``` + +```ts +// api/routes/pet/{petId}.ts +export const GET: HTTP_GET = ($) => { + if ($.context.simulatePetNotFound) { + return $.response[404].text("Not found"); + } + return $.response[200].json({ id: $.path.petId, name: "Fluffy", status: "available" }); +}; +``` + +Reach into the live context via the `contextRegistry` returned by `counterfact()` to toggle behavior per test: + +```ts +import { counterfact } from "counterfact"; + +const port = 4001; +let contextRegistry: Awaited>["contextRegistry"]; +let stop: () => Promise; + +const config = { + openApiPath: "./openapi.yaml", + basePath: "./api", + port, + startServer: true, + generate: { routes: false, types: false }, + watch: { routes: false, types: false }, +}; + +beforeAll(async () => { + const app = await counterfact(config); + contextRegistry = app.contextRegistry; + ({ stop } = await app.start(config)); +}); + +afterAll(async () => { + await stop(); +}); + +it("returns 404 when the flag is set", async () => { + contextRegistry.find("/").simulatePetNotFound = true; + const response = await fetch(`http://localhost:${port}/pet/1`); + expect(response.status).toBe(404); + contextRegistry.find("/").simulatePetNotFound = false; +}); +``` + +## Consequences + +- Tests send real HTTP requests, so they exercise routing, middleware, headers, and content negotiation — not just handler logic. +- The server starts and stops once per suite, keeping test overhead low even with many test cases. +- Context flags make it easy to test error branches without writing a separate handler file per scenario. +- Handler files remain the single source of truth for mock behavior; the test suite does not need to duplicate response logic. + +## Related Patterns + +- [Simulate Failures and Edge Cases](./simulate-failures.md) — the context-flag technique for toggling error conditions +- [Test the Context, Not the Handlers](./test-context-not-handlers.md) — unit-test context logic independently of the HTTP layer +- [Mock APIs with Dummy Data](./mock-with-dummy-data.md) — shape the responses the integration tests assert against diff --git a/docs/patterns/custom-middleware.md b/docs/patterns/custom-middleware.md new file mode 100644 index 000000000..ace4ee9f4 --- /dev/null +++ b/docs/patterns/custom-middleware.md @@ -0,0 +1,108 @@ +# Custom Middleware + +You need behavior that applies uniformly across a group of routes — authentication checks, response headers, request logging, or any cross-cutting concern — without repeating the logic in every handler file. + +## Problem + +Handlers are a poor place for cross-cutting concerns. Adding an auth check, a CORS header, or a request log to every handler creates duplication, makes the handlers harder to read, and means that a new route will silently miss the concern unless the author remembers to add it. + +## Solution + +Drop a `_.middleware.ts` file into any directory in the routes tree. It exports a standard Koa middleware function that applies to all routes in that subtree. Handlers in the subtree receive the request only after the middleware has run, so a rejected request never reaches a handler. + +Place middleware close to the routes it governs. A `_.middleware.ts` at the root applies everywhere. One under `routes/admin/` applies only to admin endpoints. + +## Example + +### Authentication gate + +Reject unauthenticated requests before any handler runs: + +```ts +// routes/_.middleware.ts +import type { Middleware } from "koa"; + +const middleware: Middleware = async (ctx, next) => { + if (!ctx.headers.authorization) { + ctx.status = 401; + ctx.body = "Unauthorized"; + return; + } + await next(); +}; + +export default middleware; +``` + +All routes now require an `Authorization` header. A missing header returns `401` immediately — no handler is called. + +### Adding response headers + +Attach headers to every response in a subtree — useful for simulating CORS, rate-limit headers, or tracing identifiers: + +```ts +// routes/_.middleware.ts +import { randomUUID } from "node:crypto"; +import type { Middleware } from "koa"; + +const middleware: Middleware = async (ctx, next) => { + await next(); + ctx.set("x-ratelimit-limit", "1000"); + ctx.set("x-ratelimit-remaining", "999"); + ctx.set("x-request-id", randomUUID()); +}; + +export default middleware; +``` + +### Request logging + +Log every request and its response status without touching any handler: + +```ts +// routes/_.middleware.ts +import type { Middleware } from "koa"; + +const middleware: Middleware = async (ctx, next) => { + const start = Date.now(); + await next(); + const elapsed = Date.now() - start; + console.log(`${ctx.method} ${ctx.path} → ${ctx.status} (${elapsed}ms)`); +}; + +export default middleware; +``` + +### Scoped middleware + +Apply middleware only to a specific domain. The `_.middleware.ts` under `routes/admin/` fires for admin routes only; all other routes are unaffected: + +```ts +// routes/admin/_.middleware.ts +import type { Middleware } from "koa"; + +const middleware: Middleware = async (ctx, next) => { + const token = ctx.headers["x-admin-token"]; + if (token !== "super-secret") { + ctx.status = 403; + ctx.body = "Forbidden"; + return; + } + await next(); +}; + +export default middleware; +``` + +## Consequences + +- Middleware applies automatically to every new route added to the subtree — no per-handler boilerplate is required. +- A single `_.middleware.ts` at the routes root applies to all paths; placing it in a subdirectory scopes it to that domain. +- Handlers remain focused on response logic; authentication, headers, and logging live in a separate file. +- Middleware runs in Koa's standard onion model — you can add logic before `next()` (pre-processing) and after (post-processing). + +## Related Patterns + +- [Simulate Failures and Edge Cases](./simulate-failures.md) — middleware can reject or modify requests without touching handlers +- [Federated Context Files](./federated-context.md) — the same directory-scoping model applies to both context files and middleware files +- [Reference Implementation](./reference-implementation.md) — add middleware to replicate the authentication and header behavior of the real API diff --git a/docs/usage-patterns.md b/docs/usage-patterns.md index 56f7d351b..4d6cb2939 100644 --- a/docs/usage-patterns.md +++ b/docs/usage-patterns.md @@ -2,7 +2,7 @@ A pattern is a reusable solution to a recurring problem when building API simulations with Counterfact. Each pattern below describes a context, the problem it addresses, the solution, and its consequences. -Most projects start with [Explore a New API](./patterns/explore-new-api.md) or [Executable Spec](./patterns/executable-spec.md) to get a running server from an OpenAPI spec with no code. From there, [Mock APIs with Dummy Data](./patterns/mock-with-dummy-data.md) and [AI-Assisted Implementation](./patterns/ai-assisted-implementation.md) are the natural next steps for adding realistic responses — the former by hand, the latter with an AI agent doing the heavy lifting. As the mock grows, [Federated Context Files](./patterns/federated-context.md) and [Test the Context, Not the Handlers](./patterns/test-context-not-handlers.md) keep the stateful logic organized and reliable. Throughout all of this, [Live Server Inspection with the REPL](./patterns/repl-inspection.md) is Counterfact's most distinctive feature: it lets you seed data, send requests, and toggle behavior in real time without restarting. [Simulate Failures and Edge Cases](./patterns/simulate-failures.md) and [Simulate Realistic Latency](./patterns/simulate-latency.md) extend any mock to cover error paths and performance characteristics that real services exhibit. [Reference Implementation](./patterns/reference-implementation.md) and [Executable Spec](./patterns/executable-spec.md) make the mock a first-class artifact that teams can rely on as the API evolves. Finally, [Agentic Sandbox](./patterns/agentic-sandbox.md) and [Hybrid Proxy](./patterns/hybrid-proxy.md) address the two common integration strategies — isolating an AI agent from the real service, or blending mock and live traffic across endpoints. +Most projects start with [Explore a New API](./patterns/explore-new-api.md) or [Executable Spec](./patterns/executable-spec.md) to get a running server from an OpenAPI spec with no code. From there, [Mock APIs with Dummy Data](./patterns/mock-with-dummy-data.md) and [AI-Assisted Implementation](./patterns/ai-assisted-implementation.md) are the natural next steps for adding realistic responses — the former by hand, the latter with an AI agent doing the heavy lifting. As the mock grows, [Federated Context Files](./patterns/federated-context.md) and [Test the Context, Not the Handlers](./patterns/test-context-not-handlers.md) keep the stateful logic organized and reliable. Throughout all of this, [Live Server Inspection with the REPL](./patterns/repl-inspection.md) is Counterfact's most distinctive feature: it lets you seed data, send requests, and toggle behavior in real time without restarting. [Simulate Failures and Edge Cases](./patterns/simulate-failures.md) and [Simulate Realistic Latency](./patterns/simulate-latency.md) extend any mock to cover error paths and performance characteristics that real services exhibit. [Reference Implementation](./patterns/reference-implementation.md) and [Executable Spec](./patterns/executable-spec.md) make the mock a first-class artifact that teams can rely on as the API evolves. Finally, [Agentic Sandbox](./patterns/agentic-sandbox.md) and [Hybrid Proxy](./patterns/hybrid-proxy.md) address the two common integration strategies — isolating an AI agent from the real service, or blending mock and live traffic across endpoints. [Automated Integration Tests](./patterns/automated-integration-tests.md) shows how to embed the mock server in a test suite using the programmatic API, while [Custom Middleware](./patterns/custom-middleware.md) covers cross-cutting concerns like authentication and response headers without touching individual handlers. | Pattern | When to use it | |---|---| @@ -18,6 +18,8 @@ Most projects start with [Explore a New API](./patterns/explore-new-api.md) or [ | [Reference Implementation](./patterns/reference-implementation.md) | You want a working, executable implementation that expresses intended API behavior in code | | [Agentic Sandbox](./patterns/agentic-sandbox.md) | You are building an AI coding agent and want to avoid rate limits and costs during development | | [Hybrid Proxy](./patterns/hybrid-proxy.md) | Some endpoints exist in the real backend; others need to be mocked | +| [Automated Integration Tests](./patterns/automated-integration-tests.md) | You want to run real HTTP tests against the mock in a CI-friendly test suite | +| [Custom Middleware](./patterns/custom-middleware.md) | You want authentication, headers, or logging applied uniformly across a group of routes | ## See also