EAI-7230: Rate-limiting PoC (RL-1) — per-key request + token enforcement

[johnl-amd]

What

Spike scaffolding (RL-1) to prove native, per-API-key request + generation-token enforcement on the AI gateway using Envoy Gateway global / cost-based rate limiting backed by Redis. No proxy or upstream code.

Changes

Change	File
Enable EG global rate-limit service, point it at in-cluster Redis	root/values.yaml (config.envoyGateway.rateLimit.backend)
Ephemeral Redis (Deployment + Service, no persistence/HA)	sources/envoy-gateway-config/templates/ratelimit-redis.yaml
Per-key BackendTrafficPolicy (request count + token budget)	sources/envoy-gateway-config/templates/backend-traffic-policy-ratelimit.yaml
Toggles + policy params	sources/envoy-gateway-config/values.yaml (rateLimit.*)
Runbook	docs/rate-limit-spike.md

Safe by default

• rateLimit.enabled: true → deploys Redis + the rate-limit service.
• rateLimit.policy.enabled: false → the per-key policy does not render until a real key is supplied.

So merging/deploying this readies the infra and enforces nothing — zero impact on inference until we plug in a key and flip the policy on.

Builds on what's already live

• Token costs are emitted as io.envoy.ai_gateway:llm_output_token via AIGatewayRoute.spec.llmRequestCosts (ai-gateway-discovery).
• x-api-key-id is injected by cluster-auth ext_authz.

The policy keys on exactly those — this PR only adds the Redis backend + policy.

Known risk to verify on cluster

The rate-limit filter must run after ext_authz or x-api-key-id is absent at limiting time. The ai-gateway EnvoyProxy already orders lua before ext_authz; live filter-chain ordering will be confirmed before enabling the policy (Step 2 in the runbook).

Test plan

See docs/rate-limit-spike.md. Once deployed to app-dev with a real key:

1. N requests/min → HTTP 429 after the limit.

2. Output tokens over budget → subsequent calls 429 (one-request overshoot, charged from prior response metadata).
3. A different key keeps working (per-key isolation).
4. Counts hold across Envoy replicas (shared Redis).

Follow-ups (RL-2+)

Redis HA/persistence posture, windows offered (min/hour/day), confirm "generation tokens" = llm_output_token vs total, narrow targetRef to per-service HTTPRoute (or add x-aim-service-id selector) for per-inference-service limits.