Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 33 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -290,6 +290,39 @@ signal the web login UI reads to switch on the **identifier-first** experience
`engineActive=false` keeps the legacy single-screen email+password form, so the
UI redesign reverts with the env flag and no web redeploy.

### Client-side embedding kill-switches — FACE + VOICE (default OFF)

Two independent gates route a precomputed client-computed embedding to the
biometric-processor's embedding endpoints instead of uploading the raw
image/audio (privacy + GPU-less; the raw media never leaves the device). Both
default OFF (legacy server-side path, byte-identical), flip WITHOUT a redeploy,
and support per-tenant canary. **Both MUST be passed via the compose
`environment:` block** (the service uses an explicit block, NOT `env_file:` — a
var only in `.env.prod` is silently dropped; this exact gap once broke the face
flag).

- **FACE** — `ClientSideEmbeddingPolicy` (`app.auth.client-side-embedding`):
`FaceVerifyMfaStepHandler` + `BiometricController.enrollFaceEmbedding` route a
512-d Facenet512 vector to bio `/verify-embedding` / `/enroll-embedding`.
Env: `APP_AUTH_CLIENT_SIDE_EMBEDDING(_TENANTS)`.
- **VOICE (audit H3, GPU-less)** — `ClientSideVoiceEmbeddingPolicy`
(`app.auth.client-side-voice-embedding`): `VoiceVerifyMfaStepHandler` (reads
the `embedding` key off the MFA `data` map when ON, else the legacy `voiceData`)
+ `BiometricController.enrollVoiceEmbedding`
(`POST /api/v1/biometric/voice/enroll-embedding/{userId}`,
`VoiceEnrollEmbeddingRequest`, 256-length-validated, fail-closed when OFF) route
a 256-d Resemblyzer speaker vector to bio `/voice/verify-embedding` /
`/voice/enroll-embedding`. Env:
`APP_AUTH_CLIENT_SIDE_VOICE_EMBEDDING(_TENANTS)`.

SECURITY: an embedding carries no media, so the bio side cannot run
liveness/replay on it — an embedding FACE/VOICE factor MUST be paired with a
liveness factor in the auth flow. **Rollout ordering:** flip this (identity) flag
ON BEFORE the web `VITE_CLIENT_SIDE_*_EMBEDDING` flag (web-ON + identity-OFF
breaks the factor). The VOICE web preprocessing port is still a documented
scaffold (browser mel+VAD not yet parity-validated — see biometric-processor
`docs/design/VOICE_CLIENT_EMBEDDING_SPEC.md`); keep the voice flag OFF until it is.

### Operator reality — 2026-05-30 stabilize-&-harden backlog (P1-1 + P1-5, DEPLOYED)

- **P1-1 — cross-tenant isolation ITs are now a CI gate (PR #155/#156).** The
Expand Down
13 changes: 13 additions & 0 deletions docker-compose.prod.yml
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,19 @@ services:
# this (identity) flag ON before web VITE_CLIENT_SIDE_EMBEDDING.
APP_AUTH_CLIENT_SIDE_EMBEDDING: ${APP_AUTH_CLIENT_SIDE_EMBEDDING:-false}
APP_AUTH_CLIENT_SIDE_EMBEDDING_TENANTS: ${APP_AUTH_CLIENT_SIDE_EMBEDDING_TENANTS:-}
# Client-side VOICE embedding (audit H3, GPU-less). MUST be listed here for
# the SAME reason as the face flag above (explicit environment: block, not
# env_file:). When ON, the browser computes the 256-d Resemblyzer speaker
# vector and the server does ONLY a pgvector cosine compare — the raw audio
# never leaves the device. Default OFF = legacy server-side Resemblyzer
# forward pass on uploaded audio. Reversible via .env.prod + `up -d` recreate
# (no rebuild). Canary one tenant via _TENANTS=<uuid>. Pair ordering: this
# (identity) flag ON before web VITE_CLIENT_SIDE_VOICE_EMBEDDING. NOTE: the
# browser preprocessing port is a documented scaffold (not yet parity-verified
# — see biometric-processor docs/design/VOICE_CLIENT_EMBEDDING_SPEC.md); keep
# OFF until the client mel+VAD is validated.
APP_AUTH_CLIENT_SIDE_VOICE_EMBEDDING: ${APP_AUTH_CLIENT_SIDE_VOICE_EMBEDDING:-false}
APP_AUTH_CLIENT_SIDE_VOICE_EMBEDDING_TENANTS: ${APP_AUTH_CLIENT_SIDE_VOICE_EMBEDDING_TENANTS:-}
# NFC serial-only login (2026-06-01). MUST be listed here — the service uses an
# explicit `environment:` block, NOT `env_file:`, so a var only in .env.prod is
# NOT passed into the container (this exact gap left the flag dead after the
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
package com.fivucsas.identity.application.dto.command;

import com.fasterxml.jackson.annotation.JsonProperty;
import jakarta.validation.constraints.NotEmpty;
import jakarta.validation.constraints.Size;

import java.util.List;

/**
* JSON request body for the CLIENT-SIDE-EMBEDDING <b>voice</b> enroll endpoint
* ({@code POST /api/v1/biometric/voice/enroll-embedding/{userId}} — audit H3,
* GPU-less).
*
* <p>Carries the precomputed 256-dim Resemblyzer speaker embedding the browser
* computed on-device, so the raw audio never leaves the client. This is the
* embedding counterpart of {@code POST /api/v1/biometric/voice/enroll/{userId}}
* (which carries base64 audio).</p>
*
* <p>The embedding MUST be exactly {@value #EMBEDDING_DIM} elements — the
* Resemblyzer output dimensionality. A wrong-length (or empty) vector is rejected
* by bean validation with {@code 400 Bad Request} (the bio side independently
* re-validates to 256, returning {@code 422}).</p>
*
* @param embedding the 256-dim client-side speaker embedding (required, exactly
* {@value #EMBEDDING_DIM} elements)
* @param tenantId optional tenant identifier; when null/blank the controller
* derives it from the authenticated principal
* @param optimize re-enroll &amp; optimize: when true and the user already has a
* voiceprint, the bio side FUSES this sample into the existing
* centroid (defaults to false via {@link #optimizeOrDefault()})
*/
public record VoiceEnrollEmbeddingRequest(
@NotEmpty(message = "embedding is required")
@Size(min = EMBEDDING_DIM, max = EMBEDDING_DIM,
message = "embedding must contain exactly " + EMBEDDING_DIM + " elements")
List<Double> embedding,

@JsonProperty("tenant_id")
String tenantId,

Boolean optimize) {

/**
* Resemblyzer speaker-embedding output dimensionality. The bio
* {@code /voice/enroll-embedding} store is a 256-dim pgvector column, so a
* vector of any other length cannot be a valid template.
*/
public static final int EMBEDDING_DIM = 256;

/** Null-safe optimize flag (absent ⇒ false, the plain append/average path). */
public boolean optimizeOrDefault() {
return Boolean.TRUE.equals(optimize);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -166,6 +166,43 @@ default Map<String, Object> enrollVoice(UUID userId, String voiceData) {
*/
Map<String, Object> verifyVoice(UUID userId, String voiceData);

/**
* Verifies a user's voice from a CLIENT-SIDE precomputed embedding (256-dim
* Resemblyzer speaker vector) instead of audio — audit-H3 GPU-less path,
* where the embedding is computed in the browser so the raw audio never
* leaves the device. Routes to the biometric-processor's
* {@code POST /voice/verify-embedding} endpoint.
*
* <p>Gated by {@code ClientSideVoiceEmbeddingPolicy} at the call site; the
* legacy audio {@link #verifyVoice} path is unchanged. The embedding alone
* carries no audio, so the server-side replay/liveness check cannot run on
* it — an embedding VOICE factor MUST be paired with a liveness factor in the
* auth flow.
*
* @param tenantId optional tenant identifier (currently unused by the bio
* voice routes, which are user-scoped; forwarded for parity
* + forward-compat, null/blank omitted)
* @param userId the user ID to verify against
* @param embedding the 256-dim client-side speaker embedding (list of floats)
* @return Map containing verification response data (e.g. {@code verified})
*/
Map<String, Object> verifyVoiceEmbedding(String tenantId, UUID userId, List<Double> embedding);

/**
* Enrolls a user's voice from a CLIENT-SIDE precomputed embedding (256-dim
* Resemblyzer speaker vector) instead of audio — audit-H3 GPU-less path.
* Routes to the biometric-processor's {@code POST /voice/enroll-embedding}
* endpoint. Mirrors {@link #enrollVoice(UUID, String, boolean)} but with the
* embedding already computed off-device.
*
* @param tenantId optional tenant identifier (see {@link #verifyVoiceEmbedding})
* @param userId the user ID to enroll
* @param embedding the 256-dim client-side speaker embedding (list of floats)
* @param optimize re-enroll &amp; optimize: fuse into the existing centroid
* @return Map containing enrollment response data (e.g. {@code success})
*/
Map<String, Object> enrollVoiceEmbedding(String tenantId, UUID userId, List<Double> embedding, boolean optimize);

/**
* Deletes a user's enrolled face biometric data.
*
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
package com.fivucsas.identity.application.service;

import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Component;

import java.util.LinkedHashSet;
import java.util.Locale;
import java.util.Set;
import java.util.UUID;

/**
* Feature gate for the CLIENT-SIDE-EMBEDDING <b>voice</b> path (audit H3 —
* GPU-less: the 256-d Resemblyzer speaker embedding is computed in the browser so
* the raw audio never leaves the device, and the CPU-only server skips the
* Resemblyzer forward pass).
*
* <p>This is the VOICE twin of {@link ClientSideEmbeddingPolicy} (face). It is a
* <b>separate</b> class on purpose so the voice and face rollouts are independent
* kill-switches — the voice client port is a documented scaffold whose browser
* preprocessing is not yet validated to parity, so it must be flippable without
* touching the (production-ready) face path.
*
* <p>When this policy is ON and a VOICE step / enrollment payload carries a
* precomputed {@code embedding} (256 floats) instead of {@code voiceData},
* Identity Core forwards to the biometric-processor's
* {@code POST /voice/verify-embedding} / {@code POST /voice/enroll-embedding}
* endpoints. When OFF (the default) the legacy audio path
* ({@code /voice/verify} / {@code /voice/enroll}) is used and behaviour is
* byte-identical to before — so a bad rollout reverts by flipping the env flag,
* WITHOUT a redeploy.
*
* <ul>
* <li><b>OFF (default)</b> — legacy audio path for every tenant.</li>
* <li><b>Global ON</b> ({@code app.auth.client-side-voice-embedding=true}) — the
* embedding path applies to every tenant whose payload carries an
* embedding.</li>
* <li><b>Per-tenant canary</b>
* ({@code app.auth.client-side-voice-embedding-tenants=<uuid>,<uuid>}) — the
* embedding path applies ONLY to the listed tenants even when the global
* flag is false, so one tenant can be canaried in production before the
* master switch is flipped.</li>
* </ul>
*
* <p>Both knobs are plain {@code @Value} env-backed properties; no DB write /
* migration is needed to flip them, keeping the kill-switch fast and the blast
* radius minimal.
*
* <p><b>SECURITY:</b> the embedding path carries NO audio, therefore the
* biometric-processor cannot run its spectral-fingerprint replay check or any
* live-capture proof. An embedding-only VOICE factor MUST be paired with a
* separate liveness factor in the auth flow; this policy only governs WHERE the
* embedding is matched.
*/
@Component
@Slf4j
public class ClientSideVoiceEmbeddingPolicy {

private final boolean globallyEnabled;
private final Set<UUID> canaryTenantIds;

public ClientSideVoiceEmbeddingPolicy(
@Value("${app.auth.client-side-voice-embedding:false}") boolean globallyEnabled,
@Value("${app.auth.client-side-voice-embedding-tenants:}") String canaryTenants) {
this.globallyEnabled = globallyEnabled;
this.canaryTenantIds = parseTenantIds(canaryTenants);
log.info("Client-side-embedding voice path: globallyEnabled={}, canaryTenantCount={}",
globallyEnabled, canaryTenantIds.size());
}

/**
* The GLOBAL master switch state, independent of any tenant. The default
* (false) means the legacy audio path is used everywhere.
*/
public boolean isEnabled() {
return globallyEnabled;
}

/**
* True when the client-side-embedding path should drive VOICE for
* {@code tenantId}. Defaults to false (legacy audio path) for every tenant
* unless the global flag is on or the tenant is explicitly canaried.
*/
public boolean isEnabledForTenant(UUID tenantId) {
if (globallyEnabled) {
return true;
}
return tenantId != null && canaryTenantIds.contains(tenantId);
}

/**
* String-tenant overload for call sites that hold the (optional, possibly
* null/non-UUID) tenant id as a {@code String}. Uses the GLOBAL master switch,
* falling back to the per-tenant canary list only when the id parses to a
* UUID. A null / blank / non-UUID tenant id is enabled ONLY under the global
* switch, never via the canary list (a malformed id cannot match a canary
* entry, and must not silently widen the rollout).
*/
public boolean isEnabledForTenant(String tenantId) {
if (globallyEnabled) {
return true;
}
if (tenantId == null || tenantId.isBlank()) {
return false;
}
try {
return isEnabledForTenant(UUID.fromString(tenantId.trim()));
} catch (IllegalArgumentException e) {
return false;
}
}

private static Set<UUID> parseTenantIds(String csv) {
Set<UUID> ids = new LinkedHashSet<>();
if (csv == null || csv.isBlank()) {
return ids;
}
for (String raw : csv.split(",")) {
String token = raw.trim();
if (token.isEmpty()) {
continue;
}
try {
ids.add(UUID.fromString(token.toLowerCase(Locale.ROOT)));
} catch (IllegalArgumentException e) {
// A typo in the canary list must NOT crash startup or silently
// widen the rollout — skip the bad token with a warning.
log.warn("Ignoring invalid tenant UUID in app.auth.client-side-voice-embedding-tenants: '{}'", token);
}
}
return ids;
}
}
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
package com.fivucsas.identity.application.service.mfa.handler;

import com.fivucsas.identity.application.port.output.BiometricServicePort;
import com.fivucsas.identity.application.service.ClientSideVoiceEmbeddingPolicy;
import com.fivucsas.identity.application.service.mfa.MfaStepResult;
import com.fivucsas.identity.application.service.mfa.VerifyMfaStepHandler;
import com.fivucsas.identity.domain.model.auth.AuthMethodType;
Expand All @@ -9,13 +10,18 @@
import lombok.RequiredArgsConstructor;
import org.springframework.stereotype.Component;

import java.util.ArrayList;
import java.util.List;
import java.util.Map;

@Component
@RequiredArgsConstructor
public class VoiceVerifyMfaStepHandler implements VerifyMfaStepHandler {

private final BiometricServicePort biometricService;
// Audit H3 (GPU-less voice): gates the client-side-embedding path. Default
// OFF means the legacy audio path below is byte-identical to before.
private final ClientSideVoiceEmbeddingPolicy clientSideVoiceEmbeddingPolicy;

@Override
public AuthMethodType supports() {
Expand All @@ -24,13 +30,66 @@ public AuthMethodType supports() {

@Override
public MfaStepResult verify(MfaSession session, User user, Map<String, Object> data) {
// Cache user-id once to keep the entity.User boundary surface
// (ArchUnit UserDomainBoundaryTest) at one call site per method —
// matches FaceVerifyMfaStepHandler.
java.util.UUID userId = user.getId();

// ROUTING (audit H3): when the client-side-voice-embedding path is ON for
// this tenant AND the payload carries a precomputed 256-d speaker
// embedding (the raw audio never left the device), match against the bio
// /voice/verify-embedding endpoint. Otherwise fall through to the
// UNCHANGED legacy audio path. Default OFF (policy + no embedding) ⇒
// identical to the prior behaviour. NOTE: an embedding carries no audio,
// so the bio processor cannot run its replay/liveness check on it — an
// embedding VOICE factor MUST be paired with a liveness factor in the
// flow; this handler only routes the match.
List<Double> embedding = extractEmbedding(data.get("embedding"));
boolean embeddingPathEnabled =
clientSideVoiceEmbeddingPolicy.isEnabledForTenant(session.getTenantId());
if (embeddingPathEnabled && embedding != null && !embedding.isEmpty()) {
String tenantId = session.getTenantId() != null ? session.getTenantId().toString() : null;
Map<String, Object> embeddingResult =
biometricService.verifyVoiceEmbedding(tenantId, userId, embedding);
return Boolean.TRUE.equals(embeddingResult.get("verified"))
? MfaStepResult.ok()
: MfaStepResult.fail();
}

// --- Legacy audio path (UNCHANGED) ---
String voiceData = (String) data.get("voiceData");
if (voiceData == null || voiceData.isBlank()) {
return MfaStepResult.fail();
}
Map<String, Object> result = biometricService.verifyVoice(user.getId(), voiceData);
Map<String, Object> result = biometricService.verifyVoice(userId, voiceData);
return Boolean.TRUE.equals(result.get("verified"))
? MfaStepResult.ok()
: MfaStepResult.fail();
}

/**
* Coerces the {@code embedding} payload field (a JSON array deserialized into
* a {@code List<?>} of {@link Number}, or absent) into a {@code List<Double>}.
* Returns null when the value is absent or not a list; a non-numeric element
* makes the whole embedding invalid (null) so a malformed payload falls back
* to the legacy audio path rather than sending garbage to the bio service.
* (Length validation to 256 is enforced bio-side, returning HTTP 422.)
*/
private static List<Double> extractEmbedding(Object raw) {
if (!(raw instanceof List<?> list)) {
return null;
}
if (list.isEmpty()) {
return List.of();
}
List<Double> out = new ArrayList<>(list.size());
for (Object el : list) {
if (el instanceof Number n) {
out.add(n.doubleValue());
} else {
return null; // malformed element → not a usable embedding
}
}
return out;
}
}
Loading
Loading