Skip to content

feat(face): client-side embedding for FACE enrollment (audit H2)#223

Merged
ahmetabdullahgultekin merged 1 commit into
mainfrom
feat/client-side-enroll-embedding-2026-06-12
Jun 12, 2026
Merged

feat(face): client-side embedding for FACE enrollment (audit H2)#223
ahmetabdullahgultekin merged 1 commit into
mainfrom
feat/client-side-enroll-embedding-2026-06-12

Conversation

@ahmetabdullahgultekin

Copy link
Copy Markdown
Contributor

What

Extends the client-side-embedding FACE path to ENROLLMENT (audit item H2) so that enrollment and verification share one aligned embedding space.

Today FACE verify computes the Facenet512 vector in-browser and uploads only the 512-float embedding, but FACE enrollment still uploaded raw images, which the CPU-only server then embedded (MTCNN + Facenet512). That meant a user's enrolled server template and their client verify probe lived in slightly different preprocessing spaces — and enrollment itself ran heavy server ML.

When VITE_CLIENT_SIDE_EMBEDDING is ON, FaceEnrollmentDialog now:

  1. computes the embedding for the best frontal capture (the centered index-0 frame) via the existing embedCapturedFace module (no preprocessing reimplemented), and
  2. submits only the 512-vector to the new BiometricService.enrollFaceEmbedding(), which POSTs JSON to the already-existing api route POST /api/v1/biometric/enroll-embedding/{userId} — the raw image never leaves the device.

This mirrors MfaStepRenderer's verify path exactly: compute → if a vector, submit { embedding }; else show a retryable error and never fall back to an image upload while the flag is ON (the CPU-only server 400s the image path when its flag is on). Flag OFF (default) is byte-identical to today's image enroll (enrollFace).

Request shape (frozen api contract)

POST /api/v1/biometric/enroll-embedding/{userId}
Content-Type: application/json
{ "embedding": number[512], "tenant_id"?: string }   // tenant_id omitted when blank

This matches the api EnrollEmbeddingRequest record (the route, port, adapter, EnrollBiometricService routing and ClientSideEmbeddingPolicy fail-closed gate already shipped in identity-core-api PR #222/#223). The api 400s a wrong-length/empty vector and 403s the tenant when the server flag is off.

Changes

  • BiometricService.enrollFaceEmbedding(userId, embedding, tenantId?) — JSON POST, tenant_id omitted when blank (mirrors the multipart helper).
  • useFaceChallenge — snapshots the 478-pt FaceLandmarker mesh with each capture (new optional + defensive updateChallenge param and ChallengeState.captureLandmarks) so the enroll embedding is eye-aligned identically to the verify probe (self-consistency). A backend without dense landmarks (BlazeFace) yields null → the embedder falls back to the unaligned crop. The param is optional/guarded so existing 3-arg callers are unaffected.
  • FaceEnrollmentFlow — threads detection.captureLandmarks through to onComplete.
  • i18nenrollmentPage.faceClientPrepFailed (en + tr).
  • TestsBiometricService.enrollFaceEmbedding (JSON body / tenant-omit / success shape) + FaceEnrollmentDialog (flag OFF → enrollFace; flag ON + vector → embedCapturedFace + enrollFaceEmbedding; flag ON + null → retryable snackbar, no enroll, no image fallback).

Notes

  • optimize/template-fusion is not supported on the embedding enroll route (the EnrollEmbeddingRequest has no optimize field); a re-enroll replaces the template. Noted in code.
  • Ordering caveat (deploy): flip the identity app.auth.client-side-embedding flag before VITE_CLIENT_SIDE_EMBEDDING, same as verify.

Reversibility

Flag OFF (the default) = unchanged server-side image enroll. No behavior change while dark.

Verification

  • vitest run (4 affected suites): 28 passed
  • tsc --noEmit: clean
  • eslint on changed files: clean

Extends the client-side-embedding FACE path to ENROLLMENT so enroll and
verify share ONE aligned embedding space. Today FACE verify computes the
Facenet512 vector in-browser and uploads only the embedding, but FACE
*enrollment* still uploaded raw images (server-side MTCNN+Facenet512 on
the CPU-only box) — so a user's enrolled server template and their client
verify probe lived in slightly different preprocessing spaces.

When VITE_CLIENT_SIDE_EMBEDDING is ON, FaceEnrollmentDialog now computes
the embedding for the best frontal capture (the centered index-0 frame)
via the EXISTING embedCapturedFace module and submits ONLY the 512-vector
to the new BiometricService.enrollFaceEmbedding(), which POSTs JSON
{ embedding, tenant_id? } to the existing api route
POST /biometric/enroll-embedding/{userId} (EnrollEmbeddingRequest). The
raw image never leaves the device. Mirrors MfaStepRenderer's verify path
exactly: compute -> if vector submit embedding, else show a retryable
error and NEVER fall back to an image upload while the flag is ON (the
CPU-only server 400s the image path when its flag is on). Flag OFF is
byte-identical to the legacy image enroll (enrollFace).

- BiometricService.enrollFaceEmbedding(userId, embedding, tenantId?):
  JSON POST, tenant_id omitted when blank (mirrors the multipart helper).
- useFaceChallenge: snapshots the 478-pt FaceLandmarker mesh WITH each
  capture (new optional, defensive updateChallenge param + ChallengeState
  .captureLandmarks) so the enroll embedding is eye-aligned identically to
  the verify probe (self-consistency). Backend backend lacking dense
  landmarks (BlazeFace) -> null -> embedder uses the unaligned crop.
- FaceEnrollmentFlow threads detection.captureLandmarks through to
  onComplete.
- i18n: enrollmentPage.faceClientPrepFailed (en + tr).
- Tests: BiometricService.enrollFaceEmbedding (JSON body, tenant omit,
  success shape) + FaceEnrollmentDialog (flag OFF/ON/null routing).

Reversible: flag OFF (default) = unchanged server-side enroll. Vitest +
tsc + eslint green.
@ahmetabdullahgultekin ahmetabdullahgultekin merged commit 10a47f5 into main Jun 12, 2026
4 checks passed
@ahmetabdullahgultekin ahmetabdullahgultekin deleted the feat/client-side-enroll-embedding-2026-06-12 branch June 12, 2026 20:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant