feat(face): client-side embedding for FACE enrollment (audit H2)#223
Merged
ahmetabdullahgultekin merged 1 commit intoJun 12, 2026
Merged
Conversation
Extends the client-side-embedding FACE path to ENROLLMENT so enroll and
verify share ONE aligned embedding space. Today FACE verify computes the
Facenet512 vector in-browser and uploads only the embedding, but FACE
*enrollment* still uploaded raw images (server-side MTCNN+Facenet512 on
the CPU-only box) — so a user's enrolled server template and their client
verify probe lived in slightly different preprocessing spaces.
When VITE_CLIENT_SIDE_EMBEDDING is ON, FaceEnrollmentDialog now computes
the embedding for the best frontal capture (the centered index-0 frame)
via the EXISTING embedCapturedFace module and submits ONLY the 512-vector
to the new BiometricService.enrollFaceEmbedding(), which POSTs JSON
{ embedding, tenant_id? } to the existing api route
POST /biometric/enroll-embedding/{userId} (EnrollEmbeddingRequest). The
raw image never leaves the device. Mirrors MfaStepRenderer's verify path
exactly: compute -> if vector submit embedding, else show a retryable
error and NEVER fall back to an image upload while the flag is ON (the
CPU-only server 400s the image path when its flag is on). Flag OFF is
byte-identical to the legacy image enroll (enrollFace).
- BiometricService.enrollFaceEmbedding(userId, embedding, tenantId?):
JSON POST, tenant_id omitted when blank (mirrors the multipart helper).
- useFaceChallenge: snapshots the 478-pt FaceLandmarker mesh WITH each
capture (new optional, defensive updateChallenge param + ChallengeState
.captureLandmarks) so the enroll embedding is eye-aligned identically to
the verify probe (self-consistency). Backend backend lacking dense
landmarks (BlazeFace) -> null -> embedder uses the unaligned crop.
- FaceEnrollmentFlow threads detection.captureLandmarks through to
onComplete.
- i18n: enrollmentPage.faceClientPrepFailed (en + tr).
- Tests: BiometricService.enrollFaceEmbedding (JSON body, tenant omit,
success shape) + FaceEnrollmentDialog (flag OFF/ON/null routing).
Reversible: flag OFF (default) = unchanged server-side enroll. Vitest +
tsc + eslint green.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Extends the client-side-embedding FACE path to ENROLLMENT (audit item H2) so that enrollment and verification share one aligned embedding space.
Today FACE verify computes the Facenet512 vector in-browser and uploads only the 512-float embedding, but FACE enrollment still uploaded raw images, which the CPU-only server then embedded (MTCNN + Facenet512). That meant a user's enrolled server template and their client verify probe lived in slightly different preprocessing spaces — and enrollment itself ran heavy server ML.
When
VITE_CLIENT_SIDE_EMBEDDINGis ON,FaceEnrollmentDialognow:embedCapturedFacemodule (no preprocessing reimplemented), andBiometricService.enrollFaceEmbedding(), whichPOSTs JSON to the already-existing api routePOST /api/v1/biometric/enroll-embedding/{userId}— the raw image never leaves the device.This mirrors
MfaStepRenderer's verify path exactly: compute → if a vector, submit{ embedding }; else show a retryable error and never fall back to an image upload while the flag is ON (the CPU-only server400s the image path when its flag is on). Flag OFF (default) is byte-identical to today's image enroll (enrollFace).Request shape (frozen api contract)
This matches the api
EnrollEmbeddingRequestrecord (the route, port, adapter,EnrollBiometricServicerouting andClientSideEmbeddingPolicyfail-closed gate already shipped in identity-core-api PR #222/#223). The api400s a wrong-length/empty vector and403s the tenant when the server flag is off.Changes
BiometricService.enrollFaceEmbedding(userId, embedding, tenantId?)— JSON POST,tenant_idomitted when blank (mirrors the multipart helper).useFaceChallenge— snapshots the 478-pt FaceLandmarker mesh with each capture (new optional + defensiveupdateChallengeparam andChallengeState.captureLandmarks) so the enroll embedding is eye-aligned identically to the verify probe (self-consistency). A backend without dense landmarks (BlazeFace) yieldsnull→ the embedder falls back to the unaligned crop. The param is optional/guarded so existing 3-arg callers are unaffected.FaceEnrollmentFlow— threadsdetection.captureLandmarksthrough toonComplete.enrollmentPage.faceClientPrepFailed(en + tr).BiometricService.enrollFaceEmbedding(JSON body / tenant-omit / success shape) +FaceEnrollmentDialog(flag OFF →enrollFace; flag ON + vector →embedCapturedFace+enrollFaceEmbedding; flag ON + null → retryable snackbar, no enroll, no image fallback).Notes
optimize/template-fusion is not supported on the embedding enroll route (theEnrollEmbeddingRequesthas nooptimizefield); a re-enroll replaces the template. Noted in code.app.auth.client-side-embeddingflag beforeVITE_CLIENT_SIDE_EMBEDDING, same as verify.Reversibility
Flag OFF (the default) = unchanged server-side image enroll. No behavior change while dark.
Verification
vitest run(4 affected suites): 28 passedtsc --noEmit: cleaneslinton changed files: clean