Skip to content

port: migrate from mp.solutions to mp.tasks.vision (FaceLandmarker)#106

Merged
ahmetabdullahgultekin merged 1 commit into
mainfrom
fix/2026-05-12-mediapipe-tasks-api-port
May 28, 2026
Merged

port: migrate from mp.solutions to mp.tasks.vision (FaceLandmarker)#106
ahmetabdullahgultekin merged 1 commit into
mainfrom
fix/2026-05-12-mediapipe-tasks-api-port

Conversation

@ahmetabdullahgultekin

Copy link
Copy Markdown
Contributor

Why

The mediapipe.solutions namespace was removed in mediapipe 0.10.35
(currently deployed in production). Every /verify call is logging:

Landmark detection failed: module 'mediapipe' has no attribute 'solutions'

…and all temporal/face signals (EAR, MAR, yaw, pitch, roll, blink, smile)
collapse to null in the liveness_calibration event. This PR closes that
gap by porting the four server-side consumers from the legacy face_mesh
API to the new mp.tasks.vision.FaceLandmarker API.

Inventory of ported call sites

File Old call New call
app/infrastructure/ml/landmarks/mediapipe_landmarks.py mp.solutions.face_mesh.FaceMesh(...).process(image) FaceLandmarker.detect(mp_image)
app/infrastructure/ml/quality/quality_assessor.py with mp.solutions.face_mesh.FaceMesh(...) as fm: fm.process(rgb) FaceLandmarker.detect(mp_image) (per-call lifecycle)
app/infrastructure/ml/proctoring/mediapipe_gaze_tracker.py mp.solutions.face_mesh.FaceMesh(static_image_mode=False, ...).process(rgb) FaceLandmarker.detect_for_video(mp_image, ts_ms) (VIDEO mode + monotonic timestamps)
app/infrastructure/ml/liveness/active_liveness_detector.py mp.solutions.face_mesh.FaceMesh(...).process(rgb) FaceLandmarker.detect(mp_image)
tests/demo_local.py dual-path (Tasks + legacy fallback) Tasks-only

5 executable call sites fixed. Result-shape adaptation throughout:

  • Old: result.multi_face_landmarks[0].landmark[i].x
  • New: result.face_landmarks[0][i].x

New shared loader

To avoid duplicating model-path + SHA + env-fallback logic across all
sites, this PR centralises everything in
app/infrastructure/ml/landmarks/face_landmarker_loader.py with:

  • FACE_LANDMARKER_MODEL_PATH env override → repo-root default → cwd fallback
  • FACE_LANDMARKER_MODEL_SHA256 integrity check (warn-and-disable on mismatch)
  • RunningMode.IMAGE vs RunningMode.VIDEO switching
  • Fail-soft on missing asset / unimportable Tasks API
  • to_mp_image(rgb) helper so call sites don't need to import mediapipe
    just for the ImageFormat enum

PR #102's _evaluate_ear_liveness_safe in app/api/routes/verification.py
already follows the same pattern — no shared-file conflicts.

Model delivery

The Dockerfile now bakes face_landmarker.task (float16/latest, ~3.7 MB)
into /app/models/ via a single RUN curl … | sha256sum -c so the image
is reproducible:

SHA256: 64184e229b263107bc2b804c6625db1341ff2bb731874b0bcc2fe6544e0bc9ff

This change is intentionally standalone (single-stage RUN) so it does
not conflict with Team B's PR #104 model-fetcher multi-stage build. If
that PR merges first, the face_landmarker.task curl in this PR will
need to be moved into the model-fetcher stage at merge time — a trivial
rebase. Otherwise this PR ships the asset on its own.

.env.example documents both FACE_LANDMARKER_MODEL_PATH and the
expected SHA. app/core/config.py makes both first-class Pydantic
settings (the SHA field was added by PR #102; this PR adds the path
field).

Tests

  • 22 new unit tests across 4 files:
    • test_face_landmarker_loader.py — path/SHA/import fail-soft (11 tests)
    • test_mediapipe_landmarks_tasks_api.py — new result shape adaption + lazy init caching (5)
    • test_active_liveness_detector_tasks_api.py — empty-face handling + mp.Image plumbing (3)
    • test_mediapipe_gaze_tracker_tasks_api.py — VIDEO mode invariants (3)
  • 1 new integration test that AST-walks app/ for any executable
    reference to mp.solutions / mediapipe.solutions and fails the build
    if any survive — regression guard.
  • Verified on this branch HEAD: zero executable references; all mp.solutions
    hits are docstrings/comments documenting the migration.
  • Existing infrastructure suite re-run: 233 passed, 1 skipped, 1 xfailed
    (no regressions).

Coordination with Team B PR #102 (Bio-Python)

File This PR PR #102 Conflict?
app/api/routes/verification.py (unchanged) adds _evaluate_ear_liveness_safe None
app/core/config.py adds FACE_LANDMARKER_MODEL_PATH field already adds FACE_LANDMARKER_MODEL_SHA256 Trivial (adjacent fields; one-line update of SHA description)
app/application/services/active_liveness_manager.py (unchanged) (unchanged) None

This PR depends on PR #102 in the sense that the SHA settings field comes
from there. If #102 lands first, this PR rebases cleanly. If this PR lands
first, #102 will need to rebase the SHA field on top of the path field.

Operator notes

  • .env.prod SHA pin: the live asset SHA is already documented in
    .env.example. On rebuild + redeploy, set FACE_LANDMARKER_MODEL_SHA256
    in .env.prod to the same value to activate integrity verification.
  • No volume mount required for /app/models/ — the asset is image-baked.
  • Verified the Dockerfile model-fetcher stage builds locally:
    face_landmarker.task: OK from sha256sum -c.

Test plan

  • Build the image, ensure face_landmarker.task ends up at
    /app/models/face_landmarker.task with SHA pin OK.
  • Boot the container, hit /verify with a valid still frame, confirm
    log line Landmark detection failed: module 'mediapipe' has no attribute 'solutions' is GONE.
  • Confirm liveness_calibration event payload now contains non-null
    ear_left, ear_right, mar, yaw, pitch, roll.
  • Run pytest tests/unit/infrastructure/test_face_landmarker_loader.py tests/unit/infrastructure/test_mediapipe_*_tasks_api.py tests/integration/test_no_mp_solutions_runtime.py — all 23 pass.

Memory-rule compliance:

  • feedback_readonly_rootfs_cache_dirs: model is image-baked + chmod 0644;
    no volume needed.
  • feedback_git_push: used bare git push.
  • feedback_verify_completion_claims: ran grep -rn "mp.solutions" app/ tests/ on branch HEAD; zero executable hits.

The mediapipe.solutions namespace was removed in mediapipe 0.10.35
(currently deployed in production). Every /verify call was logging
"Landmark detection failed: module 'mediapipe' has no attribute
'solutions'" and all temporal/face signals (EAR, MAR, yaw, pitch, roll,
blink, smile) collapsed to null in the liveness_calibration event.

Ports four server-side consumers from the legacy face_mesh API to the
new mp.tasks.vision.FaceLandmarker API:

  app/infrastructure/ml/landmarks/mediapipe_landmarks.py
  app/infrastructure/ml/quality/quality_assessor.py
  app/infrastructure/ml/proctoring/mediapipe_gaze_tracker.py    (VIDEO mode)
  app/infrastructure/ml/liveness/active_liveness_detector.py
  tests/demo_local.py                                            (test harness)

Result-shape adaptations:
  Old: result.multi_face_landmarks[0].landmark[i].x
  New: result.face_landmarks[0][i].x

The Tasks API requires a .task model asset. Centralised the
path-resolution + SHA256 integrity check in a new shared loader so the
four call sites + the verification route's _evaluate_ear_liveness_safe
(PR #102) follow one contract:

  app/infrastructure/ml/landmarks/face_landmarker_loader.py
    - FACE_LANDMARKER_MODEL_PATH override (env-var first, repo-root fallback)
    - FACE_LANDMARKER_MODEL_SHA256 verification (warn-and-disable on mismatch)
    - VIDEO running-mode with monotonic timestamp_ms (gaze tracker)
    - IMAGE running-mode (the other three)

Model delivery:
  Dockerfile bakes face_landmarker.task (float16/latest, ~3.7 MB) into
  /app/models/ via curl with build-time SHA256 verification. PIN:
    64184e229b263107bc2b804c6625db1341ff2bb731874b0bcc2fe6544e0bc9ff

  Documented in .env.example. Configured first-class in app/core/config.py
  (FACE_LANDMARKER_MODEL_PATH, FACE_LANDMARKER_MODEL_SHA256 already declared
  per PR #102; this PR adds the path default + the live SHA reference).

Tests:
  + 22 new unit tests (loader path/SHA/import fail-soft, ported sites
    asserting new result shape adaption, gaze tracker VIDEO-mode timestamp
    monotonicity)
  + 1 new integration test that AST-walks app/ for any executable
    reference to mp.solutions / mediapipe.solutions and fails the build
    if any survive (regression guard).

Verified with `grep -rn "mp.solutions" app/ tests/` on branch HEAD: zero
executable references; all hits are docstrings/comments documenting the
migration.

Depends on PR #102 for the FACE_LANDMARKER_MODEL_SHA256 settings field +
the _evaluate_ear_liveness_safe loader pattern this PR generalises. No
conflict in the shared verification.py path (that route is unchanged by
this PR).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings May 12, 2026 21:13

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@ahmetabdullahgultekin ahmetabdullahgultekin merged commit 40fab11 into main May 28, 2026
9 of 10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants