feat(control): RGB-only collision guardrail module at ~10 Hz#1748
feat(control): RGB-only collision guardrail module at ~10 Hz#1748pmg5408 wants to merge 8 commits intodimensionalOS:devfrom
Conversation
Adds a guardrail module that intercepts Twist commands and evaluates risk against RGB camera frames. Establishes the module lifecycle, runtime state, background decision thread with condition-based synchronization, and stream interfaces. Risk evaluation logic is stubbed for a follow-up commit.
Adds the single-worker guardrail loop, timed wakeup behavior, and shared runtime state for applying the latest safety assessment to incoming Twist commands. Separates module orchestration from policy logic and fails closed if policy evaluation errors out.
Adds fail-closed startup, stale input handling, and explicit INIT and SENSOR_DEGRADED decisions to the RGB guardrail loop. The worker now publishes state changes immediately while keeping policy evaluation outside the callback lock for responsiveness.
Implements a reusable RGB-only guardrail policy using optical-flow magnitude in a forward ROI with conservative clamp and stop behavior. Adds image-quality checks, hysteresis, and policy-side configuration while keeping scheduling and publishing logic in the module shell.
Tightens the guardrail's fail-closed behavior by validating frame-pair freshness, checking image quality across both flow frames, and preventing stop-threshold detections from passing through unchanged. Also improves worker-side scheduling so risk evaluation only runs on new image data and adds logging around startup, shutdown, and policy failures.
Adds focused tests for optical-flow policy transitions, fail-closed image quality handling, worker-thread command gating, and end-to-end guarded stream behavior. The suite covers startup, stale inputs, stop/clamp republication, fast-upstream handling, and concurrent stop safety.
Greptile SummaryThis PR adds a new Confidence Score: 5/5Safe to merge; all findings are minor style and maintenance issues with no operational impact The hysteresis state machine is correct and well-covered by tests. The fail-closed design (SENSOR_DEGRADED/INIT on missing/stale inputs) is exercised end-to-end. The only code-level finding is a logging-path data race that is non-crashing in CPython and carries no operational consequence. All remaining findings are P2. No files require special attention before merging Important Files Changed
Sequence DiagramsequenceDiagram
participant Camera
participant Guard as RGBCollisionGuardrail
participant RS as _GuardrailRuntimeState
participant DL as decision_loop (worker thread)
participant Policy as OpticalFlowMagnitudePolicy
participant Out as safe_cmd_vel
Camera->>Guard: color_image (Image)
Guard->>RS: shift latest→previous, store image, bump image_generation
Guard->>DL: condition.notify()
Note over DL: wakes from condition.wait()
DL->>RS: _should_recompute_risk_locked(now)?<br/>[new generation AND next_risk_time elapsed]
RS-->>DL: yes → snapshot _RiskEvaluationInput<br/>(prev_image, cur_image, cmd_vel, health)
DL->>Policy: evaluate(prev, cur, cmd_vel, health)
Note over Policy: 1. Health gates (has_previous_frame, image_fresh, frame_pair_fresh)<br/>2. Forward-motion deadband check<br/>3. Grayscale + resize to flow_downsample_width_px<br/>4. Crop central lower forward ROI<br/>5. Quality checks: occlusion + low-texture<br/>6. Farneback optical flow → mean magnitude<br/>7. Hysteresis counter update → next state
Policy-->>DL: GuardrailDecision(state, cmd_vel, reason, risk_score)
DL->>RS: _store_decision_locked()<br/>persist decision, set pending_decision_publish on state change
DL->>RS: _consume_publish_cmd_locked()
RS-->>DL: resolved Twist to publish
DL->>Out: publish(cmd_vel)
Note over Out: PASS → upstream cmd<br/>CLAMP → forward speed capped<br/>STOP_LATCHED → linear.x zeroed<br/>SENSOR_DEGRADED/INIT → Twist.zero()
Note over Guard,DL: Parallel path — incoming_cmd_vel arrives
Guard->>RS: store cmd_vel, latest_cmd_time<br/>set pending_cmd_update=True
Guard->>DL: condition.notify()
DL->>RS: _consume_publish_cmd_locked()<br/>pending_cmd_update path → _resolved_cmd_for_latest_locked()
DL->>Out: publish(resolved cmd using last stored decision)
Reviews (1): Last reviewed commit: "test(control): add RGB guardrail policy ..." | Re-trigger Greptile |
dimos/control/safety/test_rgb_collision_guardrail_integration.py
Outdated
Show resolved
Hide resolved
…ging Extracts duplicated guardrail test helpers into a shared conftest to keep the safety test suite consistent as it evolves. Also captures guardrail state under lock before exception logging so failure logs reflect a stable runtime snapshot.
Moves duplicated guardrail test helpers into a shared test utility module so the safety test suite stays consistent as coverage grows. This keeps the module and integration tests aligned without relying on duplicated local helper definitions.
Problem
Direct
cmd_velproducers do not currently have a reusable RGB-only guardrail on the motion path. This makes behaviors like person-following vulnerable to blindly forwarding commanded motion without any conservative frontal collision check from the onboard camera stream.Solution
Adds a reusable
RGBCollisionGuardrailmodule underdimos/control/safetythat consumescolor_imageandincoming_cmd_veland publishessafe_cmd_vel.The implementation is split into two parts:
PASS,CLAMP,STOP_LATCHED, orSENSOR_DEGRADEDKey design decisions:
STOP_LATCHEDThis PR also adds focused unit and integration tests for policy transitions, module fail-closed behavior, and threaded guarded-output behavior.
Breaking Changes
None.
How to Test
Contributor License Agreement