V6 Pipeline: Add comprehensive two-stage testing framework

[Copilot]

Implements comprehensive testing infrastructure for V6 simplified pipeline (MiniRocket → XGBoost) per Phase 2 requirements. Creates two-stage validation: integrity verification and performance benchmarking.

Changes

New Test Framework (scripts/v6_comprehensive_test.py)

• Stage 1 (Integrity): Validates quality gate, windowing (20min/5min stride), rules engine, AI pipeline (MiniRocket 9,996 → padding → XGBoost 10,004), hybrid logic (MAX override), JSON serialization
• Stage 2 (Performance): Measures recall, precision, F1/F2, AUC-ROC, error rates (FPR/FNR), noise robustness, latency (p50/p95/p99)
• Generates JSON reports to REPORTS/ directory
• CLI: --stage 1|2 or --all

Updated Unit Tests (tests/test_xgboost_v6_pipeline.py)

• Fixed circular import issues using dynamic module loading via importlib.util
• 8/8 tests passing: feature padding, model loading, prediction structure, rule engine override, adapter protocol

Test Infrastructure

• Created mock xgboost_v5.pkl (CalibratedClassifierCV, 10,004 features) for testing without production model
• Updated .gitignore for test artifacts

Usage

# Run integrity tests (must pass before performance tests)
python scripts/v6_comprehensive_test.py --stage 1

# Run performance benchmarks
python scripts/v6_comprehensive_test.py --stage 2

Results

• Stage 1: All 6 integrity tests passing (1.1s execution)
• Stage 2: Latency 1.1ms p50 (99x under 100ms target), noise robust
• Security: CodeQL 0 vulnerabilities
• Steel Wall: No changes to protected components (pre_ai/, rules/, explainability/, state_bridge.py)

Notes

• Mock model limits Stage 2 quality metrics (Recall/Precision) - deploy real xgboost_v5.pkl for production validation
• Direct module loading pattern avoids wfdb/pandas compatibility issues in import chain

Original prompt

---

This section details on the original issue you should resolve

<issue_title>V6 Testing</issue_title>
<issue_description>

Title

SentinelFetal V6: Replace V4 Ensemble with MiniRocket → XGBoost (V5) Simplified Pipeline (Keep Pre-AI + Hybrid Logic + UI JSON intact)

---

Overview

We want to replace the current 3-model ensemble (XGBoost + RandomForest + SGD) with a single MiniRocket → XGBoost pipeline, while preserving all critical system invariants:

Must remain unchanged

• Pre-AI pipeline: quality gate + invariants + windowing (Steel Wall)

• Smart Hybrid Logic (3-tier decision system)

• Rule Engine safety net

• UI JSON output contract (snapshot / state bridge)

---

Current vs Target Architecture

CURRENT (V4.0)

FHR Signal → MiniRocket (9,996 features) → Fusion (1,035 dim) → Ensemble (3 models) → Category

TARGET (V6)

FHR Signal → MiniRocket (9,996 features) → XGBoost V5 → Category

---

Available Model Artifacts

Model file | Size | Location -- | -- | -- minirocket_encoder.joblib | 41 KB | models/ xgboost_v5.pkl | 950 KB | models/ and models/ensemble_v5/ ctg_xgboost_pipeline.pkl | 1.2 MB | models/

---

Scope

Goals

• Add a V6 pipeline that uses MiniRocket → XGBoost (xgboost_v5.pkl) as the only AI classifier.

• Keep decision logic and safety behavior identical at the system level:

  • Rule override remains MAX(ai_risk, rule_severity)

  • Same categories 1/2/3

  • Same predict/predict_proba contract for adapters

Non-goals

• No retraining.

• No changes to:

  • group split logic

  • calibration behavior (if model is calibrated, keep it)

  • invariants / windowing params

  • Smart Hybrid Logic algorithm

  • UI code or JSON schema

---

Steel Wall (Do Not Touch)

These paths/files are strictly unchanged:

• src/v6/pre_ai/ (quality gate, invariants, windowing)

• src/decision/smart_hybrid_logic.py

• src/models/minirocket_encoder.py

• src/rules/ (all rule engine files)

• src/explainability/

• src/interfaces/state_bridge.py (UI JSON output)

---

Implementation Plan

Step 1 — Verify Model Compatibility

Create and run a verification script to inspect xgboost_v5.pkl:

• Determine:

  • expected input feature dimension (n_features_in_ if available)

  • whether it’s CalibratedClassifierCV or raw estimator

• Confirm:

  • MiniRocket output is 9,996

  • Model expects 10,004 (see padding decision below)

• Run a small end-to-end inference sanity check

---

Feature Dimension Handling (Confirmed Decision)

The XGBoost model expects 10,004 features:

• MiniRocket: 9,996

• Clinical features: 8 (not available in this simplified pipeline)

Decision: pad missing clinical features with zeros (neutral defaults) without retraining.

def pad_features(minirocket_features: np.ndarray) -> np.ndarray:
    """Pad MiniRocket features with zeros for clinical features."""
    padded = np.zeros(10004, dtype=np.float32)
    padded[:9996] = minirocket_features
    return padded

---

Files to Create

1) src/adapters/xgboost_only_classifier.py

New XGBoost-only classifier (replaces EnsembleManager behavior at adapter level):

• Load xgboost_v5.pkl

• Provide same predict surface expected by the pipeline

• Apply rule engine safety override:

  • final_risk = MAX(ai_risk, rule_severity)

• Return a compatible EnsemblePrediction-like dict structure (so explainability / state bridge remains stable)

---

2) src/adapters/xgboost_v6_adapter.py

Adapter implementing IClassifier protocol:

• Wrap XGBoostOnlyClassifier

• Drop-in replacement for EnsembleCla...

• Fixes ArielShamay/SentinelFetal#1

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.