Skip to content

Handle legacy MPHeight NaNs as NULLs (skip estimator) to ensure full well migration #541

@ksmuczynski

Description

@ksmuczynski

Summary

Legacy WellData.MPHeight contains NaN values. Current transfer logic treats NaN as a real value, bypasses estimation, and then fails at DB insert because MeasuringPointHistory.measuring_point_height is Numeric. We want
migration completeness without using the estimator (it’s not trustworthy right now). Proposed fix: treat NaN as missing and insert NULL MP height for those wells, with nullable history during migration.

Why this matters

  • NaN values prevent some wells from transferring.
  • Estimator currently uses early measurements and may be inaccurate.
  • We need all wells migrated even if MP height is unknown.

Proposed changes

  • Transfer logic: treat NaN as missing and do not call estimator for those records.
  • Allow MeasuringPointHistory.measuring_point_height to be NULL during migration (migration already exists).
  • Read‑path hardening: ensure API/property access handles NULL history values.

Files

  • transfers/well_transfer.py
    • Detect NaN (pd.isna) and set MP height to None. Skip estimator for NaN cases.
  • db/thing.py
    • measuring_point_height and measuring_point_description should skip null history entries.
  • schemas/thing.py
    • Ensure response types allow measuring_point_height: float | None.
  • db/measuring_point_history.py
    • Confirm column nullable matches migration.
  • (Optional) transfers/util.py
    • Guard estimator against NaN if still used elsewhere.

Acceptance criteria

  • All legacy wells transfer even when MPHeight is NaN.
  • No estimator calls for NaN MP height in well transfer.
  • API responses tolerate null MP height without errors.
  • Tests updated or added to validate null MP height behavior.

Notes

  • Do not change column type to Float; keep Numeric for integrity.
  • Estimator improvements (using most recent DTW/DTW‑BGS) are out of scope for this issue.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions