Skip to content

kas-BDMS-626-resolve-database-errors#605

Merged
jacob-a-brown merged 2 commits intokas-well-BDMS-626-inventory-ingestion-updates_v2from
kas-BDMS-626-resolve-database-errors
Mar 17, 2026
Merged

kas-BDMS-626-resolve-database-errors#605
jacob-a-brown merged 2 commits intokas-well-BDMS-626-inventory-ingestion-updates_v2from
kas-BDMS-626-resolve-database-errors

Conversation

@ksmuczynski
Copy link
Copy Markdown
Contributor

Why

This PR addresses the following problem / context:

  • Well inventory imports were failing with database foreign key errors when blank contact-related values were persisted as empty strings instead of missing values.
  • Valid contact organizations used by the current import set were not present in the lexicon, causing contact inserts to fail.
  • Re-running the same well inventory CSV against a database with partially imported rows caused duplicate sample_name constraint violations instead of safely skipping already imported records.

How

Implementation summary - the following was changed / added / removed:

  • Normalized blank contact organization and related text fields to None in the WellInventoryRow schema so empty strings are not written into lexicon-backed foreign key columns.
  • Normalized blank well status values to None to avoid invalid empty-string lexicon references.
  • Added the missing organization terms required by the current import dataset to the lexicon.
  • Updated the well inventory importer to detect rows that have already been imported and skip recreating related records on rerun.
  • Ensured the idempotent rerun path returns serializable results so CLI output remains stable.
  • Added regression coverage for blank-value normalization and safe reruns of the same import file.

Notes

Any special considerations, workarounds, or follow-up work to note?

  • This PR is focused on database-error resolution for contact organization/status foreign keys and duplicate inserts on rerun.
  • The added organization terms must be loaded into the target database lexicon for those contact inserts to succeed.
  • Remaining import failures after this PR are expected to come from source-data validation issues, not the database errors addressed here.
  • The current import summary counts successfully skipped rerun rows as imported; a future follow-up could distinguish newly inserted rows from already-existing rows.

…rganization terms

- Treat blank contact organization and well status values as missing instead of persisting empty strings
- Prevent foreign key failures caused by empty organization and status lexicon references during import
- Add newly encountered organization terms to the lexicon so valid contact records can persist successfully
- Detect previously imported well inventory rows before inserting related records
- Skip recreating field activity water-level samples and observations when the same row is reprocessed
- Return serializable existing-row results so CLI reruns report cleanly instead of crashing
@jacob-a-brown jacob-a-brown merged commit a4e9412 into kas-well-BDMS-626-inventory-ingestion-updates_v2 Mar 17, 2026
5 checks passed
@jacob-a-brown jacob-a-brown deleted the kas-BDMS-626-resolve-database-errors branch March 17, 2026 19:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants