Improve well-inventory CLI feedback and validation handling; add real-user CSV feature coverage by jirhiker · Pull Request #521 · DataIntegrationGroup/OcotilloAPI

jirhiker · 2026-02-15T06:22:12Z

Summary

This PR improves the well-inventory-csv CLI workflow by making output clearer and more actionable, hardening validation behavior for messy real-world CSV input, and adding feature coverage for a real user-entered dataset.

It also updates pydantic/pydantic-core to remove the FastAPIPaginationWarning compatibility warning.

Changes

1. CLI output improvements (`well-inventory-csv`)

Enhanced user-facing CLI output in cli/cli.py:
- status banner (SUCCESS vs COMPLETED WITH ISSUES)
- structured summary section
- grouped validation errors by row
- higher validation display limit (up to 10)
- styled validation field labels (orange + bold)
- clearer section separators and headings
Preserved summary compatibility line:
- Summary: processed=... imported=... rows_with_issues=...
Command exits with the adapter’s actual exit code.

2. Validation crash fixes

schemas/__init__.py
- past_or_today_validator now safely handles None for optional date fields.
schemas/well_inventory.py
- phone_validator now catches NumberParseException and raises ValueError so invalid phone values are returned as structured row validation errors instead of crashing.

3. Real-user CSV feature test coverage

Added new feature:
- tests/features/well-inventory-real-user-csv.feature
Added supporting steps:
- tests/features/steps/well-inventory-real-user-csv.py
- updates in tests/features/steps/well-inventory-csv-given.py
Added feature fixture copy:
- tests/features/data/well-inventory-real-user-entered-data.csv

4. Dependency updates (warning fix)

Updated:
- pydantic==2.12.5
- pydantic-core==2.41.5
Files updated:
- pyproject.toml
- uv.lock
- requirements.txt

5. Test updates

Updated tests/test_cli_commands.py assertions to match new grouped validation output format.

Why

Real-world CSVs are noisy; import should never crash on invalid rows.
CLI users need clear, row-grouped diagnostics to fix input quickly.
fastapi-pagination warning required aligning pydantic versions for compatibility.

Testing

Feature dry-run binding checks for new/updated Behave steps.
Mocked CliRunner checks for output/exit-code behavior in well-inventory-csv.
Updated CLI unit test expectations for new formatting.

Note: Full DB-backed test execution was limited in this environment due local Postgres availability constraints.

Reviewer Notes

This branch includes well-inventory CLI/validation/test changes.
Please pay special attention to:
- output formatting in cli/cli.py
- validation handling in schemas/well_inventory.py and schemas/__init__.py
- behavior expectations in tests/test_cli_commands.py

…er validation, and add CSV feature tests

Copilot

Pull request overview

This pull request enhances the well-inventory-csv workflow by improving CLI user experience, adding validation robustness, and expanding test coverage with real-world CSV data. The changes address practical issues that arise when importing messy, user-entered CSV files.

Changes:

Enhanced CLI output with colored summaries, grouped validation errors, and proper exit codes
Improved phone number and date validation to handle None values and parse errors gracefully
Added BDD feature tests using real user-entered CSV data to validate production-like inputs
Updated pydantic (2.11.7 → 2.12.5) and pydantic-core (2.33.2 → 2.41.5) dependencies

Reviewed changes

Copilot reviewed 11 out of 13 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
cli/cli.py	Enhanced well_inventory_csv command with formatted output, validation error grouping, and proper exit code handling
schemas/init.py	Updated past_or_today_validator to handle None values for optional date fields
schemas/well_inventory.py	Added try-catch for NumberParseException in phone_validator to return structured validation errors
tests/test_cli_commands.py	Added tests for CLI output formatting and validation error reporting
tests/features/well-inventory-real-user-csv.feature	New BDD feature for testing real user CSV data
tests/features/steps/well-inventory-real-user-csv.py	Step definitions for real user CSV validation scenarios
tests/features/steps/well-inventory-csv-given.py	Enhanced to support real user CSV file loading
tests/features/environment.py	Changed hardcoded rebuild flag to use environment variable via get_bool_env
tests/features/data/well-inventory-real-user-entered-data.csv	Real-world CSV data with 130 rows for testing
tests/data/well_inventory_real_user_entered_data.csv	Duplicate of the above CSV file
pyproject.toml, uv.lock, requirements.txt	Updated pydantic dependency versions

tests/data/well_inventory_real_user_entered_data.csv

tests/features/data/well-inventory-real-user-entered-data.csv

schemas/well_inventory.py

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1936f9a213

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

tests/test_cli_commands.py

…er validation, and add CSV feature tests

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 11 out of 13 changed files in this pull request and generated 2 comments.

cli/cli.py

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 10 out of 12 changed files in this pull request and generated no new comments.

…er validation, and add CSV feature tests

…gic for well_name_point_id

Copilot

Pull request overview

Copilot reviewed 13 out of 15 changed files in this pull request and generated 5 comments.

services/well_inventory_csv.py

tests/test_cli_commands.py

cli/cli.py

…cessing

Copilot

Pull request overview

Copilot reviewed 13 out of 15 changed files in this pull request and generated 5 comments.

tests/features/steps/well-inventory-real-user-csv.py

services/well_inventory_csv.py

cli/cli.py

…oint_id

Copilot

Pull request overview

Copilot reviewed 13 out of 15 changed files in this pull request and generated 5 comments.

tests/features/environment.py

services/well_inventory_csv.py

cli/cli.py

tests/test_cli_commands.py

tests/features/steps/well-inventory-csv-given.py

…ganizations

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot

Pull request overview

Copilot reviewed 28 out of 30 changed files in this pull request and generated 5 comments.

services/well_inventory_csv.py

tests/test_well_inventory.py

services/well_inventory_csv.py

… in CSV upload

…ontacts in CSV upload

…plicate contacts

Copilot

Pull request overview

Copilot reviewed 28 out of 30 changed files in this pull request and generated 4 comments.

services/well_inventory_csv.py

tests/features/steps/well-notes.py

tests/features/steps/api_common.py

tests/test_well_inventory.py

…x function

…ocess

Copilot

Pull request overview

Copilot reviewed 31 out of 33 changed files in this pull request and generated 2 comments.

Copilot · 2026-02-16T07:07:10Z

tests/test_well_inventory.py

+
+        # Unsupported forms
+        assert _extract_autogen_prefix("XY-001") is None
+        assert _extract_autogen_prefix("XYZ-") == "XYZ-"


The test assertion at line 845 expects _extract_autogen_prefix("XYZ-") to return "XYZ-", but the regex AUTOGEN_PREFIX_REGEX only matches 2-3 letter prefixes. The regex pattern is r"^[A-Z]{2,3}-$" which means 2 to 3 uppercase letters followed by a hyphen. "XYZ-" has 3 letters and should match, so this test appears correct. However, it would be good to add a test case for a 4-letter prefix that should return None to verify the upper bound of the regex works as expected.

Copilot · 2026-02-16T07:07:11Z

tests/test_well_inventory.py

+        # New supported form (2-3 uppercase letter prefixes)
+        assert _extract_autogen_prefix("WL-XXXX") == "WL-"
+        assert _extract_autogen_prefix("SAC-XXXX") == "SAC-"
+        assert _extract_autogen_prefix("ABC -xxxx") == "ABC-"


The test input "ABC -xxxx" (with a space before the hyphen) is expected to extract "ABC-" prefix. However, the AUTOGEN_TOKEN_REGEX pattern uses \s* which allows optional whitespace between the prefix and hyphen. This is actually being tested correctly, but it may be worth documenting that the function normalizes whitespace by removing it when extracting the prefix.

jirhiker and others added 2 commits February 14, 2026 23:19

chore: update pydantic and pydantic-core versions, enhance phone numb…

ee350ea

…er validation, and add CSV feature tests

Formatting changes

1936f9a

Copilot AI review requested due to automatic review settings February 15, 2026 06:22

Copilot started reviewing on behalf of jirhiker February 15, 2026 06:22 View session

Copilot AI reviewed Feb 15, 2026

View reviewed changes

tests/data/well_inventory_real_user_entered_data.csv Outdated Show resolved Hide resolved

tests/features/data/well-inventory-real-user-entered-data.csv Outdated Show resolved Hide resolved

schemas/well_inventory.py Show resolved Hide resolved

chatgpt-codex-connector bot reviewed Feb 15, 2026

View reviewed changes

tests/test_cli_commands.py Outdated Show resolved Hide resolved

jirhiker and others added 3 commits February 14, 2026 23:41

chore: update pydantic and pydantic-core versions, enhance phone numb…

b93b00c

…er validation, and add CSV feature tests

chore: update phone validation output format in CLI tests

40fbe54

Formatting changes

f70ec28

Copilot AI review requested due to automatic review settings February 15, 2026 06:41

jirhiker review requested due to automatic review settings February 15, 2026 06:41

Update schemas/well_inventory.py

4c1156b

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings February 15, 2026 06:42

Formatting changes

783a6ab

Copilot started reviewing on behalf of jirhiker February 15, 2026 06:42 View session

Copilot AI reviewed Feb 15, 2026

View reviewed changes

cli/cli.py Outdated Show resolved Hide resolved

DataIntegrationGroup deleted a comment from Copilot AI Feb 15, 2026

jirhiker and others added 2 commits February 14, 2026 23:48

delete file

9c06f8c

Apply suggestions from code review

70cc08c

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings February 15, 2026 06:50

Formatting changes

06c2120

Copilot started reviewing on behalf of jirhiker February 15, 2026 06:50 View session

Copilot AI reviewed Feb 15, 2026

View reviewed changes

jirhiker requested a review from marissafichera February 15, 2026 06:56

jirhiker added 2 commits February 15, 2026 00:09

chore: update pydantic and pydantic-core versions, enhance phone numb…

23ce228

…er validation, and add CSV feature tests

chore: update CSV validation scenarios and improve auto-generation lo…

d03b553

…gic for well_name_point_id

Copilot AI review requested due to automatic review settings February 15, 2026 07:49

Formatting changes

87d1315

Copilot started reviewing on behalf of jirhiker February 15, 2026 07:50 View session

Copilot AI reviewed Feb 15, 2026

View reviewed changes

services/well_inventory_csv.py Show resolved Hide resolved

tests/test_cli_commands.py Outdated Show resolved Hide resolved

cli/cli.py Outdated Show resolved Hide resolved

cli/cli.py Outdated Show resolved Hide resolved

cli/cli.py Outdated Show resolved Hide resolved

Formatting changes

b822c6f

Copilot started reviewing on behalf of jirhiker February 15, 2026 17:24 View session

feat: add validation for missing well_name_point_id column in CSV pro…

3b7c561

…cessing

Copilot AI reviewed Feb 15, 2026

View reviewed changes

tests/features/steps/well-inventory-real-user-csv.py Show resolved Hide resolved

services/well_inventory_csv.py Outdated Show resolved Hide resolved

cli/cli.py Show resolved Hide resolved

cli/cli.py Show resolved Hide resolved

cli/cli.py Show resolved Hide resolved

jirhiker added 2 commits February 15, 2026 10:34

test: update test for blank well_name_point_id to auto-generate IDs

c9d1305

test: update CSV test to include a valid row with a blank well_name_p…

6e895ca

…oint_id

Copilot AI review requested due to automatic review settings February 15, 2026 17:51

Copilot started reviewing on behalf of jirhiker February 15, 2026 17:51 View session

Copilot AI reviewed Feb 15, 2026

View reviewed changes

jirhiker and others added 2 commits February 15, 2026 10:57

feat: enhance CSV processing to handle duplicate contact names and or…

21ad925

…ganizations

Update services/well_inventory_csv.py

7c081d4

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot AI review requested due to automatic review settings February 15, 2026 18:01

Copilot started reviewing on behalf of jirhiker February 15, 2026 18:01 View session

jirhiker and others added 3 commits February 15, 2026 11:02

Update tests/features/environment.py

9765313

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update tests/test_cli_commands.py

f5d9013

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

refactor: rename step implementations for clarity and consistency

619f59f

Copilot AI reviewed Feb 15, 2026

View reviewed changes

jirhiker added 2 commits February 15, 2026 11:14

test: add test for handling multiple contacts with null organizations…

95f1426

… in CSV upload

test: remove redundant test for handling multiple null organization c…

81a324e

…ontacts in CSV upload

Copilot AI review requested due to automatic review settings February 15, 2026 18:17

Copilot started reviewing on behalf of jirhiker February 15, 2026 18:17 View session

test: streamline CSV upload tests for blank well_name_point_id and du…

1d6d697

…plicate contacts

Copilot AI reviewed Feb 15, 2026

View reviewed changes

services/well_inventory_csv.py Outdated Show resolved Hide resolved

tests/features/steps/well-notes.py Show resolved Hide resolved

tests/features/steps/api_common.py Show resolved Hide resolved

tests/test_well_inventory.py Show resolved Hide resolved

jirhiker added 2 commits February 15, 2026 11:28

fix: update type hint for well_id parameter in _extract_autogen_prefi…

729faba

…x function

fix: enhance error handling and validation reporting in CSV upload pr…

f8ceb2c

…ocess

Copilot AI review requested due to automatic review settings February 16, 2026 07:04

Copilot started reviewing on behalf of jirhiker February 16, 2026 07:04 View session

Copilot AI reviewed Feb 16, 2026

View reviewed changes

jirhiker merged commit 6599c42 into staging Feb 16, 2026
13 checks passed

TylerAdamMartinez deleted the well-inventory-csv-fix branch February 26, 2026 18:25

Conversation

jirhiker commented Feb 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

1. CLI output improvements (well-inventory-csv)

2. Validation crash fixes

3. Real-user CSV feature test coverage

4. Dependency updates (warning fix)

5. Test updates

Why

Testing

Reviewer Notes

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jirhiker commented Feb 15, 2026 •

edited

Loading

1. CLI output improvements (`well-inventory-csv`)