chore(delphi): install pandas-stubs to fix pyright false positives#2565
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR aims to reduce Pyright false positives in the Delphi polismath code by installing pandas-stubs and adding two narrowly-scoped Pyright ignores for remaining numpy-stubs typing limitations.
Changes:
- Add
pandas-stubsas a development dependency indelphi/pyproject.toml. - Add two line-scoped
# pyright: ignore[...]suppressions inrepness.pywith explanatory comments and upstream Pyright issue references.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
delphi/pyproject.toml |
Adds a dependency-groups dev group with pandas-stubs to improve Pyright typing for pandas. |
delphi/polismath/pca_kmeans_rep/repness.py |
Adds two targeted Pyright ignores to silence known numpy-stubs/Pyright typing issues without affecting runtime behavior. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
pandas 2.3 ships no `py.typed` marker, so pyright fell back to its bundled stubs and mistyped `DataFrame.__getitem__` as `ndarray`, producing 55 errors in `polismath/pca_kmeans_rep/repness.py` alone. Adding the official `pandas-stubs` package (from the pandas-dev org) drops it to 2 errors. The 2 remaining errors are numpy-stubs limitations (np.where dtype loss, np.sqrt(Series) losing Series-ness). Each is silenced with a line-scoped `# pyright: ignore[reportXxx]` comment that names the specific report category and points to the upstream pyright issue. No production code semantics changed. Verified: - `uv run pyright polismath/pca_kmeans_rep/repness.py` → 0 errors - A deliberate `int`-for-`Series` bug is still caught - 22/22 tests in `tests/test_repness_unit.py` pass Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
jucor
force-pushed
the
jc/delphi-pandas-stubs
branch
from
222553f to
d27d2b3
Compare
Delphi Coverage Report
|
jucor
added a commit
that referenced
this pull request
Jun 11, 2026
Re-records `golden_snapshot.json` for `vw` and `biodiversity` against the top of the D10/D11/D12 stack (after PRs #2564, #2566, #2567, #2568). The previous goldens were deleted in PR #2516 because they were stale; this PR brings them back at the new post-stack baseline. Sits on top of D12 (PR 11) in the spr stack. ## DEFERRED Per Julien 2026-06-11: goldens re-records belong to the Python-vs-Python refactor comparison phase, not the current Clojure-parity chase. This snapshot exists as a historical reference; DO NOT re-record until we enter that phase. Goldens add noise during Clojure-parity work — they shift on every parity fix and have to be re-recorded with no oracle (the Clojure blob is the only ground truth right now). Reactivate this PR (or re-create it) once the Python pipeline is stable and we are comparing pre-vs-post Python refactor. ## Why these goldens reflect the current state After PR 14a (scalar deletion) + ns-PASS fix + D10 (rep selection) + D11 (consensus selection) + D12 (comment priorities), the Python pipeline output now includes: - The Clojure-parity `ns` semantics (counts PASS votes). - The new Clojure-parity rep-comment selection (D10). - The new `{'agree': [...], 'disagree': [...]}` consensus shape (D11). - The newly-computed `comment_priorities` dict (D12). - The pre-existing D5/D6/D7/D8/D9/D15/K-inv fixes from the 2026-06-09/10 stack. These snapshot files capture the combined state. They are NOT a baseline for the `test_regression.py` regression suite during the Clojure-parity phase — they are reference values for the future Python-vs-Python phase. ## sklearn KMeans seeding The recorder uses sklearn's `KMeans(random_state=42)` (the default in `polismath/pca_kmeans_rep/kmeans.py`). This produces deterministic output across runs without requiring exact match with Clojure's first-k-distinct initialization. The seeding policy is unchanged from pre-D10 — this PR just re-records against the new output values. Future work (separate PR): decide whether to switch to first-k-distinct seeding for tighter Clojure parity, or accept the sklearn-vs-Clojure divergence with looser comparison tolerances. ## /goal mode Final PR in the autonomous stacked series (PR 14a → ns-PASS → D10 → D11 → D12 → goldens) per user request. All decisions documented in `~/polis/D10_D11_D12_GOLDENS_DECISIONS.md` for batch review. The stack also picks up the recent edge change `#2565` (`chore(delphi): install pandas-stubs to fix pyright false positives`), which was rebased under all prior commits before this golden record. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> commit-id:f7185ccd
jucor
added a commit
that referenced
this pull request
Jun 11, 2026
Re-records `golden_snapshot.json` for `vw` and `biodiversity` against the top of the D10/D11/D12 stack (after PRs #2564, #2566, #2567, #2568). The previous goldens were deleted in PR #2516 because they were stale; this PR brings them back at the new post-stack baseline. Sits on top of D12 (PR 11) in the spr stack. ## DEFERRED Per Julien 2026-06-11: goldens re-records belong to the Python-vs-Python refactor comparison phase, not the current Clojure-parity chase. This snapshot exists as a historical reference; DO NOT re-record until we enter that phase. Goldens add noise during Clojure-parity work — they shift on every parity fix and have to be re-recorded with no oracle (the Clojure blob is the only ground truth right now). Reactivate this PR (or re-create it) once the Python pipeline is stable and we are comparing pre-vs-post Python refactor. ## Why these goldens reflect the current state After PR 14a (scalar deletion) + ns-PASS fix + D10 (rep selection) + D11 (consensus selection) + D12 (comment priorities), the Python pipeline output now includes: - The Clojure-parity `ns` semantics (counts PASS votes). - The new Clojure-parity rep-comment selection (D10). - The new `{'agree': [...], 'disagree': [...]}` consensus shape (D11). - The newly-computed `comment_priorities` dict (D12). - The pre-existing D5/D6/D7/D8/D9/D15/K-inv fixes from the 2026-06-09/10 stack. These snapshot files capture the combined state. They are NOT a baseline for the `test_regression.py` regression suite during the Clojure-parity phase — they are reference values for the future Python-vs-Python phase. ## sklearn KMeans seeding The recorder uses sklearn's `KMeans(random_state=42)` (the default in `polismath/pca_kmeans_rep/kmeans.py`). This produces deterministic output across runs without requiring exact match with Clojure's first-k-distinct initialization. The seeding policy is unchanged from pre-D10 — this PR just re-records against the new output values. Future work (separate PR): decide whether to switch to first-k-distinct seeding for tighter Clojure parity, or accept the sklearn-vs-Clojure divergence with looser comparison tolerances. ## /goal mode Final PR in the autonomous stacked series (PR 14a → ns-PASS → D10 → D11 → D12 → goldens) per user request. All decisions documented in `~/polis/D10_D11_D12_GOLDENS_DECISIONS.md` for batch review. The stack also picks up the recent edge change `#2565` (`chore(delphi): install pandas-stubs to fix pyright false positives`), which was rebased under all prior commits before this golden record. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> commit-id:f7185ccd
jucor
added a commit
that referenced
this pull request
Jun 11, 2026
Re-records `golden_snapshot.json` for `vw` and `biodiversity` against the top of the D10/D11/D12 stack (after PRs #2564, #2566, #2567, #2568). The previous goldens were deleted in PR #2516 because they were stale; this PR brings them back at the new post-stack baseline. Sits on top of D12 (PR 11) in the spr stack. ## DEFERRED Per Julien 2026-06-11: goldens re-records belong to the Python-vs-Python refactor comparison phase, not the current Clojure-parity chase. This snapshot exists as a historical reference; DO NOT re-record until we enter that phase. Goldens add noise during Clojure-parity work — they shift on every parity fix and have to be re-recorded with no oracle (the Clojure blob is the only ground truth right now). Reactivate this PR (or re-create it) once the Python pipeline is stable and we are comparing pre-vs-post Python refactor. ## Why these goldens reflect the current state After PR 14a (scalar deletion) + ns-PASS fix + D10 (rep selection) + D11 (consensus selection) + D12 (comment priorities), the Python pipeline output now includes: - The Clojure-parity `ns` semantics (counts PASS votes). - The new Clojure-parity rep-comment selection (D10). - The new `{'agree': [...], 'disagree': [...]}` consensus shape (D11). - The newly-computed `comment_priorities` dict (D12). - The pre-existing D5/D6/D7/D8/D9/D15/K-inv fixes from the 2026-06-09/10 stack. These snapshot files capture the combined state. They are NOT a baseline for the `test_regression.py` regression suite during the Clojure-parity phase — they are reference values for the future Python-vs-Python phase. ## sklearn KMeans seeding The recorder uses sklearn's `KMeans(random_state=42)` (the default in `polismath/pca_kmeans_rep/kmeans.py`). This produces deterministic output across runs without requiring exact match with Clojure's first-k-distinct initialization. The seeding policy is unchanged from pre-D10 — this PR just re-records against the new output values. Future work (separate PR): decide whether to switch to first-k-distinct seeding for tighter Clojure parity, or accept the sklearn-vs-Clojure divergence with looser comparison tolerances. ## /goal mode Final PR in the autonomous stacked series (PR 14a → ns-PASS → D10 → D11 → D12 → goldens) per user request. All decisions documented in `~/polis/D10_D11_D12_GOLDENS_DECISIONS.md` for batch review. The stack also picks up the recent edge change `#2565` (`chore(delphi): install pandas-stubs to fix pyright false positives`), which was rebased under all prior commits before this golden record. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> commit-id:f7185ccd
jucor
added a commit
that referenced
this pull request
Jun 11, 2026
Re-records `golden_snapshot.json` for `vw` and `biodiversity` against the top of the D10/D11/D12 stack (after PRs #2564, #2566, #2567, #2568). The previous goldens were deleted in PR #2516 because they were stale; this PR brings them back at the new post-stack baseline. Sits on top of D12 (PR 11) in the spr stack. ## DEFERRED Per Julien 2026-06-11: goldens re-records belong to the Python-vs-Python refactor comparison phase, not the current Clojure-parity chase. This snapshot exists as a historical reference; DO NOT re-record until we enter that phase. Goldens add noise during Clojure-parity work — they shift on every parity fix and have to be re-recorded with no oracle (the Clojure blob is the only ground truth right now). Reactivate this PR (or re-create it) once the Python pipeline is stable and we are comparing pre-vs-post Python refactor. ## Why these goldens reflect the current state After PR 14a (scalar deletion) + ns-PASS fix + D10 (rep selection) + D11 (consensus selection) + D12 (comment priorities), the Python pipeline output now includes: - The Clojure-parity `ns` semantics (counts PASS votes). - The new Clojure-parity rep-comment selection (D10). - The new `{'agree': [...], 'disagree': [...]}` consensus shape (D11). - The newly-computed `comment_priorities` dict (D12). - The pre-existing D5/D6/D7/D8/D9/D15/K-inv fixes from the 2026-06-09/10 stack. These snapshot files capture the combined state. They are NOT a baseline for the `test_regression.py` regression suite during the Clojure-parity phase — they are reference values for the future Python-vs-Python phase. ## sklearn KMeans seeding The recorder uses sklearn's `KMeans(random_state=42)` (the default in `polismath/pca_kmeans_rep/kmeans.py`). This produces deterministic output across runs without requiring exact match with Clojure's first-k-distinct initialization. The seeding policy is unchanged from pre-D10 — this PR just re-records against the new output values. Future work (separate PR): decide whether to switch to first-k-distinct seeding for tighter Clojure parity, or accept the sklearn-vs-Clojure divergence with looser comparison tolerances. ## /goal mode Final PR in the autonomous stacked series (PR 14a → ns-PASS → D10 → D11 → D12 → goldens) per user request. All decisions documented in `~/polis/D10_D11_D12_GOLDENS_DECISIONS.md` for batch review. The stack also picks up the recent edge change `#2565` (`chore(delphi): install pandas-stubs to fix pyright false positives`), which was rebased under all prior commits before this golden record. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> commit-id:f7185ccd
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
pandas2.3 ships nopy.typedmarker, so pyright fell back to its bundled stubs and mistypedDataFrame.__getitem__asndarray, producing 55 errors inpolismath/pca_kmeans_rep/repness.pyalone. Installing the officialpandas-stubs(from thepandas-devorg) drops that to 2 errors.The 2 remaining errors are numpy-stubs limitations, not pandas:
repness.py:467—np.wherewidenspfromnp.ndarraytonp.ndarray | bool_, sonp.sum(...)is typedbool_(pyright #2811).repness.py:541—np.sqrt(Series)loses Series-ness, so.fillna()looks like an attribute error (pyright #4081).Each is silenced with a line-scoped
# pyright: ignore[reportXxx]that names the specific category and links to the upstream issue. No global rule overrides, notypeCheckingMode = "off".What changed
delphi/pyproject.toml— new[dependency-groups] dev = ["pandas-stubs>=3.0.3.260530"]with an explanatory comment.delphi/polismath/pca_kmeans_rep/repness.py— two single-line# pyright: ignore[...]comments + 2 explanatory lines above them. No runtime behavior change.Why not just disable the report categories globally
The handoff doc that prompted this work (
HANDOFF_PYRIGHT_PANDAS_STUBS.md) explicitly called that out as the wrong answer:reportArgumentType = "none"and friends would also kill real argument-type signal everywhere. Targeted ignores keep the rest of the codebase honest.Test plan
uv run pyright polismath/pca_kmeans_rep/repness.py→ 0 errors (was 55).int-where-Series-is-expected bug is still caught (verified with a 5-line scratch script — pyright flagged both args).tests/test_repness_unit.py→ 22/22 pass.Notes
pandas-stubsis an official endorsed-by-pandas-dev package and is what Pylance ships with by default.delphi/uv.lockis not tracked in this repo (onlydelphi/requirements.lock, which holds prod deps viapip-compile). Dev deps live inpyproject.tomlonly, same convention as PR chore(delphi): standardize venv as .venv and add Pyright config #2560.py.typedmarker, so PEP 561 means pyright ignores its inline annotations. Confirmed byfind .venv ... -name py.typed.