Skip to content

Evaluate adding pyright alongside mypy in CI #134

@JohnRDOrazio

Description

@JohnRDOrazio

Background

The repo already has [tool.pyright] configured in pyproject.toml (just venvPath, venv, pythonVersion), but pyright is purely IDE-side today — every editor with a Python LSP picks it up locally. Mypy is the only type checker that gates merges, configured strict with the pydantic.mypy plugin (pyproject.toml:[tool.mypy], .pre-commit-config.yaml, .github/workflows/release.yml:20).

The two checkers catch overlapping but not identical sets of issues:

  • Mypy strengths: plugin ecosystem (pydantic.mypy, sqlalchemy[mypy]) catches framework-specific patterns pyright can't see; conservative inference reduces false positives.
  • Pyright strengths: stronger bidirectional inference catches narrowing bugs and unused-symbol cases mypy misses; significantly faster (incremental TypeScript engine vs. full Python re-check); tracks PEP drafts more closely.

Real example from this session: while editing ontokit/services/ontology_index.py and tests/unit/test_ontology_index_service.py, the editor's pyright pass surfaced ~10 unused-variable diagnostics that mypy doesn't flag (e.g., mock_db parameters injected for fixture symmetry but not asserted on, _args/_kwargs placeholder names, _self in nested helpers). None are bugs, but several are signals that test fixtures or function signatures could be tightened.

Proposal

Add pyright as a non-gating CI step first, then decide whether to make it gating after seeing real-world signal/noise.

Phase 1 — non-gating (this issue's PR)

  • Add pyright to [dependency-groups].dev in pyproject.toml.
  • Add a step to .github/workflows/release.yml (or a new dedicated job) running uv run pyright ontokit/. Mark it continue-on-error: true so it surfaces issues without blocking merges.
  • Tune [tool.pyright]:
    • Likely turn off reportUnusedVariable for test files (most noise comes from fixture-symmetry parameters).
    • Decide on reportImplicitOverride, reportMissingTypeStubs, reportPrivateUsage — these are the most opinionated rules.
    • Possibly typeCheckingMode = "standard" initially rather than "strict" — strict pyright + strict mypy is overkill.

Phase 2 — evaluate (follow-up)

After ~2 weeks of non-gating runs:

  • Count how many unique issues pyright flags that mypy missed. Are they real bugs, real cleanups, or noise?
  • Decide: drop continue-on-error, keep it advisory, or remove the check.

Why not just use one or the other?

  • Drop mypy, use pyright: loses pydantic.mypy plugin (catches Pydantic field/validator typing issues pyright can't see) and pre-commit-config.yaml integration. Net regression.
  • Drop pyright, ignore the suggestion: loses the stronger inference + the editor signal that's already running locally for every contributor.

Adding pyright as a complement (not a replacement) is the path that captures the value of both.

Tradeoffs to weigh in the PR

  • CI duration: pyright is fast (likely <30s for ontokit/), so minimal time cost.
  • Maintenance: one more tool's config to keep current as the codebase grows.
  • Tuning cost: initial run will likely surface 50-100+ diagnostics. Most are noise; sorting through them is the bulk of the work.

Acceptance criteria

  • pyright added to [dependency-groups].dev in pyproject.toml
  • CI workflow runs uv run pyright ontokit/ (initially continue-on-error: true)
  • [tool.pyright] tuned to suppress noise that's clearly fixture-symmetry / placeholder rather than real issues
  • Decision documented in CLAUDE.md / AGENTS.md on which checker is authoritative for what

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions