Skip to content

e2e test - additional pyrefly coverage#12157

Open
rodrigosf672 wants to merge 3 commits intomainfrom
rsf/pyrefly01
Open

e2e test - additional pyrefly coverage#12157
rodrigosf672 wants to merge 3 commits intomainfrom
rsf/pyrefly01

Conversation

@rodrigosf672
Copy link
Member

@rodrigosf672 rodrigosf672 commented Feb 26, 2026

This pull request expands the test coverage for Python diagnostics in the LSP integration tests, aiming to catch potential issues and breakages in diagnostic functionality. It introduces several new test cases for common Python errors and edge cases, and prepares groundwork for future tests on QMD and Notebook files. Additionally, a minor documentation update clarifies a parameter name in the Problems page object (badgeCount).

Expanded Python diagnostics test coverage:

  • Added multiple new tests to lsp-diagnostics.test.ts to verify diagnostics for scenarios such as non-existent module imports, missing attributes, calling non-callables, incompatible assignments, wrong return types, list/dict element type mismatches, and type errors with indentation issues. These tests improve robustness and help detect pyrefly-related issues or delays.
  • Added skipped tests for QMD and Notebook files, noting current limitations due to pyrefly not supporting .qmd and .ipynb files. These are placeholders for future diagnostic coverage.

Test infrastructure improvement:

  • Imported path module in lsp-diagnostics.test.ts to support file path operations for future and existing tests.

Documentation update:

  • Clarified the parameter name from problemCount to badgeCount in the Problems page object, making the expected badge count meaning more explicit.

@:pyrefly @:web

@github-actions
Copy link

github-actions bot commented Feb 26, 2026

E2E Tests 🚀
This PR will run tests tagged with: @:critical @:pyrefly @:web

readme  valid tags

@testlabauto
Copy link
Contributor

testlabauto commented Feb 27, 2026

Don't forget that this is how you test in CI:
https://github.com/posit-dev/positron/actions/workflows/test-e2e-pyrefly.yml

I ran against your PR and it looks good!

For a refresher, here is what happens:
https://github.com/posit-dev/positron/blob/main/playwright.config.ts#L46
That ^ skips tests in the lsp directory unless ALLOW_PYREFLY is set. It is only set with the workflow I mentioned above and with the nightly tests.

So, its annoying but using the tag for pyrefly on your PR doesn't do anything.

@rodrigosf672 rodrigosf672 linked an issue Mar 2, 2026 that may be closed by this pull request
3 tasks
Updated the notebook diagnostics test to use the notebooksPositron fixture
instead of the legacy notebooks fixture. Key changes:

- Enable Positron notebooks in test settings
- Use notebooksPositron.newNotebook() and kernel.select()
- Replace typeInEditor with addCodeToCell
- Use proper Positron notebooks content expectations
- Enter edit mode before editing cell content

This unskips the test and makes it work correctly with Positron notebooks.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

expand pyrefly automated tests

4 participants