Skip to content

Feat/chaos tests issue 259#272

Open
divyansha12 wants to merge 2 commits into
utksh1:mainfrom
divyansha12:feat/chaos-tests-issue-259
Open

Feat/chaos tests issue 259#272
divyansha12 wants to merge 2 commits into
utksh1:mainfrom
divyansha12:feat/chaos-tests-issue-259

Conversation

@divyansha12
Copy link
Copy Markdown

@divyansha12 divyansha12 commented May 23, 2026

Adds comprehensive integration-level chaos tests to cover task execution failure modes, ensuring robust handling and recovery. Specifically:

  1. Process crash during task execution: Simulates backend command execution failure and verifies the task/report states are marked as failed rather than staying stuck in running or pending.
  2. Upsert failure after successful scan: Simulates a database failure during the final state update to verify the task is successfully marked as failed.
  3. Non-zero exit but raw artifact present: Verifies that when a scanner exit status is non-zero, the task/report states are marked failed but the raw artifact file is still written to disk to aid debugging.
  4. Concurrency slot release: Verifies that the task queue slot concurrency limiter is unconditionally released in a finally block even when a crash occurs, preventing queue starvation.

Additionally, this PR:

  • Replaced hardcoded python3 command in test_python.sh with a checker to automatically find Python 3.11+ and reconstruct venv_tests if the version is outdated.
  • Cleaned up auto-generated Playwright/test-results folders from git tracking and added them to .gitignore to comply with the project guidelines.
  • Recalculated integrity checksums for all 60 plugins.

Related Issues

Closes #259

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update

How Has This Been Tested?

  1. Ran all four new chaos integration tests: pytest -v testing/backend/integration/test_chaos_execution.py (all passed successfully).
  2. Ran the full backend test suite: ./testing/test_python.sh (all 335 tests passed successfully).
  3. Ran pre-commit run --all-files which fully succeeded across all hooks (including black, end-of-file-fixer, trailing-whitespace, and check-yaml).

Checklist

  • My code follows the code style of this project.
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • My changes generate no new warnings.

🟢 GSSoC note – I’m a GSSoC contributor. All checks are passing; please add the gssoc:approved label when merging.

…patibility

- Add new integration tests in testing/backend/integration/test_chaos_execution.py
- Fix Python version compatibility issues in testing/test_python.sh to use Python 3.11+
- Format codebase using black to comply with pre-commit static checks
- Recalculate and update integrity checksums for all 60 plugins
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[TEST] Add chaos tests for task execution failures and partial artifacts

1 participant