Skip to content

test: data-driven filter test engine with RTK_UPDATE_TEST_DATA#901

Open
osipxd wants to merge 2 commits intortk-ai:developfrom
osipxd:feat/data-driven-tests
Open

test: data-driven filter test engine with RTK_UPDATE_TEST_DATA#901
osipxd wants to merge 2 commits intortk-ai:developfrom
osipxd:feat/data-driven-tests

Conversation

@osipxd
Copy link
Copy Markdown

@osipxd osipxd commented Mar 28, 2026

Summary

  • Adds src/datatest.rs: a lightweight data-driven test engine that discovers input/expected file pairs under tests/data/ and runs them through a caller-supplied closure returning the actual output
  • Adds Rust and Python test data under tests/data/filter/ covering the full MinimalFilter and AggressiveFilter codepaths, and wires filter::tests::test_filter to use it

Update mode: set RTK_UPDATE_TEST_DATA=1 to overwrite expected files with the actual filter output instead of failing — useful when filter behaviour intentionally changes.

Known filter quirks documented by the test data (noted for follow-up):

  • pub(crate) items are silently dropped by aggressive (pub\s+ regex doesn't match pub(crate)), but Python __internal_method is kept — asymmetry between the two languages
  • Closing } is dropped from struct/mod bodies (brace depth tracking exits before emitting it)
  • #[cfg(test)] mod tests wrapper is dropped but its contents (use super::*, function signatures) are still emitted at the outer indentation level
  • Test methods inside a unittest.TestCase subclass are not dropped in Python either
  • // ... implementation placeholder is used for Python files (C-style comment)
  • Implementation placeholder is always indented 4 spaces regardless of nesting depth

Test plan

  • cargo fmt --all && cargo clippy --all-targets && cargo test
  • RTK_UPDATE_TEST_DATA=1 cargo test filter::tests::test_filter — verified update mode writes correct expected files, re-run confirms green

@CLAassistant
Copy link
Copy Markdown

CLAassistant commented Mar 28, 2026

CLA assistant check
All committers have signed the CLA.

@osipxd osipxd force-pushed the feat/data-driven-tests branch 3 times, most recently from d337976 to 86f9ffb Compare March 28, 2026 19:04
osipxd and others added 2 commits March 28, 2026 20:04
Add `src/datatest::run_tests` (compiled only under `#[cfg(test)]`) for
file-based filter tests. Pairs `<name>.<ext>` inputs with
`<name>.after.<params>.<ext>` expected outputs; collects all failures
before reporting.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add file-based test cases for the filter engine:
- tests/data/filter/rust.{rs,after.minimal.rs,after.aggressive.rs}
- tests/data/filter/python.{py,after.minimal.py,after.aggressive.py}

Wire up test_filter using datatest::run_tests("filter/*").

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@osipxd osipxd force-pushed the feat/data-driven-tests branch from 86f9ffb to 39afde0 Compare March 28, 2026 19:04
@osipxd
Copy link
Copy Markdown
Author

osipxd commented Mar 28, 2026

The failure report currently dumps full expected/actual blocks. A line-by-line diff would be much easier to read, but that requires adding a diff crate (e.g. similar). Leaving as-is for now to keep dependencies minimal.

@osipxd osipxd marked this pull request as ready for review March 28, 2026 20:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants