Conversation
Replace fixed seeds in non-documentation tests with property-generated seeds across Rust and Python test suites. Preserve simulation semantics by checking seed-stable anomaly signals for FeatureSketch, mStream, OnlineIForest, and Forest attribution/imputation behavior. Keep documentation example seeds fixed for reproducible examples.
There was a problem hiding this comment.
Pull request overview
This PR updates the Rust and Python test suites to avoid relying on fixed RNG seeds by generating seeds via property-based testing (proptest/Hypothesis), while aiming to keep anomaly-signal assertions stable and reproducible via failure-reported seeds.
Changes:
- Replaced fixed-seed unit/integration tests with property-generated seeds in Rust (proptest) and Python (Hypothesis).
- Refactored Rust simulation tests into seed-parameterized assertion helpers invoked from proptests, improving failure diagnostics by printing the seeds.
- Updated Python JSON roundtrip tests to use temporary directories within the property tests, and made detector/forest
seedparameters optional (None=> random).
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/public_api_surface.rs | Converts the public API smoke test to a proptest with generated seeds. |
| tests/onlineiforest_test.py | Generates seeds via Hypothesis and adjusts JSON roundtrip test temp-path handling. |
| tests/onlineiforest_simulation.rs | Converts the OnlineIForest simulation to proptest-driven randomized seed coverage with improved failure context. |
| tests/mstream_simulation.rs | Converts mStream simulations to proptest-driven randomized seeds and strengthens some invariants. |
| tests/forest_test.rs | Extends existing proptests to randomize forest seeds and adjusts attribution/imputation-related assertions. |
| tests/forest_test.py | Switches multiple tests to Hypothesis-provided seeds and makes seed optional in helpers. |
| tests/featuresketch_test.py | Switches selected tests to Hypothesis-provided seeds and updates JSON roundtrip temp-path handling. |
| tests/featuresketch_simulation.rs | Converts FeatureSketch simulations to proptest-driven randomized seeds with improved failure context. |
Comment on lines
+286
to
+288
| proptest! { | ||
| #![proptest_config(ProptestConfig::with_cases(32))] | ||
|
|
Comment on lines
+280
to
+282
| proptest! { | ||
| #![proptest_config(ProptestConfig::with_cases(32))] | ||
|
|
Comment on lines
+135
to
+137
| proptest! { | ||
| #![proptest_config(ProptestConfig::with_cases(32))] | ||
|
|
| *, | ||
| input_dim: int, | ||
| seed: int, | ||
| seed: int | None = None, |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replace fixed seeds in non-documentation tests with property-generated seeds across Rust and Python test suites. Preserve simulation semantics by checking seed-stable anomaly signals for FeatureSketch, mStream, OnlineIForest, and Forest attribution/imputation behavior.
Keep documentation example seeds fixed for reproducible examples.