Fix flaky zero-allocation tests on Julia 1.12+ via AllocCheck.check_allocs#66
Merged
ChrisRackauckas merged 1 commit intoJun 17, 2026
Conversation
The allocation tests in test/alloc_tests.jl used `@allocated`, which measures allocations of a single runtime call. On Julia 1.12+ this surfaced false positives that did not reflect any real allocation in the package: - `dr[8]` (DisjointRange second-range branch) was never warmed up before its `@allocated`, so it measured ~700k bytes of first-call compilation. - The iteration test accumulated into a global-scope variable inside the `@allocated begin ... end` block, so the accumulator was boxed (704 bytes) — an artifact of measuring in non-function scope, not of the iterate methods. - The first `@allocated A[50, 50]` after a single warmup still captured residual specialization allocations (16 bytes). `AllocCheck.check_allocs` (a test dependency already declared but unused) statically proves the absence of any allocating code path for the given argument types. It is immune to compilation and global-scope boxing noise and is a strictly stronger guarantee than `@allocated == 0`. `check_allocs` reports zero allocation sites for every operation on Julia 1.10, 1.12.6, and 1.13.0-rc1, confirming the package code is genuinely allocation-free; only the test methodology was flaky. Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The
Coretest group is red onmainacross every platform (macOS, Ubuntu, Windows) and on both Julia1(1.12.6) andpre(1.13.0-rc1). The failures are all intest/alloc_tests.jl:Root cause
These are
@allocatedmeasurement artifacts on Julia 1.12+, not real allocations in the package.@allocatedmeasures the allocations of a single runtime call, which conflates first-call compilation and global-scope boxing with genuine heap allocation:dr[8]) — the second-range branch ofDisjointRangegetindex is never warmed up before this@allocated(onlydr[3], the first-range branch, runs at line 20), so it measures ~700 KB of first-call compilation.slives in the global scope of the@allocated begin ... endblock, so it is boxed (704 bytes). That is an artifact of measuring outside a function, not of theiteratemethods.A[50, 50]) — even after one warmup call, the first@allocatedcaptured residual specialization allocations (16 bytes).The package targeted Julia 1.10/1.11 CI when these tests were written; they passed there and only began failing as CI moved to Julia 1.12/1.13.
Fix
Rewrite the tests to use
AllocCheck.check_allocs(a test dependency that was already declared but unused).check_allocsstatically proves the absence of any allocating code path for the given argument types — it is immune to compilation and global-scope boxing noise and is a strictly stronger guarantee than@allocated == 0. This is not a loosening: the same intended invariant (the operation never allocates) is verified more rigorously.Local verification
check_allocsreports 0 allocation sites for every operation, confirming the package code is genuinely allocation-free; only the test methodology was flaky. Runningtest/alloc_tests.jlafter the change:Full
Coregroup via the realSciMLTesting.run_tests()harness on 1.12.6:Runic format check on the changed file passes (exit 0).
Please ignore until reviewed by @ChrisRackauckas.
🤖 Generated with Claude Code