Skip to content

Fix double-counting of occupied disk space#1103

Merged
GlassOfWhiskey merged 3 commits into
masterfrom
fix/double-count-size
Jun 24, 2026
Merged

Fix double-counting of occupied disk space#1103
GlassOfWhiskey merged 3 commits into
masterfrom
fix/double-count-size

Conversation

@LanderOtto

@LanderOtto LanderOtto commented Jun 23, 2026

Copy link
Copy Markdown
Collaborator

This commit fixes the double-counting of occupied disk space when a job releases its resources. The issue arises when the Job instance uses the same directory for its output and tmp directories. In the current implementation, this can happen when the Job instance has no explicit values in its input, output, or tmp directory attributes, and the target workdir is used for all of them. Added new test.

Additionally, this commit fixes the behavior of the bind_mount_point method, which was collapsing Storage without following the correct logic, leading to missing Storage info. Improved test_bind_volumes.

…ob releases its resources. The issue arises when the `Job` instance uses the same directory for its output and tmp directories. In the current implementation, this can happen when the `Job` has no explicit input, output, or tmp directory definition, and the `target` workdir is used for all of them.
@LanderOtto LanderOtto force-pushed the fix/double-count-size branch from b02c631 to 464d019 Compare June 23, 2026 09:30
@codecov

codecov Bot commented Jun 23, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 96.29630% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 64.18%. Comparing base (50bec95) to head (6bc31d9).
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
streamflow/core/scheduling.py 94.44% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           master    #1103       +/-   ##
===========================================
+ Coverage        0   64.18%   +64.18%     
===========================================
  Files           0       93       +93     
  Lines           0    17570    +17570     
  Branches        0     2161     +2161     
===========================================
+ Hits            0    11278    +11278     
- Misses          0     6010     +6010     
- Partials        0      282      +282     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

…y indicate when a normalized instance is necessary. Additionally, fix the behavior of the `bind_mount_point` method, which was collapsing `Storage` without following the correct logic, leading to missing `Storage` info. Now, the bind method keeps the previous keys for the storages
@LanderOtto LanderOtto force-pushed the fix/double-count-size branch 7 times, most recently from 8b58b44 to 4e558e8 Compare June 23, 2026 15:20
@LanderOtto LanderOtto force-pushed the fix/double-count-size branch from 4e558e8 to 6bc31d9 Compare June 23, 2026 15:29

@GlassOfWhiskey GlassOfWhiskey left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@GlassOfWhiskey GlassOfWhiskey merged commit e8dc796 into master Jun 24, 2026
42 checks passed
@GlassOfWhiskey GlassOfWhiskey deleted the fix/double-count-size branch June 24, 2026 09:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants