Add tunable buffer size for spt3g filtering_istream#194
Merged
Conversation
mhasself
approved these changes
Dec 12, 2024
Member
mhasself
left a comment
There was a problem hiding this comment.
This is a miracle... no notes. This basically resolves the performance concerns, as far as I can tell. Let's get it out there.
|
We'll just integrate this upstream as a keyword argument to the g3_istream / G3Reader code. |
|
I'm not really sure there is much downside to setting a big global default rather than having a config parameter -- no one using this software is reading a couple KB and then going home and especially no one is reading a couple KB from something really slow where they would notice the extra reads. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This adds a small (and perhaps temporary) patch to specify the buffer size to the
filtering_istreamstack. This is a quick update to fix slow data loading and #192 will be rebased after this is merged. Also added GSL to the wheel build dependencies.For testing on a compute node on perlmutter.nersc.gov, I used a single process and 8 threads. I then loaded one wafer (ufm_mv12) of satp3 observation
obs_1714550584_satp3_1111111.I used the new
SO3G_FILESYSTEM_BUFFERenvironment variable to try a variety of buffer sizes loading this same data from the CFS and scratch filesystems. Note that the frame files in this case are each ~200MB, so the largest buffer size tested is actually larger than the frame files. The smaller values of buffer size were essentially unusable on CFS.Here is a plot of the times:

Note that CFS access is faster from a login node than a compute node, and scratch access is faster from a compute node than a login node. Here is the data in the plot: