Is there away to just append anew "slice"to an existing file. without loading all the existing data?

[pconesa]

[colinpalmer]

There isn't currently a function to do this in mrcfile, but I can consider adding one if it'd be useful.

It should be possible to do this anyway, though it would take a bit of care to get it right. You can open an MRC file without reading all the data either using mrcfile.mmap or mrcfile.open with header_only=True. Then you could adjust the header values for mz and nz to make the header consistent with the new file size, and write the extra data to the end of the file using standard Python or numpy file writing methods.

[pconesa]

Thanks Colin. Let me explain our case that may be generic enough to justify having a specific implementation. We are "motioncorrecting TiltSeriesMovies" (each movie in a tilt series)... write a final tiltserie stack we open the "tilt image" (separated) and add it to our stack... we are currenlty keeping a the numpyarrayfor all the tilts in memory... this may result in a 20GB or even more memory ocupancy. It is true that for tomography machines should be ready for this... but such an implementation to append the "last" images to an existing mrcfile will aleviate this issue.

…

________________________________ De: Colin Palmer ***@***.***> Enviado: lunes, febrero 17, 2025 10:27 PM Para: ccpem/mrcfile ***@***.***> Cc: Pablo Conesa ***@***.***>; Author ***@***.***> Asunto: Re: [ccpem/mrcfile] Is there away to just append anew "slice"to an existing file. without loading all the existing data? (Issue #67) There isn't currently a function to do this in mrcfile, but I can consider adding one if it'd be useful. It should be possible to do this anyway, though it would take a bit of care to get it right. You can open an MRC file without reading all the data either using mrcfile.mmap or mrcfile.open with header_only=True. Then you could adjust the header values for mz and nz to make the header consistent with the new file size, and write the extra data to the end of the file using standard Python or numpy file writing methods. — Reply to this email directly, view it on GitHub<#67 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAF7ZYJIPYWCPVKMW3PPJHT2QJH4XAVCNFSM6AAAAABXEUDIE6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNRUGEYDCNZQHA>. You are receiving this because you authored the thread.Message ID: ***@***.***> [colinpalmer]colinpalmer left a comment (ccpem/mrcfile#67)<#67 (comment)> There isn't currently a function to do this in mrcfile, but I can consider adding one if it'd be useful. It should be possible to do this anyway, though it would take a bit of care to get it right. You can open an MRC file without reading all the data either using mrcfile.mmap or mrcfile.open with header_only=True. Then you could adjust the header values for mz and nz to make the header consistent with the new file size, and write the extra data to the end of the file using standard Python or numpy file writing methods. — Reply to this email directly, view it on GitHub<#67 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAF7ZYJIPYWCPVKMW3PPJHT2QJH4XAVCNFSM6AAAAABXEUDIE6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDMNRUGEYDCNZQHA>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[colinpalmer]

Hello, sorry for the slow reply. I agree with your case and I can see the usefulness of being able to add extra slices to a file. I won't realistically have time to do this in the next few weeks, but I will try and look at it later.