Skip to content

Use symbolic links instead of copying image data to Images folder#1099

Open
ElpadoCan wants to merge 13 commits into
mainfrom
use_sym_links_datastruct
Open

Use symbolic links instead of copying image data to Images folder#1099
ElpadoCan wants to merge 13 commits into
mainfrom
use_sym_links_datastruct

Conversation

@ElpadoCan
Copy link
Copy Markdown
Collaborator

@ElpadoCan ElpadoCan commented May 18, 2026

This PR implements the use of "symbolic links" files instead of the classic TIFF files for image data.

A symbolic link is a special type of file that acts as a pointer or alias, referring to another file by its path rather than its content.

In this case, the symbolic link is a _symlink.ini file that contains the information on how to read the source image file.

This file can be created in the data structure module or by opening an image file in the module 3 GUI.

See issue #1072

@ElpadoCan ElpadoCan requested a review from Teranis May 18, 2026 14:28
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a “symbolic link” mechanism for image data in Cell-ACDC by using *_symlink.ini files that point to raw/source microscopy data, allowing workflows (GUI, CLI, and segmentation pipeline) to load image data without copying it into Images/ as TIFF/NPZ/H5.

Changes:

  • Add support for discovering channels and loading image data via *_symlink.ini (including BioIO/BioFormats-backed loading paths).
  • Update GUI/CLI/data-structure workflows to optionally create and use symlink ini files instead of copying/converting image data.
  • Refactor several channel-path resolution utilities to centralize image file discovery (including symlink-aware paths).

Reviewed changes

Copilot reviewed 16 out of 17 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
cellacdc/widgets.py Adds a new SegmentPushButton used by new symlink-related dialogs.
cellacdc/utils/base.py Removes large commented-out legacy segmentation-file selection logic.
cellacdc/segm.py Refactors channel selection/loading for multi-position segmentation to use shared channel selection + centralized path lookup.
cellacdc/prompts.py Extends available-channel discovery to read channels from *_symlink.ini.
cellacdc/myutils.py Routes channel file discovery through load.get_filename_from_channel and adds parsing for ;;channel tagging.
cellacdc/load.py Core support: symlink-aware channel path selection, symlink ini read/write helpers, centralized image+background loading helper.
cellacdc/gui.py Integrates symlink option into “Open image/video file…” flow; refactors fluorescence path resolution; improves some dialogs.
cellacdc/dataStruct.py Adds UI prompt and command flag plumbing to generate symlink outputs during data-structure creation.
cellacdc/dataPrep.py Minor formatting adjustment related to channel path retrieval.
cellacdc/cli.py Uses shared load.load_image_and_bkgr_data to load channel data (symlink-aware).
cellacdc/apps.py Minor UI text/layout tweaks.
cellacdc/acdc_bioio_bioformats/reader.py Adjusts reader kwargs selection/handling to better support per-extension kwargs.
cellacdc/acdc_bioio_bioformats/_utils.py Adds symlink ini generation + symlink loader; refactors plane-reading helper.
cellacdc/acdc_bioio_bioformats/_save_data.py Adds --use_symlinks CLI flag and passes it into saving pipeline.
cellacdc/acdc_bioio_bioformats/_save_data_single_channel.py Adds -u flag for symlink mode in single-channel save script.
cellacdc/acdc_bioio_bioformats/init.py Silences noisy JAVA_HOME print.
cellacdc/init.py Adds valid_image_data_ends constant.
Comments suppressed due to low confidence (2)

cellacdc/load.py:4301

  • save_symlink_ini_from_image_filepath doesn’t return the created path, but callers (e.g., GUI “Open image/video file…”) assign it to symlink_filepath and pass it to _openFolder. This will be None and break loading. Return the symlink ini path; additionally, since _openFolder infers the selected channel by substring-matching imageFilePath, consider returning f"{symlink_ini_filepath};;{channel_name}" (or otherwise ensure the channel can be inferred).
    cp_symlink = config.ConfigParser()
    cp_symlink[f'channel_name.{channel_name}'] = {
        'source_filepath': image_filepath,
        'frames_range': '0,1',
        'zslices_range': '0,1',
        'channel_index': '0',
        'series_index': '0',
        'lazy_load': 'False',
        'use_bioio': 'False',
    }
    with open(symlink_ini_filepath, 'w') as configfile:
        cp_symlink.write(configfile)

cellacdc/gui.py:31187

  • load.get_filename_from_channel returns '' on failure, not None. The check if user_ch_path is None: will therefore not catch missing channels and will append an empty path, likely causing failures later. Use if not user_ch_path: (or adjust get_filename_from_channel to return None).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread cellacdc/load.py
Comment thread cellacdc/load.py Outdated
Comment thread cellacdc/acdc_bioio_bioformats/_utils.py
Comment thread cellacdc/acdc_bioio_bioformats/_save_data_single_channel.py
Comment thread cellacdc/gui.py
Comment thread cellacdc/segm.py
Comment thread cellacdc/dataStruct.py Outdated
Comment thread cellacdc/load.py
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 16 out of 17 changed files in this pull request and generated 4 comments.

Comments suppressed due to low confidence (1)

cellacdc/load.py:987

  • When symlink_channel_filepath is set to "<path>;;<channel>", the subsequent cp_symlink.read(symlink_channel_filepath) tries to read a non-existent filename containing ;;.... This makes the symlink branch unreachable even if the section-name check is fixed. Read the actual ini filepath (before the ;; suffix) and validate the proper section instead.
    if symlink_channel_filepath:
        cp_symlink = config.ConfigParser()
        cp_symlink.read(symlink_channel_filepath)
        if channel_name in cp_symlink.sections():
            if logger is not None:
                logger(f'Using channel file ({symlink_channel_filepath})...')
            return symlink_channel_filepath

Comment thread cellacdc/load.py Outdated
Comment thread cellacdc/gui.py
Comment thread cellacdc/acdc_bioio_bioformats/_save_data_single_channel.py
Comment thread cellacdc/prompts.py Outdated
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 19 out of 20 changed files in this pull request and generated 4 comments.

Comment thread cellacdc/myutils.py Outdated
Comment thread cellacdc/load.py
Comment thread cellacdc/load.py Outdated
Comment thread cellacdc/load.py
@ElpadoCan
Copy link
Copy Markdown
Collaborator Author

Add a symbolic link pointing to the existing channel files in a previously created Position folder

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants