Add GPU-accelerated 3D z-stack renderer by keejkrej · Pull Request #1096 · SchmollerLab/Cell_ACDC

keejkrej · 2026-05-06T19:43:42Z

vispy 3d rendering of z-stacks, supports overlay channel and segmentation channel
don't merge yet, haven't thoroughly tested
feel free to checkout and test

Implements a standalone vispy-based 3D volume renderer with full Cell_ACDC integration: multi-channel overlay, segmentation mask volumes, anisotropic voxel scaling, plane/ISO/MIP depiction modes, smooth-ISO pre-filter, percentile auto-contrast, opacity/contrast controls, keyboard shortcuts, QSettings persistence, and screenshot export. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Copilot

Pull request overview

Adds a new GPU-accelerated 3D z-stack renderer (VisPy-based) and integrates it into the main GUI as a new z-projection option, alongside a large smoke-test suite intended to validate renderer behavior without requiring an active OpenGL window.

Changes:

Introduces cellacdc.renderer3d with a Qt window, controls, overlay volumes, voxel anisotropy scaling, and screenshot export.
Integrates the renderer into gui.py via a new z-projection dropdown entry and adapter-based wiring for feeding current volumes/overlays.
Adds renderer3d optional dependency group (VisPy + PyOpenGL) and a new tests/test_renderer3d.py test module.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 10 comments.

File	Description
tests/test_renderer3d.py	Adds renderer3d smoke/API/tests, including vispy-compat checks and GUI integration expectations.
pyproject.toml	Adds `renderer3d` optional dependencies and includes them in `all`.
cellacdc/renderer3d.py	New VisPy/Qt 3D volume renderer window, controls, overlays, voxel scaling, settings persistence, screenshot export.
cellacdc/gui.py	Adds “3D z-render” mode, launches/updates/hides renderer window, and supplies current z-stack + overlays + voxel sizes.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+
+def test_interpolation_modes_valid_in_vispy():
+    """Every INTERPOLATION_MODES id must be a valid vispy Volume interpolation."""
+    from cellacdc import renderer3d


+    from cellacdc import renderer3d
+    import vispy
+    from qtpy import API_NAME
+    vispy.use(API_NAME)
+    from vispy.scene.visuals import Volume


+def test_depiction_plane_configs_valid_in_vispy():
+    """_PLANE_CONFIGS normals must be unit vectors; vispy supports 'plane' mode."""
+    import vispy; from qtpy import API_NAME; vispy.use(API_NAME)
+    from vispy.scene.visuals import Volume
+    from cellacdc import renderer3d


+    """VolumeRenderer3DWindow must expose all expected public methods."""
+    from cellacdc.renderer3d import VolumeRenderer3DWindow
+    required = {
+        'update_volume', 'set_rendering_mode', 'set_colormap', 'set_clim',
+        'set_gamma', 'set_opacity', 'set_iso_threshold', 'set_attenuation',
+        'set_interpolation', 'set_step_size', 'set_smooth_iso',
+        'set_depiction', 'set_zplane_position', 'set_plane_thickness',
+        'set_voxel_scale', 'reset_view', 'save_screenshot',
+        'auto_contrast_percentile',
+    }


+        s = QSettings(TEST_ORG, TEST_APP)
+        assert s.value('mode_idx',       type=int)   == 3
+        assert s.value('colormap',       type=str)   == 'viridis'
+        assert s.value('interp_idx',     type=int)   == 1
+        assert abs(s.value('clim_min',   type=float) - 0.05) < 1e-6


+COLORMAPS: list[str] = [
+    'grays', 'viridis', 'hot', 'coolwarm', 'blues', 'reds',
+    'greens', 'plasma', 'inferno', 'magma',
+]
+


+            except ImportError:
+                pass  # scipy not available — skip smoothing silently
+        self._gpu_data_is_smoothed = smoothed
+        current_cmap = 'grays'


+        s.setValue('clim_max',       c._clim_max.value())
+        s.setValue('gamma',          c._gamma_spin.value())
+        s.setValue('step_size',      c._step_spin.value())
+        s.setValue('smooth_iso',     c._smooth_iso_cb.isChecked())


                ol_img = img.mean(axis=0)
            elif zProjHow == 'median z-proj.':
                ol_img = np.median(img, axis=0)
+            else:
+                ol_img = img[z].copy()


keejkrej · 2026-05-13T16:16:23Z

@codex review

ElpadoCan · 2026-05-18T14:33:21Z

Hi @keejkrej,

Thank you for this contribution, it looks awesome!! 🚀

Before we merge, do you plan to look at the Copilot comments, or should I also have a look?

Also, the tests are failing because the vispy module is missing. Should Cell-ACDC handle the installation of vispy the first time the user tries this feature?

Thanks!

keejkrej · 2026-05-18T19:52:49Z

Hi @ElpadoCan ,

it would be great if you can have a look at the copilot comments and make some changes. I think one has to make several important design choices to make the 3d renderer truly useful for segmentation (it's more than just model integrations).

ElpadoCan · 2026-05-19T07:24:05Z

3d renderer truly useful for segmentation

What do you mean by this? You mean displaying segmentation masks in the 3D renderer as well? Because editing segmentation masks in 3D sounds hard, I would not do it for now. What do you think?

ElpadoCan · 2026-05-19T08:50:30Z

Hi @keejkrej,

For now, I will remove the tests\test_renderer3d.py file from the automatic tests, because while it works locally, it fails on the GitHub VMs. This is likely because of GUI initialisation. I tried to fix it, but it now fails without any additional information.

keejkrej · 2026-05-19T10:12:50Z

I mean whether to show 3d render in separate window or in side by side view in the gui. Do we need real time feedbacks for the segmentation? It is currently implemented as a post-validation of segmentation.

ElpadoCan · 2026-05-19T12:04:50Z

I actually like it in a separate window, what do you think @Teranis?

ElpadoCan · 2026-05-19T12:08:01Z

I mean whether to show 3d render in separate window or in side by side view in the gui. Do we need real time feedbacks for the segmentation? It is currently implemented as a post-validation of segmentation.

I don't think we need the ability to edit the segmentation masks in the 3d renderer. The only thing I would add is to display the segmentation masks overlaid with colours. Can we do this by displaying an image for the masks as well?

ElpadoCan · 2026-05-19T13:36:02Z

Cool feature that would be nice to have:

z-anisotropy adjustments with interpolation

This way, one can visualise volumes where the classic z-anisotropy is rendered as isotropic

Teranis · 2026-05-19T14:15:20Z

Hi, this looks very cool! I like the idea that it's in a separate window since it's really just a visualisation. Along these lines, I would move the entry point to a dedicated button or menu option in the top ribbon, and if possible, make it so one could use the feature outside of the GUI as a utility.
Additionally I had the problem that there was an error (totally my fault, I forgot to install PyOpenGL), but the error only showed in the console, and nowhere on screen. I would suggest adding error handling similar to the workers and the main GUI!

ElpadoCan · 2026-05-19T14:41:19Z

Hi, this looks very cool! I like the idea that it's in a separate window since it's really just a visualisation. Along these lines, I would move the entry point to a dedicated button or menu option in the top ribbon, and if possible, make it so one could use the feature outside of the GUI as a utility.
Additionally I had the problem that there was an error (totally my fault, I forgot to install PyOpenGL), but the error only showed in the console, and nowhere on screen. I would suggest adding error handling similar to the workers and the main GUI!

Yes, I agree with everything. It would be cool if this was an easy to use python API as well.

Regarding UI, I'm working on some improvements.

For the missing packages, I will add automatic installation. I'm also planning to add a Cell ID selector to show only a specific cell (if segm is available of course).

Stay tuned 🚀

ElpadoCan · 2026-05-20T13:33:44Z

Yes, I agree with everything. It would be cool if this was an easy to use python API as well.

Note tha this is already possible like this

renderer = VolumeRenderer3DWindow()
renderer.update_volume(zstack_array)   # (Z, Y, X) numpy array
renderer.show()

We can add cellacdc._run._setup_app() before which checks if an application is already created or not. If not, it creates one and the API would be self sustained

Copilot AI review requested due to automatic review settings May 6, 2026 19:43

Copilot started reviewing on behalf of keejkrej May 6, 2026 19:44 View session

Copilot AI reviewed May 6, 2026

View reviewed changes

ElpadoCan changed the title ~~feat: add GPU-accelerated 3D z-stack renderer~~ Add GPU-accelerated 3D z-stack renderer May 18, 2026

ElpadoCan added 4 commits May 19, 2026 09:37

fix: install vispy and PyOpenGL for CI tests

6e9efa5

fix: test_rendered3d failing

0f64a81

fix: test_rendered3d failing because of GUI init

3f0d6f2

fix: test_rendered3d failing because not closing gui elements

ca5f179

ci: ignore test_renderer3d because it requires a GUI environment

5fdb923

ElpadoCan mentioned this pull request May 19, 2026

Improvements 3d renderer #1102

Draft

Conversation

keejkrej commented May 6, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

keejkrej commented May 13, 2026

Uh oh!

ElpadoCan commented May 18, 2026

Uh oh!

keejkrej commented May 18, 2026

Uh oh!

ElpadoCan commented May 19, 2026

Uh oh!

ElpadoCan commented May 19, 2026

Uh oh!

keejkrej commented May 19, 2026

Uh oh!

ElpadoCan commented May 19, 2026

Uh oh!

ElpadoCan commented May 19, 2026

Uh oh!

ElpadoCan commented May 19, 2026

Uh oh!

Teranis commented May 19, 2026

Uh oh!

ElpadoCan commented May 19, 2026

Uh oh!

ElpadoCan commented May 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants