Python

OpticalFlow3D Python Implementation

Quick Start

To get started using OpticalFlow3D, you first need a Python environment with the required dependencies, listed below. Once your Python environment is set up, a good starting point is the notebook example_processing_script.ipynb, which provides examples of calculating optical flow. If you are uncertain how to get started with a Jupyter Notebook, a code editor (e.g., VS Code) will allow for interactive use of the notebook file. If you prefer, you may alternatively start a Jupyter interface from the command line using the command jupyter notebook and navigating to the correct file in the browser window that opens.

These same instructions can be used to explore the example_analysis_script.ipynb notebook, which provides example of analyzing the calculated flow fields.

Dependencies

The calculation and visualization tools described below require the following Python packages:

• colorcet
• jupyter
• matplotlib
• natsort
• numpy
• pandas
• pathlib
• scipy
• tifffile

A Conda environment file for Windows systems is provided in opticalflow3D.yml. If you do not already have an entry point to Conda commands, Miniforge is a free tool that you can use to get started. Learn more about Miniforge and Conda in this FocalPlane blog post by Robert Haase. Once you have Conda set up, you can create an environment for OpticalFlow3D using the following command:

conda env create -f opticalflow3D.yml

Usage

Optical Flow Calculation

The actual calculation of optical flow is performed by the functions calc_flow3D and calc_flow2D, for 3D and 2D image sequences respectively. These functions are contained within the calc_flow.py module. Inputs are an image sequence and three smoothing parameters. The only differences in usage between the two functions are (1) whether a 4D or 3D matrix of images is input, and (2) whether flow in the axial direction, $v_z$, is output.

calc_flow2D(images,xySig=3,tSig=1,wSig=4)

calc_flow3D(images,xyzSig=3,tSig=1,wSig=4)

Input images should be of the format $N_t \times N_z \times N_y \times N_x$, with the $N_z$ dimension excluded for 2D processing.

xyzSig is the standard deviation used for Gaussian spatial smoothing and has a default value of 3. Larger values will remove noise but also remove spatial detail.

tSig is the standard deviation used for Gaussian temporal smoothing and has a default value of 1. Larger values remove noise but also remove temporal detail.

wSig is the standard deviation used for Gaussian weighting of the Lucas-Kanade neighborhood. Larger values create a larger neighborhood for the linear algebra solution, and will smooth over small spatial features in the flow.

Image Parsing

The function process_flow (also contained in calc_flow.py) organizes image sequences into the format required by calc_flow3D and calc_flow2D. If you have a non-tif file type, this function must be adapted to your images.

In addition to the three smoothing parameters described in Calculation, this function requires details about the images to be processed.

process_flow(imDir,imName,fileType="SequenceT",spatialDimensions=3,xyzSig=3,tSig=1,wSig=4)

imDir is the full path to folder of image(s) to process.

imName is the name of the image(s) to process. For a single multi-page tif, this is simply the name of the tif file. For an image series composed of multiple tif files, the imName must be specified with a wildcard (.*) for the time expression in the file names. For example, for a list of files with the names:

• myexperiment_t000_ch0.tif
• myexperiment_t001_ch0.tif
• myexperiment_t002_ch0.tif,

the correct ImName would be 'myexperiment_t.*_ch0'.

fileType can be either 'OneTif' or 'SequenceT'. For the 'OneTif' type, the entire timelapse and all z-slices are assumed to be saved in one single tif file, and it is assumed the metadata is in the format generated by ImageJ/FIJI. For the 'SequenceT' type, the images are assumed to be saved as a sequence, with one tif per time point (but all z-slices per time point saved in one tif file).

spatialDimensions can be either 2 or 3 based on the number of spatial dimensions of the input data.

Optical Flow Outputs

The function process_flow excludes the first and last $3 \times tSig$ time points from analysis to avoid edge effects.

For each analyzed time point, the function saves four tif files (three for 2D processing). These files correspond to the intensity changes in each directory ($v_x$, $v_y$, $v_z$), and reliability values (rel) for that time point. Each tif is saved with float precision.

Example Notebooks

The example_processing_script.ipynb file provides an example of using process flow to analyze a set of images. Example 1 shows how to analyze a series of files (i.e., the 'SequenceT' type), and Example 2 shows how to process multiple folders of 'OneTif' format files at once.

In example_analysis_script.ipynb, the output of the processing notebook is visualized and analyzed. This analysis requires the image directory and the name of the dataset, as well as metadata about the images. Analysis parameters include a reliability threshold value and parameters to control the visualization. The outputs of this notebook include visualization of the reliability mask, a quiver visualization of theta, distributions of magnitude and theta, and examples of running these analyses over time.