Getting started¶

Ultrack tracking pipeline is divided into three main steps:

segment: Creating the candidate segmentation hypotheses;
link: Finding candidate links between segmentation hypotheses of adjacent frames;
solve: Solving the tracking problem by finding the best segmentation and trajectory for each cell.

These three steps have their respective function with the same names and configurations but are summarized in the track function or the Tracker class, which are the main entry point for the tracking pipeline.

You’ll notice that these functions do not return any results. Instead, they store the results in a database. This enables us to process datasets larger than memory, and distributed or parallel computing. We provide auxiliary functions to export the results to a format of your choice.

The MainConfig.data_config provides the interface to interact with the database, so beware of using overwrite parameter when re-executing these functions, to erase previous results otherwise it will build on top of existing data.

If you want to go deep into the weeds of our backend. We recommend looking at the ultrack.core.database.py file.

Each one of the main steps will be explained in detail below, a detailed description of the parameters can be found in Configuration.

Segmentation¶

Ultrack’s canonical inputs are a foreground and a contours, there are several ways to obtain these inputs, which will be explained below. For now, let’s consider we are working with them directly.

Both foreground and contours maps must have the same shape, with the first dimension being time (T) and the remaining being the spatial dimensions (Z optional, and Y, X).

foreground is used with config.segmentation_config.threshold to create a binary mask indicating the presence of the object of interest, it’s by default 0.5. Values above the threshold are considered as foreground, and values below are considered as background.

contours indicates the confidence of each pixel (voxel) being a cell boundary (contour). The higher the value, the more likely it is a cell boundary. It couples with config.segmentation_config.min_frontier which fuses segmentation candidates separated by a boundary with an average value below this threshold, it’s by default 0, so no fusion is performed.

The segmentation is the most important step, as it will define the candidates for the tracking problem. If your cells of interest are not present in the foreground after the threshold, you won’t be able to track them. If there isn’t any faint boundary in the contours, you won’t be able to split into individual cells. That’s why it’s preferred to have a lot of contours (more hypotheses), even incorrect ones than having none.

Linking¶

The linking step is responsible for finding candidate links between segmentation hypotheses of adjacent frames. Usually, not a lot of candidates are needed (config.linking_config.max_neighbors = 5), unless you have several segmentation hypotheses (contours with several gray levels).

The parameter config.linking_config.max_distance must be at least the maximum distance between two cells in consecutive frames. It’s used to filter out candidates that are too far from each other. If this value is too small, you won’t be able to link cells that are far from each other.

Solving¶

The solving step is responsible for solving the tracking problem by finding the best segmentation and trajectory for each cell. The parameters for this step are harder to interpret, as they are related to the optimization problem. The most important ones are:

config.tracking_config.appear_weight: The penalization for a cell to appear, which means to start a new lineage;
config.tracking_config.division_weight: The penalization for a cell to divide, breaking a single tracklet into two;
config.tracking_config.disappear_weight: The penalization for a cell to disappear, which means to end a lineage;

These weights are negative or zero, as they try to balance the cost of including new lineages in the final solution. The connections (links) between segmentation hypotheses are positive and measure the quality of the tracks, so only lineages with a total linking weight higher than the penalizations are included in the final solution. At the same time, our optimization problem is finding the combination of connections that maximize the sum of weights of all lineages.

See the tracking configuration description for more information and Tuning tracking performance for details on how to select these parameters.

Exporting¶

Once the above steps have been applied, the tracking solutions are recorded in the database and they can be exported to a format of your choice, them being, to_networkx, to_trackmate, to_tracks_layer, tracks_to_zarr and others.

See the export API reference for all available options and their parameters.

Example of exporting solutions to napari tracks layer:

# ... tracking computation

# Exporting to napari format using `Tracker` class
tracks, graph = tracker.to_tracks_layer()

# Exporting using config file
tracks, graph = to_tracks_layer(config)

Post-processing¶

We also provide some additional post-processing functions, to remove, join, or analyze your tracks. Most of them are available in ultrack.tracks. Some examples are:

close_tracks_gaps: That closes gaps by joining tracklets and interpolating the missing segments;
filter_short_sibling_tracks: That removes short tracklets generated by false divisions;
get_subgraph: Which returns the whole lineage(s) of a given tracklet.

Other functionalities can be found in ultrack.utils or ultrack.imgproc, one notable example is:

tracks_properties: Which returns compute statistics from the tracks, segmentation masks and images.

For additional information, please refer to the tracks post-processing API reference.

Image processing¶

Despite being presented here last, ultrack’s image processing module provides auxiliary functions to process your image before the segmentation step. It’s not mandatory to use it, but it might reduce the amount of code you need to write to preprocess your images.

Most of them are available in ultrack.imgproc , ultrack.utils.array and ultrack.utils.cuda modules.

Refer to the image processing API reference for more information.