ultrack¶
ultrack
is general purpose 2D/3D cell tracking software.
It can track from segmentation masks or from raw images directly, specially fluorescence microscopy images.
Four interfaces are provided, depending on your needs:
napari plugin
FIJI plugin
Python API
Command line interface for batch processing, including distributed computing.
See below for additional details in each interface.
Moreover, it was originally developed to terabyte-scale zebrafish embryo images where we had few 3D annotations.
Hence, a few key features of ultrack
are:
Out-of-memory storage of intermediate results. You should not run out of memory even for large datasets. We have tracked a 3TB dataset on a laptop with 64GB of RAM.
It does not commit to a single segmentation. Instead it considers multiple segmentations per cell and it picks the best segmentation for each cell at each time point while tracking.
Zebrafish imaged using DaXi whole embryo tracking.
Quickstart¶
This quickstart guide is recommended for users who are already familiar with Python and image analysis. Otherwise, we recommend you read the Installation and Getting started sections.
Installation¶
If already have a working Python environment, you can install ultrack
using pip.
We recommend you use a conda environment to avoid any conflicts with your existing packages.
If you’re using OSX or for additional information on how to create a conda environment and install packages, see Installation.
pip install ultrack
Basic usage¶
The following example demonstrates how to use ultrack
to track cells using its canonical input, a binary image of the foreground and a cells’ contours image.
import napari
from ultrack import MainConfig, Tracker
# import to avoid multi-processing issues
if __name__ == "__main__":
# Load your data
foreground = ...
contours = ...
# Create a config
config = MainConfig()
# Run the tracking
tracker = Tracker(config=config)
tracker.track(foreground=foreground, contours=contours)
# Visualize the results
tracks, graph = tracker.to_tracks_layer()
napari.view_tracks(tracks[["track_id", "t", "y", "x"]], graph=graph)
napari.run()
If you already have segmentation labels, you can provide them directly to the tracker.
import napari
from ultrack import MainConfig, Tracker
# import to avoid multi-processing issues
if __name__ == "__main__":
# Load your data
labels = ...
# Create a config
config = MainConfig()
# this removes irrelevant segments from the image
# see the configuration section for more details
config.segmentation_config.min_frontier = 0.5
# Run the tracking
tracker = Tracker(config=config)
tracker.track(labels=labels)
# Visualize the results
tracks, graph = tracker.to_tracks_layer()
napari.view_tracks(tracks[["track_id", "t", "y", "x"]], graph=graph)
napari.run()
Installation¶
The easiest way to install the package is to use the conda (or mamba) package manager. If you do not have conda installed, we recommend to install mamba first, which is a faster alternative to conda. You can find mamba installation instructions here.
Once you have conda (mamba) installed, you should create an environment for ultrack
as follows:
conda create -n ultrack python=3.11 higra gurobi pytorch pyqt -c pytorch -c gurobi -c conda-forge
Then, you can activate the environment and install ultrack
:
conda activate ultrack
pip install ultrack
You can check if the installation was successful by running:
ultrack --help
GPU acceleration¶
Ultrack makes use of GPU for image processing operations. You can install the additional packages required for GPU acceleration by running (Linux and Windows only):
conda install pytorch-cuda -c pytorch -c nvidia
conda install cupy -c conda-forge
# linux only
conda install cucim -c rapidsai
# for windows, you can install cucim using pip
pip install git+https://github.com/rapidsai/cucim.git#egg=cucim&subdirectory=python/cucim"
See the PyTorch website for more information on how to install PyTorch with GPU support.
Gurobi setup¶
Gurobi is a commercial optimization solver that is used in the tracking module of ultrack
.
While it is not a requirement, it is recommended to install it for the best performance.
To use it, you need to obtain a license (free for academics) and activate it.
Install gurobi using conda¶
You can skip this step if you have already installed Gurobi.
In your existing Conda environment, install Gurobi with the following command:
conda install -c gurobi gurobi
Obtain and activate an academic license¶
Obtaining a license: register for an account using your academic email at Gurobi’s website. Navigate to the Gurobi’s named academic license page, and follow the instructions to get your academic license key.
Activating license: In your Conda environment, run:
grbgetkey YOUR_LICENSE_KEY
Replace YOUR_LICENSE_KEY with the key you received. Follow the prompts to complete activation.
Test the installation¶
Verify Gurobi’s installation by running:
ultrack check_gurobi
Troubleshooting¶
Depending on the operating system, the gurobi library might be missing and you need to install it from here.
If you’re still having trouble, with the installation we recommend reaching out to us or using the docker image, see Docker instructions.
Napari plugin¶
We wrapped up most of the functionality in a napari widget. The widget is already installed by default, but you must have napari installed to use it.
To use it, open napari and select the widget from the plugins menu selecting ultrack
and then Ultrack
from the dropdown menu.
The plugin is built around the concept of a tracking workflow. Any workflow is a sequence of pre-processing steps, segmentation, (candidate segments) linking, and the tracking problem solver. We explain the different workflows in the following sections.
Workflows¶
The difference between the workflows is the way the user provides the information to the plugin,
and the way it processes the information. The remaining steps are the same for all workflows.
In that sense, segmentation
, linking
, and solver
are the same for all workflows.
For each step, the widget provides direct access to the parameters of the step, and the user can
change the parameters to adapt the workflow to the specific problem. We explain how these
parameters behave in Configuration docs, and, more specifically, in the
Experiment
,
Linking
, and
Tracking
sections. Every input requested by the plugin
should be loaded beforehand as a layer in Napari
.
There are three workflows available in the plugin:
Automatic tracking from image: This workflow is designed to track cells in a sequence of images. It uses classical image processing techniques to detect the cells (foreground) and their possible contours. In this workflow, you can change the parameters of the image processing steps. Refer to the documentation of the functions used in the image processing steps:
- Manual tracking: Since ultrack is designed to work with precomputed cell detection and
contour detection, this workflow is designed for the situation where the user has already computed the cell detection and the contours of the cells. In this situation, no additional parameter is needed, you only need to provide the cell detection and the contours of the cells.
- Automatic tracking from segmentation labels: This workflow is designed to track cells
in a sequence of images where the user has already computed the segmentation of the cells. This workflow wraps the function
ultrack.utils.labels_to_contours()
to compute the foreground and contours of the cells from the segmentation labels, refer to its documentation for additional details.
Flow Field Estimation¶
Every workflow allows the use of a flow field to improve the tracking of dynamic cells.
This method estimates the movement of the cells in the sequence
of images through the function ultrack.imgproc.flow.timelapse_flow()
.
See the documentation of this function for additional details.
FIJI plugin¶
Ultrack is also available as a FIJI plugin.
Its usage and installation instructions are in FIJI’s ultrack documentation.
Getting started¶
Ultrack tracking pipeline is divided into three main steps:
segment
: Creating the candidate segmentation hypotheses;link
: Finding candidate links between segmentation hypotheses of adjacent frames;solve
: Solving the tracking problem by finding the best segmentation and trajectory for each cell.
These three steps have their respective function with the same names and configurations but are summarized in the track
function or the Tracker
class, which are the main entry point for the tracking pipeline.
You’ll notice that these functions do not return any results. Instead, they store the results in a database. This enables us to process datasets larger than memory, and distributed or parallel computing. We provide auxiliary functions to export the results to a format of your choice.
The MainConfig.data_config
provides the interface to interact with the database, so beware of using overwrite
parameter when re-executing these functions, to erase previous results otherwise it will build on top of existing data.
If you want to go deep into the weeds of our backend. We recommend looking at the ultrack.core.database.py
file.
Each one of the main steps will be explained in detail below, a detailed description of the parameters can be found in Configuration.
Segmentation¶
Ultrack’s canonical inputs are a foreground
and a contours
, there are several ways to obtain these inputs, which will be explained below. For now, let’s consider we are working with them directly.
Both foreground
and contours
maps must have the same shape, with the first dimension being time (T
) and the remaining being the spatial dimensions (Z
optional, and Y
, X
).
foreground
is used with config.segmentation_config.threshold
to create a binary mask indicating the presence of the object of interest, it’s by default 0.5. Values above the threshold are considered as foreground, and values below are considered as background.
contours
indicates the confidence of each pixel (voxel) being a cell boundary (contour). The higher the value, the more likely it is a cell boundary. It couples with config.segmentation_config.min_frontier
which fuses segmentation candidates separated by a boundary with an average value below this threshold, it’s by default 0, so no fusion is performed.
The segmentation is the most important step, as it will define the candidates for the tracking problem.
If your cells of interest are not present in the foreground
after the threshold, you won’t be able to track them.
If there isn’t any faint boundary in the contours
, you won’t be able to split into individual cells. That’s why it’s preferred to have a lot of contours (more hypotheses), even incorrect ones than having none.
Linking¶
The linking step is responsible for finding candidate links between segmentation hypotheses of adjacent frames. Usually, not a lot of candidates are needed (config.linking_config.max_neighbors = 5
), unless you have several segmentation hypotheses (contours
with several gray levels).
The parameter config.linking_config.max_distance
must be at least the maximum distance between two cells in consecutive frames. It’s used to filter out candidates that are too far from each other. If this value is too small, you won’t be able to link cells that are far from each other.
Solving¶
The solving step is responsible for solving the tracking problem by finding the best segmentation and trajectory for each cell. The parameters for this step are harder to interpret, as they are related to the optimization problem. The most important ones are:
config.tracking_config.appear_weight
: The penalization for a cell to appear, which means to start a new lineage;config.tracking_config.division_weight
: The penalization for a cell to divide, breaking a single tracklet into two;config.tracking_config.disappear_weight
: The penalization for a cell to disappear, which means to end a lineage;
These weights are negative or zero, as they try to balance the cost of including new lineages in the final solution. The connections (links) between segmentation hypotheses are positive and measure the quality of the tracks, so only lineages with a total linking weight higher than the penalizations are included in the final solution. At the same time, our optimization problem is finding the combination of connections that maximize the sum of weights of all lineages.
See the tracking configuration description for more information and Tuning tracking performance for details on how to select these parameters.
Exporting¶
Once the above steps have been applied, the tracking solutions are recorded in the database and they can be exported to a format of your choice, them being, to_networkx
, to_trackmate
, to_tracks_layer
, tracks_to_zarr
and others.
See the export API reference for all available options and their parameters.
Example of exporting solutions to napari tracks layer:
# ... tracking computation
# Exporting to napari format using `Tracker` class
tracks, graph = tracker.to_tracks_layer()
# Exporting using config file
tracks, graph = to_tracks_layer(config)
Post-processing¶
We also provide some additional post-processing functions, to remove, join, or analyze your tracks. Most of them are available in ultrack.tracks
. Some examples are:
close_tracks_gaps
: That closes gaps by joining tracklets and interpolating the missing segments;filter_short_sibling_tracks
: That removes short tracklets generated by false divisions;get_subgraph
: Which returns the whole lineage(s) of a given tracklet.
Other functionalities can be found in ultrack.utils
or ultrack.imgproc
, one notable example is:
tracks_properties
: Which returns compute statistics from the tracks, segmentation masks and images.
For additional information, please refer to the tracks post-processing API reference.
Image processing¶
Despite being presented here last, ultrack’s image processing module provides auxiliary functions to process your image before the segmentation step. It’s not mandatory to use it, but it might reduce the amount of code you need to write to preprocess your images.
Most of them are available in ultrack.imgproc
, ultrack.utils.array
and ultrack.utils.cuda
modules.
Refer to the image processing API reference for more information.
Ultrack Docker Image¶
This example shows how to use the Ultrack Docker image. The Docker image is a pre-configured environment that includes the Ultrack software and all its dependencies. The images are available on Docker Hub at royerlab/ultrack repository. You can also build the image from scratch (see the Running the Docker Image section).
In the following sections, we will discuss:
Running the Docker Image for a basic introduction to the Ultrack software inside the Docker container.
Gurobi support for using Gurobi inside the Ultrack Docker image.
GPU support for using GPU acceleration inside the Ultrack Docker image.
Running the Docker Image with Jupyter Notebook for running a Jupyter Notebook server inside the Docker container.
(Optional) Building the Docker Image for building the Ultrack Docker image from scratch.
Running the Docker Image¶
To run the Ultrack Docker image, you can use the following command:
docker run -it --rm -v /path/to/your/data:/data royerlab/ultrack:0.6.1-cpu # replace 6.0.1-cpu with the desired version and
# variant (e.g., 6.0.1-cpu, 6.0.1-cuda11.8)
This command will start the Ultrack Docker image and mount the /path/to/your/data directory to the /data directory inside the container. Replace /path/to/your/data with the path to your data directory. Then, you can use the Ultrack CLI for your analysis.
Gurobi support¶
Gurobi is a commercial optimization solver used in Ultrack for solving optimization problems. To use Gurobi inside the Ultrack Docker image, you need a valid Web License Service (WLS) license. Once you have the license file (gurobi.lic), mount it to the /opt/gurobi/gurobi.lic directory inside the container. For example:
docker run -it --rm -v /path/to/your/data:/data \
-v /path/to/your/gurobi.lic:/opt/gurobi/gurobi.lic \
royerlab/ultrack:0.6.1-cpu
Check the Gurobi support by running:
ultrack check_gurobi
This should display the Gurobi license information if the license file is valid.
GPU support¶
The Ultrack Docker image supports GPU acceleration using CUDA. To use GPU support, you need a compatible NVIDIA GPU and the NVIDIA driver installed on your host machine. Include the –gpus all flag in the docker run command. For example:
docker run -it --rm -v /path/to/your/data:/data --gpus all \
royerlab/ultrack:0.6.1-cuda11.8 # replace 6.0.1-cuda11.8 with the
# desired version and variant
Running the Docker Image with Jupyter Notebook¶
You can run the Ultrack Docker image with a Jupyter Notebook server by forwarding the default Jupyter port (8888) to the host machine using -p 8888:8888. If the port is in use, use a different port (e.g., -p 8889:8888):
docker run -it --rm -v /path/to/your/data:/data -p 8888:8888 royerlab/ultrack:0.6.1-cpu
After starting the container, install the Jupyter Notebook server:
uv pip install --no-cache --system jupyterlab # or simply `pip install jupyterlab`
Start the Jupyter Notebook server:
jupyter lab --ip=0.0.0.0 --allow-root # ip=0.0.0.0 allows access from any IP address
# allow-root allows running Jupyter as root (default user in container)
You will see a URL to access the Jupyter Notebook server in your browser. Use the Ultrack API to analyze your data.
(Optional) Building the Docker Image¶
To build the Ultrack Docker image from scratch, use the following command:
python3 docker/build_containers.py
This command requires an argument specifying the desired version and variant of the Ultrack Docker image. If not provided, the script lists all available versions and variants. For example, to build an image with basic CPU support:
python3 docker/build_containers.py cpu
You can then use the built image as described in the Running the Docker Image section.
Examples¶
Here we provide some examples of how to use Ultrack for cell tracking.
Some examples are provided as Jupyter notebooks with additional documentation, but we do not recommend using Jupyter notebooks for your day-to-day analysis.
Other examples as Python scripts can be found in here.
Additional packages might be required. Therefore, conda environment files are provided, which can be installed using:
conda env create -f <environment-file.yml>
conda activate <your new env>
pip install git+https://github.com/royerlab/ultrack
The existing examples are:
multi_color_ensemble : Multi-colored cytoplasm cell tracking using Cellpose and Watershed segmentation ensemble. Data provided by The Lammerding Lab.
flow_field_3d : Tracking demo on a cartographic projection of Tribolium Castaneum embryo from the cell-tracking challenge, using a flow field estimation to assist tracking of motile cells.
stardist_2d : Tracking demo on HeLa GPF nuclei from the cell-tracking challenge using Stardist 2D fluorescence images pre-trained model.
zebrahub : Tracking demo on zebrafish tail data from zebrahub acquired with DaXi using Ultrack’s image processing helper functions.
neuromast_plantseg : Tracking demo membrane-labeled zebrafish neuromast from Jacobo Group of CZ Biohub using PlantSeg’s membrane detection model.
micro_sam : Tracking demo with MicroSAM instance segmentation package.
Tuning tracking performance¶
Once you have a working ultrack pipeline, the next step is optimizing the tracking performance. Here we describe our guidelines for optimizing the tracking performance and up to what point you can expect to improve the tracking performance.
It will be divided into a few sections:
Pre-processing: How to make tracking easier by pre-processing the data;
Input verification: Guidelines to check if you have good labels or foreground and contours maps;
Hard constraints: Parameters must be adjusted so the hypotheses include the correct solution;
Tracking tuning: Guidelines to adjust the weights to make the correct solution more likely.
Pre-processing¶
Registration¶
Before tracking, the first question to ask yourself is, are your frames correctly aligned?
If not, we recommend aligning them. To do that, we provide the ultrack.imgproc.register_timelapse
to align translations, see the registration API.
If the movement is more complex, with cells moving in different directions, we recommend using the flow
functionalities to align individual segments with distinct transforms, see the flow tutorial.
See the flow estimation API for more information.
Deep learning¶
Some deep learning models are sensitive to the contrast of your data, we recommend adjusting the contrast and removing background before applying them to improve their predictions. See the image processing utilities API for more information.
Input verification¶
At this point, we assume you already have a labels
image or a foreground
and contours
maps;
You should check if labels
or foreground
contains every cell you want to track.
Any region that is not included in the labels
or foreground
will not be tracked and can only be fixed with post-processing.
If you are using foreground
and contours
maps, you should check if the contours induce hierarchies that lead to your desired segmentation.
This can be done by loading the contours
in napari and viewing them over your original image with blending='additive'
.
You want your contours
image to have higher values in the boundary of cells and lower values inside it.
This indicates that these regions are more likely to be boundaries than the interior of cells.
Notice, that this notion is much more flexible than a real contour map, which is we can use an intensity image as a contours map or an inverted distance transform.
In cells where this is not the case it is less likely ultrack will be able to separate them into individual segments.
If your cells (nuclei) are convex it is worth trying the ultrack.imgproc.inverted_edt
for the contours
.
If even after going through the next steps you don’t have successful results, I suggest looking for specialized solutions once you have a working pipeline. Some of these solutions are PlantSeg for membranes or GoNuclear for nuclei.
Hard constraints¶
This section is about adjusting the parameters so we have hypotheses that include the correct solution.
Please refer to the Configuration docs as we refer to different parameters.
1. The expected cell size should be between segmentation_config.min_area
and segmentation_config.max_area
.
Having a tight range assists in finding a good segmentation and significantly reduces the computation.
Our rule of thumb is to set the min_area
to half the size of the expected cell or the smallest cell, disregarding outliers.
And the max_area
to 1.25~1.5 the size of the largest cell, this is less problematic than the min_area
.
2. linking_config.max_distance
should be set to the maximum distance a cell can move between frames.
We recommend setting some tolerance, for example, 1.5 times the expected movement.
Tracking tuning¶
Once you have gone through the previous steps, you should have a working pipeline and now we can focus on the results and what can be done in each scenario.
- My cells are oversegmented (excessive splitting of cells):
Increase the
segmentation_config.min_area
to merge smaller cells;Increase the
segmentation_config.max_area
to avoid splitting larger cells;If you have clear boundaries and the oversegmentation are around weak boundaries, you can increase the
segmentation_config.min_frontier
to merge them (steps of 0.05 recommended).If you’re using
labels
as input or to create my contours you can also try to increase thesigma
parameter to create a better surface to segmentation by avoiding flat regions (full of zeros or ones).
- My cells are undersegmented (cells are fused):
Decrease the
segmentation_config.min_area
to enable segmenting smaller cells;Decrease the
segmentation_config.max_area
to remove larger segments that are likely to be fused cells;Decrease the
segmentation_config.min_frontier
to avoid merging cells that have weak boundaries;EXPERIMENTAL: Set
segmentation_config.max_noise
to a value greater than 0, to create more diverse hierarchies, the scale of this value should be proportional to thecontours
value, for example, if thecontours
is in the range of 0-1, themax_noise
around 0-0.05 should be enough. Play with it. NOTE: the solve step will take longer because of the increased number of hypotheses.
- I have missing segments that are present on the
labels
orforeground
: Check if these cells are above the
segmentation_config.threshold
value, if not, decrease it;Check if
linking_config.max_distance
is too low and increase it, when cells don’t have connections they are unlikely to be included in the solutions;Your
tracking_config.appear_weight
,tracking_config.disappear_weight
&tracking_config.division_weight
penalization weights are too high (too negative), try bringing them closer to 0.0. TIP: We recommend adjustingdisappear_weight
weight first, because when tuningappear_weight
you should balance outdivision_weight
so appearing cells don’t become fake divisions. A rule of thumb is to keepdivision_weight
equal or higher (more negative) thanappear_weight
.
- I have missing segments that are present on the
- I’m not detecting enough dividing cells:
Bring
tracking_config.division_weight
to a value closer to 0.Depending on your time resolution and your cell type, it might be the case where dividing cells move further apart, in this case, you should tune the
linking_config.max_distance
accordingly.
- I’m detecting too many dividing cells:
Make
tracking_config.division_weight
more negative.
- My tracks are short and not continuous enough:
This is tricky, once you have tried the previous steps, you can try making the
tracking_config.{appear, division, disappear}_weight
more negative, but this will remove low-quality tracks.Another option is to use
ultrack.tracks.close_tracks_gaps
to post process the tracks.
- I have many incorrect tracks connecting distant cells:
Decrease the
linking_config.max_distance
to avoid connecting distant cells. If that can’t be done because you will lose correct connections, then you should setlinking_config.distance_weight
to a value closer higher than 0, usually in very small steps (0.01).
Configuration¶
The configuration is at the heart of ultrack, it is used to define the parameters for each step of the pipeline and where to store the intermediate results. The MainConfig is the main configuration that contains the other configurations of the individual steps plus the data configuration.
The configurations are documented below, the parameters are ordered by importance, most important parameters are at the top of the list. Parameters that should not be changed in most of the cases are at the bottom of the list and contain a SPECIAL
tag.
Configuration for intermediate data storage and retrieval. |
|
Segmentation hypotheses creation configuration |
|
Candidate cell hypotheses linking configuration |
|
Tracking (segmentation & linking selection) configuration |
- class ultrack.config.MainConfig¶
- field data_config [Optional] (alias 'data')¶
Configuration for intermediate data storage and retrieval.
- field segmentation_config [Optional] (alias 'segmentation')¶
Segmentation hypotheses creation configuration
- field linking_config [Optional] (alias 'linking')¶
Candidate cell hypotheses linking configuration
- field tracking_config [Optional] (alias 'tracking')¶
Tracking (segmentation & linking selection) configuration
- class ultrack.config.DataConfig¶
Configuration for intermediate data storage and retrieval.
- field n_workers = 1¶
Number of workers for parallel processing
- field working_dir = PosixPath('.')¶
Working directory for auxiliary files (e.g. sqlite database, metadata)
- field database = 'sqlite'¶
SPECIAL
: Database typesqlite
andpostgresql
supported
- field address = None¶
SPECIAL
: Postgresql database path, for example,postgres@localhost:12345/example
- property database_path¶
Returns database path given working directory and database type.
- metadata_add(data)¶
Adds data content to metadata file.
- property metadata¶
Returns metadata as dictionary.
- dict(*args, **kwargs)¶
Generate a dictionary representation of the model, optionally specifying which fields to include or exclude.
- class ultrack.config.SegmentationConfig¶
Segmentation hypotheses creation configuration
- field min_area = 100¶
Minimum segment number of pixels, regions smaller than this value are merged or removed when there is no neighboring region
- field max_area = 1000000¶
Maximum segment’s number of pixels, regions larger than this value are removed
- field n_workers = 1¶
Number of worker threads
- field min_frontier = 0.0¶
Minimum average frontier value between candidate segmentations, regions sharing an average frontier value lower than this are merged
- field threshold = 0.5¶
Threshold used to binarize the cell foreground map
- field max_noise = 0.0¶
SPECIAL
: Upper limit of uniform distribution for additive noise on contour map
- field ws_hierarchy = <function watershed_hierarchy_by_area>¶
SPECIAL
: Watershed hierarchy function from higra used to construct the hierarchy
- field anisotropy_penalization = 0.0¶
SPECIAL
: Image graph z-axis penalization, positive values will prioritize segmenting the xy-plane first, negative will do the opposite
- dict(*args, **kwargs)¶
Generate a dictionary representation of the model, optionally specifying which fields to include or exclude.
- class ultrack.config.LinkingConfig¶
Candidate cell hypotheses linking configuration
- field max_distance = 15.0¶
Maximum distance between neighboring segments
- field n_workers = 1¶
Number of worker threads
- field max_neighbors = 5¶
Maximum number of neighbors per candidate segment
- field distance_weight = 0.0¶
Penalization weight \(\gamma\) for distance between segment centroids, \(w_{pq} - \gamma \|c_p - c_q\|_2\), where \(c_p\) is region \(p\) center of mass
- field z_score_threshold = 5.0¶
SPECIAL
: z-score threshold between intensity values from within the segmentation masks of neighboring segments
- class ultrack.config.TrackingConfig¶
Tracking (segmentation & linking selection) configuration
- field appear_weight = -0.001¶
Penalization weight for appearing cell, should be negative
- field disappear_weight = -0.001¶
Penalization for disappearing cell, should be negative
- field division_weight = -0.001¶
Penalization for dividing cell, should be negative
- field n_threads = -1¶
Number of worker threads
- field window_size = None¶
Time window size for partially solving the tracking ILP. By default it solves the entire timelapse at once. Useful for large datasets.
- field overlap_size = 1¶
Number of frames used to shared (overlap/pad) each size when
window_size
is set. This improves the tracking quality at the edges of the windows and enforce continuity of the tracks.
- field solution_gap = 0.001¶
Solver solution gap. This will speed up the solver when finding the optimal solution might taken a long time, but may affect the quality of the solution.
- field time_limit = 36000¶
Solver execution time limit in seconds
- field link_function = 'power'¶
SPECIAL
: Function used to transform the edge and node weights, identity or power
- field power = 4¶
SPECIAL
: Expoent \(\eta\) of power transform, \(w_{pq}^\eta\)
- field bias = -0.0¶
SPECIAL
: Edge weights bias \(b\), \(w_{pq} + b\), should be negative
FAQ¶
- Q: What is each configuration parameters for?
A: See the configuration page.
- Q: What to do when Qt platform plugin cannot be initialized?
A: The solution to try is to install
pyqt
usingconda
from the-c conda-forge
channel.- Q: Why my python script gets stuck when using ultrack?
A: You need to wrap your code in a
if __name__ == '__main__':
block to avoid themultiprocessing
module to run the same code in each process. For example:import ultrack def main(): # Your code here ... if __name__ == '__main__': main()
- Q: My results show strange segments with perfect lines boundaries. What is happening and how can I fix it?
A: This is a hierarchical watershed artifact. Regions with “flat” intensities create arbitrary partitions that are, most of the time, a straight line.
You have three options to fix this:
increase
min_area
parameter, so these regions get removed. However, if you have objects with varying areas, this might be challenging and lead to undersegmentation.- increase
min_frontier
; this is my preferred option when you have a high accuracy network as plants. This merges regions whose average intensity between them is below min_frontier. In this case, assuming your boundary is between 0 and 1,
min_frontier=0.1
or even0.05
should work. Be careful to not increase this too much because it could merge regions where cells are shared by a “weak” boundary.
- increase
- another option is to blur the boundary map so you avoid creating regions with “flat” intensities.
This follows the same reasoning for using EDT to run watersheds. This works better for convex objects. And remember to renormalize the intensity values if using this with
min_frontier
.
- Q: Ultrack is not working with my data. What can I do?
A: See the tracking optimizing page.
- Q: My data is isotropic. How can I take that into account?
A: Provide the the Z, Y, X scaling factors in the
scale
parameter oftrack
orlink
functions.- Q: How can I use Ultrack distributed over a cluster?
Theory¶
See our algorithm description in our computer vision paper.
Citing¶
If you use ultrack
in your research, please cite the following papers, the algorithm and the biological applications and software.
@article{bragantini2023ucmtracking,
title={Large-Scale Multi-Hypotheses Cell Tracking Using Ultrametric Contours Maps},
author={Jordão Bragantini and Merlin Lange and Loïc Royer},
year={2023},
eprint={2308.04526},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
@article{bragantini2024ultrack,
title={Ultrack: pushing the limits of cell tracking across biological scales},
author={Bragantini, Jordao and Theodoro, Ilan and Zhao, Xiang and Huijben, Teun APM and Hirata-Miyasaki, Eduardo and VijayKumar, Shruthi and Balasubramanian, Akilandeswari and Lao, Tiger and Agrawal, Richa and Xiao, Sheng and others},
journal={bioRxiv},
pages={2024--09},
year={2024},
publisher={Cold Spring Harbor Laboratory}
}
And the respective auxiliary methods (e.g. Cellpose, napari, etc) depending on your usage.
Additional resources¶
Documentation contents¶
- API
- Object Oriented API
- Core functionalities
- Image processing utilities
- Exporting
- Core functionalities
Tracker
Tracker.add_flow()
Tracker.add_links()
Tracker.add_nodes_prob()
Tracker.export_by_extension()
Tracker.get_nodes_features()
Tracker.link()
Tracker.segment()
Tracker.solve()
Tracker.to_ctc()
Tracker.to_networkx()
Tracker.to_pandas()
Tracker.to_trackmate()
Tracker.to_tracks_layer()
Tracker.to_zarr()
Tracker.track()
add_flow()
add_new_node()
export_tracks_by_extension()
link()
load_config()
segment()
solve()
to_ctc()
to_trackmate()
to_tracks_layer()
track()
tracks_to_zarr()
- Array utilities
- Image processing utilities
- Tracks utilities
add_track_ids_to_tracks_df()
close_tracks_gaps()
filter_short_sibling_tracks()
get_paths_to_roots()
get_subgraph()
inv_tracks_df_forest()
left_first_search()
sort_track_ids()
sort_trees_by_length()
sort_trees_by_max_radius()
split_tracks_df_by_lineage()
split_trees()
tracks_df_forest()
tracks_df_movement()
tracks_length()
tracks_profile_matrix()
- Exporting
- Core functionalities
- CLI
- REST API