Automatic tracking

← Back to main page

The intended workflow is as follows:

  1. Obtain nucleus positions.

  2. Obtain division likelihoods.

  3. Obtain link likelihoods.

  4. Create tracks (i.e., possible lineages made of probable links)

  5. Calculate error probabilities.

  6. Manually correct warnings to improve tracks.

  7. OR Do automated analysis only with best tracks.

Before you start, create a directory that will hold all the results of your analysis (e.g., automated_tracking inside the directory that holds your images).

Step 1: Obtain nucleus positions

Open the OrganoidTracker GUI and load your images (File -> Load images...). For the best results, the input data should somewhat resemble the training data. This can often be achieved with a little pre-processing (See here for tips).

Select Tools -> Detect cells in images.... When prompted by the dialog box, select the neural network trained for detecting nuclear positions. (See here for how to train a model or find links to pre-trained models at our Github page.) A second dialog box prompts you to create a folder for storing the detected positions by typing the new folder’s name (e.g., positions within automated_tracking).

You now have a folder (e.g. positions) containing configuration files and scripts with the name organoid_tracker_predict_positions. To run the script, double-click the BAT file (Windows) or the SH file (Linux).

After running the script, you have a folder called Automatic positions that contains a .aut file with a name like the input image you are analyzing. You can load this .aut file (File->Load tracking data...) alongside the microscopy image to assess the quality of the predictions.

If you want to debug this step, open the organoid_tracker.ini file and give the predictions_output_folder setting a value, for example test. If you run the script again, it will create a folder with that name (test in this example) and place images there indicating how likely it is to find a cell there.

Correcting for changed image offsets

When taking longer time lapse movies, the organoid can slowly slide out of the view. For this reason, the microscope user can move the imaged area so that the organoid stays in the view. In the time lapse movie, this makes the organoid appear to “teleport”: from the edges of the image it jumps back to the center.

To create links over this “jump”, the program would need to create a lot of large-distance links. The program will likely refuse to do this. To correct for this, before running the linking process, open the OrganoidTracker GUI and load the images and positions (File menu). Then edit the data (Edit -> Manually change data...) and edit the image offsets (Edit -> Edit image offsets...) of the correct time point. Instructions are on the bottom of the window.

Step 2: Obtain division likelihoods

With both the images and nuclear positions loaded into the OrganoidTracker GUI, select Tools -> Detect dividing cells.... When prompted by the dialog box, select the neural network trained for detecting cell divisions. You will also be prompted to create a folder for storing the detected divisions by typing its name (e.g., divisions within automated_tracking). As with step 1, this will generate various files and folders. Run the organoid_tracker_predict_divisions script.

After the script is finished, you have a folder called Division predictions containing a new .aut file also named after the microscopy image. Use File->Load tracking data... to visualize the results alongside the original microscopy image.

If you double-click on a cell in the main window, you can view its data, which now includes the division probability. Verify that this probability is what you would expect: high for cells that are about to divide, low otherwise.

Step 4: Create tracks

With both the images and links loaded into the OrganoidTracker GUI, select Tools -> Create tracks.... You will also be prompted to create a folder for storing the tracks by typing its name (e.g., tracks within automated_tracking). As with step 1, this will generate various files and folders. Run the organoid_tracker_create_tracks script.

This will generate a folder called Output tracks, which contains a folder with the same name as the original microscopy image. Within that folder there are four .aut files. As indicated by the name, these contain either All possible links (useful for calculating error rates) or a most probable subset (Final links). These exist as two versions, clean or raw.

Now you can also set some parameters to clean up the data after tracking. It is advisable to at least remove very short tracks for clarity.

Troubleshooting

You can modify this step of the analysis by changing the following values in the configuration file (e.g., organoid_tracker.ini within tracks).

  • maximum z depth for which we want tracks: 22 (pixels, default) Remove tracks that start or end above this pixel height, as “deeper” slices are noisier and harder to segment and therefore track. Consider decreasing this value to focus on higher quality segmentations when creating tracks.

  • margin_um: 8 (μm, default) Defines a margin at the image edge; positions within that margin have higher probabilities to disappear and appear. Consider increasing this margin if cells move a lot.

  • min appearance probability: 0.01 (default) False negative rate of cell detection. This value need not be exact to get good performance as it does not substantially affect results. Can be affected by cells “appearing” from outside the field of view, but the margin_um parameter (see above) already accounts for this by automatically increasing probabilites of appearance within a defined border region.

  • min disappearance probability: 0.01 (default) False negative rate of cell detection plus the death rate of cells. Consider adjusting to match the known death rate, but this value need not be exact to get good performance. Can be affected by cells “disappearing” to outside the field of view, but the margin_um parameter (see above) accounts for this by automatically increasing probabilites of disappearance within a defined border region.

Step 5: Calculate error rates

With both the images and Final links - clean.aut loaded into the OrganoidTracker GUI, select Tools -> Compute marginalized error rates.... When prompted, select the all possible links file (All possible links - clean.aut) and then create a folder for storing the output (e.g., error_rates). Run the resulting organoid_tracker_marginalization script.

This should be produce a file called Filtered positions.aut.

Troubleshooting

You can modify this step of the analysis by changing the following values in the configuration file (e.g., organoid_tracker.ini within error_rates).

  • maximum error rate: 0.01 (default) Links with marginal probability < (1- maximum error rate) are removed (e.g, links must exceed threshold probability of 0.99 for the default value of 0.01). Consider adjusting this value to make the threshold more or less stringent.

  • temperature (to account for shared information): 1.5 (default) Scales probabilities before calculating marginalized probabilites across all possible states; in this way, accounts for shared information between neural network predictions and can absorb miscalibrations of the neural network outputs. Nonetheless, needs to be calibrated to each model. If your data are sufficiently similar to the intestinal organoid data we trained our models on, you can use the default value. Otherwise, calibrate yourself.

Step 6: Manually correct warnings

This step is not automated. 😉

With both the images and Filtered positions.aut loaded into the OrganoidTracker GUI, go to Edit -> Manually change data.... A cross will appear over all locations where the program has detected some inconsistency in the tracking data:

Example of a warning

To view why OrganoidTracker warned you about this position, hover you mouse over a nucleus and press E. This will show you what the warning is. If the warning was a false positive, just press Delete to delete the warning. Otherwise, exit the warning screen (press E or Escape). This takes you back to the cell track editor, where you can fix the error.

To fix errors, you can edit the data using mainly the Insert and Delete keys, which are used to insert and delete links (respectively). Detected positions can also be inserted and deleted if necessary. Because correcting data works exactly the same as manual tracking, please see the manual tracking tutorial for more information.

If you press Left and Right in the warning screen, you will be taken to other warnings in the same lineage. If you press Up or Down, you will be taken to the warnings of other lineage trees.

If you press the L key from the cell track editor, a screen is opened that shows whether there are still warnings remaining in the lineage of that cell. If you hover your mouse over such a lineage and press E, the program will take you to the (first) warning in that lineage tree.

If you don’t want to correct everything, there are several options:

  • Limiting the number of time steps that you are looking at. In the warning screen, if you open the Edit menu, there’s an option to set the minimum and maximum time point for correction.

  • Limiting the area you are looking at. This is most easily done by pressing L in the cell track editor, and then observing any lineages that still have warnings. Correct any lineages that need to be corrected (press E while you hover your mouse over them). Once you are satisfied, you can then delete all lineages that still have errors in them. This is done from the cell track editor: press Edit > Batch deletion > Delete lineages with errors. See batch editing for more ways to delete a lot of lineages at once.

You can change a few settings of the error checker, to make it stricter or less strict. In the error checking screen (the screen you opened with E) there are three options available in the Edit menu: the minimimum marginalized probability a link should have, the minimum time in between two cell divisions of the same cell, and the maximum distance a cell may move per minute (only useful before marginalization).

Often correcting mistakes below a certain threshold also fixes a few mistakes that were not flagged. The increase in tracking quality might therefore be more than naively expected. To check the resulting quality you can rerun the marginalization on your corrected data while flagging in the organoid_tracker.ini file that you have checked part or all of the errors.

What to do if my results aren’t good?

That’s a difficult problem! You have a few options:

Step 7: Automated analysis

In step 5 a set of filtered high-confidence tracks is produced. You can choose to analyze these further. Generally for many application, manual corrections are not needed.

  • If you want to quantify a fluorescence reporter you can take long tracks (Edit -> Manually change data -> Batch deletion -> Delete short lineages) and work from there.

  • If you want to quantify tissue flows flawless trajectories are often overkill. Setting a lower probability threshold should still give you good enough data.

  • Analyzing cell cycle dynamics should be done using survival analysis (See here for how to do this) anyway and this can deal with cells that are lost to follow up. So it is not a problem to use the uncorrected filtered data.