Non-rigid stitching & the NRStitcher program

NRStitcher is a Python program for non-rigid stitching of terabyte-scale volume images. Algorithms that it implements are discussed in Arttu Miettinen, Ioannis Vogiatzis Oikonomidis, Anne Bonnin, Marco Stampanoni. NRStitcher: non-rigid stitching of terapixel-scale volumetric images, btz423, Bioinformatics, 2019.

Basically, NRStitcher is used to construct a large (volume) image from overlapping tiles. The program is able to account for small deformations in the tiles. NRStitcher uses pi2 system for image processing and runs on Slurm cluster or locally.

Acquisition of the tile images

In order to generate input data for the NRStitcher program, one typically uses some kind of a microscope to acquire a number of overlapping images. Large-ish overlaps are easier to stitch with the NRStitcher. For microtomographic images overlaps of 10 % - 30 % are typically used: the well-reconstructed region in the images is cylindrical, so approximately 30 % overlap between neighbouring cylinders is required in the radial direction. In the axial direction 10 % overlap seems to be fine in most cases.

The images must be saved in a format supported by the pi2 system, e.g. .raw, .tif, .nrrd, or sequence of .tif or .png slices.

The order of acquisition can be anything (snake, row-by-row, random, etc.) as long as approximate locations of the tiles are known.

Masking

Sometimes the tile images contain bad areas that must not be used in the stitching process. In order to facilitate this, pixel value 0 is handled specially in the stitching process. It is used to denote pixels with “no usable data” and can therefore be used to mask some regions of the tile image out of the final result. For example, in the case of tomographic images, pixel value 0 could be used to get rid of the badly reconstructed region near the edges of the image. In many cases this seems to be unnecessary, though.

In the case your valid pixel data contains zeroes, consider mapping the pixel values e.g. to range [1, max], where max denotes the maximal representable number in the pixel data type. After stitching, you may re-map back to the original range. The linmap command can be used for this purpose.

Stitching

In order to stitch the tiles, one must first create a new folder that will contain a stitch settings file, temporary files, and the final stitched image.

The stitch settings file must be generated according to the template. The name of the settings file must be stitch_settings.txt and it must be saved into the output folder. Most of the settings can be initially left to their default values, but the [positions] section must be modified so that it contains paths and file names of the tiles and their positions. The section contains one line for each tile. The line is formatted like image_name = x, y, z, where image_name gives the path and file name of the tile, and x, y, z is the location of that tile in pixels.

For example, positions of a four-tile image could be defined like this:

[positions]
wc1_150x150x1.raw = 0, 80, 0
wc2_150x150x1.raw = 110, 60, 0
wc3_150x150x1.raw = 220, 40, 0
wc4_150x150x1.raw = 330, 20, 0

If the stitching is to be run on a computer cluster, the cluster settings in the configuration file must be changed to match the capabilities of the available compute nodes.

If you are working at the SLS TOMCAT beamline, there is a script that converts a set of CT scan log files to a stitch settings file. Please refer to the internal documentation for details.

Often it is useful to try the stitching with downscaled versions of the tiles to see if everything works. This can be conveniently done using the parameter binning. Specify value larger than one to downscale the tiles by that amount before stitching. Usually binning 4 is a good starting point.

When the parameter file is ready, the stitching process is started by running the Python file nr_stitcher.py in the output folder.

After the script is finished, the largest .raw image in the output folder is the stitched image. Please note that if there are multiple non-connected sets of tiles in the output, multiple output images will be created.

Before running nr_stitcher.py again with different parameters, please remove all temporary files from the output folder by running rm-stitch-temp.sh or rm-stitch-temp.cmd, or specify redo=True in the configuration file.

Example

An example of tiles and the corresponding settings file can be found in the examples archive distributed along each pi2 version. See https://github.com/arttumiettinen/pi2/releases.

Known problems

  • Stitching result is not good, there are doubled details visible in the regions where one or more tiles overlap.

    This is usually caused by too large or too small values of point_spacing and coarse_block_radius parameters.

  • Error “: No such file or directory” when running in Linux.

    The error means that the line endings in the .py files have been converted to Windows format. This might happen for various not-so-obvious reasons, e.g. when transferring the Python code files by email. Easy workaround is to prepend ‘python’ to the commands, e.g.

    python /path_to_scripts/nr_stitcher.py

    In some cases it is necessary to explicitly specify python3 so that the operating system does not decide to use python 2.x versions that are incompatible:

    python3 /path_to_scripts/nr_stitcher.py
  • The same cluster jobs are being submitted over and over again.

    The cluster jobs fail for reason or another, and the NRStitcher is unable to catch the error. Check Slurm output files *-out.txt and *-err.txt for more information. Sometimes simply re-running the nr_stitcher_nongrid.py fixes the problem.

  • Large blocks of output image are black.

    If using a cluster, this problem is usually caused by failed SLURM jobs in the finalization phase. Each black block corresponds to one failed job. The most probable causes for this problem are out of memory error caused by too large max_block_size parameter in the stitch settings file, or jobs running too long and being killed by SLURM.

    In order to find out the reason for the error, please check the SLURM output files *-out.txt and *-err.txt. They will contain an indication of the reason why jobs failed if they failed.

    In any case, you might want to try again with smaller max_block_size parameter and larger job run time (using e.g. cluster_extra_params = --time=48:00:00 in the stitch settings file)

    In some cases the problems are intermittent and you will get away by simply re-running nr_stitcher.py. It tries its best not to re-calculate the good regions.