Xray Projectomic Reconstruction Extracting Segment with Skeleton Banner
Join

Evaluation Metric [code]

To evaluate performance, we compared the output segmentations with the GT skeletons in the test set to calculate the XPRESS score (Expected Run Length + Rand index).
Expected Run Length is a commonly used segmentation accuracy metric [1], it quantifies the average length of error-free neurites from the segmentation based on skeleton ground truth.

Rand Index is a widely used metric to evaluate the voxel-wise segmentation accuracy [2], we use the averaged rand split and rand merge to quantify the quality.

[1] Sheridan, Arlo, Tri Nguyen, Diptodip Deb, Wei-Chung Allen Lee, Stephan Saalfeld, Srini Turaga, Uri Manor, and Jan Funke. "Local shape descriptors for neuron segmentation." BioRxiv (2022): 2021-01.

[2] Rand, William W. "Objective criteria for the evaluation of clustering methods." Journal of the American Statistical association 66.336 (1971): 846-850.

Submission Format [code]

The challenge accepts HDF5 files for submission. A valid submission should have an HDF5 file, containing the voxel-wise prediction of the instance segmentation result for the test volume.
To make sure that your submission is evaluated correctly, please take note of the following items. You could refer to our tutorial for more guideline.
  • The submission contains a single .h5 file, the name of your file should be submission.h5. You need to zip the .h5 file into one .zip file for submission.
  • In nm, the data offset and shape in nm should be exactly (3267, 3267, 3267) and (33066, 33066, 33066) - that is, the center 33um^3 of the provided raw volume. This is different from validation where the GT skels limited us to the center 23um^3 sub-volume. Make sure that you set the ROI correctly for test v.s. validation when running segmentation.
  • The segmentation is downsampled by 3x. This means the submission resolution is 99nm, 3x larger than the raw resolution of 33nm. We use downsampling to save submission bandwidth and evaluation overheads. In 99nm resolution, the data offset and shape would be (33,33,33) and (334,334,334) voxels.