Bellos, Dimitrios
(2021)
Denoising and semantic segmentation of 4D datasets with undersampled micro X-ray CTs using deep learning.
PhD thesis, University of Nottingham.
Abstract
Over recent years, multiple imaging approaches has been proposed to depict the hidden inner structure of many different biological and non-biological systems in the form of 3D volumes. As time progressed due to multiple technological advances, these methods have become faster and more accessible, allowing in recent years the collection of time-resolved volumes that form 4D datasets. With 4D capturing techniques, the hidden inner structure of a system is collected repeatedly through time, allowing us to perceive not only its 3D structure, but temporal changes of this structure. Nevertheless, despite the technological advances in data collection there is still a major need for accurate, efficient and easy to use methods for 3D and 4D data analysis.
One of the most difficult tasks associated with 3D or 4D studies is their semantic segmentation. Unfortunately for such high-dimensional data, performing this task manually may take many days of human work, or even months in the case of 4D data. Moreover for the case of 4D data, there are other problems that make their semantic segmentation challenging. In particular, for time-series of micro X-ray Computed Tomograms (CTs), each tomogram is often undersampled in terms of projections due to time restraints. Specifically, in order to capture sufficient numbers of projections and perform the time-series collection quickly in order to capture temporal phenomena, the time allocated for each of its tomograms is low. Because of this only a limited amount of well-exposed projections can be collected. This sets an ill-posed reconstruction problem, and the resulting tomogram's reconstructions are low in signal-to-noise ratio. While computed tomography challenges grew, a new branch of computer vision approaches using deep learning have delivered promising approaches that address both the the problem of a low signal-to-noise ratio, and semantic segmentation. Inspired by these computed tomography challenges and the considerable promise of deep learning approaches, this thesis will propose approaches that address denoising and semantic segmentation of time-series, undersampled tomograms.
Initially, we propose a fast projection-upsampling method by upscaling a tomogram's sinograms using a deep learning-driven super-resolution approach. For our method works to upsample undersampled tomograms in a time-series with the use of a representative highly-sampled tomogram during training. These representative highly-sampled tomograms are usually collected either at the beginning or the end of a time-series, where their collection is not detrimental to the collection of the time-series. These tomograms are collected for spatial reference, providing essential spatial information for the restoration of the undersampled tomograms in our approach described here. Our method, named UDNN, designed to compete against analytical interpolation techniques with fast inference times is able to outperform them both on real-world and synthetic data.
Furthermore, we propose a second network for denoising and semantic segmentation of the undersampled tomograms' reconstructions using again the representative highly-sampled tomograms. Our proposed DenseUSeg approach expands upon Bui et al.'s DenseSeg architecture with a U-Net inspired decoder. Additionally, our proposed Stacked-DenseUSeg approach, by the stacking of two of our DenseUSegs, is able to produce both a denoising and a segmentation output in an end-to-end fashion. Through this process, the denoising output can be utilised to offer more accurate segmentation. Both our methods are able to outperform other state-of-the-art deep learning approaches for volume segmentation, with our Stacked-DenseUSeg achieving the highest accuracy of all. They are also compared against the earlier state-of-the-art methods for the task of denoising. While the earlier state-of-the-art methods are designed for segmentation, we decided to test our proposed networks against them also for the task of denoising since state-of-the-art tomogram denoising methods primarily employ less sophisticated CNN architectures and comparison against them would not be fair. In our experiments, DenseUSeg performs either better or equal against them for the task of denoising, and while Stacked-DenseUSeg is only slightly less accurate than DenseUSeg, the visual quality of its denoised outputs is noteworthy compared to the state-of-the-art methods solely trained for denoising.
To help handle the scarcity of real-world annotated data, we propose a novel use of knowledge transfer using parameter initialisation with networks pretrained on synthetic tomograms. During experiments it is discovered that with the use of pretrained nets on realistically constructed synthetic tomograms, it is possible improve Stacked-DenseUSeg segmentation accuracy and at the same time reduce the amount of real-world annotated data needed. Moreover, due to the use of less real-world data, it possible for the training to require less time.
Finally, having addressed the semantic segmentation of undersampled tomograms in a time-series, the temporal dimension remains though underutilised. This because our proposed Stacked-DenseUSeg provides segmentation predictions for each tomogram in a time-series independently though time. For this reason we propose a novel use of hidden Markov models for the refinement of tomogram's segmentation using temporal information. Namely, we propose our HMM-T, that use temporal information from temporally adjacent segmentations of tomograms and our HMM-TC, which also utilises confidence (probabilities) from these segmentations as produced by the network that generated them (Stacked-DenseUSeg). During experiments our HMM-T improved the accuracy of the tomogram segmentation both qualitatively and quantitatively. On the other hand, our HMM-TC demonstrated only qualitative and visual improvements, which in specific areas are even better that the ones of HMM-T. Because of that we speculate that with a more precise fine-tuning of its parameters it may even possible to surpass HMM-T.
The combination of these methods enables the processing of significant quantities of time-series of undersampled tomograms collected at Diamond Light Source. The aim of this work is to enable researchers who use time-series of undersampled CT collections (4D datasets) to quickly and easily extract useful information from the torrent of 4D data that is routinely collected in synchrotron locations.
Actions (Archive Staff Only)
 |
Edit View |
Tools
Tools
![[img]](/style/images/fileicons/application_pdf.png)