Title Page
Abstract
Contents
List of Abbreviations 17
1. Introduction 19
1.1. Motivations 23
1.2. Contributions 23
2. Background 24
2.1. Medical image segmentation using deep-learning 24
2.1.1. Convolution neural network (CNN) 24
2.1.2. Fully convolutional networks (FCN) 26
2.1.3. U-Net 26
2.1.4. Cascade U-Net 27
2.1.5. no-new-U-Net (nnU-Net) 28
2.1.6. UNEt Transformers (UNETR) 29
2.1.7. Shifted windows UNEt TRansformers (SwinUNETR) 30
2.1.8. Active learning (AL) 31
2.2. Computer aided design (CAD) modeling 32
2.2.1. Manual 3D CAD modeling 32
2.2.2. Script-based application programming interface (API) 34
2.3. 3D printing technologies with clinical application 36
2.3.1. Fused deposition modeling (FDM) 36
2.3.2. Stereolithography apparatus (SLA) 36
2.3.3. Digital Light Processing (DLP) 37
2.3.4. Color-Jet printing (CJP) 37
2.3.5. Multi-jet printing (MJP) 37
2.3.6. Photopolymer jetting (PolyJet) 38
2.3.7. Selective laser sintering (SLS) 38
3. Semi-automated and enhanced segmentation using semantic segmentation with active learning (AL) 39
3.1. Kidney substructures with RCC in Kidney 39
3.2. Mandibular condyle in dental CBCT 50
3.3. Thoracoabdominal aortic dissection in CT angiography 61
3.4. Abdominal aortic aneurysm in abdominal CT 70
4. Semi-automated design and measurement with application programming interface (API) of computer aided design (CAD) 80
4.1. Aortic graft reconstruction guides in thoracoabdominal aortic dissection 80
4.2. Landmark measurement in abdominal aortic aneurysm 91
5. Utilization of patient-specific guides fabricated using 3D printing from viewpoint of clinical application 102
5.1. 3D printed surgical guides in thoracoabdominal aortic dissection 102
6. Discussion 115
7. Conclusion 139
References 140
Abstract (In Korean) 152
Table 3-1. Dice similarity coefficient for each stage of kidney with cascade 3D U-Net in kidney CT. 48
Table 3-2. Comparison of segmentation time for kindey between manual and AL-corrected segmentation in kidney CT. 49
Table 3-3. Root mean square (RMS) evaluation from 3D modeling for kidney. 49
Table 3-4. Dice similarity coefficient and Hausdorff distance for each stage of mandibular condyles with a basic 3D U-Net and cascade 3D U-Net in CBCT. 59
Table 3-5. Segmentation time of mandibular condyles for manual, basic 3D U-Net, and cascade 3D U-Net in stage 5 of CBCT. 61
Table 3-6. Dice similarity coefficient for abdominal aortic dissection with UNETR and SwinUNETR of MONAI and 2D U-Net, 3D U-Net, 2D-3D U-Net ensemble, and cascade 3D... 68
Table 3-7. 95% Hausdorff distance for thoracoabdominal aortic dissection with UNETR and SwinUNETR of MONAI and 2D U-Net, 3D U-Net, 2D-3D U-Net ensemble, and cascade 3D... 69
Table 3-8. The segmentation time for manual and correction using SwinUNETR in thoracoabdominal aortic dissection. 70
Table 3-9. Dice similarity coefficient for abdominal aortic aneurysm with UNETR and SwinUNETR of MONAI and 2D U-Net, 3D U-Net, 2D-3D U-Net ensemble, and cascade 3D U-Net of nnU-Net. 77
Table 3-10. 95% Hausdorff distance for abdominal aortic aneurysm with UNETR and SwinUNETR of MONAI and 2D U-Net, 3D U-Net, 2D-3D U-Net ensemble, and cascade 3D U-Net of nnU-Net. 78
Table 3-11. The segmentation time for manual and correction using 3D U-Net of nnU-Net in abdominal aortic aneurysm. 79
Table 4-1. Ten patients' profiles for thoracoabdominal aortic dissection, including classification, level of segmental arteries, and range of graft reconstruction. Note. T, thoracic;... 82
Table 4-2. Hausdorff average distance of corresponding points between conventional and automated modeling methods for each patient with visualizing and marking guides. Note:... 89
Table 4-3. Modeling time for conventional and automated modeling methods with two types of patient-specific graft reconstruction guides. 91
Table 4-4. Manufacturer's guideline of patient selection for EVAR. 92
Table 4-5. The differences between manual and automatic measurements for the aortic neck diameter, aortic aneurysm, right iliac artery, left iliac artery diameter, aortic neck length, the... 98
Table 5-1. Individual profiles of the subject patients including classification, level of segmental artery, and range of replacement. Note. T, thoracic; L, lumbar; Rs, right side; Ls,... 104
Table 5-2. Measurements and time requirements of graft reconstruction using DGM, IBT, MBT, and GBT. Note. DGM, digital graft model; IBT, image-based technique; MBT, model-... 111
Table 5-3. Correlation coefficient between DGM and three techniques for diagonal length, height, and angle. Note. r12, correlation coefficient for DGM and IBT; r13, correlation... 113
Table 5-4. Retrospective survey in relation to understanding, usefulness, satisfaction, surgical outcome, and recommendability for use in other applications for IBT without 3D printing... 114
Table 6-1. The summary of each task 117
Table 6-2. Dice similarity coefficient for each institution with each stage of mandibular condyles with a basic 3D U-Net and cascade 3D U-Net in CBCT. Note: AMC, Asan Medical... 120
Table 6-3. Intra- and inter-observer variation for 4 classes in abdominal aortic aneurysm. 126
Table 6-4. The best network for each stage and class in 4 studies 127
Table 6-5. Comparison between 2D and 3D U-Net of nnU-Net in two aortic studies 129
Table 6-6. Comparison between high- and low-resolution with 6 networks in two aortic studies 130
Table 6-7. Hausdorff average distance of 5 patients for validation between conventional and automated modeling methods with visualizing and marking guides. Note: CP, corresponding point. 132
Table 6-8. Modeling time of 5 patients for validation with conventional and automated modeling methods consisted of two types of patient-specific graft reconstruction guides. 132
Table 6-9. Comparison of correlation between 3 researchers for in the aortic neck diameter, aortic neck length, right and left tortuosity including curve length, line length, and ratio. 134
Figure 1-1. The overall procedure of 3DP technology for clinical application. 19
Figure 2-1. The convolution neural network (CNN) architecture. 24
Figure 2-2. The fully convolutional networks (FCN) architecture. 26
Figure 2-3. The U-Net architecture. 27
Figure 2-4. Cascade U-Net methods (A) The first training for the location of segmentation from CT images with basic U-Net, (B) The second training for the classes from the first training. 28
Figure 2-5. The UNEt TRansformers (UNETR) architecture. 30
Figure 2-6. The Shifted windows UNEt TRansformers (SwinUNETR) architecture. 31
Figure 2-7. The process of the active learning (AL). 32
Figure 2-8. The process of the application programming interface (API). 35
Figure 2-9. The principle and structure of 3D printing (3DP) technologies. (A) the fused deposition modeling (FDM) type, (B) the stereolithography apparatus (SLA) type, (C) the... 39
Figure 3-1. The overall procedure s of cascade 3D U-Net using active learning (AL) in kidney CT. 41
Figure 3-2. The manual segmentation techniques including (A) thresholding, (B) region growing, (C) dilation and erosion, and (D) edit mask. 42
Figure 3-3. Pre-processing procedure for kidneys with a 3D U-Net and cascade 3D U-Net (A) Normalization image, (B) Division by the right and left sides, (C) Flip the right images... 44
Figure 3-4. Data distribution in each stage for active learning (AL) in kidney CT. 45
Figure 3-5. The architecture of (A) a basic 3D U-Net and (B) a cascaded 3D U-Net 46
Figure 3-6. Root mean square (RMS) evaluation from 3D model about kidney with RCC 50
Figure 3-7. The overall procedure s of a basic 3D U-Net and its based cascade 3D U-net using active learning (AL) in CBCT. 52
Figure 3-8. Pre-processing procedure for mandibular condyles with a basic 3D U-Net and cascade 3D U-Net (A) Normalization image, (B) Division by the right and left sides, (C) Flip... 54
Figure 3-9. Data distribution for mandibular condyle in each stage for active learning (AL). 56
Figure 3-10. The architecture of (A) a basic 3D U-Net and (B) a cascaded 3D U-Net. 57
Figure 3-11. The difference map between ground truth and prediction in 3D U-Net and cascaded 3D U-Net. (A) The best case and (B) worst case of 3D U-net, and (C) the best case... 60
Figure 3-12. (A) The octopod thoracoabdominal aortic aneurysm (TAAA) aortic graft. (B) Representative computed tomographic images at the visceral and intercostal level. (C)... 62
Figure 3-13. Overall flow of thoracoabdominal aortic dissection and abdominal aortic aneurysm for smart labeling with human in the loop in CT angiography. 64
Figure 3-14. Data distribution for thoracoabdominal aortic dissection in each stage for active learning (AL). 67
Figure 3-15. Data distribution in each stage for active learning (AL) for abdominal aortic aneurysm. 74
Figure 4-1. Overall procedure for the evaluation of two types of patient-specific graft reconstruction guides consisting of visualizing and marking guides for aortic dissection and... 81
Figure 4-2. The conventional modeling method for patient-specific visualizing and marking guides. (A) Automated segmentation of thoracoabdominal aorta, including major blood... 85
Figure 4-3. Comparison of the aligned difference map is shown with the absolute mean differences in the areas between conventional and automated modeling methods with (A)... 90
Figure 4-4. The clinical defined measurement landmarks for AAA models. 93
Figure 4-5. Conventional image-based measurement. 94
Figure 4-6. Bland-Altman plot indicating the distribution of the differences between manual and automatic method, divided by (A) the aortic neck diameter and (B) the aortic aneurysm diameter. 99
Figure 4-7. Bland-Altman plot indicating the distribution of the differences between manual and automatic method, divided by (A) the right iliac artery diameter, (B) the left iliac artery diameter, and (C) the aortic neck length. 100
Figure 4-8. Bland-Altman plot indicating the distribution of the differences between manual and automated method, divided by (I) the curve length, (J) the line length, and (K) the tortuosity ratio for left iliac artery. 100
Figure 4-9. Bland-Altman plot indicating the distribution of the differences between manual and automatic method, divided by (A) the curve length, (B) the line length, and (C) the tortuosity ratio for left iliac artery. 101
Figure 5-1. Overall process of graft reconstruction with the conventional IBT, MBT, and GBT. 103
Figure 5-2. Two types of patient-specific graft reconstruction guide application for open repair of thoracoabdominal aortic dissection. (A) CT angiography images and segmentation... 107
Figure 5-3. Patient-specific graft reconstruction process with MBT and GBT in operating room. 108
Figure 5-4. Guide to evaluating the accuracy of three techniques-IBT, MBT, and GBT; (A) Locating celiac artery and segmental arteries marked on graft. (b) Spreading the graft using... 110
Figure 5-5. Bland-Altman plot indicating the distribution of the differences between DGM and IBT, divided by (A) diagonal line, (B) height and (C) angle; between the DGM and... 112
Figure 6-1. A case for cascaded 3D U-Net (A) images, (B) prediction, and (C) ground truth in kidney CT 118
Figure 6-2. The errors for prediction of cascade 3D U-net including (A) noise and (B) under segmentation. 121
Figure 6-3. Best case for SwinUNETR with images, prediction (SwinUNETR and 2D U-Net), and ground truth in thoracoabdominal aortic dissection. 123
Figure 6-4. Best case for 3D U-Net with images, prediction (3D U-Net and UNETR), and ground truth 125
Figure 6-5. The subjective assessment and measurement error of elliptical aortic center and blood vessels in CT images. If a red line is selected instead of a green line, there is an error... 136
Figure 6-6. Making time according to the number of segmental arteries for IBT, MBT and GBT. 137
Algorithm 4-1. Automated modeling method for the visualizing guide and marking guide. 87
Algorithm 4-2. The module for part inspection. 88
Algorithm 4-3. The module for finding the maximum curve. 95
Algorithm 4-4. The algorithms for the automatic measurement of the aortic neck diameter, aortic aneurysm diameter, the common iliac artery diameter, aortic neck length, the tortuosity... 96