Title Page
Abstract
Contents
1. Introduction 21
1.1. Contributions and Outline of the Dissertation 25
2. Robust Visual Odometry Systems using Image Edge and Point Features 27
2.1. Introduction 28
2.1.1. Literature Review 29
2.1.2. Contributions of the Chapter 32
2.1.3. Algorithm Overview 33
2.2. Preliminaries 34
2.2.1. Notation Rules of This Chapter 34
2.2.2. 3-D Geometry of Camera Motions and Edges 34
2.2.3. ICP-based Edge Alignment 35
2.3. Edge Extraction and Culling 36
2.3.1. Edge Labeling using Overlapping Regions 36
2.3.2. Finding Salient Edgelets out of Labeled Edges 37
2.4. Robust Edge Matching via Oriented Quadtrees 39
2.4.1. Generating Multiple Oriented Quadtrees 39
2.4.2. Fast NNS Strategy storing Visited Nodes 41
2.5. Motion Tracking and 3-D Reconstruction 41
2.5.1. Stereo Point-to-edge Distances Minimization 41
2.5.2. Edge Inverse Depth Reconstruction and Propagation 45
2.6. Leveraging Feature Modalities: VO System combining Edges and Points 46
2.6.1. Selective Point and Edge Extraction with Image Binning 49
2.6.2. Edge and Point-based Hybrid Camera Motion Estimation with Illumination Compensation 50
2.7. Performance Analysis 53
2.7.1. Analysis 1: Normal Quadtree vs. Multiple Oriented Quadtrees 53
2.7.2. Analysis 2: Effect of storing the Previously Matched Nodes 55
2.7.3. Analysis 3: Effect of the Edge Culling Method 56
3. Scale-aware Monocular Visual Odometry using Vehicle Kinematic Constraint 59
3.1. Introduction 60
3.1.1. Literature Review 62
3.1.2. Contributions of the Chapter 65
3.1.3. Algorithm Overview 65
3.2. Preliminaries 66
3.2.1. Notation Rules of This Chapter 66
3.2.2. Camera Motion constrained by Vehicle Kinematics 67
3.2.3. Visual Processing Front-end and Data Structures 69
3.3. Camera-Vehicle Extrinsic Pose Calibration 70
3.3.1. Problem Formulation 70
3.3.2. Linear Initialization of Ψj and Q[이미지참조] 72
3.3.3. Full Refinement of the Initial Guesses 74
3.4. Absolute Scale Recovery between Turning Regions 76
3.4.1. Observing Absolute Scale via Kinematic Geometry 76
3.4.2. Detecting Turning Regions 79
3.4.3. Recovering Unknown Scale by Nonlinear Programming with Equality Constraints 80
3.5. Performance Analysis 84
3.5.1. Noise Sensitivity Analysis of the Scale Observer 84
3.5.2. Analysis via Implementations on Synthetic Data 87
4. Experimental Results on Real-world Scenarios 93
4.1. Implementations on Indoor Office Datasets 93
4.1.1. EuRoC MAV Datasets 93
4.1.2. ICL-NUIM Datasets 95
4.1.3. SNU Modern Building Indoor Datasets 101
4.2. Implementations on Indoor Driving Datasets 108
4.2.1. KITTI Odometry Datasets 108
4.2.2. SNU Underground Parking Lots Datasets 110
5. Conclusion 127
References 130
국문초록 139
Table 3.1. RMSE comparisons of angles and scale estimations on the synthetic dataset 92
Table 4.1. Performance comparison on EuRoC datasets 94
Table 4.2. Performance comparison on ICL-NUIM office datasets 96
Table 4.3. Quantitative comparison on the KITTI odometry datasets - scale estimation error ratio 110
Table 4.4. Quantitative comparison on the KITTI odometry datasets - translation error 113
Table 4.5. Hardware specifications of the author-collected dataset 113
Table 4.6. Results of the camera-vehicle extrinsic calibration on the author-collected dataset 115
Figure 1.1. Challenging environments for the conventional visual odometry 22
Figure 1.2. Image sensors installed in the automotive vehicles. 23
Figure 1.3. Monocular metric scale ambiguity problem and the unknown camera-vehicle pose 23
Figure 1.4. Overall flowchart of the proposed methods in the dissertation 24
Figure 2.1. Literature review for the edge and point-based robust visual odometry systems 30
Figure 2.2. Flowchart of the proposed edge-point VO system 32
Figure 2.3. Edge label bins with overlapping regions 36
Figure 2.4. Extracted salient edgelets and center points 38
Figure 2.5. Example of depth-first search for finding salient edgelets 39
Figure 2.6. Comparison between a normal single-rooted quadtree and the multiple cached quadtrees 42
Figure 2.7. Illustration of the image gradient vector on the image edge pixels 44
Figure 2.8. Illustration of the point-to-edge normal distance induced by a matched pair of points 45
Figure 2.9. Static and temporal stereo configurations 46
Figure 2.10. Ambiguous edges in the modern indoor scenes 47
Figure 2.11. Point abundant regions 48
Figure 2.12. Edges, points, and lines in edge-dominant scenes 48
Figure 2.13. Comparison of characteristics of various image features 49
Figure 2.14. Salient edges and points in the image 49
Figure 2.15. Feature bucketing strategy to maintain the number of edge and point pixels 50
Figure 2.16. Illustration of geometry of the consecutive cameras, pixels, and image patches 51
Figure 2.17. Example of the matching results of the normal quadtree and the proposed multiple quadtrees 54
Figure 2.18. Selected two images on the EuRoC V1_01 dataset for quantitative evaluations 55
Figure 2.19. ICP iterations and time consumption to align the simple image in Fig. 2.18(a). 56
Figure 2.20. ICP iterations and time consumption to align the cluttered image in Fig. 2.18(b). 57
Figure 2.21. ICP iterations and time consumption by using the edge culling method to align the cluttered image in Fig. 2.18(b). 58
Figure 2.22. Results of the edge culling method with lmin={5,15,25}[이미지참조] 58
Figure 3.1. Literature review for the scale-aware monocular visual odometry algorithms. 63
Figure 3.2. Block diagram of the proposed scale-aware monocular visual odometry and extrinsic calibration system 66
Figure 3.3. Illustration of the vehicle kinematics. 68
Figure 3.4. Singular value history of the linear initialization of qs[이미지참조] 75
Figure 3.5. Two triangles formed by the turn of the vehicle and relationship between Ψ and θ (a) Ψj is the turn angle of the vehicle, and θj is the subtended angle...[이미지참조] 77
Figure 3.6. Factor graph of a landmark and related keyframes for the absolute scale recovery 81
Figure 3.7. Noise sensitivities of the scale observer 85
Figure 3.8. Turn angle Ψj, relative distance ρj/L, and the error over the scale on the author-collected parking lots driving datasets[이미지참조] 86
Figure 3.9. Trajectory and 3-D points of the synthetic dataset, and the turning region detection results 88
Figure 3.10. Results of the camera-vehicle extrinsic calibration on the synthetic dataset 89
Figure 3.11. Results of the absolute scale recovery on the synthetic dataset 91
Figure 4.1. Calculation times of four implementations on EuRoC datasets. 94
Figure 4.2. Representative scenes of the ICL-NUIM datasets (left figures), and simulated brightness changes (right figures). 95
Figure 4.3. Results of the edge and point-based VO with various settings on office_00 of ICL-NUIM dataset. 97
Figure 4.4. Results of the edge and point-based VO with various settings on office_01 of ICL-NUIM dataset. 98
Figure 4.5. Results of the edge and point-based VO with various settings on office_02 of ICL-NUIM dataset. 99
Figure 4.6. Results of the edge and point-based VO with various settings on office_03 of ICL-NUIM dataset. 100
Figure 4.7. VO and 3-D reconstruction results on the author-collected dataset. 102
Figure 4.8. VO and 3-D reconstruction results on the author-collected dataset. 103
Figure 4.9. VO and 3-D reconstruction results on the author-collected dataset. 104
Figure 4.10. VO and 3-D reconstruction results on the author-collected dataset. 105
Figure 4.11. Edge and point-based VO and 3-D reconstruction results on the author-collected dataset - library 106
Figure 4.12. Edge and point-based VO and 3-D reconstruction results on the author-collected dataset - stair 107
Figure 4.13. Turning region detection results on KITTI odometry datasets 111
Figure 4.14. Representative trajectories of the proposed method on the KITTI odometry datasets 112
Figure 4.15. Experimental setting for the author-collected dataset - overview 114
Figure 4.16. Experimental setting for author-collected dataset - sideview 118
Figure 4.17. Representative images of the author-collected SNU underground parking lots datasets. 119
Figure 4.18. Overviews of the author-collected datasets 120
Figure 4.19. Overviews of the author-collected datasets 121
Figure 4.20. LIO-SAM results on the bldg_39 and bldg_220 datasets with the loop closure 122
Figure 4.21. LIO-SAM results on the bldg_39 and bldg_220 datasets without the loop closure 123
Figure 4.22. Limitations of using the fixed image region for ground landmarks 124
Figure 4.23. Overall trajectories on bldg_220 of the proposed method, ORB-SLAM with monocular and stereo settings, and the 3-D LiDAR odometry. 125
Figure 4.24. Overall trajectories on bldg_220 of the proposed method, ORB-SLAM with monocular and stereo settings, and the 3-D LiDAR odometry. 126
Figure 5.1. Omnidirectional imagery in indoor environment 128