Title Page
Abstract
Contents
Chapter 1. Introduction 16
Chapter 2. Related Works 20
2.1. Generative Adversarial Networks 20
2.2. Image Manipulation 22
2.3. 3D-aware GANs 24
2.4. 3D Object Manipulation 26
Chapter 3. Attribute Quantifier 30
3.1. Overview 30
3.2. Preliminaries 33
3.2.1. Contrastive Language-Image Pre-Training 33
3.2.2. CutMix 34
3.3. Method 35
3.3.1. Quantifier Design 35
3.3.2. Quantifier Training 36
3.4. Experiments 38
3.4.1. Implementation 38
3.4.2. Evaluation Metric 39
3.4.3. Benchmark of the Amount of Training Data 39
3.4.4. Benchmark of the Various Attributes 42
3.4.5. Ablation study 45
3.5. Contributions 45
Chapter 4. Quantitative Manipulation on StyleGAN 47
4.1. Overview 47
4.2. Preliminaries : StyleGAN 49
4.3. Method 51
4.3.1. Overall Scheme 51
4.3.2. Navigator Design 51
4.3.3. Navigator Training 53
4.4. Experiments 55
4.4.1. Implementation 55
4.4.2. Attribute Manipulation 59
4.4.3. Quantitative comparison 66
4.4.4. Qualitative comparison 71
4.4.5. Ablation Study 75
4.5. Contributions 78
Chapter 5. Quantitative Manipulation on 3D-Aware Image 80
5.1. Overview 80
5.2. Preliminaries : EG3D 83
5.3. Method 85
5.3.1. Custom Attribute Quantifier 85
5.3.2. Multi-Attribute Navigator Design 88
5.3.3. Multi-Attribute Navigator Training 91
5.4. Experiments 93
5.4.1. Implementation Details 93
5.4.2. Custom Attribute Manipulation 96
5.4.3. Quantitative Comparisons 99
5.4.4. Qualitative Comparisons 106
5.4.5. Ablation Study 107
5.5. Contributions 112
Chapter 6. Concluding Remarks 113
6.1. Summary of Methods and Contributions 113
6.2. Potential Negative Societal Impacts 114
6.3. Suggestions for Future Research 115
Bibliography 117
초록 130
Table 3.1. Performance of quantifier on various attributes. 44
Table 4.1. Hyper-parameters for training the navigator. 56
Table 4.2. Quantitative comparison with competing methods. We evaluated the manipulation performance on age and smile attributes using four evaluation metrics:... 68
Table 4.3. Comparison of age manipulation performance evaluated using CORAL- CNN on the FFHQ-Aging dataset. Ground truth is the performance of CORAL-CNN... 69
Table 4.4. Comparison of the quantification performance of the age attribute manipulation evaluated by varying the target quantity from 0.5 to 1.0 at 0.1 intervals. 70
Table 4.5. Ablation study on quantitative manipulation. We experimented on face age attribute and evaluated the performance on the manipulation range (R), accuracy(σ),... 76
Table 4.6. Ablation study on quantity representation vector of the navigator. We experimented on face age attribute and evaluated the performance on the manipulation range... 77
Table 5.1. Specifications of attribute datasets for training quantifiers. Includes the number of groups (M) and the number of samples for each group (Ωa) in constructing...[이미지참조] 94
Table 5.2. Quantitative comparison with competing methods in EG3D. We evaluated the manipulation performance on age, smile, gender, and eyeglass attributes using five... 101
Table 5.3. Quantitative comparison with competing methods in EG3D. We evaluated the age manipulation performance using MWR method. The MWR method used models... 103
Table 5.4. Quantitative comparison with competing methods in EG3D. We evaluated the smile manipulation performance using DMUE method. 104
Table 5.5. Ablation study on quantitative manipulation. We experimented on four attributes (age, smile, gender, and eyeglasses) and reported the mean values of exactness... 106
Table 5.6. Ablation study on color bias. Difference in average intensities of red and blue channels according to gender manipulation. 109
Figure 3.1. Example of numerical inference in the vision-language model. The logit scores for the phrases captioned in each subfigure are output from the CLIP model. 31
Figure 3.2. Example of assigning densely quantified labels by augmentation using CutMix (object: face; attribute: age, M = 10, N = 1). Synthetic samples' labels densely distributed in the quantity range [0,1]. 37
Figure 3.3. Performance of the quantifier depending on the number of images per group, N (correlation: higher is better; MAE: lower is better). 41
Figure 3.4. Examples of dataset for six emotions. From the left figure to the right figure, the quantity of the emotion gradually increases. 43
Figure 3.5. Performance impact of Cutmix according to the M value 45
Figure 4.1. Example of proposed image manipulation for the face age attribute. The age attribute is normalized into a quantity value between 0 (for two-year-olds) and 1... 48
Figure 4.2. Overall scheme of the proposed method: the navigator manipulates the latent feature for StyleGAN to generate the target image from a source image given along... 52
Figure 4.3. Quantitative manipulation results of faces for happiness attribute. The first row shows the samples of training dataset, and the other rows show the generated images. The first image of the generated images is the source image, and the ten images after that... 57
Figure 4.4. Quantitative manipulation results of faces for age attribute. The first row shows the samples of training dataset, and the other rows show the generated images. The first image of the generated images is the source image, and the ten images after that are... 58
Figure 4.5. Manipulation results of faces for various attributes. 60
Figure 4.6. Quantitative manipulation results of cars for year, type, and chromatic color attributes. The image in the first column is the source, and the ten following images are the images manipulated with the target quantity increased from 0.0 to 1.0 at 0.1 intervals.... 62
Figure 4.7. Quantitative manipulation results of churches for darkness and cloud attributes. The image in the first column is the source, and the ten following images are the images manipulated with the target quantity increased from 0.0 to 1.0 at 0.1 intervals. 63
Figure 4.8. Fine-grained manipulation results of the face happiness attribute. Manipula- tion result images are generated with the target quantity increased from 0.01 to 0.99 at... 64
Figure 4.9. Fine-grained manipulation results of the face age attribute. Manipulation result images are generated with the target quantity increased from 0.01 to 0.99 at 0.01 intervals. 65
Figure 4.10. Qualitative comparison with competing methods: StyleFlow and Talk- to-Edit. The leftmost column is the source, and the six following images are the... 72
Figure 4.11. Comparison between models regarding the degree of image manipulation according to the manipulation strength (left: age attribute; right: smile attribute). Red boxes denote unnatural manipulated images due to excessive manipulation strength. 73
Figure 4.12. Face identity loss ablation study. 78
Figure 5.1. An example of quantitative image manipulation for the face tiredness attribute. Attributes expressed as complex facial features, such as tiredness, are not easy... 81
Figure 5.2. Overall scheme of the attribute quantifier. The quantifier that estimates the attribute quantities of the image is trained using the custom dataset and consistency... 86
Figure 5.3. Overall scheme of the proposed method. The quantifiers that estimate the attribute quantities of the source and target images are first trained using the custom dataset. The navigator then manipulates the latent feature for efficient geometry-aware 3D... 89
Figure 5.4. Quantitative manipulation results of faces for age, smile, and tiredness attributes. The image in the first column is the source, and the following four images... 95
Figure 5.5. Various custom attribute manipulation results. The image in the first column is the source, and the following three images are the results of each manipulation for... 97
Figure 5.6. Quantitative manipulation results of the face for custom attributes: drunken- ness, pain, eyebrow, and thinness. The image in the first column is the source image,... 98
Figure 5.7. Quantitative manipulation results of cats for custom attributes (fur, eye, and age). The image in the first column is the source, and following five are the manipulated images. 99
Figure 5.8. Attribute manipulation on real images. The image in the first column is the source, the image in the second column is the image inverted from the source using pivotal tuning inversion (PTI), and the following six images are the results of each manipulation... 100
Figure 5.9. Correlation analysis on attribute disentanglement. The x-axis and y-axis of each matrix represent the target quantities for smile and age, respectively. The values in... 105
Figure 5.10. Qualitative comparison for age and smile attributes manipulation in EG3D. Competing methods fail to preserve identity when manipulated to Old and No-smile. 107
Figure 5.11. Ablation study on view consistency. We experimented on two attributes (age, smile) and reported the viewpoint-wise standard deviation σv (a lower value is better).[이미지참조] 108
Figure 5.12. Examples of bias in the face age manipulation using Ωa = 1 quantifier. For the two different attribute datasets, quantifiers were trained using each attribute...[이미지참조] 110
Figure 5.13. Examples of the face age manipulation depending on the number of images per group, Ωa, of the quantifier attribute datasets, Da.Ωa = 1 examples are the...[이미지참조] 111