Title Page
Contents
ABSTRACT 10
요약 12
Chapter 1. Introduction 14
1.1. Stable Federated Learning with Dataset Condensation 15
1.2. Better Generalized Few-Shot Learning Despite No Base Data 18
Chapter 2. Related Works 22
2.1. Related problems in Stable Federated Learning 22
2.1.1. Federated Learning 22
2.1.2. Dataset Compression 23
2.1.3. Class imbalance 23
2.2. Related problems in Generalized Few-Shot Learning 24
2.2.1. Few-Shot Learning 24
2.2.2. Generalized Few-Shot Learning 25
Chapter 3. Stable Federated Learning with Dataset Condensation 28
3.1. Analysis on Training Instability of Federated Learning 28
3.2. Method 30
3.2.1. The Framework of Federated Learning 30
3.2.2. Dataset Condensation 32
3.2.3. FedDC: Stable Federated Learning 34
3.2.4. FedDC: Stable and Privacy-protecting Federated Learning 36
Chapter 4. Better Generalized Few-Shot Learning Despite No Base Data 37
4.1. Preliminary 37
4.1.1. Basic Framework of GFSL 37
4.1.2. Problem Statement of Zero-Base GFSL 39
4.2. Analysis and Methodology on Zero-Base GFSL 39
4.2.1. Hardness of Zero-Base GFSL 40
4.2.2. Analysis on Weight Distributions of Classifiers 40
4.2.3. Solution: Classifier Normalization 44
Chapter 5. Experiments 46
5.1. FedDC 46
5.1.1. Experimental Settings 46
5.1.2. Experimental Results 48
5.2. Classifier Normalization 53
5.2.1. Experimental Settings 53
5.2.2. Experimental Results 53
Chapter 6. Conclusion 56
Chapter 7. Appendix of "Better Generalized Few-Shot Learning Despite No Base Data" 57
7.1. Implementation Details 57
7.2. Additional Experiments 59
7.2.1. Ablation Study 59
7.2.2. Comparison with Naïve Approaches 60
7.2.3. Mean Centering for Both vs. Novel Only 62
7.2.4. Performance Comparison With and Without Base Data 62
7.2.5. Utilization of More Powerful Feature Extractors 63
7.2.6. Effect of Additional Learnable Parameters 64
Bibliography 65
이력서 78
TABLE 5.1. Comparison of accuracy, average class variance, and average round variance on CIFAR-10 52
TABLE 5.2. Comparison to prior works on mini-ImageNet. 54
TABLE 5.3. Comparison to prior works on tiered-ImageNet. 54
TABLE 7.1. Ablation study on mini-imagenet to analyze the effectiveness of different components in our method. MC is the mean centering, VB is the variance balancing,... 60
TABLE 7.2. Ablation study on ImageNet-800 to analyze the effectiveness of different components in our method. MC is the mean centering, VB is the variance balancing,... 60
TABLE 7.3. Comparisons of naïve approaches on Mini-ImageNet 63
TABLE 7.4. Comparing the accuracy with mean centering for novel and all classifier in zero-base GFSL and GFSL. 63
TABLE 7.5. Comparisons of different feature extractors on Tiered-ImageNet. 64
FIGURE 1.1. Performance of federated learning depending on data heterogeneity on CIFAR-10. 16
FIGURE 1.2. (a) Conceptual visualization of the decision boundaries in the feature space of the model trained on base classes and then fine-tuned on novel classes, respectively,... 20
FIGURE 2.1. Illustration of data given in the fine-tuning phase for each few-shot learning scenario, where the height and width of each box is proportional to the amount of data... 26
FIGURE 3.1. Comparison of class-wise accuracy after learning the 120th round and 121st round, and the round-class distribution. 29
FIGURE 4.1. Plots of the weight distributions of base and novel classifiers where dotted lines represent mean of the distributions 41
FIGURE 4.2. Overview of our solution for zero-base GFSL, Classifier Normalization. In the pre-training stage, we train the entire model on all base classes. In the fine-tuning... 44
FIGURE 5.1. Label distribution of CIFAR-10 across clients in different data partitions: (a) α=0.05, (b) α=0.1, and (c) α=1. 47
FIGURE 5.2. Comparison of the accuracy for each non-IID on CIFAR-10 using E=1: (a) α=0.05, (b) α=0.1, and (c) α=1. 48
FIGURE 5.3. Comparison of the round-wise class variance: (a) FedAvg (E=1), (b) FedDC (E=1), (c) FedAvg (E=10), and (d) FedDC (E=10). 49
FIGURE 5.4. Comparison of the accuracy in VGG. 50
FIGURE 5.5. Visualization of raw data and condensed images by DC and DC+ of the 5th client. 51
FIGURE 5.6. (a) Confusion matrices with and without adaptation normalization on Mini-ImageNet, where the rightmost ones after red lines are only novel classes. (b) Plots... 55
FIGURE 7.1. Evaluation on different iterations of additional parameters. 64