Title Page
Contents
초록 9
Abstract 10
Ⅰ. Introduction 11
Ⅱ. Proposed Method 17
2.1. Consensus ADMM 17
2.2. Problem Formulation 18
2.3. Proposed FedAND under partial participation 20
2.4. Convergence Proof of Fed AND 26
2.5. Analysis of Server Drift 28
2.6. Gradient of Global Loss Function 32
Ⅲ. Experimental Results 33
3.1. Experimental Setup 33
3.2. Experiments on Statistical Heterogeneity 34
3.3. Experiments on System Heterogeneity 36
3.4. Evaluating against Different Baselines 39
Ⅳ. Conclusion 44
Ⅴ. appendix 45
References 50
Table 1. Comparison of federated learning algorithms. 14
Table 2. Performance comparisons under systems heterogeneity (stragglers). 37
Table 3. Comparison of computation time. 40
Table 4. Speed of reaching the target accuracy. 40
Figure 1. Learning curves of FedDyn and FedAND under full participation. 19
Figure 2. System model of FedAND. 23
Figure 3. The normalized drift of consensus ADMM. 30
Figure 4. The impact of server drift. 31
Figure 5. The gradient of aggregated Lagrangian. 32
Figure 6. Performance comparisons under statistical heterogeneity. 35
Figure 7. Primal residual and dual residual (Synthetic (0.5,0.5) dataset). 36
Figure 8. The impact of various straggler ratio on FedAND. 38
Figure 9. Synthetic (1,1) dataset. 39
Figure 10. MNIST dataset. 39
Figure 11. Communication round to reach target accuracy by epoch. 41
Figure 12. Top-1 accuracy based on the stragglers ratio. 42
Figure 13. Normalized Drift of FedAND with respect to variations in ρ. 43