Title Page

Abstract

초록

Preface

Contents

Chapter 1. Introduction 19

1.1. Introduction to Data Privacy 19

1.1.1. Syntactic Privacy Models and Their Limitations 21

1.1.2. Differential Privacy 24

1.2. Motivation 30

1.3. Contributions of the Thesis 31

1.4. Organization of the Thesis 32

Chapter 2. Part I: Trajectory data analysis under local differential privacy 33

2.1. Introduction 33

2.2. Preliminaries and Related work 37

2.2.1. Differentail Privacy and Local Differential Privacy 37

2.2.2. Hidden Markov Model(HMM) 39

2.2.3. Data Correlation and Differential Privacy 41

2.3. The Proposed Method 43

2.3.1. Problem Definition 44

2.3.2. Client Side 45

2.3.3. Server Side 46

2.4. Performance Evaluation 48

2.4.1. Environment 49

2.4.2. Experimental results 50

2.5. Conclusion 60

Chapter 3. Part II: An activation function for differentially private deep learning 61

3.1. Introduction 61

3.2. Preliminaries 63

3.2.1. Rényi Differential Privacy (RDP) 63

3.2.2. DP-SGD and Gradient Clipping 65

3.3. The proposed method 66

3.3.1. Bounded activation function (BELU) 66

3.3.2. Differentially private stochastic gradient descent with BELU 68

3.4. Experiments 69

3.5. Conclusion 70

Chapter 4. Part III: Federated learning with local differential privacy 76

4.1. Introduction 76

4.2. Preliminaries 81

4.2.1. Local Differential Privacy (LDP) 81

4.2.2. Direct Feedback Alignment (DFA) 83

4.2.3. Federated Learning (FL) 85

4.3. Related work 87

4.4. The proposed method 89

4.4.1. Overview 89

4.4.2. LFL-DFALS: Local-differentially private Federated learning with Direct Feedback Alignment and Layer Sampling 92

4.4.3. AFL-LMTGR: Asynchronous Federated Learning with Local Model Training and Gradient Rebalancing 99

4.5. Performance Evaluation 104

4.5.1. Efficiency of direct feedback alignment and layer sampling 107

4.5.2. Experiments on varying communication probability 109

4.5.3. Experiments on varying the number of clients 111

4.5.4. Experiments on varying ε 112

4.6. Conclusion 112

Chapter 5. Conclusion 125

References 129

Table 2.1. Correspondence between LDP-based time-series location data collection/analysis and the Hidden Markov Model 43

Table 2.2. Parameter combinations in the experiments in trajectory data collection and analysis under local differential privacy with Hidden Markov Model 50

Table 3.1. Average accuracy of 10 epochs and accuracy in the last epoch 75

Table 4.1. Notations for LAFD 94

Table 4.2. Decription of Datasets for experiments in LAFD 105

Table 4.3. Experiment parameters for experiments in LAFD 105

Figure 2.1. Average absolute error of starting location distribution and transition probability with Simple Graph, f＝0, p＝0.5, q＝0.75 on varying the number of trajectories. 51

Figure 2.2. Average absolute error of starting location distribution and transition probability with Simple Graph, f＝0, p＝0.25, q＝0.75 on varying the number of trajectories. 52

Figure 2.3. Average absolute error of starting location distribution and transition probability with Simple Graph, f＝0.5, p＝0.5, q＝0.75 on varying the number of trajectories. 53

Figure 2.4. Average absolute error of starting location distribution and transition probability with Simple Graph, f＝0.5, p＝0.25, q＝0.75 on varying the number of trajectories. 54

Figure 2.5. Average absolute error of starting location distribution and transition probability with Seoul Subway, f＝0, p＝0.5, q＝0.75 on varying the number of trajectories. 55

Figure 2.6. Average absolute error of starting location distribution and transition probability with Seoul Subway, f＝0, p＝0.25, q＝0.75 on varying the number of trajectories. 56

Figure 2.7. Average absolute error of starting location distribution and transition probability with Seoul Subway, f＝0.5, p＝0.5, q＝0.75 on varying the number of trajectories. 57

Figure 2.8. Average absolute error of starting location distribution and transition probability with Seoul Subway, f＝0.5, p＝0.25, q＝0.75 on varying the number of trajectories. 58

Figure 3.1. An example of BELU (α＝1, β＝2) 71

Figure 3.2. Average accuracy of 10 epochs and accuracy in the last epoch varying activation function 72

Figure 3.3. Average accuracy of 10 epochs and accuracy in the last epoch varying β 73

Figure 3.4. Average accuracy of 10 epochs varying β and clipping size 74

Figure 4.1. The overview of LAFD between the server and clients 88

Figure 4.2. The structure of LFL-DFALS in clients 91

Figure 4.3. The structure of LFL-DFALS augmented with AFL-CLMTGR in clients 91

Figure 4.4. A example workflow of federated learning in an ideal environment 100

Figure 4.5. A example workflow of AFL-CMTGR in a realistic environment 100

Figure 4.6. Binary classification of Mushroom and Breast Cancer Wisconsin by epochs with training DFA with layer sampling, DFA without layer sampling, backpropagation... 114

Figure 4.7. Top-1 accuracy of classification by epochs with training DFA with layer sampling,DFA without layer sampling, backpropagation with layer sampling, and backprop-... 115

Figure 4.8. Accuracy of binary classification of Mushroom by epochs on varying pcomm[이미지참조] 116

Figure 4.9. Accuracy of binary classification of Breast Cancer Wisconsin by epochs on varying pcomm[이미지참조] 117

Figure 4.10. Top-1 accuracy of classification of USPS by epochs on varying pcomm[이미지참조] 118

Figure 4.11. Top-1 accuracy of classification of MNIST by epochs on varying pcomm[이미지참조] 119

Figure 4.12. Top-1 accuracy of classification of Fashion-MNIST by epochs on varying pcomm[이미지참조] 120

Figure 4.13. Top-1 accuracy of binary classification by epochs on varying the number of clients 121

Figure 4.14. Top-1 accuracy of classification by epochs on varying the number of clients 122

Figure 4.15. Top-1 accuracy of binary classification by epochs on varying the number of clients 123

Figure 4.16. Top-1 accuracy of image classification by epochs on varying ϵ 124