Title Page
Abstract
초록
Contents
List of Dataset Icons 19
Chapter 1. Introduction 21
1.1. Background and Motivation 21
1.2. Problem definitions 24
1.3. Research Goals 25
1.4. Research Scope 25
1.5. Contributions 26
1.5. Framework 27
1.6. Organization of This thesis 29
Chapter 2. Literature Review 31
2.1. Ergonomics Aspect 31
2.2. Risk Level Prediction 32
2.3. Wrist Kinematics 35
2.4. Imbalanced Data 37
2.5. Explainable Artificial Intelligence (XAI) 39
Chapter 3. Data Collection and Preparation 42
3.1. Data Collection 42
3.2. Data Preparation 47
Chapter 4. Posture Risk Measurement for Ergonomics Assessment in Assembly Tasks 48
4.1. Introduction 48
4.2. Method 51
4.2.1. RULA Score Calculation 52
4.2.2. Wrist Kinematics Measurement 54
4.2.3. Body Posture Measurement 60
4.2.4. Evaluation 62
4.3. Experiment and Results 63
4.3.1. Similarity Measurement 63
4.3.2. Personalized Measurement 66
4.4. Discussion 68
Chapter 5. RULA Risk Level Prediction with Deep Learning for Imbalanced Class Data 73
5.1. Introduction 73
5.2. Method 77
5.2.1. Preliminaries 78
5.2.2. Missing Value Imputation 79
5.2.3. Data Oversampling using KD-SMOTE 79
5.2.4. Risk Prediction using HyNet-CB 83
5.2.5. Evaluation 89
5.3. Experiment and Results 90
5.3.1. Risk Level Estimator 90
5.3.2. Missing value Imputation Experiment Results 91
5.3.3. Evaluation HyNet-CB Using The Imbalanced Data 95
5.3.4. Evaluation of HyNet-CB Using The Balanced Data 99
5.4. Discussion 101
Chapter 6. XAI Utilization for Ergonomics Risk Level Classification 104
6.1. Introduction 104
6.2. Method 107
6.2.1. Data Preparation 108
6.2.2. XAI Approaches for Neural Network Model 109
6.2.3. Feature Importance Summarization 114
6.2.4. Feature Importance Ranking Correlation 115
6.3. Experiment and Discussion 115
6.3.1. Performance Evaluation 116
6.3.2. Feature Importance Analysis 118
6.3.3. Feature Importance Ranking Correlation Analysis 124
6.3.4. The Experiment in Different Domains 124
Chapter 7. Final Remarks 129
7.1. Conclusions 129
7.2. Future work 130
References 131
Appendices 144
Appendix A. RULA Score Annotation by expert 144
Appendix B. Time execution to calculate joint angle and RULA score 144
Appendix C. Feature list and rank 145
Table 2.1. Risk level prediction related works 34
Table 4.1. RULA Risk level 54
Table 4.2. RULA wrist posture scoring criteria 55
Table 4.3. RULA body posture score criteria 61
Table 4.4. Similarity measurement results for section A (right and left). 64
Table 4.5. Similarity measurement results for section A (Max score). 65
Table 4.6. Previous Study similarity measurement results for wrist section. 65
Table 4.7. Similarity measurement results for section B. 66
Table 4.8. Similarity measurement results for grand RULA score. 66
Table 4.9. T-test results for subjects 1-6 67
Table 4.10. T-test results for subjects 7-12 67
Table 4.11. Number of samples 68
Table 5.1. RULA risk level 79
Table 5.2. Missing value pattern 92
Table 5.3. Missing value simulation results 95
Table 5.4. Evaluation results using the imbalanced data (simple imputer) 97
Table 5.5. Evaluation results using the imbalanced data (KNN imputer) 97
Table 5.6. F1-score for each risk level 100
Table 5.7. Confusion metrics of the additional experiment 101
Table 5.8. Confusion metrics of HyNet-CB performance 103
Table 6.1. HyNet-CBA setup parameter for risk level prediction. 112
Table 6.2. List of important features 118
Table 6.3. Ranking correlation results 124
Table 6.4. HyNet-CBA parameter setup for activity recognition 126
Appendix B 14
Table B.1. The execution time required to determine the RULA score 144
Appendix C 15
Table C.1. List of feature name acronyms 145
Table C.2. Summary of Feature Importance Rank 147
Figure 1.1. Risk Level Prediction Framework 27
Figure 1.2. Organization of this thesis 30
Figure 3.1. Data collection and preparation process 42
Figure 3.2. Sensors: (a) Body tracking sensor, (b) Finger tracking sensor 43
Figure 3.3. User interface of data collection application 44
Figure 3.4. List of the activities in three levels of activity 45
Figure 3.5. Laboratory setup 46
Figure 3.6. (a) Drawer part, (b) Panel placement on the workbench 46
Figure 4.1. Posture risk measurement process 51
Figure 4.2. RULA assessment for sections A and B 52
Figure 4.3. RULA scoring mechanism 53
Figure 4.4. Finger joints 55
Figure 4.5. Wrist's flexion and extension position 56
Figure 4.6. Calculation of wrist radial-ulnar deviation angle 57
Figure 4.7. Wrist pronation-supination: (a) wrist is twisted in mid-range and (b) wrist is at or near the end of the range 58
Figure 4.8. Initial position 58
Figure 4.9. Wrist twist (pronation-supination) angle calculation 59
Figure 4.10. Wrist twist position in the range 59
Figure 4.11. Body joints 61
Figure 4.12. RULA score distribution 68
Figure 4.13. Installation of the side panel to the main panel 70
Figure 5.1. Examples of assembly activities 76
Figure 5.2. RULA risk prediction process 78
Figure 5.3. HyNet-BC architecture 84
Figure 5.4. 1D Convolution operation 84
Figure 5.5. The structure of one-layer LSTM 86
Figure 5.6. A LSTM cell 86
Figure 5.7. Bidirectional LSTM structure 88
Figure 5.8. Risk level during the assembly process from participant 1. 90
Figure 5.9. Class distribution. 91
Figure 5.10. Experimental procedure of missing value imputation 91
Figure 5.11. Missing value percentage in assemble main panel activity 92
Figure 5.12. Missing value percentage in assemble side panel activity 93
Figure 5.13. Missing value percentage in integrate panel activity 93
Figure 5.14. Missing value percentage in prepare the workspace activity 94
Figure 5.15. Missing value percentage in slide the mid panel activity 94
Figure 5.16. The confusion matrices of HBU-LSTM (a) and HyNet-CB (b) using the imbalanced data and cross-entropy loss function... 98
Figure 5.17. The confusion matrices of HBU-LSTM (a) and HyNet-CB (b) using the imbalanced data and focal loss function... 99
Figure 5.18. The confusion matrices of HBU-LSTM (a) and HyNet-CB (b) with the KD-SMOTE oversampling method... 102
Figure 5.19. The confusion matrices of Bi-LSTM (a) and HyNet-CB (b) with the KD-SMOTE oversampling method (KNN imputer data). 103
Figure 6.1. XAI analysis process 107
Figure 6.2. Data transformation 108
Figure 6.3. HyNet-CBA architecture 110
Figure 6.4. Performance evaluation with other deep learning models 117
Figure 6.5. Performance evaluation with traditional machine learning algorithms 117
Figure 6.6. Ten most important features of the HyNet-CBA model in overall prediction 119
Figure 6.7. Ten most important features for instance in class "Fair" 120
Figure 6.8. Ten most important features for instance in class "Poor" 121
Figure 6.9. Ten most important features for instance in class "Bad" 121
Figure 6.10. Waterfall plot of random sample in class "Fair" 122
Figure 6.11. Waterfall plot of random sample in class "Poor" 122
Figure 6.12. Waterfall plot of random sample in class "Bad" 123
Figure 6.13. Force plot from 500 instances 124
Figure 6.14. Class distribution of manufacturing dataset 125
Figure 6.15. Confusion matrix of activity recognition 127
Figure 6.16. Ten most important features of HyNet-CBA model in activity recognition 128
Figure A.1. RULA score annotation by expert 144