Title Page
Contents
LIST OF ABBREVIATIONS 10
Chapter 1. Introduction 14
Chapter 2. Linguistic Features in Proficiency Evaluation 16
2.1. Accuracy 17
2.2. Fluency 18
Chapter 3. Experiment Procedures 20
3.1. Data 20
3.1.1. Speakers 20
3.1.2. Sentences 21
3.2. Experiment I: Preliminary Experiment 21
3.2.1. Data Subset 21
3.2.2. Participants 22
3.2.3. Methods 22
3.3. Experiment II: Main Experiment 23
3.3.1. Data Subset 23
3.3.2. Participants 24
3.3.3. Methods 24
3.4. Speech Segmentation 25
Chapter 4. Results 27
4.1. Experiment I: Preliminary Experiment 27
4.1.1. Proficiency Scores of Each Learner 27
4.1.2. Reliability: Correlation between the Raters 28
4.1.3. Analysis of Speech Rate 29
4.2. Experiment II: Main Experiment 31
4.2.1. Proficiency Scores of Each Learner 31
4.2.2. Reliability: Correlation between the Raters 33
4.2.3. Naturalness Analysis 35
4.2.4. Survey Results 36
4.3. Linguistic Features in Proficiency Evaluation 39
4.3.1. Linguistic Features: Accuracy 39
4.3.2. Linguistic Features: Fluency 41
4.4. Rhythm Metrics Reflecting Segmental or Suprasegmental Elements 52
4.5. Statistical Analysis of Metrics 55
Chapter 5. Conclusion 57
References 60
Appendices 65
Appendix A. List of 36 Sentences 65
Appendix B. Means (standard deviations) of Each Speaker's Proficiency Score 67
Appendix C. Survey Results of Linguistic Elements of Accuracy in Evaluating English Proficiency (in Korean) 69
Appendix D. Survey Results of Linguistic Elements of Fluency in Evaluating English Proficiency (in Korean) 70
국문 초록 71
Table 1. Linguistic Features Corresponding to Accuracy 18
Table 2. Linguistic Features Corresponding to Fluency 19
Table 3. List of Sentences for Proficiency Evaluation in the First Experiment 22
Table 4. Means (standard deviations) of Each Speaker's Proficiency Score 28
Table 5. Correlation between the Raters 29
Table 6. Means (standard deviations) of Speech Rate 30
Table 7. Means (standard deviations) of Each Speaker's Proficiency Score 32
Table 8. Correlation between the Raters (graduate students) 34
Table 9. Correlation between the Raters (teachers) 34
Table 10. Correlation between the Experiments (S: graduate students; T: teachers) 34
Table 11. Accuracy, Hit Rates, and False Alarm Rates of Graduate Students' Responses 35
Table 12. Accuracy, Hit Rates, and False Alarm Rates of Teachers' Responses 36
Table 13. Survey Results on Linguistic Elements of Accuracy in Evaluating English Proficiency 37
Table 14. Survey Results on Linguistic Elements of Fluency in Evaluating English Proficiency 38
Table 15. Means (standard deviations) of the Number of Segmental Errors 40
Table 16. The Number (ratio) of Segmental Error Types per Group 40
Table 17. Means (standard deviations) of Vowel Intensity and Duration per Group (sampled data) 43
Table 18. Means (standard deviations) of Vowel Intensity and Duration per Group (entire data) 44
Table 19. Definitions and Formulas of Rhythm Metrics and Expectations of the Rhythmic Characteristics for Korean EFL Learners 53
Table 20. Means (standard deviations) of Rhythm Metrics per Group 54
Table 21. Mean Value Difference, Parameters, and Significance of Fixed Effect 56
Figure 1. Example of Speech Segmentation in English 26
Figure 2. Boxplot of Speech Rate per Group in the First Experiment 30
Figure 3. Boxplots of Proficiency Score per Group in Each Experiment 33
Figure 4. Boxplots of Stressed and Unstressed Vowel Duration per Group (sampled data) 43
Figure 5. Boxplots of Stressed and Unstressed Vowel Duration per Group (entire data) 45
Figure 6. Pitch Contour Lines and a Linear Regression Line of a Declarative Sentence Spoken by Native Speakers (N), Advanced Learners (A), and Low-Level Learners (L) 47
Figure 7. Pitch Contour Lines and a Linear Regression Line of a Wh- Question Spoken by Native Speakers (N), Advanced Learners (A), and Low-Level Learners (L) 49
Figure 8. Pitch Contour Lines and a Linear Regression Line of a Yes/No Question Spoken by Native Speakers (N), Advanced Learners (A), and Low-Level Learners (L) 50
Figure 9. Boxplot of the Number of Pauses per Group 51
Figure 10. Boxplots of Rhythm Metrics per Group 54