Title page
Contents
Acknowledgement 4
Abstract 5
1. Design of international large-scale assessment 8
1.1. Purposes and outcomes of international large-scale assessments 8
1.2. Growing demands for multi-purpose utilisation of assessment 9
1.3. Item bank development in large-scale assessment 10
1.4. Objective and Overview 12
2. Towards more diverse assessments 13
2.1. Outcomes of cognitive assessments at different layers 13
2.2. Approaches to estimating outputs 16
2.3. Optimising instrument design 23
3. Towards more flexible assessment 27
3.1. Process of test implementation 27
3.2. Periodic assessment and sporadic assessment 29
3.3. Technical standard for item validation and parameter estimation 30
3.4. In-test trialling 33
3.5. Item cloning 34
4. Data illustration: PIAAC Education and Skills Online 35
4.1. Education and Skills Online 35
4.2. Prototype of new Education & Skills Online 38
4.3. Summary 42
5. Data illustration: PISA Household Survey Module 43
5.1. PISA for Development 43
5.2. Prototype of PISA Household Survey Module 46
5.3. Summary 49
6. Discussions 50
6.1. Towards more diverse assessments 50
6.2. Towards more flexible assessments 52
6.3. New tools and their limitations 54
6.4. Future development 54
References 56
A. Appendix 58
A.1. Evaluating the effects of sample size on LRM through numerical simulation 58
Table 1. Outcomes of cognitive assessments at different layers 15
Table 2. Approaches to estimating and reporting test scores 22
Table 3. Optimal instrument designs for the defined outcomes 26
Table 4. Test form design of the core part of the original E&S Online 36
Table 5. New Education & Skills Online: Certification version and Distribution version 39
Table 6. Test form design of Certification version of new E&S Online (Prototype) 40
Table 7. Test form design of Distribution version of new E&S Online (Prototype) 40
Table 8. Test form design of PISA-D 44
Table 9. Test form design of the 30-minute version of PISA-HSM 47
Table 10. Test form design of the 45-minute version of PISA-HSM 47
Figure 1. Standard errors (left figure) and 95% confidence intervals (right figure) of proficiency estimates in the Literacy domain of the original E&S Online 38
Figure 2. Standard errors (left figure) and 95% confidence intervals (right figure) of proficiency estimates in the Numeracy domain of het original E&S Online 38
Figure 3. Standard errors (left figure) and 95% confidence intervals (right figure) of proficiency estimates in the Literacy domain of the Certification and... 42
Figure 4. Standard errors (left figure) and 95% confidence intervals (right figure) of proficiency estimates in the Numeracy domain of the Certification and... 42
Figure 5. Standard errors (left figure) and 95% confidence intervals (right figure) of proficiency estimates in the READ domain of the PISA-D 45
Figure 6. Standard errors (left figure) and 95% confidence intervals (right figure) of proficiency estimates in the MATH domain of the PISA-D 46
Figure 7. Standard errors (left figure) and 95% confidence intervals (right figure) of proficiency estimates in the READ domain of the PISA-HSM 48
Figure 8. Standard errors (left figure) and 95% confidence intervals (right figure) of proficiency estimates in the MATH domain of the PISA-HSM 49
Figure 9. Residual standard deviations of the estimated latent regression models in MATH 59
Figure 10. Residual standard deviations of the estimated latent regression models in READ 59
Figure 11. Residual standard deviations of the estimated latent regression models in SCIE 59