1887

OECD Education Working Papers

This series is designed to make available to a wider readership selected studies drawing on the work of the OECD Directorate for Education. Authorship is usually collective, but principal writers are named. The papers are generally available only in their original language (English or French) with a short summary available in the other.

English, French

Item characteristics and test-taker disengagement in PISA

If test-takers do not engage with the assessment, the reliability of test scores and the validity of inferences about their proficiency may suffer. Test-taker disengagement is particularly likely in low-stakes assessments and, according to prior research, for certain types of students. But levels of engagement may also be related to aspects that test developers can manipulate, such as item characteristics. This paper investigates which item characteristics are associated with two indicators of test-taker disengagement, rapid guessing and breakoffs, in an international assessment of reading. Analyses of data from almost 500 000 students from 67 countries and economies that took part in the 2018 Programme for International Student Assessment (PISA) show that rapid guessing was observed mainly on simple multiple-choice questions. Breakoffs were more likely in the presence of idiosyncratic selected-response formats, such as hot spot or matching tasks. Both rapid guessing and breakoffs were more frequent on tasks involving long and complex texts.

English

This is a required field
Please enter a valid email address
Approval was a Success
Invalid data
An Error Occurred
Approval was partially successful, following selected items could not be processed due to error