Taking PISA Seriously: How Accurate are Low Stakes Exams?

PISA is seen as the gold standard for evaluating educational outcomes worldwide. Yet, being a low-stakes exam, students may not take it seriously resulting in downward biased scores and inaccurate rankings. This paper provides a method to identify and account for non-serious behavior in low-stakes exams by leveraging information in computer-based assessments in PISA 2015. We compare the score/rankings with no corrections to those generated using the PISA approach as well as our method which fully corrects for the bias. We show that the total bias is large and that the PISA approach corrects for only about half of it.