Invariance of the response processes between gender and modes in an assessment of reading

  • In this paper, we developed a method to extract item-level response times from log data that are available in computer-based assessments (CBA) and paper-based assessments (PBA) with digital pens. Based on response times that were extracted using only time differences between responses, we used the bivariate generalized linear IRT model framework (B-GLIRT, [1]) to investigate response times as indicators for response processes. A parameterization that includes an interaction between the latent speed factor and the latent ability factor in the cross-relation function was found to fit the data best in CBA and PBA. Data were collected with a within-subject design in a national add-on study to PISA 2012 administering two clusters of PISA 2009 reading units. After investigating the invariance of the measurement models for ability and speed between boys and girls, we found the expected gender effect in reading ability to coincide with a gender effect in speed in CBA. Taking this result as indication for the validity of the time measures extracted from time differences between responses, we analyzed the PBA data and found the same gender effects for ability and speed. Analyzing PBA and CBA data together we identified the ability mode effect as the latent difference between reading measured in CBA and PBA. Similar to the gender effect the mode effect in ability was observed together with a difference in the latent speed between modes. However, while the relationship between speed and ability is identical for boys and girls we found hints for mode differences in the estimated parameters of the cross-relation function used in the B-GLIRT model.

Download full text files

Export metadata

Author:Joachim Ulf Kröhne, Carolin Hahnel, Frank GoldhammerORCiDGND
Parent Title (English):Frontiers in applied mathematics and statistics
Publisher:Frontiers Media
Place of publication:Lausanne
Contributor(s):Qiwei He
Document Type:Article
Year of Completion:2019
Date of first Publication:2019/02/18
Publishing Institution:Universitätsbibliothek Johann Christian Senckenberg
Release Date:2020/05/13
Tag:computer-based assessment; log data; mode-effects; paper-based assessment; reading ability; response times
Issue:Art. 2
Page Number:16
First Page:1
Last Page:16
Copyright © 2019 Kroehne, Hahnel and Goldhammer. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
Institutes:Erziehungswissenschaften / Erziehungswissenschaften
Psychologie und Sportwissenschaften / Psychologie
Angeschlossene und kooperierende Institutionen / Deutsches Institut für Internationale Pädagogische Forschung (DIPF)
Dewey Decimal Classification:3 Sozialwissenschaften / 37 Bildung und Erziehung / 370 Bildung und Erziehung
Licence (German):License LogoCreative Commons - Namensnennung 4.0