TY  - JOUR
A1  - Engelhardt, Lena
A1  - Naumann, Johannes
A1  - Goldhammer, Frank
A1  - Frey, Andreas
A1  - Horz, Holger
A1  - Hartig, Katja
A1  - Wenzel, S. Franziska C.
T1  - Development and evaluation of a framework for the performance-based testing of ICT skills
T2  - Frontiers in education
N2  - This paper addresses the development of performance-based assessment items for ICT skills, skills in dealing with information and communication technologies, a construct which is rather broadly and only operationally defined. Item development followed a construct-driven approach to ensure that test scores could be interpreted as intended. Specifically, ICT-specific knowledge as well as problem-solving and the comprehension of text and graphics were defined as components of ICT skills and cognitive ICT tasks (i.e., accessing, managing, integrating, evaluating, creating). In order to capture the construct in a valid way, design principles for constructing the simulation environment and response format were formulated. To empirically evaluate the very heterogeneous items and detect malfunctioning items, item difficulties were analyzed and behavior-related indicators with item-specific thresholds were developed and applied. The 69 item’s difficulty scores from the Rasch model fell within a comparable range for each cognitive task. Process indicators addressing time use and test-taker interactions were used to analyze whether most test-takers executed the intended processes, exhibited disengagement, or got lost among the items. Most items were capable of eliciting the intended behavior; for the few exceptions, conclusions for item revisions were drawn. The results affirm the utility of the proposed framework for developing and implementing performance-based items to assess ICT skills.
KW  - information and communication technology skills
KW  - assessment framework
KW  - performance items
KW  - validation
KW  - behavioral indicators
Y1  - 2021
UR  - http://publikationen.ub.uni-frankfurt.de/frontdoor/index/index/docId/62468
UR  - https://nbn-resolving.org/urn:nbn:de:hebis:30:3-624682
SN  - 2504-284X
N1  - This work was supported by the German Federal Ministry of Education and Research (grant numbers: 01LSA010, 01LSA010A, 01LSA010B). The publication of this article was funded by the Open Access Fund of the Leibniz Association.
VL  - 6
IS  - art. 668860
SP  - 1
EP  - 15
PB  - Frontiers Media
CY  - Lausanne
ER  -