Development of the automatic item generation system for the diagnosis of misconceptions about force and laws of motion
More details
Hide details
Faculty of Education, Chulalongkorn University, Bangkok, THAILAND
Online publication date: 2023-05-08
Publication date: 2023-06-01
EURASIA J. Math., Sci Tech. Ed 2023;19(6):em2282
The understanding of force and laws of motion is a fundamental foundation for learning mechanics and understanding other complex physics-related subjects. Automatic item generation (AIG) is also suitable for generating items and able to reduce the chance of item exposure. We, thus, developed an AIG system for the diagnosis of misconceptions about force and laws of motion in order to create a large number of quality items that would be used to diagnose students’ misconceptions. AIG system that has been developed contains 18 item models; it can generate 320-3,200 test items. The system contains six menus, i.e., (1) users’ data, (2) item models, (3) item generation, (4) test generation, (5) the users’ guide, and (6) the system’s developer. Based on the examination of AIG system’s quality by experts on educational assessment and experts on information technology, AIG’s quality in terms of utility, feasibility, propriety, and accuracy is at the highest level. The system was improved using the two dimensions of users’ experiences with physics instructors, i.e., (1) pragmatic dimension and (2) hedonic dimension. This research offers an approach to developing AIG system that responds to users’ needs.
Aini, F. N., Sutopo, & Suyudi, A. (2021). Teaching integrated Newton’s laws of motion for high school students. AIP Conference Proceedings, 2330, 050013.
de la Torre, J., & Minchen, N. (2014). Cognitively diagnostic assessments and the cognitive diagnosis model framework. Psicología Educativa [Educational Psychology], 20(2), 89-97.
Embretson, S., & Yang, X. (2006). Automatic item generation and cognitive psychology. In C. R. Rao, & S. Sinharay (Eds.), Handbook of statistics (pp. 747-768). North Holland.
Gierl, M. J., & Lai, H. (2013). Instructional topics in educational measurement (ITEMS) module: Using automated processes to generate test items. Educational Measurement: Issues and Practice, 32(3), 36-50.
Gierl, M. J., & Lai, H. (2016). A process for reviewing and evaluating generated test items. Educational Measurement: Issues and Practice, 35(4), 6-20.
Gierl, M. J., & Lai, H. (2018). Using automatic item generation to create solutions and rationales for computerized formative testing. Applied Psychological Measurement, 42(1), 42-57.
Gierl, M. J., Zhou, J., & Alves, C. (2008). Developing a taxonomy of item model types to promote assessment engineering. The Journal of Technology, Learning and Assessment, 7(2), 1-51.
Graf, E. A., Peterson, S., Steffen, M., & Lawless, R. (2005). Psychometric and cognitive analysis as a basis for the design and revision of quantitative item models. ETS.
Gurel, D. K., Eryılmaz, A., & McDermott, L. C. (2015). A review and comparison of diagnostic instruments to identify students’ misconceptions in science. EURASIA Journal of Mathematics, Science and Technology Education, 11(5), 989-1008.
Hassenzahl, M. (2003). The thing and I: Understanding the relationship between user and product. In M. A. Blythe, K. Overbeeke, A. F. Monk, & P. C. Wright (Eds.), Funology: From usability to enjoyment (pp. 31-42). Kluwer Academic.
Hussain, A., Hussein, I., Mkpojiogu, E. O., & Sarlan, A. (2019). The state of user experience design (UXD) practice in Malaysia: An in-situ interview approach. International Journal of Innovative Technology and Exploring Engineering, 8(8S), 498-505.
Javidanmehr, Z., & Sarab, M. R. A. (2017). Cognitive diagnostic assessment: Issues and considerations. International Journal of Language Testing, 7(2), 73-98.
Kaniawati, I., Fratiwi, N. J., Danawan, A., Suyana, I., Samsudin, A., & Suhendi, E. (2019). Analyzing students’ misconceptions about Newton’s laws through four-tier Newtonian test (FTNT). Journal of Turkish Science Education, 16(1), 110-122.
Krueger, A. E., Pollmann, K., Fronemann, N., & Foucault, B. (2020). Guided user research methods for experience design–A new approach to focus groups and cultural probes. Multimodal Technologies and Interaction, 4(3), 1-22.
Lai, H., Gierl, M. J., Byrne, B. E., Spielman, A. I., & Waldschmidt, D. M. (2016). Three modeling applications to promote automatic item generation for examinations in dentistry. Journal of Dental Education, 80(3), 339-347.
Latifi, S., Gierl, M., Wang, R., Lai, H., & Wang, A. (2017). Information-based methods for evaluating the semantics of automatically generated test items. Artificial Intelligence Research, 6(1), 69-79.
Narjaikaew, P. (2013). Alternative conceptions of primary school teachers of science about force and motion. Procedia-Social and Behavioral Sciences, 88, 250-257.
National Research Council. (1997). Science teaching reconsidered: A handbook. National Academies Press.
Pellegrino, J. W., & Hilton, M. L. (2012). Education for life and work: Developing transferable knowledge and skills in the 21st century. National Academies Press.
Saglam-Arslan, A., & Devecioglu, Y. (2010). Student teachers’ levels of understanding and model of understanding about Newton’s laws of motion. Asia-pacific Forum on Science Learning and Teaching, 11(1), Article 7.
Sinharay, S., & Johnson, M. S. (2008). Use of item models in a large-scale admissions test: A case study. International Journal of Testing, 8(3), 209-236.
Sinharay, S., & Johnson, M. S. (2013). Statistical modeling of automatically generated items. In M. J. Gierl, & T. Haladyna (Eds.), Automatic item generation: Theory and practice (pp.183-195). Routledge.
Sornkhatha, P., & Srisawasdi, N. (2013). Supporting conceptual development in Newton’s laws of motion using an interactive computer-simulated laboratory environment. Procedia-Social and Behavioral Sciences, 93, 2010-2014.
Tang, R., Hu, Z., Henry, N., & Thomas, A. (2021). A usability evaluation of research data management librarian academy (RDMLA): Examining the impact of learner differences in pedagogical usability. Journal of Web Librarianship, 15(3), 154-193.
Thibaut, L., Ceuppens, S., De Loof, H., De Meester, J., Goovaerts, L., Struyf, A., Boeve-de Pauw, J., Dehaene, W., Deprez, J., De Cock, M., Hellinckx, L., Knipprath, H., Langie, G., Struyven, K., van de Velde, D., van Petegem, P., & Depaepe, F. (2018). Integrated STEM education: A systematic review of instructional practices in secondary education. European Journal of STEM Education, 3(1), 02.
Wancham, K., Tangdhanakanond, K., & Kanjanawasee, S. (2022). The construction and validation of the cognitive model of force and motion for a diagnosis of misconceptions. Journal of Education Naresuan University, 24(3), 60-70.
Wancham, K., Tangdhanakanond, K., & Kanjanawasee, S. (2023). Sex and grade issues in influencing misconceptions about force and laws of motion: An application of cognitively diagnostic assessment. International Journal of Instruction, 16(2), 437-456.
Yarbrough, D. B., Shula, L. M., Hopson, R. K., & Caruthers, F. A. (2011). The program evaluation standards: A guide for evaluators and evaluation users. SAGE.
Yasri, P. (2014). A systematic classification of student misconceptions in biological evolution. International Journal of Biology, 3(2), 31-41.
Journals System - logo
Scroll to top