RESEARCH PAPER
A phenomenological study of challenges that prospective mathematics teachers face in developing mathematical problems that require higher-order thinking skills
 
More details
Hide details
1
Faculty of Mathematics and Natural Sciences, Yogyakarta State University, Special Region of Yogyakarta, INDONESIA
 
2
Faculty of Education, University of Malaya, Kuala Lumpur, MALAYSIA
 
3
Faculty of Management, Education and Humanities, University College MAIWP International, Kuala Lumpur, MALAYSIA
 
4
Faculty of Teacher Training and Education, Muhammadiyah University of Surakarta, Central Java, INDONESIA
 
 
Online publication date: 2023-09-02
 
 
Publication date: 2023-10-01
 
 
EURASIA J. Math., Sci Tech. Ed 2023;19(10):em2339
 
KEYWORDS
ABSTRACT
Assessment in mathematics education in the 21st century should be more directed at higher-order thinking skills (HOTS). Not only teachers but also prospective mathematics teachers should be supported to design and carry out HOTS assessments. This qualitative study applying a phenomenology approach was mainly conducted to investigate the challenges that prospective mathematics teachers face in developing HOTS questions, including their strategies for dealing with these challenges. Our informants were 20 students of master’s program in mathematics education. They were enrolled in the assessment of mathematics learning course, which was designed with a project-based learning model. Considering the learning model used, the purpose of this study then was extended to explore the benefits of implementing a project-based learning model in supporting student competence in developing HOTS questions. Data collection was carried out by (1) administering open-ended questionnaires; (2) observing the end product in the form of mathematics learning achievement tests and test blueprints; and (3) involving two experts who worked independently to judge the questions posed by students based on levels in the cognitive process dimension in the revised Bloom’s taxonomy. Data collected from the open-ended questionnaire were analyzed qualitatively, while data from observation and judgment by experts on the end product yielded by students were analyzed descriptively. Our study supports previous studies, which demonstrate that the application of a project-based learning model that involves students to develop HOTS questions deepens students’ knowledge of assessment. In developing HOTS questions, students struggled more in matching action verbs, item indicators, and test items to the level of cognitive process they defined, as well as constructing multiple-choice HOTS questions. The strategies that students took in dealing with challenges that arose in developing HOTS questions indicate that students regulate their learning.
 
REFERENCES (88)
1.
Al-Busaidi, S., & Al-Seyabi, F. (2021). Project-based learning as a tool for student-teachers’ professional development: A study in an Omani EFL teacher education program. International Journal of Learning, Teaching and Educational Research, 20(4), 116-136. https://doi.org/10.26803/ijlte....
 
2.
Almulla, M. A. (2020). The effectiveness of the project-based learning (PBL) approach as a way to engage students in learning. SAGE Open, 10(3), 1-15. https://doi.org/10.1177/215824....
 
3.
Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. Addison-Wesley Longman.
 
4.
Apino, E., & Retnawati, H. (2017). Developing instructional design to improve mathematical higher order thinking skills of students. Journal of Physics: Conference Series, 812(1), 1-7. https://doi.org/10.1088/1742-6....
 
5.
Barak, M., & Shakhman, L. (2008). Reform-based science teaching: Teachers’ instructional practices and conceptions. EURASIA Journal of Mathematics, Science and Technology Education, 4(1), 11–20. https://doi.org/10.12973/ejmst....
 
6.
Bell, S. (2010). Project-based learning for the 21st century: Skills for the future. The Clearing House: A Journal of Educational Strategies, Issues and Ideas, 83(2), 39-43. https://doi.org/10.1080/000986....
 
7.
Blumenfeld, P. C., Soloway, E., Marx, R. W., Krajcik, J. S., Guzdial, M., & Palincsar, A. (1991). Motivating project-based learning: Sustaining the doing, supporting the learning. Educational Psychologist, 26(3-4), 369-398. https://doi.org/10.1080/004615....
 
8.
Bogdan, R. C., & Biklen, S. K. (2007). Qualitative research for education: An introduction to theory and methods (5th edn.). Pearson Education.
 
9.
Brookhart, S. M. (2010). How to assess higher-order thinking skills in your classroom. ASCD.
 
10.
Chen, S.-Y., Lai, C.-F., Lai, Y.-H., & Su, Y.-S. (2022). Effect of project-based learning on development of students’ creative thinking. The International Journal of Electrical Engineering & Education, 59(3), 232-250. https://doi.org/10.1177/002072....
 
11.
Cohen, R. J., & Swerdlik, M. E. (2018). Psychological testing and assessment: An introduction to tests and measurement. McGraw-Hill Education.
 
12.
Corden, A., & Sainsbury, R. (2006). Exploring ‘quality’: Research participants’ perspectives on verbatim quotations. International Journal of Social Research Methodology, 9(2), 97-110. https://doi.org/10.1080/136455....
 
13.
Crespí, P., García-Ramos, J. M., & Queiruga-Dios, M. (2022). Project-based learning (PBL) and its impact on the development of interpersonal competences in higher education. Journal of New Approaches in Educational Research, 11(2), 259-276. https://doi.org/10.7821/naer.2....
 
14.
Creswell, J. W. (2013). Qualitative inquiry and research design: Choosing among five approaches. SAGE.
 
15.
Dahlan, D., Permana, L., & Oktariani, M. (2020). Teacher’s competence and difficulties in constructing HOTS instruments in economics subject. Jurnal Cakrawala Pendidikan [Journal of Educational Horizons], 39(1), 111-119. https://doi.org/10.21831/cp.v3....
 
16.
DiBattista, D., & Kurzawa, L. (2011). Examination of the quality of multiple-choice items on classroom tests. The Canadian Journal for the Scholarship of Teaching and Learning, 2(2), 1-23. https://doi.org/10.5206/cjsotl....
 
17.
Djidu, H., Jailani, J., & Retnawati, H. (2021). Higher-order thinking skills among male and female students: An experimental study of the problem-based calculus learning model in secondary schools. Beta: Jurnal Tadris Matematika [Beta: Journal of Mathematics Education], 14(2), 107-125. https://doi.org/10.20414/betaj....
 
18.
Drijvers, P., Kodde-Buitenhuis, H., & Doorman, M. (2019). Assessing mathematical thinking as part of curriculum reform in the Netherlands. Educational Studies in Mathematics, 102(3), 435-456. https://doi.org/10.1007/s10649....
 
19.
Evidiasari, S., Subanji, S., & Irawati, S. (2019). Students’ spatial reasoning in solving geometrical transformation problems. Indonesian Journal on Learning and Advanced Education, 1(2), 38-51. https://doi.org/10.23917/ijola....
 
20.
Forster, M. (2004). Higher order thinking skills. Research Developments, 11(11), 10-15.
 
21.
Frank, M., Lavy, I., & Elata, D. (2003). Implementing the project-based learning approach in an academic engineering course. International Journal of Technology and Design Education, 13(3), 273-288. https://doi.org/10.1023/A:1026....
 
22.
Gierl, M. J., Bulut, O., Guo, Q., & Zhang, X. (2017). Developing, analyzing, and using distractors for multiple-choice tests in education: A comprehensive review. Review of Educational Research, 87(6), 1082-1116. https://doi.org/10.3102/003465....
 
23.
Gudmundsdottir, S., & Shulman, L. (1987). Pedagogical content knowledge in social studies. Scandinavian Journal of Educational Research, 31(2), 59-70. https://doi.org/10.1080/003138....
 
24.
Gumartifa, A., Syahri, I., Siroj, R. A., Nurrahmi, M., & Yusof, N. (2023). Perception of teachers regarding problem-based learning and traditional method in the classroom learning innovation process. Indonesian Journal on Learning and Advanced Education (IJOLAE), 5(2), 151-166. https://doi.org/10.23917/ijola....
 
25.
Guo, P., Saab, N., Post, L. S., & Admiraal, W. (2020). A review of project-based learning in higher education: Student outcomes and measures. International Journal of Educational Research, 102, 1-13. https://doi.org/10.1016/j.ijer....
 
26.
Guo, S., & Yang, Y. (2012). Project-based learning: An effective approach to link teacher professional development and students learning. Journal of Educational Technology Development and Exchange, 5(2). https://doi.org/10.18785/jetde....
 
27.
Herawati, A. (2018). The application of project-based learning in teachers’ professional development program. In Proceedings of the 6th International Conference on Information and Education Technology (pp. 119-123). https://doi.org/10.1145/317815....
 
28.
Heron, M., & Palfreyman, D. M. (2021). Exploring higher-order thinking in higher education seminar talk. College Teaching. https://doi.org/10.1080/875675....
 
29.
Hill, H. C., Ball, D. L., & Schilling, S. G. (2008). Unpacking pedagogical content knowledge: Conceptualizing and measuring teachers’ topic-specific knowledge of students. Journal for Research in Mathematics Education, 39(4), 372-400. https://doi.org/10.5951/jresem....
 
30.
Hmelo, C. E., & Ferrari, M. (1997). The problem-based learning tutorial: Cultivating higher order thinking skills. Journal for the Education of the Gifted, 20(4), 401-422. https://doi.org/10.1177/016235....
 
31.
Holmes, V.-L., & Hwang, Y. (2016). Exploring the effects of project-based learning in secondary mathematics education. The Journal of Educational Research, 109(5), 449-463. https://doi.org/10.1080/002206....
 
32.
Ijirana, I., Mansyur, J., Rizal, M., & Aminah, S. (2021). Longitudinal study of metacognitive skills and external representation of students in the context of problem-solving. Indonesian Journal on Learning and Advanced Education, 3(3), 194-206. https://doi.org/10.23917/ijola....
 
33.
Jailani, J., Sugiman, S., & Apino, E. (2017). Implementing the problem-based learning in order to improve the students’ HOTS and characters. Jurnal Riset Pendidikan Matematika [Journal of Mathematics Education Research], 4(2), 247-259. https://doi.org/10.21831/jrpm.....
 
34.
Kaur, S. (2019). Role of a teacher in student learning process. International Journal of Business and Management Invention, 8(12), 41-45.
 
35.
King, B., & Smith, C. (2020). Using project-based learning to develop teachers for leadership. The Clearing House: A Journal of Educational Strategies, Issues and Ideas, 93(3), 158-164. https://doi.org/10.1080/000986....
 
36.
Krathwohl, D. R. (2002). A revision of Bloom’s taxonomy: An overview. Theory into Practice, 41(4), 212-218. https://doi.org/10.1207/s15430....
 
37.
Kruszewska, A. (2021). Teachers’ understanding of the role of key competences in working with children: A study of the perceptions of early years educators in Southern Poland. Education 3-13: International Journal of Primary, Elementary and Early Years Education, 49(6), 650-660. https://doi.org/10.1080/030042....
 
38.
Kyriacou, C. (2007). Essential teaching skills. Nelson Thornes.
 
39.
Lewis, A., & Smith, D. (1993). Defining higher order thinking. Theory into Practice, 32(3), 131-137. https://doi.org/10.1080/004058....
 
40.
Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. SAGE. https://doi.org/10.1016/0147-1....
 
41.
Liu, J., Ma, Y., Sun, X., Zhu, Z., & Xu, Y. (2022). A systematic review of higher-order thinking by visualizing its structure through HistCite and CiteSpace software. The Asia-Pacific Education Researcher, 31(6), 635-645. https://doi.org/10.1007/s40299....
 
42.
Machromah, I., Setyaningsih, R., Sari, C. K., Swastika, A., & Khoirunnisa, M. (2019). HOTS problem posing on mathematics pre-service teachers. In Proceedings of the 4th Progressive and Fun Education International Conference (pp. 1-11). https://doi.org/10.4108/eai.7-....
 
43.
Magnusson, S., Krajcik, J., & Borko, H. (1999). Nature, sources, and development of pedagogical content knowledge for science teaching. In J. Gess-Newsome, & N. G. Lederman (Eds.), Examining pedagogical content knowledge (pp. 95-132). Kluwer Academic Publishers. https://doi.org/10.1007/0-306-....
 
44.
Marczewska, M., Weresa, M. A., & Lachowicz, M. (2023). Towards creativity and innovation in universities: Study on Central and Eastern Europe. Journal of the Knowledge Economy. https://doi.org/10.1007/s13132....
 
45.
Miller, E. C., & Krajcik, J. S. (2019). Promoting deep learning through project-based learning: A design problem. Disciplinary and Interdisciplinary Science Education Research, 1(1), 7. https://doi.org/10.1186/s43031....
 
46.
Mishra, P., & Koehler, M. J. (2006). Technological pedagogical content knowledge: A framework for teacher knowledge. Teachers College Record, 108(6), 1017-1054.
 
47.
Newmann, F. M. (1991). Organizational features and classroom thoughtfulness in secondary school social studies departments. Theory & Research in Social Education, 19(4), 391-409. https://doi.org/10.1080/009331....
 
48.
Newton, P. M., Da Silva, A., & Peters, L. G. (2020). A pragmatic master list of action verbs for Bloom’s taxonomy. Frontiers in Education, 5, 1-6. https://doi.org/10.3389/feduc.....
 
49.
Nikolaeva, S. (2012). Improving initial teacher education by using the project-based approach. Educational Research, 1(1), 51-60. https://doi.org/10.5838/erej.2....
 
50.
Nitko, A. J., & Brookhart, S. M. (2011). Educational assessment of students. Pearson Education.
 
51.
Park, S., & Oliver, J. S. (2008). Revisiting the conceptualization of pedagogical content knowledge (PCK): PCK as a conceptual tool to understand teachers as professionals. Research in Science Education, 38(3), 261-284. https://doi.org/10.1007/s11165....
 
52.
Purwasih, J. H. G. (2020). Kendala calon pendidik dalam membuat soal pilihan ganda higher order thinking (HOT) [Obstacles of prospective educators in constructing higher order thinking (HOT) multiple-choice questions]. Jurnal Sosial Humaniora [Journal of Social Humanities], 13(1), 12-22. https://doi.org/10.12962/j2443....
 
53.
Radmehr, F., & Vos, P. (2020). Issues and challenges of 21st century assessment in mathematics education. In L. Leite, E. Oldham, A. S. Afonso, F. Viseu, L. Dourado, & H. Martinho (Eds.), Science and mathematics education for 21st century citizens: Challenges and ways forward (pp. 437-462). Nova Science Publishers.
 
54.
Radović, S., Firssova, O., Hummel, H. G. K., & Vermeulen, M. (2021). Strengthening the ties between theory and practice in higher education: An investigation into different levels of authenticity and processes of re- and de-contextualization. Studies in Higher Education, 46(12), 2710-2725. https://doi.org/10.1080/030750....
 
55.
Rafi, I., & Sugiman, S. (2019). Problem posing ability among prospective mathematics teachers. Pythagoras: Jurnal Pendidikan Matematika [Pythagoras: Journal of Mathematics Education], 14(2), 188-198. https://doi.org/10.21831/pg.v1....
 
56.
Rafi, I., Retnawati, H., Apino, E., Hadiana, D., Lydiati, I., & Rosyada, M. N. (2023). What might be frequently overlooked is actually still beneficial: Learning from post national-standardized school examination. Pedagogical Research, 8(1), 1-15. https://doi.org/10.29333/pr/12....
 
57.
Retnawati, H. (2016). Analisis kuantitatif instrumen penelitian [Quantitative analysis of research instrument]. Parama Publishing.
 
58.
Retnawati, H., Djidu, H., Kartianom, K., Apino, E., & Anazifa, R. D. (2018). Teachers’ knowledge about higher-order thinking skills and its learning strategy. Problems of Education in the 21st Century, 76(2), 215-230. https://doi.org/10.33225/pec/1....
 
59.
Reynolds, C. R., Livingston, R. B., & Willson, V. (2010). Measurement and assessment in education. Pearson.
 
60.
Rianasari, V. F., & Apriani, M. S. (2019). Pre-service mathematics teachers’ knowledge about higher-order thinking skills. Journal of Physics: Conference Series, 1366(1), 1-9. https://doi.org/10.1088/1742-6....
 
61.
Richland, L. E., & Simms, N. (2015). Analogy, higher order thinking, and education. Wiley Interdisciplinary Reviews: Cognitive Science, 6(2), 177-192. https://doi.org/10.1002/wcs.13....
 
62.
Sa’adah, L., & Anjarwati, R. (2022). High order thinking skill in pre-service teachers’ lesson plan. International Journal of Education, Language, and Religion, 4(1), 37-46. https://doi.org/10.35308/ijelr....
 
63.
Saepuzaman, D., Retnawati, H., Istiyono, E., & Haryanto, H. (2022). Developing HOTS instruments: Is it difficult for physics teachers? Jurnal Pendidikan MIPA [MIPA Education Journal], 23(4), 1802-1826. https://doi.org/10.23960/jpmip....
 
64.
Saimon, M., Lavicza, Z., & Dana-Picard, T. (2022). Enhancing the 4Cs among college students of a communication skills course in Tanzania through a project-based learning model. Education and Information Technologies, 28, 6269-6285. https://doi.org/10.1007/s10639....
 
65.
Schulz, H., & FitzPatrick, B. (2016). Teachers’ understandings of critical and higher order thinking and what this means for their teaching and assessments. Alberta Journal of Educational Research, 62(1), 61-86.
 
66.
Scouller, K. (1998). The influence of assessment method on students’ learning approaches: Multiple choice question examination versus assignment essay. Higher Education, 35(1), 453-472. https://doi.org/10.1023/A:1003....
 
67.
Scully, D. (2017). Constructing multiple-choice items to measure higher-order thinking. Practical Assessment ,Research, and Evaluation, 22(4), 1-13. https://doi.org/10.7275/swgt-r....
 
68.
Shin, J., Guo, Q., & Gierl, M. J. (2019). Multiple-choice item distractor development using topic modeling approaches. Frontiers in Psychology, 10(1), 1-14. https://doi.org/10.3389/fpsyg.....
 
69.
Shulman, L. S. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15(2), 4-14. https://doi.org/10.3102/001318....
 
70.
Shulman, L. S. (1987). Knowledge and teaching: Foundations of the new reform. Harvard Educational Review, 57(1), 1-23. https://doi.org/10.17763/haer.....
 
71.
Sideeg, A. (2016). Bloom’s taxonomy, backward design, and Vygotsky’s zone of proximal development in crafting learning outcomes. International Journal of Linguistics, 8(2), 158-186. https://doi.org/10.5296/ijl.v8....
 
72.
Simkin, M. G., & Kuechler, W. L. (2005). Multiple-choice tests and student understanding: What is the connection? Decision Sciences Journal of Innovative Education, 3(1), 73-98. https://doi.org/10.1111/j.1540....
 
73.
Singer, F. M., Ellerton, N., & Cai, J. (2013). Problem-posing research in mathematics education: New questions and directions. Educational Studies in Mathematics, 83(1), 1-7. https://doi.org/10.1007/s10649....
 
74.
Sinta, U. A., Roebyanto, G., & Nuraini, N. L. S. (2022). Analisis kesulitan guru dalam menyusun soal evaluasi berbasis HOTS pada pembelajaran matematika di SDN Torongrejo 2 [Analysis of teacher difficulties in compiling HOTS-based evaluation questions in mathematics learning at SDN Torongrejo 2]. Jurnal Pembelajaran, Bimbingan, dan Pengelolaan Pendidikan [Journal of Learning, Guidance, and Education Management], 2(1), 45-53. https://doi.org/10.17977/um065....
 
75.
Stahl, N. A., & King, J. R. (2020). Expanding approaches for research: Understanding and using trustworthiness in qualitative research. Journal of Developmental Education, 44(1), 26-28.
 
76.
Stanger-Hall, K. F. (2012). Multiple-choice exams: An obstacle for higher-level thinking in introductory science classes. CBE-Life Sciences Education, 11(3), 294-306. https://doi.org/10.1187/cbe.11....
 
77.
Stefanou, C., Stolk, J. D., Prince, M., Chen, J. C., & Lord, S. M. (2013). Self-regulation and autonomy in problem- and project-based learning environments. Active Learning in Higher Education, 14(2), 109-122. https://doi.org/10.1177/146978....
 
78.
Suherman, S., Prananda, M. R., Proboningrum, D. I., Pratama, E. R., Laksono, P., & Amiruddin, A. (2020). Improving higher order thinking skills (HOTS) with project based learning (PjBL) model assisted by GeoGebra. Journal of Physics: Conference Series, 1467, 012027. https://doi.org/10.1088/1742-6....
 
79.
Takiddin, T., Jalal, F., & Neolaka, A. (2020). Improving higher order thinking skills through project-based learning in primary schools. Tarbiya: Journal of Education in Muslim Society, 7(1), 16-28. https://doi.org/10.15408/tjems....
 
80.
Teherani, A., Martimianakis, T., Stenfors-Hayes, T., Wadhwa, A., & Varpio, L. (2015). Choosing a qualitative research approach. Journal of Graduate Medical Education, 7(4), 669-670. https://doi.org/10.4300/JGME-D....
 
81.
Thompson, T. (2008). Mathematics teachers’ interpretation of higher-order thinking in Bloom’s taxonomy. International Electronic Journal of Mathematics Education, 3(2), 96-109. https://doi.org/10.29333/iejme....
 
82.
Tsybulsky, D., & Muchnik-Rozanov, Y. (2021). Project-based learning in science-teacher pedagogical practicum: The role of emotional experiences in building preservice teachers’ competencies. Disciplinary and Interdisciplinary Science Education Research, 3(9), 9. https://doi.org/10.1186/s43031....
 
83.
Van den Berg, G. (2004). The use of assessment in the development of higher-order thinking skills. Africa Education Review, 1(2), 279-294. https://doi.org/10.1080/181466....
 
84.
Winarti, W., Hairida, H., & Lestari, I. (2021). Deskripsi kemampuan guru membuat soal berdasarkan pada kurikulum 2013 di sekolah menengah atas Kabupaten Landak [A description of the teacher’s ability to create questions based on the curriculum 2013 in high schools in Landak District]. Jurnal Ilmiah Wahana Pendidikan [Scientific Journal of Educational Forum], 7(2), 108-115. https://doi.org/10.5281/zenodo....
 
85.
Wisrance, M. W., & Semiun, T. T. (2020). LOTS and HOTS of teacher-made test in junior high school level in Kefamenanu. Journal of English Education, 6(2), 62-76. https://doi.org/10.30606/jee.v....
 
86.
Wrenn, J., & Wrenn, B. (2009). Enhancing learning by integrating theory and practice. International Journal of Teaching and Learning in Higher Education, 21(2), 258-265.
 
87.
Zhou, Y., Gan, L., Chen, J., Wijaya, T. T., & Li, Y. (2023). Development and validation of a higher-order thinking skills assessment scale for pre-service teachers. Thinking Skills and Creativity, 48(2), 1-20. https://doi.org/10.1016/j.tsc.....
 
88.
Zohar, A., & Dori, Y. J. (2003). Higher order thinking skills and low-achieving students: Are they mutually exclusive? Journal of the Learning Sciences, 12(2), 145-181. https://doi.org/10.1207/S15327....
 
eISSN:1305-8223
ISSN:1305-8215
Journals System - logo
Scroll to top