The Effect of Using AI in Learning on Critical Thinking Skills Through Students’ Digital Literacy

  • Sutarsi Suhaeb Makassar State University, Indonesia
  • Nurfauziah Makassar State University, Indonesia
  • Nur Athiyyah Fadhilah Makassar State University, Indonesia
Keywords: Artificial Intelligence, Digital Literacy, Critical Thinking, SEM-PLS, College Students

Abstract

The increasing use of generative AI in university learning has raised important questions regarding its contribution to students’ cognitive development. Although applications such as ChatGPT, Gemini, and Claude can facilitate personalized and efficient learning, their educational value may be constrained when students lack sufficient digital literacy. To investigate these relationships, this study employed a quantitative cross-sectional survey involving active university students with prior experience using generative AI for learning. Data were collected through a structured five-point Likert questionnaire and examined through Partial Least Squares Structural Equation Modeling (PLS-SEM). The analysis involved evaluating the measurement properties of the constructs, examining the structural model, and estimating both direct and mediated effects. The findings confirmed satisfactory measurement quality for AI use, digital literacy, and critical thinking. AI utilization was positively related to students’ critical thinking, but its direct effect was comparatively small. A considerably stronger association was observed between AI utilization and digital literacy. In turn, digital literacy was positively associated with critical thinking. The mediation results further indicate that the contribution of AI utilization to critical thinking operates partly through students’ level of digital literacy, suggesting that the development of digital competencies is an important pathway connecting AI-supported learning with higher-order cognitive skills. This suggests that the contribution of AI to students’ critical thinking is strengthened when students possess adequate digital literacy. Therefore, higher education institutions should integrate digital literacy development into AI-supported learning, particularly by strengthening students’ abilities to verify information, recognize potential algorithmic bias, and regulate their use of AI. In this way, AI can function as a tool that supports intellectual development rather than replacing students’ critical thinking processes.

References

Andani, P., & Friyatmi. (2025). Pengaruh Pemanfaatan Artificial Intelligence dan Literasi Digital terhadap Kemampuan Berpikir Kritis. 8(1952), 6241–6246.

Andrian, R. D., Widianti, Y. M., & Wati, D. A. R. (2025). on Smart Learning Technologies The Role of Artificial Intelligence in Enhancing Critical Thinking Skills of Educational Technology Students : The Moderating Influence of Digital Literacy and Usage Regulation. Journal on Smart Learning Technologies, 1(2), 109–120.

Anita, Mulawarman, W. G., & Susilo. (2025). Cognitive Leap or Digital Divide? A Comparative Study on AI-Driven Learning and Student Analytical Capacity in Samarinda and Aceh. JPP Jurnal Pendidikan Progresif, 15(03), 1811–1828. https://doi.org/10.23960/jpp.v15i3.pp1811-1828

Arinto, M., Septian, Y., & Ihsani, H. R. (2026). STRENGTHENING STUDENTS’ READING LITERACY AND CRITICAL THINKING AS AN ADAPTIVE STRATEGY TO THE DEVELOPMENT OF ARTIFICIAL INTELLIGENCE IN THE DIGITAL AGE: A LITERATURE STUDY. Jurnal Inovasi Dan Teknologi Pendidikan JURINOTEP, 4(3), 735–751.

Arnu, A. P., & Sobandi, A. (2025). THE ROLE OF ARTIFICIAL INTELLIGENCE (AI) ON CRITICAL THINKING SKILLS IN A CASE STUDY OF STUDENTS OF THE FACULTY OF ECONOMICS AND BUSINESS (FEB) OF UNIVERSITAS SINGAPERBANGSA KARAWANG. PERSPEKTIF Ilmu Pendidikan, 39(1), 27–34.

Ateeq, A., & Alfiras, M. (2025). The impact of adaptive and interactive ai tools on student learning : From digital literacy to advanced skills. International Journal of Innovative Research and Scientific Studies, 8(6), 961–973. https://doi.org/10.53894/ijirss.v8i6.9775

Börekci, C., & Çelik, Ö. (2024). Exploring The Role of Digital Literacy in University Students’ Engagement with AI through the Technology Acceptance Model. Sakarya University Journal of Education, 14(Special Issue-AI in Education), 228–249. https://doi.org/10.19126/suje.1468866

Braz˜ao, P., & Tinaco, L. (2025). Artificial intelligence and critical thinking : a case study with educational chatbots. Frontiers in Education, 10(September), 1–12. https://doi.org/10.3389/feduc.2025.1630493

Chai, Ching Sing, Yu, Ding, King, Ronnel B, & Zhou, Ying. (2024). Development and Validation of the Artificial Intelligence Learning Intention Scale (AILIS) for University Students. Sage Open, 14(2), 21582440241242188. https://doi.org/10.1177/21582440241242188

Demir, R., & Akar, C. (2025). Transforming Clicks into Critical Thinking: An AI- Based Media Literacy Program for Children. Research Square. https://doi.org/10.21203/rs.3.rs-6786882/v1

Fakour, H., & Imani, M. (2024). Socratic wisdom in the age of AI : a comparative study of ChatGPT and human tutors in enhancing critical thinking skills. Frontiers in Education, 10(Januari), 01–12.

Fan, Y., Tang, L., Le, H., Shen, K., Tan, S., Zhao, Y., Shen, Y., Li, X., & Gašević, D. (2025). Beware of metacognitive laziness: Effects of generative artificial intelligence on learning motivation, processes, and performance. British Journal of Educational Technology, 56(2), 489–530. https://doi.org/https://doi.org/10.1111/bjet.13544

Fatih, M. (2025). Interaction Problem Based Learning of AI Assisted on Digital Literacy and Critical Thinking Abilities. Journal of Development Research, 9(1), 164–173.

Firdaus, J. A., Ummah, R. I., Aprialini, R. R., & ... (2025). Ketergantungan penggunaan kecerdasan buatan (AI) pada tugas akademik mahasiswa terhadap kemampuan berpikir kritis dan kreatif. Didaktika: Jurnal …, 14(1), 1203–1214. https://mail.jurnaldidaktika.org/contents/article/view/1634

Gerlich, M. (2025). AI Tools in Society: Impacts on Cognitive Offloading and the Future of Critical Thinking. Societies, 15(1). https://doi.org/10.3390/soc15010006

Handayani, J. R., Marsofiyati, M., & Adha, M. A. (2026). Pengaruh Penggunaan Artificial Intelligence terhadap Berpikir Kritis dengan Literasi Digital sebagai Variabel Mediasi Siswa di Era Society 5.0. Jurnal Pendidikan Dan Kebudayaan (JURDIKBUD), 6(1 SE-Articles), 44–64. https://doi.org/10.55606/jurdikbud.v6i1.10910

Hwang, H. S., Zhu, L. C., & Cui, Q. (2023). Development and Validation of a Digital Literacy Scale in the Artificial Intelligence Era for College Students. 17(8), 2241–2258.

Jasmadi, Saddam, & Lasri. (2024). Critical thinking: An analytical study on the impact of Artificial Intelligence (AI) usage on students at Al Washliyah Darussalam University, Banda Aceh. Sekumpul: Journal of Multidisciplinary Education Sciences, 2(1). https://doi.org/10.62568/jomes.v2i1.168

Kurniawati, S., Nuriansyah, F., Sukamto, R., Fadhillah, S., & Febriyanti, A. (2025). MEDIATING ROLE OF DIGITAL LITERACY ON THE EFFECT OF AI GENERATED USAGE ON CRITICAL THINKING SKILL. The International Conference on Sustainable Economics Management and Accounting Proceeding, 1, 3534–3542. https://doi.org/10.32424/icsema.1.1.458

Lawasi, M., Rohman, V., & Shoreamanis, M. (2024). The Use of AI in Improving Student’s Critical Thinking Skills. Proceedings Series on Social Sciences & Humanities, 18, 366–370. https://doi.org/10.30595/pssh.v18i.1279

Li, S., Tang, S., Geng, X., & Liu, Q. (2022). Constructing a critical thinking evaluation framework for college students majoring in the humanities. Frontiers in Psychology, Volume 13-2022. https://doi.org/10.3389/fpsyg.2022.1017885

Marengo, A., Karaoglan-Yilmaz, F. G., Yılmaz, R., & Ceylan, M. (2025). Development and validation of generative artificial intelligence attitude scale for students. Frontiers in Computer Science, Volume 7-2025. https://doi.org/10.3389/fcomp.2025.1528455

Nasution, J. S., Siregar, A. M., Hasibuan, E. S., Difla, F., Azizah, T. N., Negeri, I., & Utara, S. (2025). Dampak Negatif Penggunaan AI Terhadap Mahasiswa Dalam Proses Pembelajaran. AMI Jurnal Pendidikan Dan Riset, 3(1), 35–42.

Ng, D. T. K., Wu, W., Leung, J. K. L., Chiu, T. K. F., & Chu, S. K. W. (2024). Design and validation of the AI literacy questionnaire: The affective, behavioural, cognitive and ethical approach. British Journal of Educational Technology, 55(3), 1082–1104. https://doi.org/https://doi.org/10.1111/bjet.13411

Nurfauziah, Atmasani, D., Patawari, A. M., Fadhilah, N. A., & Saleh, H. (2025). Artificial Intelligence in Higher Education : A Systematic Review of its Impact on Student Learning. Journal of Information Technology Education, 4(2), 205–216.

Panduwinata, L., & Setiawati, F. (2024). Digital Literacy’s Influence on Critical Thinking Skills of Office Administration Education Students at UNESA. Jurnal Simki Pedagogia, 7(2 SE-Articles), 609–618. https://doi.org/10.29407/jsp.v7i2.733

Payan-Carreira, R., Sacau-Fontenla, A., Rebelo, H., Sebastião, L., & Pnevmatikos, D. (2022). Development and Validation of a Critical Thinking Assessment-Scale Short Form. In Education Sciences (Vol. 12, Issue 12, p. 938). https://doi.org/10.3390/educsci12120938

Putri, J. A., Arpizal, A., & Ulfah, F. (2025). Pengaruh Penggunaan Artificial Intelligence (AI) dan Literasi Digital terhadap Keterampilan Berfikir Kritis Mahasiswa Pendidikan Ekonomi Universitas Jambi Angkatan 2023 di Era Digital. Jurnal Pendidikan Tambusai, 9(2 SE-Articles of Research), 13008–13016. https://doi.org/10.31004/jptam.v9i2.26908

Ramadhan, M. A., Gunawan, A., Lorenza, S., Ainy, Z., & Subhan, M. (2025). Analisis Dampak Penggunaan Artificial Intelligence Terhadap Kemampuan Berpikir Kritis Mahasiswa. JURNAL MUDABBIR, 5(2), 241–249.

Rodríguez-Rojas, María-Paula, Ramírez-Segura, Ana-María, Valenzuela-Mora, Pedro, Dampuré, Julien, & Riveros Munévar, Fernando. (2024). Critical Thinking Evaluation Scale: Design and Validation in a Colombian Population. Sage Open, 14(4), 21582440241297416. https://doi.org/10.1177/21582440241297418

Santana, A., Agnes, A., Yani, D., Ananda, D., Bella, E., Harahap, F. I., & Amalia, N. (2025). PENGARUH PENGGUNAAN ARTIFICIAL INTELEGENCE (AI) TERHADAP KEMAMPUAN BERPIKIR KRITIS DAN KREATIF MAHASISWA JURUSAN BAHASA DAN SASTRA INDONESIA 1Richo. Pendas : Jurnal Ilmiah Pendidikan Dasar, 10(2), 385–396.

Sardi, J. (2025). How Generative AI Influences Students’ Self-Regulated Learning and Critical Thinking Skills? A Systematic Review. IJEP International Journal Of Engineering Pedagogy, 15(1), 94–108.

Sitompul, B., Purba, S., Tamba, P. M., & Sianturi, F. (2025). Kemampuan Berpikir Kritis terhadap Literasi Digital Mahasiswa. Jurnal Pendidikan Tambusai, 9(1 SE-Articles of Research), 10154–10161. https://doi.org/10.31004/jptam.v9i1.26241

Supriyadi, & Nasution, Z. (2024). Teknologi Artificial Intelligence (AI) dan Literasi Digital Mahasiswa terhadap Hasil Belajar Mata Kuliah Evaluasi Pembelajaran: Artificial Intelligence, Literasi Digital, Evaluasi Pembelajaran. Jurnal TEKNODIK, 28(2), 113–118. https://doi.org/10.32550/teknodik.vi.1185

Vernanda, C., Dewi, V. C., Jayanti, W. E., Bina, U., & Informatika, S. (2025). PENGARUH ARTIFICIAL INTELLEGENCE ( AI ) TERHADAP KETERAMPILAN BERFIKIR KRITIS PELAJAR ATAU MAHASISWA. Journal of Information Systems Management and Digital Business (JISMDB), 2(4), 346–357.

Wang, B., Rau, P.-L. P., & Yuan, T. (2023). Measuring user competence in using artificial intelligence: validity and reliability of artificial intelligence literacy scale. Behaviour & Information Technology, 42(9), 1324–1337. https://doi.org/10.1080/0144929X.2022.2072768

Zhou, X., Sun, K., Zhu, K., Feng, L., Sun, Q., & Zhong, D. (2025). The impact of digital literacy on university students’ innovation capability: evidence from Ningbo, China. Frontiers in Psychology, Volume 16-2025. https://doi.org/10.3389/fpsyg.2025.1548817

Zhou, X., Teng, D., & Al-Samarraie, H. (2024). The Mediating Role of Generative AI Self-Regulation on Students’ Critical Thinking and Problem-Solving. Education Sciences, 14(12). https://doi.org/10.3390/educsci14121302
Published
2026-10-11
How to Cite
Sutarsi Suhaeb, Nurfauziah, & Nur Athiyyah Fadhilah. (2026). The Effect of Using AI in Learning on Critical Thinking Skills Through Students’ Digital Literacy. International Journal of Education, Vocational and Social Science , 5(04), 2909-2933. https://doi.org/10.63922/ijevss.v5i04.6839