Acta Informatica Pragensia 2026, 15(1), 126-134 | DOI: 10.18267/j.aip.293305

Evaluating AI Text Detection Tools for Distinguishing Human-Written from AI-Generated Abstracts in Persian-Language Journals of Library and Information Science

Amrollah Shamsi ORCID...1, Ting Wang ORCID...2, Maryam Amraei ORCID...3, Narayanaswamy Vasantha Raju ORCID...4
1 Clinical Research Development Center, The Persian Gulf Martyrs Hospital, Bushehr University of Medical Sciences, Bushehr, Iran
2 School of Library and Information Management, Emporia State University, Emporia, Kansas, USA
3 Department of Information Science and Epistemology, Shahid Chamran University, Ahvaz, Iran
4 Department of Library and Information Science, Government First Grade College, Talakadu, India

Background: Researchers are using artificial intelligence (AI) tools in academic writing. However, their use may compromise the integrity and originality of the work. Hence, AI text detection tools have come to increase transparency. Objective: This study aims to evaluate the accuracy of AI text detection tools in recognizing human-written and AI-written abstracts in library and information science (LIS).

Methods: Seven Persian academic journals in LIS were selected. ZeroGPT and GPTZero as AI text detectors were used. AI-generated abstracts were produced by AI chatbots (ChatGPT 4.0, DeepSeek and Qwen).

Results: Despite performing strongly in detecting AI-generated text, especially from models such as DeepSeek and Qwen, ZeroGPT and GPTZero struggle to accurately identify human-written content, resulting in high false positive rates and raising concerns about their reliability.

Conclusion: The findings highlight the need for culturally and linguistically inclusive AI detection tools, as current systems such as ZeroGPT and GPTZero show limitations in diverse language contexts, underscoring the importance of improved algorithms and human-involved evaluation to ensure fairness and reliability in academic settings.

Keywords: Artificial intelligence; AI; AI detection; ZeroGPT; GPTZero; ChatGPT; DeepSeek; Qwen.

Received: August 18, 2025; Revised: October 16, 2025; Accepted: October 17, 2025; Prepublished online: December 29, 2025; Published: January 3, 2026  Show citation

ACS AIP APA ASA Harvard Chicago Chicago Notes IEEE ISO690 MLA NLM Turabian Vancouver
Shamsi, A., Wang, T., Amraei, M., & Raju, N.V. (2026). Evaluating AI Text Detection Tools for Distinguishing Human-Written from AI-Generated Abstracts in Persian-Language Journals of Library and Information Science. Acta Informatica Pragensia, 15(1), 126-134. doi: 10.18267/j.aip.293
Download citation
  • BibTeX (.bib)
  • Bookends (.ris)
  • EasyBib (.ris)
  • EndNote (.enw)
  • EndNote 8 (.xml)
  • ISI WoS (.isi)
  • Medlars (.medlars)
  • Mendeley (.ris)
  • MODS (.xml)
  • Papers (.ris)
  • RefWorks (.txt)
  • RefManager (.ris)
  • RIS (.ris)
  • MS Word (.xml)
  • Zotero (.ris)
    • Open full article

    References

    1. Akram, A. (2024). Quantitative Analysis of AI-Generated Texts in Academic Research: A Study of AI Presence in Arxiv Submissions using AI Detection Tool. arXiv:2403.13812. https://doi.org/10.48550/ARXIV.2403.13812 Go to original source...
    2. Alshammari, H., El-Sayed, A., & Elleithy, K. (2024). AI-Generated Text Detector for Arabic Language Using Encoder-Based Transformer Architecture. Big Data and Cognitive Computing, 8(3), Article 32. https://doi.org/10.3390/bdcc8030032 Go to original source...
    3. Amirarjmandi, Z., Navabakhsh, M., & Sarookhani, B. (2022). Scientific authority and eco-centric values in the comprehensive scientific map of the country. Applied Issues in Islamic Education, 7(3), 33-64. https://doi.org/10.52547/qaiie.7.3.33 Go to original source...
    4. Ardito, C. G. (2024). Generative AI detection in higher education assessments. New Directions for Teaching and Learning, 2025(182), 11-28. https://doi.org/10.1002/tl.20624 Go to original source...
    5. Batta, A. (2024). Transforming higher education through generative AI: Opportunity and challenges. Paradigm, 28(2), 241-243. https://doi.org/10.1177/09718907241286221 Go to original source...
    6. Bhattacharjee, A., & Liu, H. (2024). Fighting fire with fire: can ChatGPT detect AI-generated text?. ACM SIGKDD Explorations Newsletter, 25(2), 14-21. https://doi.org/10.1145/3655103.3655106 Go to original source...
    7. Canyakan, S. (2025). Comparative accuracy of AI-based plagiarism detection tools: An enhanced systematic review. Journal of AI, Humanities, and New Ethics, 1(1), 5-18.
    8. Chaka, C. (2024a). Accuracy pecking order - How 30 AI detectors stack up in detecting generative artificial intelligence content in university English L1 and English L2 student essays. Journal of Applied Learning & Teaching, 7(1), 127-139. https://doi.org/10.37074/jalt.2024.7.1.33 Go to original source...
    9. Chaka, C. (2024b). Reviewing the performance of AI detection tools in differentiating between AI-generated and human-written texts: A literature and integrative hybrid review. Journal of Applied Learning & Teaching, 7(1), 115-126. https://doi.org/10.37074/jalt.2024.7.1.14 Go to original source...
    10. Chelli, M., Descamps, J., Lavoué, V., Trojani, C., Azar, M., Deckert, M., Raynier, J., Clowez, G., Boileau, P., & Ruetsch-Chelli, C. (2024). Hallucination Rates and reference Accuracy of CHATGPT and BARD for Systematic Reviews: Comparative analysis. Journal of Medical Internet Research, 26, e53164. https://doi.org/10.2196/53164 Go to original source...
    11. Chen, V. (2024). How AI content detectors work to spot AI. https://gptzero.me/news/how-ai-detectors-work
    12. Dwivedi, Y. K., Kshetri, N., Hughes, L., Slade, E. L., Jeyaraj, A., Kar, A. K., Baabdullah, A. M., Koohang, A., Raghavan, V., Ahuja, M., Albanna, H., Albashrawi, M. A., Al-Busaidi, A. S., Balakrishnan, J., Barlette, Y., Basu, S., Bose, I., Brooks, L., Buhalis, D., … Wright, R. (2023). Opinion Paper: "So what if ChatGPT wrote it?" Multidisciplinary perspectives on opportunities, challenges and implications of generative conversational AI for research, practice and policy. International Journal of Information Management, 71, 102642. https://doi.org/10.1016/j.ijinfomgt.2023.102642 Go to original source...
    13. Elek, A., Yildiz, H. S., Akca, B., Oren, N. C., & Gundogdu, B. (2025). Evaluating the Efficacy of Perplexity Scores in Distinguishing AI-Generated and Human-Written Abstracts. Academic Radiology, 32(4), 1785-1790. https://doi.org/10.1016/j.acra.2025.01.017 Go to original source...
    14. Elkhatat, A. M., Elsaid, K., & Almeer, S. (2023). Evaluating the efficacy of AI content detection tools in differentiating between human and AI-generated text. International Journal for Educational Integrity, 19(1), Article 17. https://doi.org/10.1007/s40979-023-00140-5 Go to original source...
    15. Else, H. (2023). Abstracts written by ChatGPT fool scientists. Nature, 613(7944), Article 423. https://doi.org/10.1038/d41586-023-00056-7 Go to original source...
    16. Foltýnek, T., Meuschke, N., & Gipp, B. (2020). Academic Plagiarism Detection: A Systematic Literature Review. ACM Computing Surveys, 52(6), 1-42. https://doi.org/10.1145/3345317 Go to original source...
    17. Fraser, K. C., Dawkins, H., & Kiritchenko, S. (2025). Detecting AI-generated text: Factors influencing detectability with current methods. Journal of Artificial Intelligence Research, 82, 2233-2278. https://doi.org/10.1613/jair.1.16665 Go to original source...
    18. Gao, C. A., Howard, F. M., Markov, N. S., Dyer, E. C., Ramesh, S., Luo, Y., & Pearson, A. T. (2023). Comparing scientific abstracts generated by ChatGPT to real abstracts with detectors and blinded human reviewers. Npj Digital Medicine, 6(1), Article 75. https://doi.org/10.1038/s41746-023-00819-6 Go to original source...
    19. Golan, R., Reddy, R., Muthigi, A., & Ramasamy, R. (2023). Artificial intelligence in academic writing: a paradigm-shifting technological advance. Nature reviews urology, 20(6), 327-328. https://doi.org/10.1038/s41585-023-00746-x Go to original source...
    20. Gotoman, J. E. J., Luna, H. L. T., Sangria, J. C. S., Santiago Jr., C. S., & Barbuco, D. D. (2025). Accuracy and reliability of AI-generated text detection tools: A literature review. American Journal of IR 4.0 and Beyond, 4(1), 1-9. https://doi.org/10.54536/ajirb.v4i1.3795 Go to original source...
    21. Howard, F. M., Li, A., Riffon, M. F., Garrett-Mayer, E., & Pearson, A. T. (2024). Characterizing the Increase in Artificial Intelligence Content Detection in Oncology Scientific Abstracts From 2021 to 2023. JCO Clinical Cancer Informatics, 8, e2400077. https://doi.org/10.1200/CCI.24.00077 Go to original source...
    22. Hua, S., Jin, S., & Jiang, S. (2024). The limitations and ethical considerations of ChatGPT. Data intelligence, 6(1), 201-239. https://doi.org/10.1162/dint_a_00243 Go to original source...
    23. Huang, J., & Tan, M. (2023). The role of ChatGPT in scientific communication: Writing better scientific review articles. American Journal of Cancer Research, 13(4), 1148-1154.
    24. Kar, S. K., Bansal, T., Modi, S., & Singh, A. (2025). How sensitive are the free AI-detector tools in detecting AI-generated texts? A comparison of popular AI-detector tools. Indian journal of psychological medicine, 47(3), 275-278. https://doi.org/10.1177/02537176241247934 Go to original source...
    25. Khalifa, M. & Albadawy, M. (2024). Using artificial intelligence in academic writing and research: An essential productivity tool. Computer Methods and Programs in Biomedicine Update, 5, 100145. https://doi.org/10.1016/j.cmpbup.2024.100145 Go to original source...
    26. Kobak, D., González-Márquez, R., Horvát, E. Á., & Lause, J. (2025). Delving into LLM-assisted writing in biomedical publications through excess vocabulary. Science Advances, 11(27), eadt3813. https://doi.org/10.1126/sciadv.adt3813 Go to original source...
    27. Kwon, D. (2025). Is it OK for AI to write science papers? Nature survey shows researchers are split. Nature, 641(8063), 574-578. https://doi.org/10.1038/d41586-025-01463-8 Go to original source...
    28. Legaspi, J. B., Licuben, R. J. O., Legaspi, E. A., & Tolentino, J. A. (2024). Comparing AI detectors: evaluating performance and efficiency. International Journal of Science and Research Archive, 12(2), 833-838. https://doi.org/10.30574/ijsra.2024.12.2.1276 Go to original source...
    29. Liang, W., Zhang, Y., Wu, Z., Lepp, H., Ji, W., Zhao, X., ... & Zou, J. Y. (2024). Mapping the increasing use of LLMs in scientific papers. arXiv:2404.01268. https://doi.org/10.48550/arXiv.2404.01268 Go to original source...
    30. Liang, W., Yuksekgonul, M., Mao, Y., Wu, E., & Zou, J. (2023). GPT detectors are biased against non-native English writers. Patterns, 4(7), 100779. https://doi.org/10.1016/j.patter.2023.100779 Go to original source...
    31. Lindell, T. L., & Utterberg Modén, M. (2025). Conflicting motives: challenges of generative AI in education. Learning, Media and Technology, 50, 324-337. https://doi.org/10.1080/17439884.2024.2438933 Go to original source...
    32. Malik, M. A., & Amjad, A. I. (2025). AI vs AI: How effective are Turnitin, ZeroGPT, GPTZero, and Writer AI in detecting text generated by ChatGPT, Perplexity, and Gemini? Journal of Applied Learning & Teaching, 8(1), 91-101. https://doi.org/10.37074/jalt.2025.8.1.9 Go to original source...
    33. Mannuru, N. R., Shahriar, S., Teel, Z. A., Wang, T., Lund, B. D., Tijani, S., Pohboon, C. O., Agbaji, D., Alhassan, J., Galley, J., Kousari, R., Ogbadu-Oladapo, L., Saurav, S. K., Srivastava, A., Tummuru, S. P., Uppala, S., & Vaidya, P. (2023). Artificial intelligence in developing countries: The impact of generative artificial intelligence (AI) technologies for development. Information Development, 41(3), 1036-1054. https://doi.org/10.1177/02666669231200628 Go to original source...
    34. Mese, I. (2024). Tracing the Footprints of AI in Radiology Literature: A Detailed Analysis of Journal Abstracts. RöFo - Fortschritte auf dem Gebiet der Röntgenstrahlen und der bildgebenden Verfahren, 196(8), 843-849. https://doi.org/10.1055/a-2224-9230 Go to original source...
    35. Naddaf, M. (2025). How are researchers using AI? Survey reveals pros and cons for science. Nature. https://doi.org/10.1038/d41586-025-00343-5 Go to original source...
    36. Nguyen, T. T., Hatua, A., & Sung, A. H. (2023). How to detect AI-generated texts?. In 2023 IEEE 14th Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON), (pp. 464-471). IEEE. https://doi.org/10.1109/UEMCON59035.2023.10316132 Go to original source...
    37. Odri, G.-A., & Ji Yun Yoon, D. (2023). Detecting generative artificial intelligence in scientific articles: Evasion techniques and implications for scientific integrity. Orthopaedics & Traumatology: Surgery & Research, 109(8), 103706. https://doi.org/10.1016/j.otsr.2023.103706 Go to original source...
    38. Rafiq, S., & Qurat-ul-Ain, D. A. A. (2025). The role of AI detection tools in upholding academic integrity: An evaluation of their effectiveness. Contemporary Journal of Social Science Review, 3(1), 901-915.
    39. Sani, B., Soy, A., Imam, S. H., Mustapha, A., Aliyu, L. J., Abdulmumin, I., Ahmad, I. S., & Muhammad, S. H. (2025). Who wrote this? Identifying machine vs human-generated text in Hausa. arXiv:2503.13101. https://doi.org/10.48550/arXiv.2503.13101 Go to original source...
    40. Shah, S. (2024). The role of artificial intelligence in research writing: A critical analysis. Journal of Universal College of Medical Sciences, 12(3), 1-2. https://doi.org/10.3126/jucms.v12i03.73724 Go to original source...
    41. Siegel, M. G., Brond, J. C., Rossi, M. J., & Lubowitz, J. H. (2018). "Publish or perish" promotes medical literature quantity over quality. Arthroscopy, 34(11), 2941-2942. https://doi.org/10.1016/j.arthro.2018.08.029 Go to original source...
    42. SJR. (2025). Scimago Journal & Country Rank. https://www.scimagojr.com/countryrank.php?region=Middle%20East
    43. Subramaniam, R. (2023). Identifying Text Classification Failures in Multilingual AI-Generated Content. International Journal of Artificial Intelligence & Applications, 14(5), 57-63. https://doi.org/10.5121/ijaia.2023.14505 Go to original source...
    44. Walters, W. H. (2023). The effectiveness of software designed to detect AI-generated writing: A comparison of 16 AI text detectors. Open Information Science, 7(1), 20220158. https://doi.org/10.1515/opis-2022-0158 Go to original source...
    45. Weber-Wulff, D., Anohina-Naumeca, A., Bjelobaba, S., Foltýnek, T., Guerrero-Dib, J., Popoola, O., ... & Waddington, L. (2023). Testing of detection tools for AI-generated text. International Journal for Educational Integrity, 19(1), 1-39. https://doi.org/10.1007/s40979-023-00146-z Go to original source...
    46. Wiley. (2025). An artificial intelligence study by Wiley. https://www.wiley.com/en-us/ai-study

    This is an open access article distributed under the terms of the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits use, distribution, and reproduction in any medium, provided the original publication is properly cited. No use, distribution or reproduction is permitted which does not comply with these terms.


    Return to the content