Acta Informatica Pragensia 2026, 15(1), 253-273 | DOI: 10.18267/j.aip.297109

Antecedents of Test Automation Adoption in DevOps Continuous Testing: A Systematic Literature Review Through the TOE Framework

Anuji Isara Vithana ORCID...1, Kelum Asanga Akurugoda Gamage ORCID...2, Dilani Wickramaarachchi ORCID...1, Ruwan Wickramarachchi ORCID...1, Shan Jayasinghe ORCID...3
1 Department of Industrial Management, Faculty of Science, University of Kelaniya, Kelaniya, Sri Lanka
2 James Watt School of Engineering, University of Glasgow, Glasgow, United Kingdom
3 Department of Business and Law, Southampton Solent University, Southampton, United Kingdom

Background: The rapid evolution of software engineering has positioned DevOps practices and Continuous Testing (CT) as critical approaches for achieving speed, quality, and reliability in software delivery. Test automation is central to CT, yet its adoption remains inconsistent due to a complex interplay of technological, organizational, and environmental conditions.

Objective: This study employs a systematic literature review guided by the Technology–Organization–Environment (TOE) framework to identify, categorize, and synthesize the antecedents that influence the test automation adoption in DevOps continuous testing.

Methods: Using the PRISMA protocol, 49 peer-reviewed studies published between 2015 and 2025 were systematically analyzed, yielding 61 distinct factors comprising 29 technological, 19 organizational, and 13 environmental antecedents. These factors were further consolidated into thematic clusters to enhance analytical clarity and reduce fragmentation.

Results: The findings demonstrate that technological and organizational drivers, including relative advantage, compatibility, top management support, and employee competence, dominate the literature, while environmental influences such as competitive pressure, regulatory requirements, and vendor ecosystems are comparatively underexplored. This imbalance indicates that although the TOE framework is widely applied in technology adoption studies, empirical research has given greater attention to internal adoption enablers than to external pressures. By simplifying and synthesizing the factors into coherent sub-themes, this study contributes to both theory and practice by offering a structured lens through which test automation adoption can be examined in the DevOps CT context. Theoretically, it validates the relevance of TOE for analysing multidimensional adoption dynamics, while practically, it provides managers with evidence-based insights to prioritize critical factors when planning automation initiatives. Methodologically, it demonstrates the importance of transparent and replicable review processes for advancing cumulative knowledge.

Conclusion: Overall, the study bridges fragmented findings into a coherent framework and strengthens understanding of adoption strategies in continuous testing environments.

Keywords: Test Automation; DevOps; Continuous Testing; Technology organization environment; TOE Framework; Technology Adoption.

Received: September 19, 2025; Revised: November 10, 2025; Accepted: November 24, 2025; Prepublished online: December 31, 2025; Published: January 3, 2026  Show citation

ACS AIP APA ASA Harvard Chicago Chicago Notes IEEE ISO690 MLA NLM Turabian Vancouver
Vithana, A.I., Gamage, K.A.A., Wickramaarachchi, D., Wickramarachchi, R., & Jayasinghe, S. (2026). Antecedents of Test Automation Adoption in DevOps Continuous Testing: A Systematic Literature Review Through the TOE Framework. Acta Informatica Pragensia, 15(1), 253-273. doi: 10.18267/j.aip.297
Download citation
  • BibTeX (.bib)
  • Bookends (.ris)
  • EasyBib (.ris)
  • EndNote (.enw)
  • EndNote 8 (.xml)
  • ISI WoS (.isi)
  • Medlars (.medlars)
  • Mendeley (.ris)
  • MODS (.xml)
  • Papers (.ris)
  • RefWorks (.txt)
  • RefManager (.ris)
  • RIS (.ris)
  • MS Word (.xml)
  • Zotero (.ris)
    • Open full article

    References

    1. Aligarh, F., Sutopo, B., & Widarjo, W. (2023). The antecedents of cloud computing adoption and its consequences for MSMEs' performance: A model based on the Technology-Organization-Environment (TOE) framework. Cogent Business and Management, 10(2), 1-16. https://doi.org/10.1080/23311975.2023.2220190 Go to original source...
    2. Alshahrani, S. M., Mohamed, H., Mukhtar, M., & Mokhtar, U. A. (2023). The adoption of the e-portfolio management system in the Technical and Vocational Training Corporation (TVTC) in Saudi Arabia. International Journal of Information Management Data Insights, 3(1), 100148. https://doi.org/10.1016/j.jjimei.2022.100148 Go to original source...
    3. Amaro, R., & Dias, I. U. L. (2024). DevOps metrics and KPIs: A multivocal literature review. ACM Computing Surveys, 56(9), Article 3652508. https://doi.org/10.1145/3652508 Go to original source...
    4. Asante Boakye, E., Zhao, H., Coffie, C. P. K., & Asare-Kyire, L. (2024). Seizing technological advancement; determinants of blockchain supply chain finance adoption in Ghanaian SMEs. Technology Analysis and Strategic Management, 36(10), 2774-2790. https://doi.org/10.1080/09537325.2022.2163384 Go to original source...
    5. Awa, H. O., Ojiabo, O. U., & Orokor, L. E. (2017). Integrated technology-organization-environment (T-O-E) taxonomies for technology adoption. Journal of Enterprise Information Management, 30(6), 893-921. https://doi.org/10.1108/JEIM-03-2016-0079 Go to original source...
    6. Azad, N., & Hyrynsalmi, S. (2023). DevOps critical success factors - A systematic literature review. Information and Software Technology, 157, 107150. https://doi.org/10.1016/j.infsof.2023.107150 Go to original source...
    7. Bahari, M., Arpaci, I., Der, O., Akkoyun, F., & Ercetin, A. (2024). Driving Agricultural Transformation: Unraveling Key Factors Shaping IoT Adoption in Smart Farming with Empirical Insights. Sustainability , 16(5), 2129. https://doi.org/10.3390/su16052129 Go to original source...
    8. Bany Mohammad, A., Al-Okaily, M., Al-Majali, M., & Masa'deh, R. (2022). Business Intelligence and Analytics (BIA) Usage in the Banking Industry Sector: An Application of the TOE Framework. Journal of Open Innovation: Technology, Market, and Complexity, 8(4). https://doi.org/10.3390/joitmc8040189 Go to original source...
    9. Bhanushali, A. (2023). Ensuring software quality through effective quality assurance testing: Best practices and case studies. International Journal of Advances in Engineering Research, 26(4), 1-18.
    10. Bouteraa, M. (2024). Mixed-methods approach to investigating the diffusion of FinTech services: enriching the applicability of TOE and UTAUT models. Journal of Islamic Marketing, 15(8), 2036-2068. https://doi.org/10.1108/JIMA-12-2022-0343 Go to original source...
    11. Chong, J. L. L., & Olesen, K. (2017). A technology-organization-environment perspective on eco-effectiveness: A meta-analysis. Australasian Journal of Information Systems, 21, 1-26. https://doi.org/10.3127/ajis.v21i0.1441 Go to original source...
    12. Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319-339. https://doi.org/10.2307/249008 Go to original source...
    13. DeLone, W. H., & McLean, E. R. (2003). The DeLone and McLean model of information systems success: A ten-year update. Journal of Management Information Systems, 19(4), 9-30. https://doi.org/10.1080/07421222.2003.11045748 Go to original source...
    14. Dwivedi, Y. K., Wade, M. R., & Schneberger, S. L. (2012). Information Systems Theory. Springer. https://doi.org/10.1007/978-1-4419-6108-2 Go to original source...
    15. El-Haddadeh, R. (2020). Digital Innovation Dynamics Influence on Organisational Adoption: The Case of Cloud Computing Services. Information Systems Frontiers, 22(4), 985-999. https://doi.org/10.1007/s10796-019-09912-2 Go to original source...
    16. Enemosah, A. (2025). Enhancing DevOps Efficiency through AI-Driven Predictive Models for Continuous Integration and Deployment Pipelines. International Journal of Research Publication and Reviews, 6(1), 871-887. https://doi.org/10.55248/gengpi.6.0125.0229 Go to original source...
    17. Estdale, J., & Georgiadou, E. (2018). Applying the ISO/IEC 25010 Quality Models to Software Product. In Systems, Software and Services Process Improvement (EuroSPI 2018), (pp. 492-503). Springer. https://doi.org/10.1007/978-3-319-97925-0_42 Go to original source...
    18. Eze, S. C., Olatunji, S., Chinedu-Eze, V. C., & Bello, A. O. (2018). Key success factors influencing SME managers' information behaviour on emerging ICT (EICT) adoption decision-making in UK SMEs. Bottom Line, 31(3-4), 250-275. https://doi.org/10.1108/BL-02-2018-0008 Go to original source...
    19. Fu, H. P., Chang, T. S., Yeh, H. P., & Chen, Y. X. (2019). Analysis of factors influencing hospitals' implementation of a green e-procurement system using a cloud model. International Journal of Environmental Research and Public Health, 16(24). https://doi.org/10.3390/ijerph16245137 Go to original source...
    20. Fülöp, M. T., Ionescu, C. A., Măgdaș, N., Topor, D. I., & Breaz, T. O. (2024). Acceptance of digital instruments in the accounting profession. Journal of East European Management Studies, 29(2), 283-313. https://doi.org/10.5771/0949-6181-2024-2-283 Go to original source...
    21. Gangwar, H., Hema, D., & Ramaswamy, R. (2015). Journal of Enterprise Information Management Understanding determinants of cloud computing adoption using an integrated TAM-TOE model. Journal of Enterprise Information Management, 28(1), 107-130. https://doi.org/10.1108/JEIM-08-2013-0065 Go to original source...
    22. Gota, L., Gota, D., & Miclea, L. (2020). Continuous Integration in Automation Testing. In 2020 22nd IEEE International Conference on Automation, Quality and Testing, Robotics - THETA, AQTR 2020. IEEE. https://doi.org/10.1109/AQTR49680.2020.9129990 Go to original source...
    23. Grünewald, D., Ramler, R., & Felderer, M. (2023). Hawk: DevOps-driven transparency and accountability in cloud-native systems. arXiv:2306.02496. https://arxiv.org/abs/2306.02496 Go to original source...
    24. Gui, A., Pangestu, K. Y., Rugoswanto, F. H., Ahmarofi, A. A., & Suryanto. (2021). Factors affecting the adoption of cloud computations on company performance. Journal of Theoretical and Applied Information Technology, 99(14), 3489-3502.
    25. Gutierrez, A., Boukrami, E., & Lumsden, R. (2015). Technological, organisational and environmental factors influencing managers' decision to adopt cloud computing in the UK. Journal of Enterprise Information Management, 28(6), 788-807. https://doi.org/10.1108/JEIM-01-2015-0001 Go to original source...
    26. Hadi, H. J., Omar, M. A., & Osman, W. R. S. (2021). Investigating the determinants of CC-SaaS adoption in Iraqi's public organisations from the perspective of IT professionals. International Journal of Engineering Research and Technology, 14(2), 130-143.
    27. Hsu, C. L., & Lin, J. C. C. (2016). Factors affecting the adoption of cloud services in enterprises. Information Systems and E-Business Management, 14(4), 791-822. https://doi.org/10.1007/s10257-015-0300-9 Go to original source...
    28. Ifinedo, P. (2011). An empirical analysis of factors influencing internet/e-business technologies adoption by SMEs in Canada. International Journal of Information Technology and Decision Making, 10(4), 731-766. https://doi.org/10.1142/S0219622011004543 Go to original source...
    29. Ilias, A., Ghani, E. K., Baidi, N., & Azhar, Z. (2020). Experiences to voluntarily adopt malaysian business reporting system MBRS: A case study of SMPs. International Journal of Financial Research, 11(3), 1-17. https://doi.org/10.5430/ijfr.v11n3p1 Go to original source...
    30. Isiaku, L., & Adalier, A. (2025). Investigating the determinants of continuous use intention of business intelligence systems in Nigerian banks: the role of regulatory support. Management Decision, (in press). https://doi.org/10.1108/MD-08-2024-1848 Go to original source...
    31. Islam, M., Khan, F., Alam, S., & Hasan, M. (2023). Artificial Intelligence in Software Testing: A Systematic Review. In IEEE Region 10 Annual International Conference, Proceedings/TENCON, (pp. 524-529). IEEE. https://doi.org/10.1109/TENCON58879.2023.10322349 Go to original source...
    32. Jain, V. (2023). Continuous testing in CI/CD pipelines. https://doi.org/10.5281/zenodo.14883221 Go to original source...
    33. Jha, R., Dutta, S., & Basu, A. (2024). An innovative model on intention to adopt cloud-based ERP among SMEs. International Journal of Intelligent Enterprise, 11(1), 33-56. https://doi.org/10.1504/ijie.2022.10047198 Go to original source...
    34. Jia, J., Xu, Y., & Li, W. (2024). A study on the strategic momentum of SMEs' digital transformation: Evidence from China. Technological Forecasting and Social Change, 200, 123038. https://doi.org/10.1016/j.techfore.2023.123038 Go to original source...
    35. Khsroo, I. T. N., Burhanuddin, M. A., Ali, M. A., & Shihab Ahmed, M. (2024). The Influence of Technological Factors on Adoption E-Commerce in SME: The Role of Trust. Journal of Global Information Technology Management, 27(2), 148-167. https://doi.org/10.1080/1097198X.2024.2327945 Go to original source...
    36. Kitchenham, B., & Charters, S. (2007). Guidelines for performing systematic literature reviews in software engineering (EBSE Technical Report, Ver. 2.3). Evidence-Based Software Engineering (EBSE). https://doi.org/10.48550/arXiv.1012.0001 Go to original source...
    37. Lahtinen, A. (2020). Test Automation Strategy in Devops Environment: an IT Management Viewpoint. https://jyx.jyu.fi/handle/123456789/69926
    38. Li, H., Dai, J., Gershberg, T., & Vasarhelyi, M. A. (2018). Understanding usage and value of audit analytics for internal auditors: An organizational approach. International Journal of Accounting Information Systems, 28, 59-76. https://doi.org/10.1016/j.accinf.2017.12.005 Go to original source...
    39. Low, C., Chen, Y., & Wu, M. (2011). Understanding the determinants of cloud computing adoption. Industrial Management and Data Systems, 111(7), 1006-1023. https://doi.org/10.1108/02635571111161262 Go to original source...
    40. Lustenberger, M., Maleąević, S., & Spychiger, F. (2021). Ecosystem Readiness: Blockchain Adoption is Driven Externally. Frontiers in Blockchain, 4, 720454. https://doi.org/10.3389/fbloc.2021.720454 Go to original source...
    41. Lwakatare, L. E., Kuvaja, P., & Oivo, M. (2019). Relationship of DevOps to agile, lean and continuous deployment: A multivocal literature review study. Empirical Software Engineering, 24(2), 765-801. https://doi.org/10.1007/s10664-017-9555-0 Go to original source...
    42. Madeyski, L., & Kawalerowicz, M. (2019). Empirical evaluation of continuous test-driven development in industrial settings. Journal of Intelligent and Fuzzy Systems, 37(6), 7643-7655. https://doi.org/10.3233/JIFS-179369 Go to original source...
    43. Marei, A. (2023). The Moderating Role of Big Data and User Satisfaction in the Predictors of Generalized Audit Software among Jordanian Auditing Firms. WSEAS Transactions on Business and Economics, 20(June), 1357-1367. https://doi.org/10.37394/23207.2023.20.121 Go to original source...
    44. Martins, R., Oliveira, T., & Thomas, M. A. (2016). An empirical analysis to assess the determinants of SaaS diffusion in firms. Computers in Human Behavior, 62, 19-33. https://doi.org/10.1016/j.chb.2016.03.049 Go to original source...
    45. Mascheroni, M. A., & Irrazábal, E. (2018). Continuous testing and solutions for testing problems in continuous delivery: A systematic literature review. Computacion y Sistemas, 22(3), 1009-1038. https://doi.org/10.13053/CyS-22-3-2794 Go to original source...
    46. Masood, T., & Egger, J. (2020). Adopting augmented reality in the age of industrial digitalisation. Computers in Industry, 115, 103112. https://doi.org/10.1016/j.compind.2019.07.002 Go to original source...
    47. Mehta, S., Oberoi, A., & Bhatia, S. S. (2018). A Cloud based Assurance Framework for Implementation of ERP in SMEs A Literature Survey. International Journal of Computer Sciences and Engineering, 6(9), 819-829. https://doi.org/10.26438/ijcse/v6i9.819829 Go to original source...
    48. Mikalef, P., Boura, M., Lekakos, G., & Krogstie, J. (2019). Big data analytics and firm performance: Findings from a mixed-method approach. Journal of Business Research, 98, 261-276. https://doi.org/10.1016/j.jbusres.2019.01.044 Go to original source...
    49. Mongeon, P., & Paul-Hus, A. (2016). The journal coverage of Web of Science and Scopus: a comparative analysis. Scientometrics, 106(1), 213-228. https://doi.org/10.1007/s11192-015-1765-5 Go to original source...
    50. Najihi, S., Elhadi, S., Abdelouahid, R. A., & Marzak, A. (2022). Software Testing from an Agile and Traditional view. Procedia Computer Science, 203, 775-782. https://doi.org/10.1016/j.procs.2022.07.116 Go to original source...
    51. Okoli, C., & Schabram, K. (2012). A Guide to Conducting a Systematic Literature Review of Information Systems Research. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.1954824 Go to original source...
    52. Oliveira, T., & Martins, M. F. (2010). Information technology adoption models at Firm Level: Review of literature. Electronic Journal of Information Systems Evaluation, 14(1), 110-121.
    53. Oni, A. A., Musa, U., & Oni, S. (2020). E-revenue adoption in state internal revenue service: Interrogating the institutional factors. Journal of Organizational and End User Computing, 32(1), 41-61. https://doi.org/10.4018/JOEUC.2020010103 Go to original source...
    54. Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., Chou, R., Glanville, J., Grimshaw, J. M., Hróbjartsson, A., Lalu, M. M., Li, T., Loder, E. W., Mayo-Wilson, E., McDonald, S., … Moher, D. (2021). The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ, 372, n71. https://doi.org/10.1136/bmj.n71 Go to original source...
    55. Palos-Sanchez, P., Reyes-Menendez, A., & Saura, J. R. (2019). Models of adoption of information technology and cloud computing in organizations. Informacion Tecnologica, 30(3), 3-12. https://doi.org/10.4067/S0718-07642019000300003 Go to original source...
    56. Pillai, R., & Sivathanu, B. (2020). Adoption of artificial intelligence (AI) for talent acquisition in IT/ITeS organizations. Benchmarking, 27(9), 2599-2629. https://doi.org/10.1108/BIJ-04-2020-0186 Go to original source...
    57. Pisirir, E., Chouseinoglou, O., Sevgi, C., & Uçar, E. (2023). Factors Affecting the Adoption of Cloud for Software Development: A Case from Turkey. International Journal of Information Technology and Decision Making, 24(4), 997-1033. https://doi.org/10.1142/S0219622023500517 Go to original source...
    58. Ramu, V. B. (2023). Optimizing DevOps Pipelines with Performance Testing: A Comprehensive Approach. International Journal of Computer Trends and Technology, 71(6), 35-41. https://doi.org/10.14445/22312803/ijctt-v71i6p106 Go to original source...
    59. Raut, R., Priyadarshinee, P., & Jha, M. (2017). Understanding the mediation effect of cloud computing adoption in Indian organization: Integrating taM-toe- Risk model. International Journal of Service Science, Management, Engineering, and Technology, 8(3), 40-59. https://doi.org/10.4018/IJSSMET.2017070103 Go to original source...
    60. Razzaq, A., Asmai, S. A., Abidin, Z. Z., Talib, M. S., Ali, M. F., & Mohammed, A. A. (2021). Propose a conceptual framework for the cloud ERP adoption among malaysian SMEs. Journal of Engineering Science and Technology, 16(4), 3387-3406.
    61. Sai, S., & Poola, P. (2024). A Systematic Approach of Introducing Test Automation in a DevOps Environment at a Large-scale Software Development Organization: A Case Study at Scania. Master thesis. Blekinge Institute of Technology.
    62. Salah, O. H., Yusof, Z. M., & Mohamed, H. (2021). The determinant factors for the adoption of CRM in the Palestinian SMEs: The moderating effect of firm size. PLoS ONE, 16(3), 1-25. https://doi.org/10.1371/journal.pone.0243355 Go to original source...
    63. Sezen, C., Ghamgosar, A., Ioini, N. El, Pahl, C., & Gunduz, N. G. (2025). Infrastructure as Code: Technology Review and Research Challenges. In Proceedings of the 15th International Conference on Cloud Computing and Services Science CLOSER - Volume 1,, (pp. 151-158). ScitePress. https://doi.org/10.5220/0013247700003950 Go to original source...
    64. Siradhana, N. K., & Arora, R. G. (2024). Examining the Influence of Artificial Intelligence Implementation in HRM Practices Using T-O-E Model. Vision, (in press). https://doi.org/10.1177/09722629241231458 Go to original source...
    65. Sulaiman, T. T., Mahomed, A. S. B., Rahman, A. A., & Hassan, M. (2023). Understanding Antecedents of Learning Management System Usage among University Lecturers Using an Integrated TAM-TOE Model. Sustainability, 15(3), 1885. https://doi.org/10.3390/su15031885 Go to original source...
    66. Teh, R., Subramaniam, A., Ho, J. A., & Basha, N. K. (2024). The mediation role of top management support in the adoption of cloud computing in Malaysian SMEs. International Journal of Management and Enterprise Development, 23(1), 73-96. https://doi.org/10.1504/IJMED.2024.138452 Go to original source...
    67. Thanabalan, P., Vafaei-Zadeh, A., Hanifah, H., & Ramayah, T. (2024). Big Data Analytics Adoption in Manufacturing Companies: The Contingent Role of Data-Driven Culture. Information Systems Frontiers, 27, 1061-1087. https://doi.org/10.1007/s10796-024-10491-0 Go to original source...
    68. Wang, Y., Pyhajarvi, M., & Mantyla, M. V. (2020). Test Automation Process Improvement in a DevOps Team: Experience Report. In Proceedings 2020 IEEE 13th International Conference on Software Testing, Verification and Validation Workshops, (pp. 314-321). IEEE. https://doi.org/10.1109/ICSTW50294.2020.00057 Go to original source...
    69. Xiang, T. C., Talib, Z. M., & Md Johar, M. G. (2023). An analysis of sustainable change management for quality 4.0: Evidence from hybrid project management adoption in the Malaysian FinTech context. Journal of Project Management, 8(4), 253-272. https://doi.org/10.5267/j.jpm.2023.6.001 Go to original source...
    70. Yarram, S., & Rao Bittla, S. (2023). Predictive Test Automation: Shaping the Future of Quality Engineering in Enterprise Platforms. Journal of Computational Analysis and Applications, 31(4), 877-883.
    71. Yaseen, H., Al-Adwan, A. S., Nofal, M., Hmoud, H., & Abujassar, R. S. (2023). Factors Influencing Cloud Computing Adoption Among SMEs: The Jordanian Context. Information Development, 39(2), 317-332. https://doi.org/10.1177/02666669211047916 Go to original source...
    72. Zaib, S., & Lakshmisetty, P. K. (2021). A systematical literature review and industrial survey in addressing the possible impacts with the continuous testing and delivery during DevOps transformation. Master Thesis. Blekinge Institute of Technology.
    73. Zheng, J., & Khalid, H. (2022). The Adoption of Enterprise Resource Planning and Business Intelligence Systems in Small and Medium Enterprises: A Conceptual Framework. Mathematical Problems in Engineering, 2022, 1829347. https://doi.org/10.1155/2022/1829347 Go to original source...
    74. Zhu, K., Kraemer, K. L., & Xu, S. (2006). The process of innovation assimilation by firms in different countries: A technology diffusion perspective on e-business. Management Science, 52(10), 1557-1576. https://doi.org/10.1287/mnsc.1050.0487 Go to original source...

    This is an open access article distributed under the terms of the Creative Commons Attribution 4.0 International License (CC BY 4.0), which permits use, distribution, and reproduction in any medium, provided the original publication is properly cited. No use, distribution or reproduction is permitted which does not comply with these terms.


    Return to the content