This repository curates the best AI tools for mobile and web app automation testing alongside seminal and recent research on AI-driven software testing. It covers open-source frameworks and commercial platforms that leverage AI (machine learning, computer vision, NLP) to improve test creation, execution, and maintenance. Key topics include AI-powered test case generation, visual UI testing, self-healing automation, defect prediction, and CI/CD integration.
-
Microsoft OmniParser
An AI tool for visual UI parsing and automation that detects interactive elements from screenshots.
GitHub: microsoft/OmniParser -
askUI VisionAgent
An automation agent that "sees" the UI to perform operations based on plain English or Python API commands.
GitHub: askui/vision-agent -
Healenium
A self-healing test automation library for Selenium that uses ML to dynamically update broken locators.
Healenium Documentation -
EvoSuite
Automatically generates JUnit test suites for Java classes using genetic algorithms to achieve high code coverage.
EvoSuite Website -
SikuliX
A GUI automation tool based on image recognition. Though it uses basic computer vision, it laid the groundwork for modern visual testing.
SikuliX -
Robot Framework
A generic test automation framework that now supports AI-driven libraries.
Robot Framework -
Appium
A mobile automation framework that can integrate with image recognition plugins for visual locator strategies.
Appium -
TestGPT
An emerging project using large language models to generate test cases from requirements.
(Link coming soon)
-
Applitools Eyes
Uses proprietary visual AI to detect UI regressions intelligently by comparing screenshots beyond pixel-level differences.
Applitools Eyes -
Functionize
Converts plain English test steps into executable scripts using ML and NLP, with self-healing capabilities when the UI changes.
Functionize -
Mabl
A cloud-based service that auto-generates tests by crawling your web application, applying ML for self-healing and maintenance.
Mabl -
Testim (Harness)
Employs dynamic weighted locators and natural language-based test creation to maintain test stability despite UI changes.
Testim on Harness -
testRigor
Enables writing tests in plain English and leverages GPT-4 for automated test script generation and maintenance across platforms.
testRigor -
AccelQ
A no-code platform that uses generative AI for creating test scenarios and self-healing automation for web, API, and mainframe testing.
AccelQ -
Rainforest QA
Combines AI with crowd-testing to convert plain English tests into automated steps, with fallback human verification if needed.
Rainforest QA -
Other Platforms:
- OpenText UFT One – Traditional enterprise testing enhanced with AI-based visual recognition.
OpenText UFT One - Autify – A no-code solution using AI for element recognition and self-healing maintenance.
Autify - Reflect – Lightweight web UI testing using plain language and some AI-driven maintenance.
Reflect - Meticulous – Automatically generates regression tests by recording real user interactions.
Meticulous - ProdPerfect – Generates tests autonomously from production traffic and user behavior analytics.
ProdPerfect - Tricentis Tosca (Vision AI) – Uses neural networks to recognize UI elements in a human-like way.
Tricentis Tosca
- OpenText UFT One – Traditional enterprise testing enhanced with AI-based visual recognition.
-
YOLOv5 & YOLOv8
State-of-the-art object detection models used for identifying UI elements in screenshots in real time.
YOLOv5 | YOLOv8 -
OpenAI GPT-4
Leverages natural language processing to generate test cases, write unit tests, and analyze logs.
OpenAI API -
Vision AI Libraries:
- Tesseract OCR:
Tesseract OCR - Azure Computer Vision:
Azure Cognitive Services - Computer Vision
- Tesseract OCR:
-
Reinforcement Learning Agents
Agents that explore GUIs and learn optimal testing strategies (e.g., via OpenAI Gym). -
Code Analysis Models:
Models like code2vec, Graph Neural Networks, and transformer-based models (e.g., Codex) assist in generating tests and predicting defects.
Also see: Diffblue Cover
-
EvoSuite: Automatic Test Suite Generation for Object-Oriented Software (Fraser & Arcuri, 2011)
Pioneering work on generating JUnit tests using genetic algorithms.
Read more -
The Future of Software Testing: AI-Powered Test Case Generation and Validation (Baqar & Khanda, 2024)
A survey of AI techniques for automatic test creation and maintenance.
arXiv -
Reinforcement Learning for Test Case Prioritization (Spieker et al., 2017)
Uses RL to dynamically order and generate tests in a CI environment.
More info -
ChatGPT and Test Generation (2023)
Early explorations leveraging LLMs to generate test scenarios and unit tests.
Read more
-
Vision-Based Mobile App GUI Testing: A Survey (Wang et al., 2023)
Reviews image-based techniques using CNNs and OCR for mobile UI testing.
arXiv -
GUI Element Detection from Mobile UI Images Using YOLOv5 (Altinbas & Serif, 2022)
Demonstrates accurate detection of UI components using YOLOv5.
arXiv -
Intelligent System for Visual Testing of Software Products (2021)
Explores neural network approaches to enhance visual regression detection.
CEUR-WS -
Visual GUI Testing in Practice: Challenges and Benefits (Alégroth et al., 2018)
An empirical study on the advantages and challenges of visual testing in CI.
IEEE Xplore
-
Self-Healing Test Automation Frameworks Using Reinforcement Learning (Dey et al., 2022)
Proposes an RL-based approach for dynamically repairing test scripts.
Online Scientific Research | ResearchGate -
Multi-Year Grey Literature Review on AI-assisted Test Automation (Corradini et al., 2024)
Surveys industry trends and implementations of self-healing in test automation.
arXiv -
Self-Healing Test Automation using AI and ML (2021)
A case study demonstrating effective use of ML to update test selectors automatically.
ResearchGate
-
A Survey on Software Defect Prediction using Deep Learning (Akimova et al., 2021)
Reviews deep learning methods for predicting defect-prone code areas.
MDPI -
DeepLineDP: Towards a Deep Learning Approach for Line-Level Defect Prediction (Wang et al., 2020)
Applies CNNs for fine-grained defect prediction at the code line level.
IEEE Xplore -
Just-In-Time Defect Prediction with Bidirectional LSTMs (Hoang et al., 2019)
Uses deep learning on commit messages and diffs to forecast risky commits.
ACM ASE -
Software Defect Prediction: Do Classifiers Matter? (Lessmann et al., 2008)
A classic study comparing various classifiers for defect prediction.
IEEE Xplore -
Bugram: Bug Detection with N-gram Language Models (Kang et al., 2019)
Utilizes N-gram models to detect anomalous code sequences that may indicate bugs.
IEEE Xplore
-
Reinforcement Learning for Test Case Prioritization in CI (Bagherzadeh et al., 2021)
Demonstrates how RL can optimize test ordering in CI pipelines.
arXiv -
AI-Driven Continuous Integration and Delivery (Pattanayak et al., 2024)
Explores predictive analytics in CI/CD using AI for test selection and build failure prediction.
IJSRA -
Test Flakiness Prediction with Machine Learning (2019)
Develops models to identify and mitigate flaky tests in CI environments.
IEEE Xplore -
Continuous Test Optimization: An Industrial Survey (2020)
Surveys ML-driven test selection strategies to enhance CI efficiency.
ACM Digital Library -
Autonomous Test Orchestration in CI (IBM Research, 2021)
Combines AI planning and rule-based systems to optimize test execution pipelines.
IBM Research
-
Implementation and Comparison of Artificial Intelligence Techniques in Software Testing
Published in: 2023 6th International Conference on Information Systems and Computer Networks (ISCON)
Summary: Discusses how AI (specifically ML and DL) enhances testing efficiency by reducing manual efforts and compares techniques for faster application testing. -
Artificial Intelligence in Software Test Automation: A Systematic Literature Review
Published in: Not specified; part of a systematic review study
Summary: Categorizes AI techniques across testing activities—including test case reusability, coverage, and fault detection—demonstrating improved efficiency and broader test coverage. -
Accelerating Software Quality: Generative AI for Automated Test-Case Generation
Published in: International Journal for Research in Applied Science and Engineering Technology
Summary: Explores the use of generative AI for automatic test-case creation and bug detection by analyzing codebases and execution traces, highlighting significant improvements in test coverage and efficiency despite challenges like data quality. -
AI Techniques in Software Engineering Paradigm
Published in: Proceedings of the 2018 ACM/SPEC International Conference on Performance Engineering
Summary: Discusses AI's role in automating various software engineering phases, including defect prediction and log analysis, thereby enhancing reliability and prediction accuracy. -
Artificial Intelligence in Software Testing: A Systematic Review
Published in: TENCON 2023 - IEEE Region 10 Conference
Summary: Analyzes 20 studies on the role of AI in software testing, covering areas such as test case generation, defect prediction, and prioritization. -
AI for Testing Today and Tomorrow: Industry Perspectives
Published in: IEEE International Conference on Artificial Intelligence Testing (AITest)
Summary: Reviews insights from an industry expert panel on strategies and visions for applying AI in testing, including the testing of AI systems and self-testing methodologies.
-
AI-Powered Test Automation: A Practical Guide for QA Engineers
Summary: A comprehensive guide on selecting tools, setting up test environments, and integrating AI models into test frameworks—with code examples and real-world case studies.
Read More -
Building Self-Healing Test Automation with Machine Learning
Summary: Explores resilient automation strategies using ML algorithms that adapt to UI changes, with detailed implementation strategies and case studies.
Read More -
Practical Applications of GPT Models in Software Testing
Summary: Demonstrates how to leverage GPT models for generating test cases, API testing, and documentation, including prompt engineering examples and integration patterns.
Read More -
Machine Learning for Test Case Prioritization: A Developer's Guide
Summary: Provides a walkthrough for implementing ML-based test case prioritization—from feature engineering to CI/CD integration—with practical code samples and performance metrics.
Read More -
Visual Testing with AI: Beyond Traditional Automation
Summary: Covers advanced techniques in visual regression testing using AI, addressing challenges like dynamic content and cross-browser compatibility, along with practical implementation tips.
Read More
Contributions, suggestions, and improvements are welcome! Please open an issue or submit a pull request to help enhance this repository.
Note: The links provided lead to additional resources, academic papers, and GitHub repositories for further exploration of each topic.