Security practitioners face twin pressures as artificial intelligence enters everyday penetration testing. Teams use AI to discover weaknesses faster while also checking the AI systems their organizations introduce. New findings from Pentest-Tools.com show demand already outstrips capacity for most groups. Nearly nine out of ten practitioners who have generated findings with AI report that the results need substantial manual checking.[1]
Among 147 respondents who had used such tools, 87.8% said the output required significant human review. Common problems included duplicate entries, issues that could not be exploited, and invented CVEs that simply do not exist. One security manager summed up the resulting loss of trust: “Confidence that turns out to be just a big lie.” Roughly 30% of free-text answers listed hallucinated exploits or fabricated findings as their single greatest frustration with the tools, ranking above cost or integration difficulties.
Only 20.3% of those surveyed said they possessed a workflow capable of handling 500 or more AI-generated candidates from one engagement. A further 38.6% stated the volume would strain their team, while 29.7% described it as unmanageable. The pattern is clear: discovery accelerates, but confirming which findings are genuine has become the limiting step.
Respondents repeatedly pointed to business logic as the largest technical shortcoming. AI systems can locate a SQL injection, yet they rarely grasp how an application is meant to behave. Examples given included a discount coupon that should apply only once per customer, the ability to add a negative quantity to a shopping cart and obtain free goods, or the simple act of swapping a user identifier in a URL to view another person’s data. These flaws leave no error signature and require an understanding of intended workflow that current tools lack.
At the same time, examinations of AI-powered and large-language-model systems is moving into routine work. Three-quarters of practitioners (75.3%) already include such systems in their engagements. Another 17.1% expect to begin within twelve months, bringing the combined figure to 92.4%. Internal pressure is also growing. More than one-third (37.3%) reported that stakeholders now request more frequent testing than a year earlier, driven by greater awareness of AI-assisted attacks.
In an expert comment, Adrian Furtuna, chief executive and founder of Pentest-Tools.com, commented: “There have been significant advances in how AI is accelerating vulnerability discovery. The challenge now is making sure those findings are accurate enough to limit manual work, instead of creating more of it.”
The survey of 158 security practitioners, including penetration testers, security engineers, application-security specialists, DevSecOps staff, consultants, and managed-service providers, took place in June 2026. All respondents already employed AI-assisted tools in assessment and validation. The research focuses on the daily experience of those performing the tests rather than on buyers or executives. AI has shifted the bottleneck from finding candidates to proving which ones matter. Teams that can triage large volumes efficiently are better placed to keep pace; those without formal processes risk spending more time on verification than they previously spent on discovery itself.This article is shared at no charge for educational and informational purposes only.
Red Sky Alliance is a Cyber Threat Analysis and Intelligence Service organization. We provide indicators of compromise information (CTI) via a notification/Tier I analysis service (RedXray) or an analysis service (CTAC). For questions, comments, or assistance, please contact the office directly at 1-844-492-7225 or feedback@redskyalliance.com
- Reporting: https://www.redskyalliance.org/
- Website: https://www.redskyalliance.com/
- LinkedIn: https://www.linkedin.com/company/64265941
Weekly Cyber Intelligence Briefings:
REDSHORTS - Weekly Cyber Intelligence Briefings
https://attendee.gotowebinar.com/register/7855487668891299929
[1] https://www.cybersecurityintelligence.com/blog/ai-pentesting-floods-teams-with-unreliable-vulnerability-findings-9603.html
Comments