State-backed hackers and cybercriminals are increasingly exploiting Google’s Gemini AI to streamline their attacks from initial reconnaissance to post-compromise operations. According to new research, actors linked to China, Iran, North Korea, and Russia used the model for target profiling, phishing lure generation, translation, coding, vulnerability testing, command-and-control development, and data exfiltration.
Some operatives even posed as cybersecurity experts to trick the AI tool into producing detailed exploitation plans, including remote code execution (RCE) and web-application (WAF) firewall bypass techniques against specific targets in the US.[1]
Model extraction attack (Source: GTIG)
Iranian-linked threat group APT42 leveraged the model to accelerate social engineering campaigns and tailor malicious tools, while others integrated AI-assisted capabilities into malware such as a CoinBait phishing kit and HonestCue malware launcher. Criminal groups also used generative AI in ClickFix campaigns that delivered infostealing malware through deceptive troubleshooting ADs. Researchers also noted signs of AI-generated code in malware artifacts, indicating that generative platforms are already shaping attacker toolchains.
Beyond direct abuse, the report observed attempts to extract and replicate Gemini itself through large-scale querying and “knowledge distillation” techniques, in which actors use data from one model to train fresh, more advanced ones. While this mostly threatens AI vendors’ intellectual property, it could also eventually affect end users of the tool as AI-as-a-Service continues to rise.
Google says it has disabled the malicious accounts and continues to harden its defenses to limit misuse and make it more difficult to exploit. However, researchers warn that AI integration will likely accelerate threat actor capabilities across cybercrime ecosystems, lowering barriers to entry and increasing the speed, scale, and sophistication of future attacks.
This article is shared at no charge for educational and informational purposes only.
Red Sky Alliance is a Cyber Threat Analysis and Intelligence Service organization. We provide indicators of compromise information via a notification service (RedXray) or an analysis service (CTAC). For questions, comments or assistance, please contact the office directly at 1-844-492-7225, or feedback@redskyalliance.com
Weekly Cyber Intelligence Briefings:
- Reporting: https://www.redskyalliance.org/
- Website: https://www.redskyalliance.com/
- LinkedIn: https://www.linkedin.com/company/64265941
Weekly Cyber Intelligence Briefings:
REDSHORTS - Weekly Cyber Intelligence Briefings
https://register.gotowebinar.com/register/5207428251321676122
[1] https://www.sentinelone.com/blog/the-good-the-bad-and-the-ugly-in-cybersecurity-week-7-7/
Comments