Microsoft's Digital Crimes Unit is pursuing legal action to disrupt cybercriminals who create malicious tools that evade the security guardrails and guidelines of generative AI (GenAI) services to create harmful content. According to a spokesman, it has observed a threat group seeking out vulnerable customer accounts using generative AI, then creating tools to abuse these services.[1]
See: https://redskyalliance.org/xindustry/microsoft-s-new-copilot-ai-agents
According to an unsealed complaint in the US Court of the Eastern District of Virginia, the company goes to great lengths to create and enhance secure AI products and services, cybercriminals continue to innovate their tactics and bypass security measures. "With this action, we are sending a clear message: the weaponization of our AI technology by online actors will not be tolerated," said Microsoft in a blog post about the lawsuit.
In the court filings that were unsealed on 13 January 2025, Microsoft noted that it had "observed a foreign-based threat-actor group develop sophisticated software that exploited exposed customer credentials scraped from public websites." The group tried to access accounts with generative AI services to alter the capabilities of those services, then resold this unlawful access to other malicious actors, providing instructions on how to use the tools to create harmful content.
Since discovering the group's actions, Microsoft has revoked access and enhanced safeguards to mitigate this kind of activity in the future. As the company continues to seek out proactive measures it can take alongside legal action, it highlights a report, "Protecting the Public From Abusive AI-Generated Content," that provides recommendations for organizations and governments to protect the public from AI-created threats.
This article is shared at no charge for educational and informational purposes only.
Red Sky Alliance is a Cyber Threat Analysis and Intelligence Service organization. We provide indicators of compromise information via a notification service (RedXray) or an analysis service (CTAC). For questions, comments or assistance, please contact the office directly at 1-844-492-7225, or feedback@redskyalliance.com
Weekly Cyber Intelligence Briefings:
- Reporting: https://www.redskyalliance.org/
- Website: https://www.redskyalliance.com/
- LinkedIn: https://www.linkedin.com/company/64265941
Weekly Cyber Intelligence Briefings:
REDSHORTS - Weekly Cyber Intelligence Briefings
https://register.gotowebinar.com/register/5378972949933166424
[1] https://www.darkreading.com/application-security/microsoft-cracks-down-malicious-copilot-ai-use
Comments