Attackers Are Learning to Live Off the AI Toolchain

31209119484?profile=RESIZE_400xAttackers are beginning to hide malicious activity inside trusted AI coding assistants and CI pipelines, mimicking routine developer and automation behavior so closely that some attack techniques can evade current detection tools entirely.  One early manifestation of the emerging threat is Sandworm_Mode, a self-propagating worm that spreads through malicious npm packages.  Researchers at Socket Security who discovered the threat earlier this year have described it as a Shai-Hulud-style worm that hijacks CI workflows and poisons AI toolchains.[1]

See:  https://redskyalliance.org/xindustry/new-sandworm-mode

CrowdStrike subsequently analyzed Sandworm_Mode's known behaviors to identify the ones the company could detect using existing telemetry and to see how many new detection rules it could develop for the malware.  CrowdStrike's findings, detailed in a report this week, offer a sobering picture of the challenges organizations will face as attackers increasingly abuse the AI toolchain in a manner like living-off-the-land attacks.

"The Sandworm_Mode campaign forces a recalibration of expectations for endpoint detection in AI-augmented environments," CrowdStrike security researcher John Prieto wrote.  Of the 14 behaviors CrowdStrike investigated, only nine generated detectable signals and just two produced signals reliable enough to trigger customer alerts.  The remaining behaviors so closely resembled legitimate development and automation behavior that it was not possible to reliably distinguish them from legitimate activity.

"Without understanding what 'normal' looks like for MCP server deployments, AI assistant configuration writes, and LLM API key usage in a given environment, there is no foundation for anomaly-based detection," Prieto said. "This telemetry class is new, and baselines are still being established across the industry."

The Sandworm_Mode campaign spread through 19 malicious npm packages and exploited the normal runtime behaviors of AI coding assistants, CI automation, and LLM toolchains that organizations are increasingly deploying in their development pipelines.  The worm is designed to steal npm, GitHub, cloud, cryptocurrency, and LLM-provider credentials, and exfiltrate them across three channels, including DNS tunneling. It propagates by infecting packages and repositories and establishes persistence through Git hooks. Sandworm_Mode also compromises AI assistants such as Cursor and Claude Code via a rogue MCP server that uses prompt injections to trick them into silently reading and passing credentials to the attacker. Sandworm_Mode features a 48- to 96-hour de

CrowdStrike found that some Sandworm_Mode behaviors did produce usable detection signals. These included behaviors such as malicious package activity, certain persistence mechanisms, and specific forms of credential or data theft.  The security vendor found that several other behaviors were effectively indistinguishable from legitimate developer and automation activity. Sandworm_Mode executed commands, accessed files, modified configurations, interacted with repositories, and called APIs in ways that closely resembled the normal behavior of AI assistants, CI/CD systems, and other development tools.

CrowdStrike framed the behavior as the AI-era version of living off the land, where instead of leveraging trusted native tools like PowerShell and certutil, attackers are abusing trusted AI coding assistants and CI pipelines.  "As AI coding agents become the standard for software development, adversaries are learning to live off the AI toolchain by exploiting the same trusted workflows developers use every day," says Adam Meyers, head of counter adversary operations at CrowdStrike, in comments to Dark Reading. "Think of it like discovering a novel pathogen. It exposes a new attack vector and a weakness in the AI development ecosystem that the security community needs to understand and address together."  The key takeaway is not just finding better indicators of compromise but having the right telemetry and understanding of behavioral context to separate malicious activity from normal activity.

"The challenge is that you're looking for a needle in a needle stack," Meyers says. "AI coding assistants and attacks like Sandworm_Mode generate much of the same telemetry, making it extremely difficult to distinguish legitimate behavior from malicious behavior using traditional approaches alone. AI has poured gasoline on an existing supply chain problem, and security has to evolve alongside it."

CrowdStrike characterized Sandworm_Mode as more of a proof of concept for an emerging attack class that organizations need to be aware of. Organizations need to extend security into the AI development life cycle," Meyers notes. That starts with protecting developer identities and credentials, increasing visibility across package repositories and CI/CD pipelines, and isolating AI development environments where appropriate, he says.  "The challenge is that these attacks deliberately blend into normal development activity. They're publishing packages, creating commits, and opening pull requests, so defenders need behavioral visibility that can separate legitimate automation from adversary automation."

 

This article is shared at no charge for educational and informational purposes only.

Red Sky Alliance is a Cyber Threat Analysis and Intelligence Service organization.  We provide indicators of compromise information (CTI) via a notification/Tier I analysis service (RedXray) or an analysis service (CTAC).  For questions, comments, or assistance, please contact the office directly at 1-844-492-7225 or feedback@redskyalliance.com    

Weekly Cyber Intelligence Briefings:
REDSHORTS - Weekly Cyber Intelligence Briefings
https://attendee.gotowebinar.com/register/7855487668891299929

 

[1] https://www.darkreading.com/cyber-risk/attackers-live-off-ai-toolchain

E-mail me when people leave their comments –

You need to be a member of Red Sky Alliance to add comments!