AI Agents Turned Into Attackers: Hugging Face Reveals Autonomous Intrusion Campaign

Hugging Face says an autonomous AI agent breached part of its production infrastructure and accessed internal data and service credentials.

Hugging Face is one of the world’s leading open-source AI companies. It provides a platform where developers and organizations can build, share, and deploy machine learning and generative AI models.

Hugging Face disclosed that an autonomous AI agent breached part of its production infrastructure last week. The company detected the intrusion, contained it, and found unauthorized access to a limited number of internal datasets and service credentials. The investigation is still ongoing, but there is no evidence the attackers modified public AI models, datasets, Spaces, or the company’s software supply chain.

“We identified unauthorized access to a limited set of internal datasets and to several credentials used by our services. We are still completing our assessment of whether any partner or customer data was affected, and we will contact any affected parties directly as required.”  states the security incident disclosure. “We have found no evidence of tampering with public, user-facing models, datasets, or Spaces, and our software supply chain (container images and published packages) was verified clean.”

The attack began in Hugging Face’s data-processing pipeline, where a malicious dataset exploited two code execution flaws to compromise a processing worker. The attackers escalated privileges, stole cloud and cluster credentials, and moved laterally across internal systems.

Hugging Face said the operation was driven by an autonomous AI agent framework that executed thousands of actions across short-lived sandboxes and used public services for self-migrating command-and-control, reflecting the rise of AI-powered, agentic attacks.

“The campaign was run by an autonomous agent framework (appearing to be built on an agentic security-research harness – used LLM still not known) executing many thousands of individual actions across a swarm of short-lived sandboxes, with self-migrating command-and-control staged on public services.” continues the company. “This matches the “agentic attacker” scenario the industry has been forecasting.”

Hugging Face closed the vulnerabilities that allowed the initial compromise, removed the attackers’ access, and rebuilt the affected systems. The company revoked and rotated compromised credentials, launched a broader secrets rotation, strengthened security controls across its clusters, and improved monitoring to detect similar attacks within minutes.

The company is investigating the incident with the help of external cybersecurity forensic experts, reviewing its security practices, and has already notified law enforcement.

As a precaution, the firm advises users to rotate their access tokens and review recent account activity for suspicious behavior. Anyone who believes they may have been affected can contact the company’s security team. The company apologized for the disruption, thanked its incident response teams, and said it will continue strengthening its security defenses.

Hugging Face used AI-based security tools to detect and investigate the intrusion. Its anomaly detection system identified suspicious activity, while LLM-powered analysis agents reviewed more than 17,000 attacker actions to reconstruct the attack timeline, identify compromised credentials, and assess the real impact within hours instead of days.

During the investigation, the company found that commercial AI models blocked forensic analysis because their safety controls flagged real attack data as potentially harmful. The company used an open-weight model, Z.ai’s GLM 5.2, running on its own infrastructure, keeping sensitive information inside its environment.

The incident highlights a growing challenge: attackers can use autonomous AI agents without restrictions, while defenders need secure AI tools ready to analyze threats quickly. AI-driven attacks are becoming a real risk, making data and AI systems a critical part of the security perimeter.

“We do not know which model powered the attacker’s agents, whether a jailbroken hosted model or an unrestricted open-weight one; either way, the attacker was bound by no usage policy, while our own forensic work was blocked by the guardrails of the hosted models we first tried.” concludes the statement. “The practical lesson for defenders: have a capable model you can run on your own infrastructure vetted and ready before an incident, both to avoid guardrail lockout and to keep attacker data and credentials from leaving your environment. This is not an argument against safety measures on hosted models, and we are sharing this feedback with the providers concerned.”

Follow me on Twitter: @securityaffairs and Facebook and Mastodon

Pierluigi Paganini

(SecurityAffairs – hacking, artificial intelligence)