A few days ago (July 16, 2026) Hugging Face, the world’s largest AI model platform, released aInformation Security Announcement, revealed an unprecedented attack: a fully automated AI agent system invaded their infrastructure from the data processing pipeline and performed more than 17,000 automated operations in one weekend, moving laterally between multiple internal clusters. What is even more surprising is the subsequent processing. The Hugging Face team initially tried to use the API of American closed-source cutting-edge models to analyze attack logs, but the real attack instructions, vulnerability payloads and C2 data triggered the security guardrails of those models, causing the models to directly refuse to execute (which models were not announced). In the end, they could only deploy the open source model GLM 5.2 of China’s Z.ai (Wisdom Spectrum AI) on their own servers to complete the analysis.
Attack techniques: from malicious data sets to global lateral movement
The starting point of this intrusion is the most vulnerable link of the AI platform, the data processing pipeline. The attacker uploaded a malicious dataset that exploited two code execution vulnerabilities in the Hugging Face dataset processing system (one in the remote code dataset loader and the other in a template injection in the dataset configuration) to execute arbitrary code on the processing node.
After gaining initial access, the attacker further advanced to node-level control, captured cloud and cluster authentication credentials, and moved laterally into multiple internal clusters over the course of a weekend. The entire attack is driven by an autonomous Agent framework, which appears to be based on some kind of Agentic security research tool, but the specific model used is still unclear.
The scale of the attack is remarkable. This agent system operated in a large number of short-lived sandbox environments, performed thousands of individual operations, and was ultimately recorded by Hugging Face as more than 17,000 attack events. Its command and control (C2) system is even self-migrating, hiding the C2 infrastructure in public network services.
Scope of damage: limited but real
According to Hugging Face, the attackers did gain access to a portion of the internal data set, as well as authentication credentials used by several services. However, the company says it has found no evidence that public models, datasets or Spaces have been tampered with, and its software supply chain (container images and released packages) has been verified to be clean.
Hugging Face is still evaluating whether any partners or customers’ profiles were affected and pledges to proactively contact any affected parties. As a precautionary measure, the company recommends that all users rotate access tokens and review recent account activity.
In the aftermath, Hugging Face has closed the exploited vulnerability, rebuilt the compromised machine nodes, rotated all affected credentials, and deployed tighter cluster admission controls. The company has also reported the incident to law enforcement agencies and introduced external security forensic experts to investigate.
Forensic Dilemma: Closed Source Model’s Security Guardrails Block Defenders
The most noteworthy twist in this incident occurred during the investigation phase. Hugging Face’s anomaly detection pipeline itself used LLM to classify security telemetry data. It was the signal correlation function of this system that first flagged the intrusion. However, to understand what an Agent that performed tens of thousands of automated operations actually did, the team needed to submit real attack instructions, exploit code, and other offensive samples to the model for analysis.
The Hugging Face team initially tried to use the cutting-edge model of a commercial API to perform this forensic work, but the security guardrails of the commercial model were unable to distinguish between “incident responders” and “attackers” and directly rejected the analysis request. As Hugging Face said in the announcement: “The attackers were not subject to any usage policies, and our own forensic efforts were blocked by the security guardrails of the hosting model we tried in the first place.”
In fact, many security researchers have complained in the past that some cutting-edge models (such as Anthropic’s Mythos and Fable) are overly restrictive and even block security queries for defensive and investigative purposes. Even Anthropic was eventually forced to withdraw Fable from public use because the U.S. government imposed export controls on the model.
After the commercial API model refused to cooperate, Hugging Face turned to Z.ai’s open source model GLM 5.2, which was deployed and executed on its own infrastructure. Results GLM 5.2 analyzed more than 17,000 attack records in a matter of hours, reconstructing a complete attack timeline, locating stolen credentials, and distinguishing between real attacks and decoy campaigns used to waste responder time.
According to Zhipu published on GitHubTechnical documents, GLM 5.2 is the flagship model of Zhipu AI. It has 744B total parameters (40B active parameters), adopts MoE (Mixed Expert) architecture, and supports a context window of 1 million tokens. In the standard code benchmark test, GLM 5.2 is the strongest open source model: Terminal-Bench 2.1 scored 81.0 (Claude Opus 4.8 is 85.0), and SWE-bench Pro scored 62.1, significantly ahead of the 62.0 and 58.4 of the previous generation GLM-5.1.
Another key advantage of using an open source model is data security: the attacker’s data and exposed credentials remain entirely within Hugging Face’s internal environment, and no sensitive information is leaked to third-party servers.
Asymmetric Dilemma: The attacker is unlimited and the defender is bound by guardrails
Hugging Face points to a central contradiction in its announcement. They don’t know what model the attackers were using, whether it was a jailbroken hosted model or an unrestricted open source model. In either case, attackers are completely exempt from usage policies, while defenders are blocked by the security guardrails of the business model they use.
The pragmatic advice given by Hugging Face is: before an incident occurs, prepare a powerful model that can be executed on your own infrastructure. Doing so avoids guardrails and ensures that attacker data and exposed credentials do not leave your environment. The company also emphasizes that this is not a security measure against hosted models and has communicated feedback to model providers.
Hugging Face bluntly stated in the announcement: “Autonomous, AI-driven attack tools are no longer theoretical deductions. It reduces the cost of executing patient, multi-stage attack campaigns and operates at machine speed.” For anyone operating an online platform, the data layer and model layer are now the front-line attack surface.
Conclusion
The Hugging Face incident is the first case where a large-scale AI platform has publicly confirmed that it has been attacked end-to-end by an autonomous AI Agent. It is also the first time that forensic analysis has been completed in an AI-driven manner. The attacker and defender entered the Agent era at the same time, but they faced completely different rules. However, when attackers are free to use any unrestricted model, but defenders are tied by the security mechanisms of the business model, whoever can independently execute, audit, and control the model will have the security initiative in the era of machine speed. This is no longer just a technical issue, but about the strategic choice between openness and closure in the AI ecosystem, which may also be something the US government will consider when restricting the use of models in the future.
Source: KOCPC Chinese