Taming the Wild West of ML: Pr... Note

Taming the Wild West of ML: Practical Model Signing with Sigstore

The Google Open Source Security Team, in partnership with NVIDIA and HiddenLayer, has launched the first stable version of a model signing library as part of the Open Source Security Foundation. This library allows users to verify that the model used by an application is the same one created by the developers, using digital signatures like those from Sigstore. The rapid evolution of large language models (LLMs) has opened the door to new security threats, including model and data poisoning, prompt injection, and prompt evasion. The ML supply chain process is vulnerable to tampering, as models are an uninspectable collection of weights that can be altered by attackers. To achieve trust in models, users need to verify their integrity and provenance, which can be done through cryptographic signing. The ML supply chain involves three stages: training, fine-tuning, and embedding into an application, each handled by different teams or companies, creating opportunities for tampering. Model signing can prevent tampering by verifying the model's integrity at each stage. The released model signing library is a Python package that supports Sigstore and traditional signing methods, and can handle the scale of ML models. The goal is to extend model signing to include datasets and other ML-related artifacts, and to build tamper-proof metadata records that can automate incident response. The project aims to create a trust ecosystem for ML, and invites the open source community to join and shape its future.