Yuval Elovici

Data owners often entrust third parties with private datasets to develop machine learning (ML) models for various purposes (ML as a service). This raises a critical concern: how can data owners verify that the model has been trained exclusively on the authorized dataset, without any unauthorized data additions? Unauthorized data additions can lead to significant specification violations and potential breaches of privacy regulations. Additionally, they can introduce unwanted bias into the outcomes, compromising the model's integrity. To address this, we introduce TSV - Training Set Verification - a technique that verifies whether unauthorized data has been added to the training set of an ML pipeline. Given an examined model (a suspected model which was allegedly only trained on the authorized data), TSV leverages a trusted verifier to provide verification services for the data owner. The verifier first trains an out-of-distribution detector and shadow models using the authorized dataset. Next, prototypes are extracted from both the shadow and examined models. By analyzing these prototypes, TSV can diagnose whether the training set of the examined model contains unauthorized data. Our evaluation, conducted on both tabular and image datasets, demonstrates TSV's effectiveness in detecting even minor unauthorized data additions while maintaining a controlled false positive of 10 percent.