Yuval Elovici

Although diffusion models are widely used in tasks such as image synthesis and data augmentation, they may raise privacy concerns. Recent studies examining privacy-related risks to diffusion models have mainly focused on membership inference attacks (MIAs), i.e., predicting whether a particular instance is a member of a target model’s training or fine-tuning data. However, most MIAs assume white/gray-box access to the target model or focus on denoising diffusion probabilistic models. Moreover, none of them make use of information that can be inferred when models are fine-tuned on an individual instance; such information can provide evidence as to whether the image is a member of the training set. This paper introduces a novel MIA approach designed for generative models, which we implement in the TuneMIA attack targeted at diffusion models. TuneMIA leverages the fine-tuning process, utilizing memorization in weights and gradients, without the need for direct access to the model. In our evaluation of TuneMIA, performed on two datasets and attack scenarios, TuneMIA outperformed SOTA MIAs on diffusion models in all examined cases, indicating that the fine-tuning process exposes information about the target model that can contribute to an MIA’s success. We also show that TuneMIA can be used to detect whether an entire dataset was used to fine-tune a model, making it useful for data owners who want to identify unauthorized use of their data. The code is publicly available at https://github.com/noamazulay274/TuneMIA.