Diffusionminva

[henrikfo]

Description

Summary of changes

• This pull request introduces the DiffMI attack with an example for CelebA. The main changes include adding a new input handler for the DiffMI attack, the Diff-Mi attack itself and extending the audit configuration with new parameters for the DiffMI attack.

DiffMI Attack Implementation:

• Instead of implementing a new CelebA_InputHandler in a file called celebA_diffmi_handler.py for the diffusion model training, a specific file called train_utils.py is created in attacks/utils/diffmi_utils/ with training specific to the DiffMI attack. The reason being that the training procedure and all its functions is very complex. IN LATER UPDATES TO THE PULL REQUEST CUSTOM LOSS FUNCTIONS MIGHT BE SUPPORTED?

Configuration Updates for DiffMI:

• Extended audit.yaml to include a new diffmi attack section with parameters for fine-tuning, preprocessing, pretraining, and attack-specific settings.

Evaluation Pipeline:

• Added a MinvResults class.

WiP

• Unclear if all unused function are completely removed. image_datasets.py for instance seems to be redundant and subject to deletion. Needs to verify for the remaining files and functions.
• 1200 ruff check issues left to fix. Most of them are function description and function input/output annotations.
• ....

How Has This Been Tested?

The attack has been tested with and without minibatch for H100 and 2080ti respectively. Fine-tuning on a 2080ti is not recommended since only a batch_size of 1 is possible.

[review-notebook-app]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[TheColdIce]

I have gone through all of the code now @henrikfo. Nice work!

• I have made some comments that needs to be fix/clarified.
• I think I have flagged all code that is not used.
• I have not flagged ruff errors. I also noticed some inconsistency in the docstring format, I dont know if ruff will flag this.
• I think also the wiki needs to be updated with how to run the attack. There is some params, use_fp16 for example, that is not clarified in the audit.yaml. Alternatively, the audit file can be flashed out.

If the comments are resolved, ruff checks are fixed and there is some clarification regarding config params, I think we can merge with main.

[henrikfo]

• I have made some comments that needs to be fix/clarified.

I believe that all of them are now resolved or responded to!

• I think I have flagged all code that is not used.

Those files/line have been removed!

• I have not flagged ruff errors. I also noticed some inconsistency in the docstring format, I dont know if ruff will flag this.

All ruff checked have passed and the inconsistencies in docstrings should be resolved aswell!

• I think also the wiki needs to be updated with how to run the attack. There is some params, use_fp16 for example, that is not clarified in the audit.yaml. Alternatively, the audit file can be flashed out.

Yes, the use_fp16 bug is fixed and I will create an new Issue about creating a comprehensive wiki for the attack!

If the comments are resolved, ruff checks are fixed and there is some clarification regarding config params, I think we can merge with main.

Great!