Does a backdoor defense still work after the model is quantized, and does the order of the two steps matter? This project reproduces and extends "Towards robustness evaluation of backdoor defense on quantized models" on DistilBERT / IMDb sentiment classification.
- Backdoor injection: trigger-phrase poisoning with forced label.
- Four defenses: Fine-Pruning, ONION, STRIP and Neural Cleanse.
- Three quantization schemes: post-training quantization (PTQ), quantization-aware training (QAT) and layer-adaptive quantization (LAQ) at 4, 6 and 8 bits, plus mixed precision driven by per-layer sensitivity.
- Two pipelines: defense then quantization, and quantization then defense, evaluated on clean accuracy, attack success rate and robust accuracy.
- Layer sensitivity analysis: nine metrics (Hessian trace, activation range, gradient norm, weight statistics and more) to rank layers by quantization fragility and allocate bits accordingly.
| File | Role |
|---|---|
improved_poison_test.py |
Main experiment driver: dataset poisoning, defenses, quantization manager, experiment runner and LayerSensitivityAnalyzer |
quantization_analysis.py |
Standalone quantization-error analysis across bit widths with adversarial input engineering |
add_baseline_analysis.py |
Adds full-precision baselines to the insight reports |
fine_tune_model.py |
Fine-tunes the clean DistilBERT classifier |
test_framework.py, test_defense_implementations.py |
Smoke tests for the pipeline and defenses |
advers/ |
Earlier iteration of the framework (Poison_test.py, quantization_framework.py, layer-wise attack scripts) with its CSV/JSON results and plots |
*.md |
Design notes: quantization improvements, Neural Cleanse / Fine-Pruning fixes, PyTorch quantization limitations, sensitivity analysis |
quantization_analysis_{4,6,8}bit.png, quantization_insights_report*.json |
Results |
pip install -r requirements.txt
python fine_tune_model.py # clean model
python improved_poison_test.py # full defense x quantization matrix
python quantization_analysis.py # per-bit-width error analysisA single GPU is sufficient; DistilBERT fine-tuning takes a few minutes. Third-party baselines were run from upstream clones of BackdoorBench and OpenBackdoor, which are not vendored here.
Course project (advanced computer architecture), done with a partner whose quantization-sensitivity codebase lives in a separate repository.