Cross-model jailbreak defense via representation repulsion. Companion code release for the AnchorRep paper, accepted at NeurIPS 2026.
Paper: AnchorRep: Defending LLMs Against Cross-Model Adversarial Transfer via Representation Repulsion (NeurIPS 2026). Adapters: huggingface.co/galwert.
AnchorRep trains a lightweight LoRA adapter that pushes the defended model's internal representations of harmful prompts away from those of a frozen anchor model, reducing cross-model adversarial transfer to under 2% across five defended models without attack-specific training.
| Path | Contents |
|---|---|
src/anchor_rep/ |
Training code (train.py) and YAML config wrapper (run_with_config.py). |
eval/ |
Evaluation scripts: cross-model GCG transfer, adaptive attacks, benchmarks, WildGuard judge. |
configs/ |
Per-defender YAML configs reproducing each row of tab:comparison. |
data/ |
AdvBench-30 training split (seed 42) + 10 refusal templates. |
attack_artifacts/ |
GCG suffixes used in evaluation (AdvBench-derived for cross-model transfer; HarmBench-derived for OOD). |
logs/ |
All evaluation logs and the manual-verification audit log. |
scripts/ |
Reproduction shell scripts. |
docs/ |
Reproducibility recipe, config reference, usage guidelines. |
rebuttal_neurips_2026/ |
NeurIPS 2026 review-response materials: final rebuttal text, the full manual-verification audit (1,090 rated items, verified Table 3), SLURM logs and per-run configs/metrics for every rebuttal-round experiment. |
Companion HuggingFace adapters (5 trained defenders): https://huggingface.co/galwert.
git clone https://github.com/galwert/AnchorRep
cd AnchorRep
pip install -e .CUDA-capable GPU is required for training and most eval scripts. Adapter training fits on a single L40S (48 GB).
Four common workflows are wrapped in single shell commands:
Train a defender from scratch with paper hyperparameters:
bash scripts/train_one.sh mistralOr with a custom hyperparameter config (copy configs/custom.yaml.example to configs/custom.yaml and edit):
bash scripts/train_one.sh customOr override individual flags on the command line:
bash scripts/train_one.sh mistral --gamma 0.5 --stage2_steps 400Evaluate the released HuggingFace adapter for a defender (default):
bash scripts/eval_only.sh mistralEvaluate a locally-trained adapter:
bash scripts/eval_only.sh mistral runs/mistral/adapterEvaluate any HuggingFace adapter compatible with the base model:
bash scripts/eval_only.sh mistral some-user/SomeOtherAdapterbash scripts/train_and_eval.sh mistral
bash scripts/train_and_eval.sh customThis runs train_one.sh followed immediately by eval_one.sh on the freshly-trained adapter.
bash scripts/reproduce_main.shLoops over all 5 defenders and produces per-row outputs that aggregate into tab:comparison.
More examples: docs/usage_examples.md is a copy-paste cookbook covering every common workflow (eval-only, train-only, train+eval, custom configs, CLI overrides, full reproduction, anchor caching, batch sweeps).
Step-by-step recipes mapped to each paper table: docs/reproducibility.md.
Full YAML key reference: docs/config_reference.md.
- CKA repulsion against a frozen anchor model on harmful prompts.
- Refusal-direction projection (Arditi et al. 2024).
- Coherency loss preserving benign hidden states.
- KL preservation on borderline benign prompts (XSTest safe subset).
- LM loss on harmful prompts (refusal targets).
Implemented as a single LoRA adapter applied at the mid-stream residual layer (50% of model depth). All five losses combine into one objective minimized over 200 to 600 gradient steps. Full method: paper Section 2.
Training uses 30 prompts sampled uniformly at random from the 520-prompt AdvBench pool with seed 42 (predetermined split, disjoint from the held-out evaluation pool). The exact 30 prompts are in data/advbench_train_split.json. Borderline prompts come from XSTest (200 safe prompts), benign prompts from WikiText-2.
- Code (
src/,eval/,scripts/,pyproject.toml): MIT License (seeLICENSE-CODE). - Data artifacts (
configs/,data/,attack_artifacts/,logs/, model card text): CC BY 4.0 (seeLICENSE-DATA).
External assets (datasets, base models, anchor models, judge classifier) are cited at point of use; see the paper appendix Asset Licenses table for full attribution.
This release is intended for defensive security research on open-weight LLMs. The released GCG suffixes are derived from publicly available attack methods on public prompts and represent a routine extension of existing practice (attack_artifacts/README.md). Do not deploy adapters to production without independent safety validation.
@inproceedings{anchorrep2026,
title={AnchorRep: Defending LLMs Against Cross-Model Adversarial Transfer via Representation Repulsion},
author={Wertheizer, Gal and Himelstein, Rom and Peretz, Tomer and Mendelson, Avi},
booktitle={Advances in Neural Information Processing Systems (NeurIPS)},
year={2026},
url={https://neurips.cc/virtual/2026/poster/149182}
}