Adversarial Graph Neural Network Benchmarks
Towards Practical and Fair Evaluation

About the project

GAB is a benchmarking framework for evaluating adversarial attacks and defenses on Graph Neural Networks (GNNs) under standardized and rigorous experimental settings.

Our goal is to eliminate inconsistencies in prior evaluations by providing a unified framework that enables fair comparison across methods. We systematically re-evaluate widely used attacks and defenses under both poisoning and evasion scenarios across multiple graph datasets, revealing critical factors that significantly impact reported performance.

Benchmarking Framework Highlights

• 🧪 Unified benchmark for adversarial GNN evaluation across diverse settings
• 📊 Comprehensive study covering attacks, defenses, and multiple datasets
• 🔍 Reveals hidden factors (e.g., node selection, training procedures) affecting robustness
• ⚖️ Standardized protocols for fair and reproducible comparisons
• 🚀 Scalable experimental pipeline (400K+ runs) for robust analysis

To setup the environment, please check out setup.md

Experiments

To run evaluation on adversarial attacks and defense. An example command can be used:

python experiments/adversarial_attack_evaluation.py --model=GCN --adversarial=l1d_rnd_attack 

Custom args:

• num_run : define number of runs with different seed
• num_split : number of random split per dataset for evaluation
• purification : select purification methods from list of purifications.
• config_setting: Select either best_config or default. best_config use the best configuration of victim model selected from model selection.
• model : name of victim model selected from list of models.
• adversarial : name of adversarial attack selected from list of adversarial attacks.
• evasion or no-evasion : to perform evaluation in evasion setting (True or evasion by default)
• poison or no-poison : to perform evaluation in poison setting (True or poison by default)
• use-node-degree or no-use-node-degree : to perform evaluation with node degree as an extra criteria to select target node (True or use-node-degree by default)
• dataset: dataset to perform experience on, select from list of supporting datasets.

Model selection

To perform model selection, use hyper_tuning_all_split.py to perform hyperparameters search on GNN backbone. An example command as follows:

python experiments/hyper_tuning_all_split.py --model=GCN --dataset=cora

Custom args:

• dataset: dataset to perform experience on
• model : name of victim model selected from list of models.

Or model selection on purification as follows:

python experiments/hyper_tuning_filter_all_split.py --model=GCN --dataset=cora --purification=GARNET

Custom args:

• dataset: dataset to perform experience on
• model : name of victim model selected from list of models.
• purification: purification method from list of models.

Supporting Purification methods

Method	Paper
GARNET	GARNET: reduced-rank topology learning for robust and scalable graph neural networks
Jaccard	Adversarial Examples on Graph Data: Deep Insights into Attack and Defense

Supporting Victim Models

Method	Paper
GCN	GARNET: reduced-rank topology learning for robust and scalable graph neural networks
GSAGE	Adversarial Examples on Graph Data: Deep Insights into Attack and Defense
GIN	Adversarial Examples on Graph Data: Deep Insights into Attack and Defense
PNA	Adversarial Examples on Graph Data: Deep Insights into Attack and Defense
GRAND	Graph random neural networks for semi-supervised learning on graphs
GCORN	Bounding the expected robustness of graph neural networks subject to node feature attacks
NoisyGNN	A simple and yet fairly effective defense for graph neural networks
GNNGuard	GNNGuard: Defending Graph Neural Networks against Adversarial Attacks
RobustGCN	Robust graph convolutional networks against adversarial attack
ElasticGNN	Elastic graph neural networks
RUNG	Robust graph neural networks via unbiased aggregation

Supporting Adversarial Attacks

Method	Paper
Nettack	Adversarial attacks on neural networks for graph data
FGA	Fast gradient attack on network embedding
SGA	Adversarial attack on large scale graph.
GOttack	GOttack: Universal Adversarial Attacks on Graph Neural Networks via Graph Orbits Learning
PR-BCD	Robustness of Graph Neural Networks at Scale
PGD	Topology attack and defense for graph neural networks: An optimization perspective
L1D-RND	Adversarial Graph Neural Network Benchmarks: Towards Practical and Fair Evaluation

Supporting datasets

Dataset Name
citeseer
citeseer_full
cora
cora_ml
cora_full
amazon_cs
amazon_photo
coauthor_cs
coauthor_phy
polblogs
karate_club
pubmed
flickr
blogcatalog
dblp
acm
uai
pdn
Roman-empire
Amazon-ratings
Minesweeper
Tolokers
Questions
chameleon
crocodile
squirrel

and OGB dataset.

Tool

Orbit discovery

GOttack requires precompute orbit discovery on the dataset, PyORCA is available in utility. Please check out PyORCA Github for setting up instruction.

Reference

Our adversarial benchmark is built upon GreatX and DeepRobust. We appreciate their contribution to graph adversarial learning.