CapST

Reference implementation of CapST: Leveraging Capsule Networks and Temporal Attention for Accurate Model Attribution in Deep-fake Videos (ACM Transactions on Multimedia Computing, Communications, and Applications, 21(4), Article 123, 2025. https://doi.org/10.1145/3715138).

CapST attributes a deep-fake image or video to the generative model that produced it (as opposed to plain real-vs-fake detection). A VGG19 backbone extracts low-level features, a Capsule Network with dynamic routing turns them into a set of per-class capsules, and — for video input — a temporal attention module fuses per-frame predictions into a single video-level decision.

Repository layout

CapST/
├── models/
│   ├── capsule_network.py   # Primary capsules + dynamic routing (Fig. 5)
│   └── capst.py              # VGG19 backbone + CapsuleNetwork + temporal fusion (Fig. 3/4)
├── configs/
│   ├── dfdm_config.py        # Hyperparameters for the DFDM experiment
│   └── gangen_config.py      # Hyperparameters for the GANGen-Detection experiment
├── datasets/
│   ├── dfdm_dataset.py       # Video-clip loader, 5 autoencoder classes
│   └── gangen_dataset.py     # Image loader, 9 GAN classes
├── training/
│   ├── train_dfdm.py         # Train/evaluate on DFDM
│   └── train_gangen.py       # Train/evaluate on GANGen-Detection
├── utils/
│   └── split_gangen_data.py  # 70/30 split for GANGen-Detection
└── gradcam/
    └── visualize_capst.py    # Grad-CAM visualization (Fig. 7)

Two model variants live in models/capst.py:

Class	Dataset	Input	Classes
CapST	DFDM	10-frame face-crop clip	5
CapSTStatic	GANGen-Detection	single face/image	9

Both share the same CapsuleNetwork module; only the classifier head and the presence of temporal fusion differ.

Installation

pip install -r requirements.txt

Tested with PyTorch 2.x and Python 3.10+. A CUDA GPU is required (training and Grad-CAM scripts move tensors to .cuda() directly).

Data

DFDM — request access from the dataset authors (see the DFDM dataset paper cited below), extract faces with a tool such as OpenFace, and lay out the frames as:

<train_dir>/df_fs/<video_id>/000001.png ...      # FaceSwap
<train_dir>/df_lw/<video_id>/000001.png ...      # Lightweight
<train_dir>/df_iae/<video_id>/000001.png ...     # IAE
<train_dir>/df_dfaker/<video_id>/000001.png ...  # Dfaker
<train_dir>/df_dfl/<video_id>/000001.png ...     # DFL-H128

GANGen-Detection — download from the GANGen-Detection repository, then split it:

python -m utils.split_gangen_data --source /path/to/GANGen-Detection --output /path/to/split

which produces <output>/train/<GAN_name>/*.png and <output>/test/<GAN_name>/*.png for the 9 classes (AttnGAN, BEGAN, CramerGAN, InfoMaxGAN, MMDGAN, RelGAN, S3GAN, SNGAN, STGAN).

Training

# DFDM (5-class video model attribution)
python -m training.train_dfdm --train_dir /path/to/dfdm/train --test_dir /path/to/dfdm/test

# GANGen-Detection (9-class image model attribution)
python -m training.train_gangen --train_dir /path/to/split/train --test_dir /path/to/split/test

Both entry points default to the hyperparameters reported in the paper (configs/dfdm_config.py, configs/gangen_config.py) and accept overrides as command-line flags, e.g. --lr, --max_epochs, --train_bs. Run with --help for the full list. Checkpoints are written to --checkpoint_dir whenever test accuracy improves; TensorBoard logs go to --log_dir.

Run the scripts from the repository root so the models, configs, and datasets packages resolve correctly.

Grad-CAM visualization

python -m gradcam.visualize_capst \
    --checkpoint /path/to/checkpoint.pth \
    --images dfaker.png dfl_h128.png faceswap.png iae.png lightweight.png \
    --titles Dfaker DFL-H128 FaceSwap IAE Lightweight

Produces the capsule-module and combined (VGG + capsule) activation maps shown in Fig. 7 of the paper for a set of still face crops.

Citation

@article{ahmad2025capst,
  author  = {Ahmad, Wasim and Peng, Yan-Tsung and Chang, Yuan-Hao and Ganfure, Gaddisa Olani and Khan, Sarwar},
  title   = {CapST: Leveraging Capsule Networks and Temporal Attention for Accurate Model Attribution in Deep-fake Videos},
  journal = {ACM Transactions on Multimedia Computing, Communications, and Applications},
  volume  = {21},
  number  = {4},
  articleno = {123},
  year    = {2025},
  doi     = {10.1145/3715138}
}

The DFDM dataset is introduced by Jia, S., Li, X., and Lyu, S., "Model attribution of face-swap deepfake videos," ICIP 2022. The GANGen-Detection dataset is from Tan, C., Li, J., and Ye, F., "AIGC Dataset: A Dataset for Detection of AI-Generated Content (GANGen-Detection)," GitHub, 2023.

License

Released for academic research purposes. No pretrained model weights are included in this repository.