Window-Granular Activation Caching for Swin Transformers via Statistical Similarity with Bounded Approximation Error
SwinCache exploits the Swin Transformer's windowed attention to cache and reuse activations at the window level. When processing similar images, windows whose input patches haven't changed significantly skip attention computation entirely, yielding 40–70% compute savings with controllable accuracy degradation.
# Install
pip install -e ".[dev]"
# Run basic test
python -m pytest tests/ -v
# Benchmark on ImageNet val
python scripts/benchmark.py --data-dir /path/to/imagenet --epsilon 0.01In Swin's W-MSA layers, each M×M window computes attention independently. If a window's input is within ε of a cached version (Frobenius norm), we reuse the cached output. For SW-MSA layers, we require all dependency windows to hit. Stage 4 (single global window) is always recomputed.
See docs/SwinCache_Paper.docx for the full theoretical analysis including error propagation bounds, Lipschitz analysis, and feasibility study.
MIT