⚡ Cross-KD: ANN to SNN Distillation for Imbalanced Tabular Data

Cross-modal knowledge distillation (Cross-KD) has shown immense promise in transferring rich representations from high-capacity Artificial Neural Networks (ANNs) to Spiking Neural Networks (SNNs). While typically used for vision or neuromorphic data, this project applies Cross-KD to highly imbalanced tabular datasets, enabling low-power, robust classification for applications like fraud detection, healthcare diagnostics, and IoT analytics.

---

💡 Problem Statement

In tabular domains with rare classes, SNNs provide power-efficient inference but suffer from limited capacity. This framework distills a well-trained ANN into an SNN using three progressive phases:

1. Logit Mimicry (KL Divergence)
2. Semantic Feature Alignment (MSE/Cosine similarity)
3. Fine-tuning for Imbalance Sensitivity (Cross-Entropy + F1 optimization)

The ANN teacher is trained with SMOTE, focal loss, or class-weighted CE. Inputs are encoded as spike trains via rate or time-to-first-spike coding.

---

🔧 Project Structure

crosskd-snn-tabular/
├── teacher_ann.py            # ANN teacher model
├── student_snn.py            # SNN student model (SpikingJelly)
├── distillation_loss.py      # KL + MSE + CE loss
├── spike_encoding.py         # Tabular → spike encoding
├── train_crosskd.py          # Pipeline: train, align, evaluate
├── evaluate_metrics.py       # Balanced evaluation metrics
├── requirements.txt
├── LICENSE
├── .gitignore
└── README.md

---

🚀 Getting Started

1️⃣ Install Dependencies

pip install -r requirements.txt

Ensure you have Python ≥3.8 and install spikingjelly from GitHub if not in PyPI: pip install git+https://github.com/fangwei123456/spikingjelly

2️⃣ Train + Distill ANN → SNN

python train_crosskd.py

3️⃣ Evaluate Performance

python evaluate_metrics.py

---

📘 Key Techniques

• Rate Coding for continuous features
• Time-to-First-Spike or One-Hot Spike for categorical/binary
• Sliding Semantic Replacement during distillation
• Focal Loss + SMOTE on ANN to combat imbalance
• KL Divergence + MSE + CE hybrid loss on SNN

---

📊 Metrics

• Macro F1-Score
• Balanced Accuracy
• PR-AUC
• Recall@Minority Class

---

🧠 Research References

• Li et al. (2025), Cross-Modal Distillation with Boundary-Aware Alignment, Neural Networks
• Fang et al. (2023), SpikingJelly: An Open-Source Framework for SNNs, Frontiers in Neuroscience
• Liu et al. (2024), Distillation from Vision Transformers to SNNs, arXiv preprint
• Cai et al. (2022), Temporal Knowledge Transfer for Spiking Networks, NeurIPS
• He et al. (2023), Focal Loss for Highly Imbalanced Data, IEEE TPAMI
• Zhang et al. (2025), Sliding Semantic Replacement in Multi-Modal Learning, ACL

---

🤖 Hugging Face Deployment

You can turn this into a live interactive demo via Hugging Face Spaces:

huggingface-cli login
huggingface-cli repo create crosskd-tabular --type=space --sdk=streamlit
git remote add hf https://huggingface.co/spaces/<username>/crosskd-tabular
git push hf main

---

📄 License

This project is licensed under the MIT License.

---

🧑‍💻 Author

Saurav Singla
🔗 github.com/sauravsingla

---

🙌 Acknowledgments

Special thanks to the authors of SpikingJelly, and the Cross-KD researchers pioneering the adaptation of semantic distillation across modalities.