Restructure paper for multi-venue submissions

- Add venue-specific subdirectories (tmlr, neurips-workshop, etc)
- Extract shared components (preamble, macros, bibliography)
- Move TMLR version to paper/tmlr/ as primary submission
- Update analysis scripts to write to paper/tmlr/tables
- Archive old paper structure in paper/archive/
- Each venue has independent main.tex and Makefile

Author: dariocazzani

analysis/generate_figures.py

@@ -123,7 +123,7 @@ def augmentation_gap_plot(df: pd.DataFrame, output_path: Path) -> None:
     plt.close()
 
 
-def save_figures(df: pd.DataFrame, output_dir: str = "paper/figures") -> None:
+def save_figures(df: pd.DataFrame, output_dir: str = "paper/tmlr/figures") -> None:
     """Generate and save all figures."""
     output = Path(output_dir)
     output.mkdir(parents=True, exist_ok=True)
@@ -136,8 +136,9 @@ def save_figures(df: pd.DataFrame, output_dir: str = "paper/figures") -> None:
         accuracy_comparison_plot(df, output, augment=augment)
         accuracy_delta_heatmap(df, output, augment=augment)
 
-    # Augmentation ablation plot
-    augmentation_gap_plot(df, output)
+    # Augmentation ablation plot (exclude imagenet)
+    df_no_imagenet = df[df["dataset"] != "imagenet"] if "dataset" in df.columns else df
+    augmentation_gap_plot(df_no_imagenet, output)
 
     print(f"Figures saved to {output}")
 

analysis/generate_tables.py

@@ -235,7 +235,7 @@ def layer_ablation_table(df: pd.DataFrame, dataset: str = "cifar10", augment: st
     return "\n".join(lines)
 
 
-def save_tables(df: pd.DataFrame, comparisons: pd.DataFrame, output_dir: str = "paper/tables") -> None:
+def save_tables(df: pd.DataFrame, comparisons: pd.DataFrame, output_dir: str = "paper/tmlr/tables") -> None:
     """Save all tables to files."""
     output = Path(output_dir)
     output.mkdir(parents=True, exist_ok=True)

paper/Makefile

@@ -0,0 +1,208 @@
+# BitNet CNN Paper — TMLR Submission
+
+**Title**: *When Augmentation Fails: Knowledge Distillation for Ternary CNNs*
+
+Single-venue focus on TMLR (Transactions on Machine Learning Research) with 3-reviewer simulation system.
+
+---
+
+## Quick Start
+
+```bash
+# Build paper
+cd paper/tmlr && make
+
+# Regenerate tables and figures from experiments
+uv run python -m analysis.generate_tables
+uv run python -m analysis.generate_figures
+
+# Review workflow (3 reviewers per round)
+# 1. Paste paper to Claude with persona_1_theory.md prompt
+# 2. Save review to reviews/tmlr/round_N/reviewer_1.md
+# 3. Repeat for persona_2 and persona_3
+# 4. Read all 3 reviews and create implementation plan
+# 5. Fix issues → rebuild → next round
+```
+
+---
+
+## Project Structure
+
+```
+paper/
+├── tmlr/                        # The paper (self-contained)
+│   ├── main.tex                 # Main paper file
+│   ├── preamble.tex             # LaTeX packages
+│   ├── macros.tex               # Custom commands
+│   ├── bibliography.tex         # References
+│   ├── figures/                 # Generated by analysis scripts
+│   ├── tables/                  # Generated by analysis scripts
+│   └── Makefile
+│
+├── reviews/
+│   └── tmlr/
+│       ├── prompts/             # 3 reviewer personas
+│       │   ├── persona_1_theory.md       # ML theory researcher
+│       │   ├── persona_2_expert.md       # Quantization expert
+│       │   └── persona_3_practitioner.md # Systems/deployment
+│       ├── round_0/             # Ready for first 3 reviews
+│       │   ├── (awaiting reviewer_1.md)
+│       │   ├── (awaiting reviewer_2.md)
+│       │   └── (awaiting reviewer_3.md)
+│       ├── round_N/             # Future rounds (3 reviews each)
+│       └── archive_*/           # Old single-reviewer rounds (Feb 14, 2026)
+│
+├── research/                    # Research prompts and notes
+├── archive/                     # Old multi-venue work (if needed)
+│   ├── neurips-workshop/        # 7.5/10 Accept (can resurrect)
+│   ├── other-venues/            # bmvc, cvpr, iclr, neurips, wacv
+│   └── reviewers/               # Old multi-venue review system
+│
+├── venues.md                    # Deadline tracker & strategy
+├── notes.md                     # Research notes and experiment log
+└── README.md                    # This file
+```
+
+---
+
+## Current Status
+
+**Target Venue:** TMLR (rolling deadline, submit ~March 2026)
+**Review Status:** Ready for Round 0 (3-reviewer system)
+**Paper Length:** 18 pages (no page limit for TMLR)
+
+### Key Results
+- 3-9% accuracy gap between ternary and FP32 on CIFAR
+- Augmentation widens gap: FP32 benefits 1.5-3.7× more
+- conv1 accounts for 54-74% of gap (only 0.08% params)
+- Recipe (FP32 conv1 + KD): 89% recovery on CIFAR-10, exceeds FP32 on harder tasks
+
+---
+
+## Review Workflow (Simplified)
+
+### 3-Reviewer System
+
+Each round uses **3 personas** to simulate diverse reviewer feedback:
+
+1. **ML Theory Researcher** - Statistical rigor, theoretical grounding
+2. **Quantization Expert** - Field knowledge, baseline comparisons
+3. **Systems Practitioner** - Reproducibility, deployment practicality
+
+### Per-Round Process
+
+```bash
+# Round N workflow
+cd paper/tmlr && make
+
+# Get 3 independent reviews:
+# - Paste paper + persona_1 prompt to Claude → save to round_N/reviewer_1.md
+# - Paste paper + persona_2 prompt to Claude → save to round_N/reviewer_2.md
+# - Paste paper + persona_3 prompt to Claude → save to round_N/reviewer_3.md
+
+# Read all 3 reviews
+cat reviews/tmlr/round_N/*.md
+
+# Create implementation plan (in PLAN.md or similar)
+# - Synthesize MAJOR/MINOR issues across all 3 reviews
+# - Prioritize: fix MAJOR issues first, MINOR if time permits
+# - Document decisions
+
+# Fix issues in main.tex
+
+# Rebuild and repeat
+make clean && make
+```
+
+---
+
+## Analysis Pipeline
+
+When experiments change:
+
+```bash
+# From project root
+uv run python -m analysis.aggregate_results   # results/raw/ → results/processed/
+uv run python -m analysis.generate_tables     # → paper/tmlr/tables/
+uv run python -m analysis.generate_figures    # → paper/tmlr/figures/
+
+# Rebuild paper
+cd paper/tmlr && make
+```
+
+**Important:** Analysis scripts now write directly to `paper/tmlr/` (updated Feb 14, 2026).
+
+---
+
+## Venue Strategy
+
+See [venues.md](venues.md) for full details.
+
+**Primary:** TMLR (rolling, ~70% acceptance probability with current work)
+
+**Backup Options:**
+- NeurIPS Workshop (Aug 2026) - draft at 7.5/10 Accept available in archive
+- BMVC, WACV (if TMLR requests major revisions)
+- Top venues (ICLR, NeurIPS, CVPR) require ImageNet validation
+
+**Archived Work:**
+- NeurIPS Workshop paper (7.5/10 Accept, 5 pages) in `archive/neurips-workshop/`
+- Other venue drafts in `archive/other-venues/`
+- Multi-venue review automation in `archive/reviewers/`
+
+---
+
+## Tips
+
+### Building the Paper
+```bash
+cd paper/tmlr
+make           # Build PDF
+make clean     # Remove build artifacts
+```
+
+### Common Issues
+
+**"No rule to make target"**
+- Check that preamble.tex, macros.tex, bibliography.tex exist in `tmlr/`
+- Ensure figures/ and tables/ directories exist in `tmlr/`
+
+**"File not found" in LaTeX**
+- Verify `\input{}` statements use local paths (no `../`)
+- Check `\graphicspath{{figures/}}` points to local directory
+
+**Stale tables/figures**
+- Re-run analysis scripts after new experiments
+- Check that scripts write to `paper/tmlr/tables/` and `paper/tmlr/figures/`
+
+---
+
+## Key Files
+
+| File | Purpose |
+|------|---------|
+| `tmlr/main.tex` | Main paper (18 pages) |
+| `reviews/tmlr/prompts/` | 3 reviewer personas for simulation |
+| `venues.md` | Venue deadlines and strategy |
+| `notes.md` | Detailed experiment log |
+| `research/` | Deep research prompts |
+
+---
+
+## Next Steps
+
+1. **Expand reviewer prompts** - Add full TMLR review instructions to 3 personas
+2. **Run Round 2** - Get 3 reviews, synthesize feedback, fix issues
+3. **Final polish** - Proofread, formatting, consistency check
+4. **Submit to TMLR** - Target early March 2026
+
+---
+
+## Archive
+
+Multi-venue work (7 venues, automated sync, workflow automation) archived in `archive/`:
+- Kept neurips-workshop (7.5/10 Accept) as backup
+- Can resurrect other venues if needed
+- Focused on TMLR as primary target
+
+**Rationale:** Single-venue focus reduces complexity, allows deeper iteration on one paper rather than spreading effort across 7 venues.

paper/README.md

@@ -0,0 +1,13 @@
+MAIN = main
+LATEX = pdflatex
+
+all: $(MAIN).pdf
+
+$(MAIN).pdf: $(MAIN).tex ../shared/*.tex
+	$(LATEX) $(MAIN)
+	$(LATEX) $(MAIN)
+
+clean:
+	rm -f *.aux *.bbl *.blg *.log *.out *.toc *.lof *.lot *.fls *.fdb_latexmk *.synctex.gz
+
+.PHONY: all clean

paper/archive/neurips-workshop/Makefile

@@ -0,0 +1,56 @@
+% Minimal bibliography for NeurIPS Workshop (12 references only)
+
+\bibliographystyle{plain}
+
+\begin{thebibliography}{99}
+
+\bibitem{ma2024bitnetb158}
+S.~Ma et al.
+\newblock The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits.
+\newblock \emph{arXiv:2402.17764}, 2024.
+\bibitem{courbariaux2015binaryconnect}
+M.~Courbariaux et al.
+\newblock BinaryConnect: Training Deep Neural Networks with Binary Weights during Propagations.
+\newblock \emph{NeurIPS}, 2015.
+\bibitem{rastegari2016xnornet}
+M.~Rastegari et al.
+\newblock XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks.
+\newblock \emph{ECCV}, 2016.
+\bibitem{li2016ternary}
+F.~Li et al.
+\newblock Ternary Weight Networks.
+\newblock \emph{arXiv:1605.04711}, 2016.
+\bibitem{rw2019timm}
+R.~Wightman.
+\newblock PyTorch Image Models.
+\newblock \url{https://github.com/rwightman/pytorch-image-models}, 2019.
+\bibitem{hinton2015distilling}
+G.~Hinton, O.~Vinyals, and J.~Dean.
+\newblock Distilling the Knowledge in a Neural Network.
+\newblock \emph{NeurIPS Deep Learning Workshop}, 2015.
+\bibitem{zhu2017ttq}
+C.~Zhu, S.~Han, H.~Mao, and W.~Dally.
+\newblock Trained Ternary Quantization.
+\newblock \emph{ICLR}, 2017.
+\bibitem{kim2019qkd}
+J.~Kim, Y.~Bhalgat, J.~Lee, C.~Patel, and N.~Kwak.
+\newblock QKD: Quantization-aware Knowledge Distillation.
+\newblock \emph{arXiv:1911.12491}, 2019.
+\bibitem{wang2019haq}
+K.~Wang, Z.~Liu, Y.~Lin, J.~Lin, and S.~Han.
+\newblock HAQ: Hardware-Aware Automated Quantization with Mixed Precision.
+\newblock \emph{CVPR}, 2019.
+\bibitem{zhou2016dorefa}
+S.~Zhou, Y.~Wu, Z.~Ni, X.~Zhou, H.~Wen, and Y.~Zou.
+\newblock DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients.
+\newblock \emph{arXiv:1606.06160}, 2016.
+\bibitem{dong2019hawq}
+Z.~Dong, Z.~Yao, A.~Gholami, M.~Mahoney, and K.~Keutzer.
+\newblock HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precision.
+\newblock \emph{ICCV}, 2019.
+\bibitem{dong2020hawqv2}
+Z.~Dong, Z.~Yao, D.~Arfeen, A.~Gholami, M.~W.~Mahoney, and K.~Keutzer.
+\newblock HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networks.
+\newblock \emph{NeurIPS}, 2020.
+
+\end{thebibliography}