Add inference/generate pipeline to dynamic training

training/README.md

@@ -76,10 +76,11 @@ Weights passed via IOSurface spatial dimension — compile 10 kernels once at st
 | `stories_cpu_ops.h` | vDSP-vectorized RMSNorm, cross-entropy, Adam |
 | `ane_classifier.h` | ANE classifier fwd (32K conv), softmax kernels |
 | `ane_rmsnorm_bwd.h` | ANE rmsnorm backward kernel |
-| `dashboard.py` | TUI dashboard — loss curve, power/CPU/memory graphs |
-| `Makefile` | Build targets |
-
-## Usage
+| `dashboard.py` | TUI dashboard — loss curve, power/CPU/memory graphs |
+| `Makefile` | Build targets |
+| `training_dynamic/generate.m` | Text generation (inference) — forward-only, 3 kernels |
+
+## Usage
 
 ### 1. Download Training Data
 
@@ -110,7 +111,7 @@ make MODEL=qwen3_06b           # default — Qwen3-0.6B (28L, GQA, 596M)
 make MODEL=stories110m         # Stories110M (12L, MHA, 109M)
 ./train --scratch              # train from random init
 ./train --resume               # resume from checkpoint
-./train --steps 200 --lr 1e-4  # custom steps/lr
+./train --scratch --steps 200 --lr 1e-4  # custom steps/lr from random init
 ```
 
 **CLI flags (`train_large` / `train_large_ane`):**
@@ -122,15 +123,55 @@ make MODEL=stories110m         # Stories110M (12L, MHA, 109M)
 - `--resume` — resume from checkpoint
 - `--no-ane-extras` — (train_large_ane only) disable ANE classifier/softmax/rmsnorm_bwd
 
-### 3. Monitor with Dashboard
-
-```bash
-pip install blessed psutil numpy
-sudo python3 dashboard.py          # static pipeline
-sudo python3 dashboard.py --dynamic # dynamic pipeline
-```
-
-### 4. Benchmarking
+**CLI flags (`training_dynamic/train`):**
+- `--scratch` — start from random initialization
+- `--resume` — resume from the model-specific checkpoint, if present
+- `--steps N`, `--lr F`, `--accum N`, `--warmup N`, `--clip F`
+- `--data PATH` — tokenized TinyStories `.bin` file
+- `--help` — print usage
+
+The dynamic pipeline does not load pretrained weights directly yet; use `--scratch` for a new run or `--resume` after a checkpoint exists.
+
+### 3. Generate Text (Inference)
+
+After training, generate text from a checkpoint using the dynamic pipeline's forward kernels:
+
+```bash
+cd training_dynamic
+make MODEL=stories110m generate    # build for Stories110M
+make MODEL=qwen3_06b generate      # build for Qwen3-0.6B
+
+# Greedy decoding (deterministic)
+./generate --ckpt ane_stories110M_dyn_ckpt.bin --tokens 100
+
+# Temperature sampling with top-k
+./generate --ckpt ane_stories110M_dyn_ckpt.bin --tokens 200 --temp 0.8 --topk 40
+
+# Use a different data file for prompt context
+./generate --ckpt ane_stories110M_dyn_ckpt.bin --data ../tinystories_data00.bin --tokens 100
+```
+
+**CLI flags (`training_dynamic/generate`):**
+- `--ckpt PATH` — checkpoint file (required, from `train --resume`)
+- `--tokens N` — number of tokens to generate (default: 100)
+- `--temp F` — sampling temperature, 0 = greedy (default: 0.0)
+- `--topk K` — top-K sampling, 0 = disabled (default: 0)
+- `--data PATH` — tokenized data for prompt context (default: `../tinystories_data00.bin`)
+- `--seed N` — random seed for sampling (default: 42)
+
+The generator loads checkpoint weights (skipping Adam state), compiles only the 3 forward kernels (sdpaFwd, woFwd, ffnFused), and runs a sliding-window forward pass. Prompt context is taken from the first `SEQ` tokens of the data file. A llama2.c tokenizer (`tokenizer.bin`) placed in the working directory enables text output; without it, token IDs are printed.
+
+For detokenization, place a llama2.c format `tokenizer.bin` in the working directory. It can be obtained from the [llama2.c tokenizer.bin](https://github.com/karpathy/llama2.c/raw/master/tokenizer.bin) (Llama 2, 32K vocab) or the Qwen3 tokenizer for the 152K vocab model.
+
+### 4. Monitor with Dashboard
+
+```bash
+pip install blessed psutil numpy
+sudo python3 dashboard.py          # static pipeline
+sudo python3 dashboard.py --dynamic # dynamic pipeline
+```
+
+### 5. Benchmarking
 
 All programs print an **Efficiency Report** at completion:
 

training/training_dynamic/Makefile

@@ -7,11 +7,15 @@ CFLAGS = -O2 -DACCELERATE_NEW_LAPACK -framework Foundation -framework IOSurface
 MODEL ?= qwen3_06b
 MODEL_HDR = models/$(MODEL).h
 
-train: train.m config.h io.h cpu_ops.h mil_dynamic.h $(MODEL_HDR)
-	@echo "Building for model: $(MODEL)"
-	$(CC) $(CFLAGS) -include $(MODEL_HDR) -o train train.m
-
-clean:
-	rm -f train
-
-.PHONY: clean
+train: train.m config.h io.h cpu_ops.h mil_dynamic.h $(MODEL_HDR)
+	@echo "Building for model: $(MODEL)"
+	$(CC) $(CFLAGS) -include $(MODEL_HDR) -o train train.m
+
+generate: generate.m config.h io.h cpu_ops.h mil_dynamic.h $(MODEL_HDR)
+	@echo "Building generate for model: $(MODEL)"
+	$(CC) $(CFLAGS) -include $(MODEL_HDR) -o generate generate.m
+
+clean:
+	rm -f train generate
+
+.PHONY: clean