feat: add exact autoregressive KV caching

[DivyamTalwar]

Summary

• add a typed, per-layer dynamic KV cache for exact autoregressive decoding
• preserve RoPE offsets, rectangular causal masks, and optional one-token K/V shift state across decode calls
• add deterministic/sampled generate_tokens, an uncached reference path, parity tests, a CPU/GPU benchmark, and README documentation

Competitive review

Current inference stacks make cache semantics explicit and testable:

• Hugging Face cache guide: https://huggingface.co/docs/transformers/en/kv_cache
• Hugging Face cache implementations: https://github.com/huggingface/transformers/blob/main/src/transformers/cache_utils.py
• LitGPT generation/benchmark API: https://github.com/Lightning-AI/litgpt/blob/main/tutorials/python-api.md

This PR is an original repository-local implementation. It copies no competitor code. The adopted product principle is exact cached/reference parity with an explicit cache lifecycle.

Verification

• python -m compileall -q model tests benchmarks
• pytest -q — 13 passed
• git diff --check
• CPU fixture, prompt 64 / 32 new tokens / 4-layer 128-hidden model:
  • full-prefix: 170.621 ms/generation
  • dynamic-cache: 87.829 ms/generation

The benchmark is observational and hardware/model-size dependent; it is not a universal speed claim.

Safety and compatibility

• caching is inference-only and fails closed during training
• prompt + generated length cannot exceed block_size
• cached and uncached logits are tested with shifted K/V both disabled and enabled
• the existing two-value forward return is unchanged unless use_cache=True

Please review only; do not merge as part of this research wave.

[DivyamTalwar]

Validation checkpoint

The latest head afdef4b6f96d276e66e030541e6ee58b03aafc69 passed authoritative GitHub Actions CI run 32662964197.

Local review evidence:

• 13 tests passed
• compileall and whitespace checks passed
• cached token-by-token logits match full-sequence logits with shifted K/V both off and on
• CPU fixture (64-token prompt, 32 generated, 4×128 model): 170.621 ms full-prefix vs 87.829 ms dynamic cache

The timing is intentionally treated as one observational fixture, not a universal performance claim. The implementation was informed by the public cache lifecycle/parity patterns in Hugging Face Transformers and LitGPT; no third-party code was copied.

Per the requested workflow, this PR remains open and must not be merged.