Checkpoint writes are not atomic, a crash mid save destroys the previous one #13

src/serialize/checkpoint.rs:79 and src/serialize/safetensors.rs:78:

let mut out = BufWriter::new(File::create(path)?);

File::create truncates immediately. And the usage pattern we document in
serialize/training.rs is overwriting one path every N steps:

if step % 500 == 0 { save_training(&model, &opt, step, "run.fdl")?; }

A kill, an OOM, or a full disk during that write leaves a truncated run.fdl and
no copy of the good one. This module exists specifically to survive a long
unattended run and this is the one failure it doesn't survive. There's also no
fsync before returning, so a power cut can leave a file that looks complete and
isn't.

Write to path.tmp, sync_all(), then rename (atomic on the same filesystem).
Worth a robustness test that truncates a checkpoint and asserts the previous one
still loads.