Hi, thanks for releasing the code!
While reading it, I found two places where it seems to differ from the paper, and I'd like to confirm which version was used for the reported results.
- No re-encoding in the memory update
The paper (Sec. 4.4, Algorithm 1) re-encodes decoded frames into clean latents (z̃ₜ = E(Iₜ)) before back-projection. In the code, update_latent_memory() passes the generated latents directly to lpc.update() with no vae.encode call. Which one matches the released recipe?
- Side branch frozen in Stage 2
The paper (Sec. 4.5) says LoRA is jointly optimized with the side branch in Stage 2. In scripts/train.py, set_stage_trainable("lora") freezes the side branch (model.vace.eval()) and the optimizer only receives LoRA parameters. Was the side branch actually frozen in Stage 2?
Apologies if I've misread anything — thanks in advance!
Hi, thanks for releasing the code!
While reading it, I found two places where it seems to differ from the paper, and I'd like to confirm which version was used for the reported results.
The paper (Sec. 4.4, Algorithm 1) re-encodes decoded frames into clean latents (z̃ₜ = E(Iₜ)) before back-projection. In the code, update_latent_memory() passes the generated latents directly to lpc.update() with no vae.encode call. Which one matches the released recipe?
The paper (Sec. 4.5) says LoRA is jointly optimized with the side branch in Stage 2. In scripts/train.py, set_stage_trainable("lora") freezes the side branch (model.vace.eval()) and the optimizer only receives LoRA parameters. Was the side branch actually frozen in Stage 2?
Apologies if I've misread anything — thanks in advance!