Evaluation Report: diff_iter4_512d12_wd001

DIFFUSION ITERATION 4: Wider model (512d) + weight decay 0.01 for regularization ================================================================================= Changes from Iteration 3: - embed_dim: 384 -> 512 (wider model) - depth: 12 (same) - num_heads: 16 (matched to embed_dim) - weight_decay: 0.0 -> 0.01 (light regularization to reduce overfitting) - epochs: 500 -> 300 (iter3 best was at ~210, didn't improve after) - lr: 5e-4 -> 3e-4 (slightly lower for larger model) Rationale: Iteration 3 finally got diffusion working with sample prediction + fixed inference. The model achieves competitive results with GPT but motor accuracy is weaker (pos MAE 0.954 vs GPT's 0.612). The iter3 model overfitted heavily (train/val gap grew to -0.013). Adding light weight decay should help regularize. A wider model should provide more capacity for precise predictions. Key: iter3 showed diffusion models need NO weight decay to start training, but too little regularization causes overfitting after ~200 epochs. Weight decay 0.01 is a middle ground. Results so far: - diff_iter1: 256d8, epsilon, wd=0.05 -> NOISE (total failure) - diff_iter2: 384d12, epsilon, wd=0.0, cosine -> NOISE (DDIM broken) - diff_iter3: 384d12, sample, wd=0.0, cosine -> SSIM=0.730 (working!) - This run: 512d12, sample, wd=0.01, cosine -> targeting SSIM>0.75