Evaluation Report: diff_iter2_384d12_cosine

DIFFUSION ITERATION 2: Larger model + training fixes ===================================================== Changes from Iteration 1 (total failure - produced noise): - embed_dim: 256 -> 384 (50% increase) - depth: 8 -> 12 (50% more layers) - num_heads: 8 -> 12 - weight_decay: 0.05 -> 0.0 (critical for diffusion models) - lr: 1e-4 -> 3e-4 (higher peak with warmup) - lr_schedule: plateau -> cosine with 10-epoch warmup - grad_clip: 1.0 (stability for larger model) - epochs: 200 -> 300 (more training time) - ~22M parameters Rationale: Iteration 1 completely failed (SSIM=0.008, basically noise). The denoising loss was ~0.675 uniformly across all timestep buckets, meaning the model didn't learn to denoise at ANY noise level. Root causes: 1. Model too small: 7M params is insufficient for denoising. The GPT model also needed 22M to work well. DiT models in the literature are typically 100M+, but 22M should be enough for this canvas size. 2. Weight decay 0.05: DiT/diffusion models typically use 0.0 weight decay. High weight decay fights the zero-initialization of adaLN modulation layers, preventing the model from learning. 3. No warmup: DiT training is sensitive to early learning rate. Warmup provides stability during initial weight updates. 4. No gradient clipping: Large gradients from random noise targets can destabilize early training.