Evaluation Report: iter5_gpt_512d12_wider

ITERATION 5: Wider GPT (embed_dim=512) instead of deeper ======================================================== Changes from Iteration 2 (best so far): - embed_dim: 384 -> 512 (33% wider) - depth: 12 (same) - num_heads: 16 (matched to embed_dim) - ~38M parameters (up from 22M) - lr: 1e-4 (slightly lower for larger model) - batch_size: 4 Rationale: Iteration 4 showed that depth=16 hurts — the extra layers amplify exposure bias in autoregressive generation (TF/FR gap went from 0.006 to 0.009). Width scaling is generally more stable than depth for ViTs, as shown in scaling law papers. Architecture comparison so far: - iter1: GPT 256d8 (6.9M) -> val_mse=0.0127, SSIM=0.697 - iter2: GPT 384d12 (22M) -> val_mse=0.0102, SSIM=0.756 *** BEST *** - iter3: MAE 384d8 -> val_mse=0.0147, SSIM=0.616 (worst) - iter4: GPT 384d16 (29M) -> val_mse=0.0133, SSIM=0.730 (worse than iter2) Hypothesis: wider representations (512 dims) will capture richer features per token without the instability of more sequential layers. The attention heads can attend to more diverse features with 512d/16 heads vs 384d/12 heads.