Evaluation Report: iter4_gpt_384d16_lowlr

ITERATION 4: Deeper GPT with gradient clipping + lower LR ========================================================== Changes from Iteration 2 (best GPT so far): - depth: 12 -> 16 (33% more layers) - lr: 1.5e-4 -> 1e-4 (lower for stability with deeper network) - batch_size: 4 (same, limited by VRAM) - epochs: 150 (more epochs since lower LR converges slower) Rationale: Iteration 3 showed MAE is clearly worse than GPT for this task. Going back to GPT and pushing capacity further. Key insight from iter3: The autoregressive approach, despite error compounding, produces more coherent and detailed predictions than the parallel reconstruction approach (MAE). This suggests the sequential dependencies between patches ARE useful for prediction quality. Summary of architecture comparison so far: - GPT 256d8 (6.9M params): val_mse=0.0127, SSIM=0.697 - GPT 384d12 (22M params): val_mse=0.0102, SSIM=0.756 (BEST) - MAE 384d8: val_mse=0.0147, SSIM=0.616 (WORST) This iteration pushes depth from 12->16 to test if more transformer blocks improve representation quality, especially for dynamic regions. Lower LR should help the deeper network converge without instability. Watching for: - Whether the train/val gap widens (overfitting with more capacity) - Whether grad norms remain stable with 16 layers - Whether SSIM continues improving