PROMETHEUS v3 -- Weight Surgery Map

Visualizing the weight-level modifications made by 21 autonomous self-improvement generations on Mistral-Nemo-12B. Each point represents the L2 norm of the difference between the modified and vanilla weights for a specific layer and component.

21
Accepted Modifications
29
Rejected
40
Transformer Layers
12B
Parameters
4h 53m
Total Runtime
3D Weight Delta Landscape
Surgery Heatmap -- Layer x Component
Attention vs MLP Modification by Layer