Two weeks of fine-tuning VLAs in sim: what worked, what broke
ACT → SmolVLA → harder scene on one 5070 Ti. Peaked at 60% on a 4-color shelf, dropped to 17.5% when I shuffled the books (0/54 on non-canonical positions). The textbook fix made it strictly worse, and the harder scene went 0/10. Three architectures, four scenes, zero clean wins. Full log.