Zero-Cost Proxies: How to find the best neural network without training

Neural networks are becoming massive monsters that are hard to train (without the "regular" 12 last-generation GPUs).
Is there a way to skip that?

Let me introduce you to Zero-Cost proxies

References

1. https://www.technologyreview.com/2022/08/05/1056814/automation-ai-machine-learning-automl/
2. iclr-blog-track.github.io/2022/03/25/zero-cost-proxies