The concept known as the 'information bottleneck' is proposed as a theory to help explain why deep neural networks perform successfully.
Notes on verification
Well-corroborated by original Tishby & Zaslavsky paper, Quanta Magazine coverage, and subsequent academic literature. Note the theory remains debated within the ML research community, but the claim only asserts it is 'proposed as a theory,' which is accurate.
Sources
- New Theory Cracks Open the Black Box of Deep Neural Networks (wired.com)
- https://www.quantamagazine.org/new-theory-cracks-open-the-black-box-of-deep-learning-20170921/ (quantamagazine.org)
- https://arxiv.org/abs/1503.02406 (arxiv.org)
- https://arxiv.org/pdf/1903.10563 (arxiv.org)