A 2019 paper coauthored by Harvard's Boaz Barak showed that double descent can occur not only as models gain parameters, but also when they receive more training data or are trained for longer.
Sources
- Large language models can do jaw-dropping things. But nobody ... (technologyreview.com)
- https://arxiv.org/abs/1912.02292 (arxiv.org)
- https://openai.com/index/deep-double-descent/ (openai.com)
- https://windowsontheory.org/2019/12/05/deep-double-descent/ (windowsontheory.org)
- https://quantum.harvard.edu/boaz-barak (quantum.harvard.edu)
More in Science
- The Elevation Science Institute conducts paleontological field work in the Bighorn Basin of Montana and Wyoming.
- At a 2009 FDA advisory-panel meeting, members voted 36-1 to recommend that children's acetaminophen products be made in a single concentration, but the FDA never made that requirement mandatory.
- The 'information bottleneck' theory has been suggested as a possible explanation for how human brains learn.
- The fault slip on the southern San Andreas Fault triggered by the 2017 Chiapas earthquake began minutes after that earthquake and continued for more than a year.
- NIES researchers established induced pluripotent stem cells (iPSCs) from the Okinawa rail, Japanese ptarmigan, Blakiston's fish owl, and Japanese golden eagle, all endangered avian species.
- Matthew Holman, an astrophysicist at the Center for Astrophysics, found that the Vera C. Rubin Observatory had captured multiple images of 2026 ED138 between March 4 and July 11.