OP-0048Deep learning theoryOpen
Derive neural scaling-law exponents from properties of the data
Loss follows a power law in compute, data and parameters. No theory predicts the exponent from the data distribution.
arXiv preprints· 2020
2 entries matching the current filters. Sorted by recently added.
Loss follows a power law in compute, data and parameters. No theory predicts the exponent from the data distribution.
For linear models the answer is a clean minimum-norm statement. For anything deeper, there is no general theory.