OP-0049Deep learning theoryOpen
Depth separation for functions learnable by gradient descent
Depth-3 networks can express functions depth-2 cannot. Whether training can find them is a different question, and it is open.
Community consensus· 2016
2 entries matching the current filters. Sorted by recently added.
Depth-3 networks can express functions depth-2 cannot. Whether training can find them is a different question, and it is open.
For linear models the answer is a clean minimum-norm statement. For anything deeper, there is no general theory.