Gradient descent on separable data with a linear model and exponential-type loss converges in direction to the maximum-margin separator. For homogeneous networks, gradient flow converges to a KKT point of a margin-maximisation problem, but this does not identify which one.
Give a characterisation of the solution selected by gradient descent — as a function of the architecture, initialisation scale, and step size — for a non-linear network class where the answer is not already determined by a kernel approximation.