http://bjlkeng.github.io/enTue, 10 Mar 2026 20:54:59 GMTNikola (getnikola.com)http://blogs.law.harvard.edu/tech/rsshttp://bjlkeng.github.io/posts/universal-resnet-the-one-neuron-approximator/Brian Keng<div><p><em>"In theory, theory and practice are the same. In practice, they are not."</em></p> <p>I read a very interesting paper titled <em>ResNet with one-neuron hidden layers is a Universal Approximator</em> by Lin and Jegelka [1]. The paper describes a simplified Residual Network as a universal approximator, giving some theoretical backing to the wildly successful ResNet architecture. In this post, I'm going to talk about this paper and a few of the related universal approximation theorems for neural networks. Instead of going through all the theoretical stuff, I'm simply going introduce some theorems and play around with some toy datasets to see if we can get close to the theoretical limits.</p> <p>(You might also want to checkout my previous post where I played around with ResNets: <a class="reference external" href="http://bjlkeng.github.io/posts/residual-networks/">Residual Networks</a>)</p> <p><a href="http://bjlkeng.github.io/posts/universal-resnet-the-one-neuron-approximator/">Read more…</a> (11 min remaining to read)</p></div>hidden layersmathjaxneural networksresidual networksResNetuniversal approximatorhttp://bjlkeng.github.io/posts/universal-resnet-the-one-neuron-approximator/Fri, 03 Aug 2018 12:03:28 GMT