Bounded Rationality (Posts about lagrange multipliers)http://bjlkeng.github.io/enSat, 03 Aug 2024 01:42:48 GMTNikola (getnikola.com)http://blogs.law.harvard.edu/tech/rssThe Calculus of Variationshttp://bjlkeng.github.io/posts/the-calculus-of-variations/Brian Keng<div><p>This post is going to describe a specialized type of calculus called
variational calculus.
Analogous to the usual methods of calculus that we learn in university,
this one deals with functions <em>of functions</em> and how to
minimize or maximize them. It's used extensively in physics problems such as
finding the minimum energy path a particle takes under certain conditions. As
you can also imagine, it's also used in machine learning/statistics where you
want to find a density that optimizes an objective <a class="footnote-reference brackets" href="http://bjlkeng.github.io/posts/the-calculus-of-variations/#id4" id="id1">1</a>. The explanation I'm
going to use (at least for the first part) is heavily based upon Svetitsky's
<a class="reference external" href="http://julian.tau.ac.il/bqs/functionals/functionals.html">Notes on Functionals</a>, which so far is
the most intuitive explanation I've read. I'll try to follow Svetitsky's
notes to give some intuition on how we arrive at variational calculus from
regular calculus with a bunch of examples along the way. Eventually we'll
get to an application that relates back to probability. I think with the right
intuition and explanation, it's actually not too difficult, enjoy!</p>
<p><a href="http://bjlkeng.github.io/posts/the-calculus-of-variations/">Read more…</a> (16 min remaining to read)</p></div>differentialsentropylagrange multipliersmathjaxprobabilityvariational calculushttp://bjlkeng.github.io/posts/the-calculus-of-variations/Sun, 26 Feb 2017 15:08:38 GMTLagrange Multipliershttp://bjlkeng.github.io/posts/lagrange-multipliers/Brian Keng<div><p>This post is going to be about finding the maxima or minima of a function
subject to some constraints. This is usually introduced in a multivariate
calculus course, unfortunately (or fortunately?) I never got the chance to take
a multivariate calculus course that covered this topic. In my undergraduate class, computer
engineers only took three half year engineering calculus courses, and the
<a class="reference external" href="http://www.ucalendar.uwaterloo.ca/1617/COURSE/course-ECE.html#ECE206">fourth one</a>
(for electrical engineers) seems to have covered other basic multivariate
calculus topics such as all the various theorems such as Green's, Gauss', Stokes' (I
could be wrong though, I never did take that course!). You know what I always imagined Newton
saying, "It's never too late to learn multivariate calculus!".</p>
<p>In that vein, this post will discuss one widely used method for finding optima
subject to constraints: Lagrange multipliers. The concepts
behind it are actually quite intuitive once we come up with the right analogue
in physical reality, so as usual we'll start there. We'll work through some
problems and hopefully by the end of this post, this topic won't seem as
mysterious anymore <a class="footnote-reference brackets" href="http://bjlkeng.github.io/posts/lagrange-multipliers/#id3" id="id1">1</a>.</p>
<p><a href="http://bjlkeng.github.io/posts/lagrange-multipliers/">Read more…</a> (11 min remaining to read)</p></div>calculuslagrange multipliersmathjaxhttp://bjlkeng.github.io/posts/lagrange-multipliers/Tue, 13 Dec 2016 12:48:31 GMT