http://bjlkeng.github.io/enTue, 10 Mar 2026 20:54:58 GMTNikola (getnikola.com)http://blogs.law.harvard.edu/tech/rsshttp://bjlkeng.github.io/posts/a-note-on-using-log-likelihood-for-generative-models/Brian Keng<div><p>One of the things that I find is usually missing from many ML papers is how they relate to the fundamentals. There's always a throwaway line where it assumes something that is not at all obvious (see my post on <a class="reference external" href="http://bjlkeng.github.io/posts/importance-sampling-and-estimating-marginal-likelihood-in-variational-autoencoders/">Importance Sampling</a>). I'm the kind of person who likes to understand things to a satisfactory degree (it's literally in the subtitle of the blog) so I couldn't help myself investigating a minor idea I read about in a paper.</p> <p>This post investigates how to use continuous density outputs (e.g. a logistic or normal distribution) to model discrete image data (e.g. 8-bit RGB values). It seems like it might be something obvious such as setting the loss as the average log-likelihood of the continuous density and that's <em>almost</em> the whole story. But leaving it at that skips over so many (interesting) and non-obvious things that you would never know if you didn't bother to look. I'm a curious fellow so come with me and let's take a look!</p> <p><a href="http://bjlkeng.github.io/posts/a-note-on-using-log-likelihood-for-generative-models/">Read more…</a> (15 min remaining to read)</p></div>generative modelslog-likelihoodmathjaxhttp://bjlkeng.github.io/posts/a-note-on-using-log-likelihood-for-generative-models/Tue, 27 Aug 2019 11:50:09 GMT