http://bjlkeng.github.io/enTue, 10 Mar 2026 20:54:58 GMTNikola (getnikola.com)http://blogs.law.harvard.edu/tech/rsshttp://bjlkeng.github.io/posts/lossless-compression-with-asymmetric-numeral-systems/Brian Keng<div><p>During my undergraduate days, one of the most interesting courses I took was on coding and compression. Here was a course that combined algorithms, probability and secret messages, what's not to like? <a class="footnote-reference brackets" href="http://bjlkeng.github.io/posts/lossless-compression-with-asymmetric-numeral-systems/#id2" id="id1">1</a> I ended up not going down that career path, at least partially because communications systems had its heyday around the 2000s with companies like Nortel and Blackberry and its predecessors (some like to joke that all the major theoretical breakthroughs were done by Shannon and his discovery of information theory around 1950). Fortunately, I eventually wound up studying industrial applications of classical AI techniques and then machine learning, which has really grown like crazy in the last 10 years or so. Which is exactly why I was so surprised that a <em>new</em> and <em>better</em> method of lossless compression was developed in 2009 <em>after</em> I finished my undergraduate degree when I was well into my PhD. It's a bit mind boggling that something as well-studied as entropy-based lossless compression still had (have?) totally new methods to discover, but I digress.</p> <p>In this post, I'm going to write about a relatively new entropy based encoding method called Asymmetrical Numeral Systems (ANS) developed by Jaroslaw (Jarek) Duda [2]. If you've ever heard of Arithmetic Coding (probably best known for its use in JPEG compression), ANS runs in a very similar vein. It can generate codes that are close to the theoretical compression limit (similar to Arithmetic coding) but is <em>much</em> more efficient. It's been used in modern compression algorithms since 2014 including compressors developed by Facebook, Apple and Google [3]. As usual, I'm going to go over some background, some math, some examples to help with intuition, and finally some experiments with a toy ANS implementation I wrote. I hope you're as excited as I am, let's begin!</p> <p><a href="http://bjlkeng.github.io/posts/lossless-compression-with-asymmetric-numeral-systems/">Read more…</a> (32 min remaining to read)</p></div>Arithmetic Codingasymmetric numeral systemscompressionentropyHuffman codingmathjaxhttp://bjlkeng.github.io/posts/lossless-compression-with-asymmetric-numeral-systems/Sat, 26 Sep 2020 14:37:43 GMT