A Probabilistic View of Linear Regression

Brian Keng

2016-05-14 20:43

One thing that I always disliked about introductory material to linear regression is how randomness is explained. The explanations always seemed unintuitive because, as I have frequently seen it, they appear as an after thought rather than the central focus of the model. In this post, I'm going to try to take another approach to building an ordinary linear regression model starting from a probabilistic point of view (which is pretty much just a Bayesian view). After the general idea is established, I'll modify the model a bit and end up with a Poisson regression using the exact same principles showing how generalized linear models aren't any more complicated. Hopefully, this will help explain the "randomness" in linear regression in a more intuitive way.

Normal Approximation to the Posterior Distribution

Brian Keng

2016-04-02 15:22

In this post, I'm going to write about how the ever versatile normal distribution can be used to approximate a Bayesian posterior distribution. Unlike some other normal approximations, this is not a direct application of the central limit theorem. The result has a straight forward proof using Laplace's Method whose main ideas I will attempt to present. I'll also simulate a simple scenario to see how it works in practice.

The Empirical Distribution Function

Brian Keng

2016-03-12 20:08

This post is going to look at a useful non-parametric method for estimating the cumulative distribution function (CDF) of a random variable called the empirical distribution function (sometimes called the empirical CDF). We'll talk a bit about the mechanics of computing it, some theory about its confidence intervals and also do some simulations to gain some intuition about how it behaves.

Elementary Statistics for Direct Marketing

Brian Keng

2016-02-27 20:40

This post is going to look at some elementary statistics for direct marketing. Most of the techniques are direct applications of topics learned in a first year statistics course hence the "elementary". I'll start off by covering some background and terminology on the direct marketing and then introduce some of the statistical inference techniques that are commonly used. As usual, I'll mix in some theory where appropriate to build some intuition.

A Primer on Statistical Inference and Hypothesis Testing

Brian Keng

2016-01-09 11:22

This post is about some fundamental concepts in classical (or frequentist) statistics: inference and hypothesis testing. A while back, I came to the realization that I didn't have a good intuition of these concepts (at least not to my liking) beyond the mechanical nature of applying them. What was missing was how they related to a probabilistic view of the subject. This bothered me since having a good intuition about a subject is probably the most useful (and fun!) part of learning a subject. So this post is a result of my re-education on these topics. Enjoy!