Email: [email protected]

Dubious Soup

#2 The Student’s t-Distribution is Fancy~

About a month or two ago, my AP statistics teacher, Mr. Wylder, presented us with the t-distribution (Student’s t-distribution for long). He laid out the concept simply: the t-distribution is nothing but a normal distribution that is thicker on the tails to better approximate small samples. I looked at his nice explanation and then at the bizarre equation in front of me, and asked, “Why?” and now we are here! While I can’t promise an exact answer to why the t-distribution is the way it is, I hope you can appreciate the journey I went on these past few weeks.

To Start…

We start our journey with clearly… NOT MATH BUT HISTORY! Yeah, I got you, you silly math nerd. In English literature at least, the Student’s t-distribution is attributed to its namesake William Sealy Gosset. This English statistician worked for a brewery when he noticed that his confidence intervals and z tests (which we will get into later) were slightly less accurate than predicted (total scam!). He then invented the t-distribution (t for test statistic t) to correct for the error by giving the normal distribution thicker tails (or higher kurtosis). He wanted to publish a paper on his new invention but his hire wanted him to conceal his name for fear of competing breweries learning of their secret weapon of error correction. Gosset decided to use his pseudonym, Student, and that was how, in 1908, the Student’s t-distribution was born!

Or so we are told. The full story of the Student’s t-distribution starts with German mathematicians Friedrich Robert Helmert and Jacob Lüroth who originally derived the t-distribution’s formula (derived as a Posterior Distribution at the time) in 1876. Additionally, in 1895, Karl Pearson published the special case Pearson IV function, the Pearson VII function, which was just a non-standardised Student’s t-distribution! Only in 1908 did Gosset improve on the Student’s t-distribution in his paper.

William S. Gosset
Robert Helmert
Jacob Lüroth
Karl Pearson

Conceptualizing the Student’s T-Distribution

Nice story, you may be thinking, but what IS the Student’s t-distribution? Now we talk about the confidence intervals and z/t tests. The t-distribution is used primarily when we don’t know the population’s standard deviation (which is most of the  time) but want to use CLT (central limit theorem), such as in confidence intervals and z/t tests to approximate the mean or proportion.

CLT itself states that if we have a big enough sample of continuous variables, then the sampling distribution of means (probability density function of all possible means given that size) will be a normal distribution. The Student’s t-distribution, then, is the sample approximation of the population’s sampling distribution.

What does that mean exactly? Well, if we look at a sample variance and compare it to the population variance, we will notice that the sample variance has a correction factor; the correction factor (or degrees of freedom) is used to better approximate the population variance given the sample. In light of this, the t-distribution is just the statistic to the sampling distribution parameter.

Behind the Student’s t-Distribution’s PDF

But now, let’s get to why I really looked into the Student’s t-distribution. Its fancy probability density function: $$f(x) = \dfrac{\Gamma\left(\dfrac{\nu + 1}{2}\right)}{\sqrt{\nu\pi}\Gamma\left(\dfrac{\nu}{2}\right)}\left(1 + \dfrac{t^{2}}{\nu}\right)^{-\dfrac{\nu + 1}{2}}$$From this article that, in my opinion, best explains why the t-distribution’s PDF is what it is, I will try to summarize. The Student’s t-distribution is the ratio function of a normal distribution and the square root of a chi squared distribution divided by the chi squared degrees of freedom.$$t_{k} \sim \frac{\mathcal{N}\left( 0,1 \right)}{\sqrt{\chi^{2}_{k}/k}}$$Remember that the t-distribution is just the statistic sampling distribution normal distribution, so you can think of the divisor as the ‘corrector.’ The more mathematical approach to understand the above formula is to understand that the t statistic and z statistic are computed using the same formula; the only difference is that the t statistic is computed using sample variance while the z statistic is computed using population variance.$$Z = \frac{\overline{X}-\mu}{\sigma/\sqrt{n}} \; T = \frac{\overline{X}-\mu}{S/\sqrt{n}}$$ After doing the algebra, we find that the t statistic equals a normal distribution divided by the square root of the sample variance divided by the population variance. The fraction of sample variance and population variance happens to be the chi squared divided by degrees of freedom. With the path from concept to function carved out, we can, with a clear conscience, evaluate the ratio PDF to find the fancy Student’s t-distribution PDF! Whew, that was a little workout! At least now you can brag that you kind of know the Student’s t-distribution.
But if my journey piqued your interests, please take your own voyage of discovery to obtain a more holistic understanding of the Student’s t-distribution. Read about the Cauchy Distribution (a special t-distribution) and its uses in physics. Or take a deep dive into posterior distributions and how they play a role in the t-distribution. You may even want to check out Pearson’s Probability Types (which I will likely write on in the future). Regardless, I hope you were entertained and can spend the day appreciating this cool part of math (if you want to, of course).

Ivanna P.