Now that you learned about probability mass functions, probability density functions, and cumulative density functions (PMFs, PDFs, and CDFs, respectively), let's dive into the world of distributions!
In this section, you'll learn about two foundational probability distributions that are extremely useful and have an endless amount of applications: the Bernoulli distribution and the Binomial distribution. You'll notice that these distributions formalize a lot of the theory you learned in the probability theory section!
You will be able to:
- Describe the components of a Bernoulli distribution
- Describe how a Binomial Distribution is related to a Bernoulli Distribution
- Use
numpy
to randomly generate Binomial and Bernoulli trials - Use
matplotlib
to show the output of generated Binomial and Bernoulli trials
In the previous sections, we discussed several probability theory situations regarding throwing a dice or flipping a coin. The Bernoulli distribution is a discrete distribution that formalizes the idea of a coin flip.
The Bernoulli experiment is a simple experiment in which there is a binary outcome: 0-1, success-failure, heads-tails, etc.
If we were to model a coin flip with a Bernoulli distribution, we could say that 0 means heads, and 1 tails. With a fair coin, obtaining either value when performing a coin toss would have an assigned probability of 0.5. The Bernoulli experiment can also describe events with different probability structures. For example, let's say that the chance of scoring a penalty goal is 80%. Where
The distribution is defined by 1 parameter, the parameter
Let's look at the PMF and the CDF when
As you can see these functions look pretty straightforward when you plot them.
Now, what is the mean and the variance of these functions? Recall that
$E(X) = \mu = \sum_i p(x_i)x_i= 0.20+0.81=0.8$
$ E((X-\mu)^2) = \sigma^2 = \sum_i p(x_i)(x_i-\mu)^2 = 0.2*(-0.8)^2+0.8*(0.2)^2 = (0.80.2)(0.2+0.8) = 0.16$
A general rule for the Bernoulli distribution is that:
Note how the Bernoulli distribution describes a single coin flip, a single penalty shot, etc. What if we repeat this process multiple times and are interested in the probability of obtaining a certain number of 1s/successes/tails? This process is described by the binomial distribution.
The binomial distribution describes the process of performing
When we say that events are independent, this means that an event is not affected by previous events.
Applying this to our penalty goal example, this means that the assumption is that, when a soccer player misses a penalty and then tries again, the fact that he missed it the previous time does not affect his chances of making it now: the probability is still 80% (
As we have a repeated Bernoulli experiment, the binomial distribution has two parameters:
Now we're interested in finding an expression that gives us the probability to find each possible amount of successes
-
$P(Y=0)$ (or the soccer player doesn't score a single time)? -
$P(Y=1)$ (or the soccer player scores exactly once)? -
$P(Y=2)$ (or the soccer player scores exactly twice)? -
$P(Y=3)$ (or the soccer player scores exactly three times)?
Calculating
Getting to
Where
$P(Y=0)=0.008$ $P(Y=1)=0.096 $ $P(Y=2)=0.384 $ $P(Y=3)=0.512$
Note how they sum to one, which is exactly what's expected! Let's look at their PMF and CDF.
Just like before, let's look at the mean and variance:
$E(X) = \mu = \sum_i p(x_i)x_i= 0.0080+0.0961+0.3842+0.5123 = 2.4 $
$ E((X-\mu)^2) = \sigma^2 = \sum_i p(x_i)(x_i-\mu)^2 =0.008*(-2.4)^2+0.096*(-1.4)^2+ 0.384*(-0.4)^2+0.512 *0.6^2 = 0.48$
Very similarly to Bernoulli, a general rule for the Binomial distribution is that: $E(X)=np$ and $\sigma^2=np*(1-p)$. You simply multiply your results with the number of trials
In the first part, you learned about Bernoulli and Binomial using their formulas to obtain the probability distributions. You can also perform random sampling. What you're basically doing then is selecting a sample from a statistical population in a way that every possible sample has a predetermined probability of being selected.
Applied to our example, imagine that 1 represents scoring a penalty goal and 0 represents missing. If np.random.binomial
to generate how many successful penalties we have when shooting 100 penalties.
import numpy as np
np.random.seed(123) # set a seed to get the same results
np.random.binomial(100, 0.8)
78
Now, let's try this again:
np.random.binomial(100, 0.8)
82
And again:
np.random.binomial(100, 0.8)
83
You can see how this number changes slightly every time and fluctuates around 80. If you'd repeat this many times, and then divide the final result by the number of times you've repeated this, you could expect that the amount of successes will converge to 80. The for loop below does this 500 times.
iteration = []
for loop in range(500):
iteration.append(np.random.binomial(100, 0.8))
np_it = np.array(iteration)
sum(np_it)/500
80.052
Now we'll use np.random.binomial
to illustrate our findings regarding penalties above. Let's keep track of how many times we observe 0 goals, 1 goal, 2 goals, and 3 goals and find the probabilities through simulation. Now, let's repeat our experiment 10000 times.
n = 10000
iteration = []
for loop in range(n):
iteration.append(np.random.binomial(3, 0.8))
np_it = np.array(iteration)
np_it
stores the total penalty goal outcomes (0 to 3) for each of the 10000 iterations. Now using np.unique()
with the optional argument return_counts
, you get the levels of the
values, counts = np.unique(np_it, return_counts=True)
print(values)
print(counts)
[0 1 2 3]
[ 82 938 3905 5075]
Now, let's use these results and visualize them in terms of fractions. You'll see that these fractions are approximations of the values as calculated in the Binomial distribution formula.
import matplotlib.pyplot as plt
plt.bar(values, counts/10000, align='center', alpha=0.9)
plt.xticks(values)
plt.ylabel('Fraction')
plt.title('Number of penalty goals')
plt.show()
Let's now look at the values and compare them with the theoretical result. Recall that the theoretical result was:
counts/10000
array([0.0082, 0.0938, 0.3905, 0.5075])
This seems pretty close to our theoretical result! Try using many more trials (50,000 or 100,000) and see how the sampling result changes!
In this lecture, you learned about the Bernoulli and Binomial distributions, you learned how to use the formula for the Binomial distribution, and how to simulate Binomial trials to get to approximations of the Binomial distribution probabilities.