AdalbertoCq / Dirichlet_processes

Brief introduction and implementations of related concepts to Dirichlet Processes: GEM distribution, Polya Urn, Chinese restaurant process, Stick-Breaking construction, and Posterior of a DP.

Dirichlet Process

Summary

Brief introduction and implementations of related concepts to Dirichlet Processes: GEM distribution, Polya Urn, Chinese restaurant process, Stick-Breaking construction, and Posterior of a DP.

1. Griffiths-Engen-McCloskey (GEM) Distribution:
1. Definition.
2. Function to construct samples
3. Function to construct sample distribution
4. GEM Figures for different values:
1. Figure 1: Distribution of weight values for each dimension of , shows the behavior of for different values
2. Figure 2: Sample vectors and the decreasing trend in average of the weights as we increase dimension
3. Figure 3: Stick representation for 15 samples, from the whole probability vector show each dimension weight with a different color. Another way of representing Figure 1.
2. Polya urn.
1. Definition.
2. Function to model Polya urn.
3. Figure 1: Distribution of independent samples of Polya urns, only high number of draws show to be distributed as a Beta distribution.
3. Chinese Restaurant process.
1. Definition.
2. Function to model table assignations.
3. Function to construct the Chinese restaurant process
4. Figure 1: CRP mean and variance on number of tables for different values.
4. Dirichlet Process:.
1. Definitions:
1. Stick-breaking representation.
2. Ferguson's definition.
2. Function to construct samples using the stick-breaking representation:
3. Function to construct sample distribution
4. DP Figures for different values:
1. Figure 1: Draws from a DP using the stick-breaking representation.
2. Figure 2: Visualization of a DP through Ferguson's definition: , , and the impact of the concentration parameter .
3. Figure 3: Visualization of a DP through Ferguson's definition: Cumulative distributions of the random probability measure and the base measure .
5. Posterior DP:
1. Posterior construction through stick-breaking.
2. Function to construct the DP posterior from obseravations.
3. Figure 1: DP posterior visualizations for different number of observations of an 'assumed true' DP posterior and values in the prior.

Notebooks:

1. DP Visualizations
2. Variational Inference GMM and DPMM