next up previous
Next: Markov Chain Monte Carlo Up: MCMC Previous: Bayesian Inference

Statistical Sampling

The average height of all people in Cambridge can be obtained by measuring the heights of all people in Cambridge and then averaging them:

\begin{displaymath}E_{p\in C} [h(p)] =\frac{1}{\vert C\vert} \sum_{p\in C} h(p) \end{displaymath}

Alternatively, this can be approximated by sampling the population:

\begin{displaymath}E_{p\in C} [h(p)] \approx \frac{1}{S} \sum_{s=1}^S h(p^{(s)}) \end{displaymath}

where $\{ p^{(s)}\vert s=1,\cdots,S) \} \in C$ is a subset of all people in Cambridge $C$.

Similarly, the following expectation can be approximated by

\begin{displaymath}E_{p(x)} f(x)=\int f(x) p(x) dx \approx \frac{1}{S} \sum_{s=1}^S
f(x^{(s)}) \end{displaymath}

where $x^{(s)}$ are drawn from the distribution $p(x)$. This is called simple Monte Carlo method. But how to sample a given distribution $p(x)$?

Importance sampling and rejection sampling work well only if the proposal density $q(x)$ is similar to $p(x)$, which may be difficult in practice.


next up previous
Next: Markov Chain Monte Carlo Up: MCMC Previous: Bayesian Inference
Ruye Wang 2018-03-26