Bayesian Analysis¶
In this post we perform a simple but explicit analysis of a curve fitting using Bayesian techniques.
import mmf_setup;mmf_setup.nbinit()
Table of Contents
The Model¶
Consider the problem of curve fitting:
$$ Y = h(t, a) + X $$where $X$ is a random variable representing errors with some probability density function (PDF) $f_X(x)$. Within this model, given $t$ and $a$, $Y$ is a random variable with PDF:
$$ f_Y(y) = P(y|t,a) = f_X\bigl(y - h(t, a)\bigr). $$Maximum Likelihood¶
Give a set of data $D=(\vect{t}, \vect{y})$, one can precisely formulate the question: what is the probability (likelihood) $P(D|a)$ that this set of data would be obtained from our model given a parameter $a$:
$$ P(D|a) = \prod_{i} f_X\bigl(y_i - h(t_i, a)\bigr). $$Maximum likelihood techniques choose the best fit for the parameter $a$ to maximize the likelihood:
$$ \sup_{a} P(D|a). $$Bayes' theorem allows us to compute the a posteriori distribution of the parameter $a$ given the observation of data $D$, updating the prior distribution $P(a)$ normalized by the probability of obtaining the data $D$:
$$ P(a|D) = \frac{P(D|a)P(a)}{P(D)}. $$