# Normal univariate

Suppose ${\displaystyle {\mathbf {x}}=(x_{1},x_{2},\cdots ,x_{n})}$ where ${\displaystyle x_{i}}$ are iid observations from the Normal (also known as Gaussian) distribution with density:

${\displaystyle {\mathcal {N}}(x\mid \mu ,\sigma ^{2})={\frac {1}{\sqrt {2\pi \sigma ^{2}}}}\exp {\Big \{}-{\frac {(x-\mu )^{2}}{2\sigma ^{2}}}{\Big \}},\;-\infty 0)}$.

In the parameterization above the parameters ${\displaystyle \mu }$ and ${\displaystyle \sigma ^{2}}$ represent the mean and variance population, respectively. This distribution may come alternatively parameterized in terms of ${\displaystyle \mu }$ and the standard deviation, ${\displaystyle \sigma }$, or in terms of ${\displaystyle \mu }$ and the precision ${\displaystyle \tau }$, defined as ${\displaystyle \tau =1/\sigma ^{2}}$.

## Case 1: Mean and variance unknown

Overall Recommended (OR) prior
${\displaystyle \pi (\mu ,\sigma ^{2})\propto \sigma ^{-2}}$
In other parameterizations
${\displaystyle \pi (\mu ,\sigma )\propto \sigma ^{-1}}$
${\displaystyle \pi (\mu ,\tau )\propto \tau ^{-1}}$
Recommended prior if ${\displaystyle \mu /\sigma \,}$ is of main interest
${\displaystyle \pi (\mu ,\sigma )\propto {\Big (}1+{\frac {1}{2}}{\big (}{\frac {\mu }{\sigma }}{\big )}^{2}{\Big )}^{-1/2}\sigma ^{-2}}$

### Formal endorsements

[1] showed that the Overall Recommended (OR) prior is the reference prior for any ordering of the parameters ${\displaystyle (\mu ,\sigma ^{2})}$. This observation led to [2] to suggest this prior for general use or as these authors describe in "situations where one is simultaneously interested in all the parameters of the model or perhaps in several functions of them."

The OR prior was first proposed by Harold Jeffreys as the prime example of what we now call Jeffreys independence prior and was later recommended by [3] based on invariance considerations.

A different recommendation arises when the parameter of interest is the standardized normal mean, that is, ${\displaystyle {\frac {\mu }{\sigma }}}$. In this situation [1] showed that the prior in the table is the reference prior.

### Posterior propriety

The posterior distribution of the OR prior is

${\displaystyle \pi (\mu ,\sigma ^{2}\mid {\mathbf {x}})={\mathcal {N}}(\mu \mid {\bar {x}},\sigma ^{2}/n)\,{{\mathcal {I}}G}{\Big (}\sigma ^{2}\mid (n-1)/2,\,\sum _{i=1}^{n}(x_{i}-{\bar {x}})^{2}/2{\Big )}}$

where ${\displaystyle {{\mathcal {I}}G}}$ is the inverse gamma distribution. It can be easily seen that this posterior distribution is proper if ${\displaystyle n\geq 2}$.

### Other properties

Other important properties of the OR prior have to be with invariance arguments and the fact that the normal distribution is location-scale invariant and hence the OR prior equals the right-Haar prior [4].

### Alternatives

• The Jeffreys prior ${\displaystyle \pi (\mu ,\sigma ^{2})\propto \sigma ^{-3/2}}$, but this prior was considered much less appealing starting from the original author [5].
• A proper prior that has the OR prior as the limiting case is ${\displaystyle {\mathcal {N}}(\mu \mid 0,E){{\mathcal {I}}G}(\sigma ^{2}\mid \epsilon ,\epsilon )}$ with (${\displaystyle E\rightarrow \infty ,\epsilon \rightarrow 0}$).

## Case 2: Mean unknown; variance known

Case 2: Recommended prior
${\displaystyle \pi (\mu )\propto 1}$

### Formal endorsements

The recommended prior is the constant prior over the real line. [1] showed that the recommended prior is the reference prior and is also the Jeffreys prior.

### Posterior propriety

The posterior distribution associated with the recommended prior is

${\displaystyle \pi (\mu \mid {\mathbf {x}})={\mathcal {N}}(\mu \mid {\bar {x}},\sigma ^{2}/n)}$,

which is always proper.

### Other properties

The distribution is location invariant and hence the recommended prior coincides with the right-Haar prior [4].

### Alternatives

• A proper prior that has the recommended prior as the limiting case is ${\displaystyle {\mathcal {N}}(\mu \mid 0,E)}$ with ${\displaystyle E\rightarrow \infty }$.

## Case 3: Variance unknown; mean known

Case 3: Recommended prior
${\displaystyle \pi (\sigma ^{2})\propto 1/\sigma ^{2}}$
In other parameterizations
${\displaystyle \pi (\sigma )\propto 1/\sigma }$
${\displaystyle \pi (\tau )\propto 1/\tau }$
${\displaystyle \pi (\ell )\propto 1,\,\ell =\log(\sigma )}$

### Formal endorsements

[1] showed that the recommended prior is the reference prior and is also the Jeffreys prior.

### Posterior propriety

The posterior distribution of the recommended prior is

${\displaystyle \pi (\sigma ^{2}\mid {\mathbf {x}})={{\mathcal {I}}G}{\Big (}\sigma ^{2}\mid n/2,\,\sum _{i=1}^{n}(x_{i}-\mu )^{2}/2{\Big )}}$

which is always proper.

### Other properties

The distribution is scale invariant and hence the recommended prior coincides with the right-Haar prior [4].

### Alternatives

• A proper prior that has the recommended prior as the limiting case is ${\displaystyle {{\mathcal {I}}G}(\sigma ^{2}\mid \epsilon ,\epsilon )}$ with ${\displaystyle \epsilon \rightarrow 0}$.

## References

1. Bernardo, J. and Smith, A. (1994), Bayesian Theory, John Wiley and Sons, London.
2. Berger, J., Bernardo, J., and Sun, D. (2015), Overall objective priors, Bayesian Analysis, 10, 189-221.
3. Barnard, G. A. (1949), Statistical inference, Journal of the Royal Statistical Society, Series B, Methodological 11, 115–149.
4. Berger, J.O. (1985), Statistical Decision Theory and Bayesian Analysis (2nd Ed.), Springer.
5. Jeffreys, H. (1961), Theory of Probability (3rd Ed.), Clarendon Press.