statistical model

Let $\text{𝐗}=(X_1,\mathrm{\dots },X_n)$ be a random vector with a given realization$\text{𝐗}(\omega )=(x_1,\mathrm{\dots },x_n)$, where $\omega$ is the outcome (of an observation or an experiment) in the sample space $\mathrm{\Omega }$. A statistical model $𝒫$ based on X is a set of probability distribution functions of X:

$$𝒫=\{F_{\text{𝐗}}\}.$$

If it is known in advance that this family of distributions comes from a set of continuous distributions, the statistical model $𝒫$ can be equivalently defined as a set of probability density functions:

$$𝒫=\{f_{\text{𝐗}}\}.$$

As an example, a coin is tossed $n$ times and the results are observed. The probability of landing a head during one toss is $p$. Assume that each toss is independent of one another. If $\text{𝐗}=(X_1,\mathrm{\dots },X_n)$ is defined to be the vector of the $n$ ordered outcomes, then a statistical model based on X can be a family of Bernoulli distributions

$$𝒫=\{\prod _{i=1}^n{p}^{x_i}{(1-p)}^{1-x_i}\},$$

where $X_i(\omega )=x_i$ and $x_i=1$ if $\omega$ is the outcome that the $i$th toss lands a head and $x_i=0$ if $\omega$ is the outcome that the $i$th toss lands a tail.

Next, suppose $X$ is the number of tosses where a head is observed, then a statistical model based on $X$ can be a family binomial distributions:

$$𝒫=\{\left(\genfrac{}{}{0pt}{}{n}{x}\right)p^x{(1-p)}^{n-x}\},$$

where $X(\omega )=x$, where $\omega$ is the outcome that $x$ heads (out of $n$ tosses) are observed.

A statistical model is usually parameterized by a function, called a parameterization

$$\mathrm{\Theta }\to 𝒫\text{ given by }\theta \mapsto F_{\theta }\text{ so that }𝒫=\{F_{\theta }\mid \theta \in \mathrm{\Theta }\},$$

where $\mathrm{\Theta }$ is called a parameter space. $\mathrm{\Theta }$ is usually a subset of ${ℝ}^n$. However, it can also be a function space.

In the first part of the above example, the statistical model is parameterized by

$$p\mapsto \prod _{i=1}^n{p}^{x_i}{(1-p)}^{1-x_i}.$$

If the parameterization is a one-to-one function, it is called an identifiable parameterization and $\theta$ is called a parameter. The $p$ in the above example is a parameter.