Definition: a **parametric probabilistic model** or, in short, **parametric model** is a family of probability distributions that can be described using a finite number of parameters.
The set of all allowable values for the parameter, or the **parameter space**, is denoted as $\Theta \subseteq \mathbb{R}^k$, and the parametric model is denoted as

$$\mathcal{P} = \{ P_{\boldsymbol \theta}\ |\ \boldsymbol \theta \in \Theta \}$$

## Categories of Parametric Models

### Exponential Family

Definition: A parametric model is called an **exponential family**, if it has the form

$$f( \mathbf{x} \ | \ \boldsymbol \theta ) = c(\boldsymbol \theta) h(\mathbf{x}) \exp \left\{ \sum_{i=1}^l w_i(\boldsymbol \theta) t_i(\mathbf{x}) \right\}$$

Here $c(\cdot), h(\cdot)$ are nonnegative, and all functions are real-valued.

In addition,

- If k < l, the parametric model is called a
**curved exponential family**.
- If k = l, it is called a
**full exponential family**.

Examples of exponential families:

- Discrete distributions: Binomial, Poisson, negative binomial;
- Continuous distributions: Normal, Gamma, Beta.

Theorem: (**Natural Parameterization**) For an exponential family, denote **natural parameters** $\eta_i = w_i(\boldsymbol \theta)$, then the parametric model can be rewritten as

$$f( \mathbf{x} \ | \ \boldsymbol \eta ) = c^{*}(\boldsymbol \eta) h(\mathbf{x}) \exp \left\{ \sum_{i=1}^l \eta_i t_i(\mathbf{x}) \right\}$$

And the **natural parameter space** defined as $\left\{ \boldsymbol \eta \ \big| \ \int_{\mathbb{R}} h(\mathbf{x}) \exp \left\{ \sum_{i=1}^l \eta_i t_i(\mathbf{x}) \right\} \text{d}x < \infty \right\}$ is convex.

### Location-Scale Family

Definition: the **location-scale family** with standard PDF $f(x)$ is the parametric model

$$\mathcal{P} = \left\{ f_{(\mu,\sigma)} = \frac{1}{\sigma} f(\frac{x-\mu}{\sigma}) \ \bigg| \ (\mu,\sigma) \in \mathbb{R}\times\mathbb{R}^{+} \right\}$$

Here, $\mu$ is called the **location parameter**, and $\sigma$ is called the **scale parameter**.

Theorem:

$$X \sim f_{(\mu,\sigma)} \Leftrightarrow \exists Z \sim f(z), \text{ s.t. } X = \sigma Z + \mu$$

🏷 Category=Statistics