Definition: a parametric probabilistic model or, in short, parametric model is a family of probability distributions that can be described using a finite number of parameters. The set of all allowable values for the parameter, or the parameter space, is denoted as \( \Theta \subseteq \mathbb{R}^k \), and the parametric model is denoted as
\[ \mathcal{P} = \{ P_{\boldsymbol \theta}\ |\ \boldsymbol \theta \in \Theta \} \]
Definition: A parametric model is called an exponential family, if it has the form
\[ f( \mathbf{x} \ | \ \boldsymbol \theta ) = c(\boldsymbol \theta) h(\mathbf{x}) \exp \left\{ \sum_{i=1}^l w_i(\boldsymbol \theta) t_i(\mathbf{x}) \right\} \]
Here \( c(\cdot), h(\cdot) \) are nonnegative, and all functions are real-valued.
In addition,
Examples of exponential families:
Theorem: (Natural Parameterization) For an exponential family, denote natural parameters \( \eta_i = w_i(\boldsymbol \theta) \), then the parametric model can be rewritten as
\[ f( \mathbf{x} \ | \ \boldsymbol \eta ) = c^{*}(\boldsymbol \eta) h(\mathbf{x}) \exp \left\{ \sum_{i=1}^l \eta_i t_i(\mathbf{x}) \right\} \]
And the natural parameter space defined as \( \left\{ \boldsymbol \eta \ \big| \ \int_{\mathbb{R}} h(\mathbf{x}) \exp \left\{ \sum_{i=1}^l \eta_i t_i(\mathbf{x}) \right\} \text{d}x < \infty \right\} \) is convex.
Definition: the location-scale family with standard PDF \( f(x) \) is the parametric model
\[ \mathcal{P} = \left\{ f_{(\mu,\sigma)} = \frac{1}{\sigma} f(\frac{x-\mu}{\sigma}) \ \bigg| \ (\mu,\sigma) \in \mathbb{R}\times\mathbb{R}^{+} \right\} \]
Here, \( \mu \) is called the location parameter, and \( \sigma \) is called the scale parameter.
Theorem:
\[ X \sim f_{(\mu,\sigma)} \Leftrightarrow \exists Z \sim f(z), \text{ s.t. } X = \sigma Z + \mu \]