[@Red-Horse2009]

Two types of uncertainty: inherent uncertainty; reducible uncertainty (ignorance).

Inherent uncertainty is uncertainty that cannot be reduced by gathering more information. The origins of this class of uncertainty typically occur at scales that are smaller than either the model or the observations in a given study. Other names for this uncertainty are irreducible uncertainty, aleatoric uncertainty, and variability.

Ignorance is typically associated with limitations in models or available experimental data.

Two approaches to probability:

  1. Statistical descriptions of probabilistic entity: distribution, moments.
  2. Function analytic approach: random variables

Generalizing the concept of RV

For a probability space $(\Omega, \Sigma, P)$ and an arbitrary measurable space $(H, \mathcal{H})$, random variable $X: \Omega \to H$ is a $(\Sigma, \mathcal{H})$-measuralbe function. It has induced distribution $\eta_X (X \in A) = P(X^{-1}(A)), \quad A \in \mathcal{H}$.

Metric space

$H$ be a metric space.

  • A Borel sigma-algebra on $H$ can be induced by its metric, denote as $\mathcal{H}$.
  • Denote the space of bounded continuous functions from $H$ to $\mathbb{R}$ as $C(H)$. It is a Banach space under the supremum norm. If $H$ is compact then any positive linear functional on $C(H)$ can be uniquely identified with the expectation operator relative to some probability measure on $H$.
  • Denote the space of all probability measures on $H$ as $\mathcal{M}(H)$. It can be metrized as a separable metric space if and only if $H$ is a separable metric space.

Topological group

Weakly complete topological linear space

Topological dual is the space of continuous linear functionals defined on a topological linear space.

Weak topology: the topological dual of a topological linear space induce a topology in the original space where neighborhoods are defined in terms of their images under the duality pairing. It is a subset of the original topology, i.e. a weaker topology.

duality pairing

Reproducing kernel Hilbert spaces (RKHS): The operator $S_X^{-1}$ is a reproducing kernel in $H_X$.

Separable Hilbert space

A countably-normed space is a topological vector space over $C$ with topology given by a countable family of norms (i.e. intersection of all norm-induce neighborhoods). A countably-Hilbert space is a countably-normed space complete with respect to its topology, and each norm is induced from an inner product.

The compatibility requirement on the norms associated with a countably-Hilbert space implies an ordering in these norms.

Nuclear operator

Both the covariance operator and the characteristic functional of a stochastic process are nuclear operators.

Nuclear space

canonical embedding

Kolmogorov consistency requirements

$L^2$ space

When $H$ is a Hilbert space, the covariance opertator $S_X$ is Hilbert–Schmidt, self-adjoint, positive, and completely continuous, and therefore has a finite or countable set of eigenvalues and corresponding orthonormal eigenfunctions ${ (\lambda_n, \phi_n) }$.

The RKHS structure ensures that the evaluation functional in this space is continuous.

Thus we have Karhunen–Loeve expansion for such defined RVs.

Multi-dimensional PCEs

Questions

  1. Often times people only use limited description of a random entity, such as mean and variance in the case of random variable, and mean process and correlation function in the case of random process. If only this extent of information is needed, why do we need to approximate r.v. with PCEs?

🏷 Category=Probability