Karhunen–Loève expansion

Karhunen–Loève expansion

Polynomial Chaos Expansion (PCE)

PCE method was first conceived by Wiener [Wiener, 1938] as a means to integrate differential equation-type operators where differential Brownian motion, at the time viewed as chaotic, was an external forcing influence.

Polynomial Chaos Expansion is a generalized Fourier expansion on the \(L^2(\Omega, \Sigma, P)\) space of random variables, distinguished by the stochastic bases used, to approximate a random variable.

PCE-based approximations have the form: \[ X^{(n)} = \sum_{i=0}^{n} g_i \Gamma_i (\xi) \]

(Strictly speaking, it only applies to random variables that are transformations of that standard Gaussian random variable.)

So we are essentially approximating distributions and assuming the transformation from a base RV (e.g. standard normal) to an arbitrarily distributed RV exists.

Note:

  • transformations among absolutely continuous RVs, say, are only guaranteed to make sense in distribution.
  • for PCE, we require the Hilbert space to be separable.

Strengths of PCE:

  1. Many of the fundamental conservation principles and equations governing physical processes describe the behavior and evolution of field variables themselves and not their statistics or probability measures. PCE expansions provide representations that are well suited for these mathematical models.

Common RV/orthogonal polynomial pairings

  1. Standard Gaussian RV/Hermite polynomials
  2. Exponentially distributed RV/Laguerre polynomials

The Hermite polynomials can be defined as:

\[ \Gamma_n(x)=(-1)^n e^{\frac{x^2}{2}}\tfrac{\mathrm{d}^n }{\mathrm{d}x^n} e^{-\frac{x^2}{2}} = \bigg (x-\frac{\mathrm{d}}{\mathrm{d}x} \bigg )^n\cdot 1, \quad n \in \mathbb{N} \]

The first four polynomials are \( 1, x, x^2-1, x^3 - 3x \).

Hermite polynomials is the set of complete orthogonal polynomials when the weighting function of the inner product on a Hilbert space is standard Gaussian PDF.

Stochastic dimension

References

  1. Wiener N. The homogeneous chaos. American Journal of Mathematics, 1938. 60:897–936.