Karhunen–Loève expansion

Karhunen–Loève expansion

Polynomial Chaos Expansion (PCE)

The polynomial chaos expansion (PCE) method was first conceived by Wiener [@Wiener1938] as a means to integrate differential equation-type operators where differential Brownian motion, at the time viewed as chaotic, was an external forcing influence.

Polynomial Chaos Expansion is a generalized Fourier expansion on the $L^2(\Omega, \Sigma, P)$ space of random variables, distinguished by the stochastic bases used, to approximate a random variable.

PCE-based approximations have the form: $$X^{(n)} = \sum_{i=0}^{n} g_i \Gamma_i (\xi)$$

(Strictly speaking, it only applies to random variables that are transformations of the standard Gaussian.)

So we are essentially approximating distributions and assuming that the transformation from a base RV (e.g. the standard normal) to an arbitrarily distributed RV exists.


  • Transformations among absolutely continuous RVs, say, are only guaranteed to make sense in distribution.
  • For PCE, we require the Hilbert space to be separable.

Strengths of PCE: Many of the fundamental conservation principles and governing equations of physical processes describe the behavior of field variables themselves but not their statistics or probability measures. PCE provide representations that are well suited for these mathematical models.

Common RV/orthogonal polynomial pairings

  1. Standard Gaussian RV/Hermite polynomials
  2. Exponentially distributed RV/Laguerre polynomials

The Hermite polynomials can be defined as:

$$\Gamma_n(x)=(-1)^n e^{\frac{x^2}{2}}\tfrac{\mathrm{d}^n }{\mathrm{d}x^n} e^{-\frac{x^2}{2}} = \bigg (x-\frac{\mathrm{d}}{\mathrm{d}x} \bigg )^n\cdot 1, \quad n \in \mathbb{N}$$

The first four polynomials are $1, x, x^2-1, x^3 - 3x$.

Hermite polynomials is the set of complete orthogonal polynomials when the weighting function of the inner product on a Hilbert space is standard Gaussian PDF.

Stochastic dimension


Norbert Wiener. The homogeneous chaos. American Journal of Mathematics, 1938. 60:897–936.

🏷 Category=Probability