A spectral decomposition, expansion, or representation, of a random function approximates the random function by a series or integral w.r.t. a particular set of functions, such that the series coefficients are pairwise uncorrelated random variables. (Here, "spectral" may come from frequency spectrum via Fourier transform, or from spectral theory of self-adjoint operator.) The set of functions may be generic, e.g. harmonics $e^{i \lambda t}$ and polynomials, or they can be specific to the problem or estimated from data.
Mercer's theorem (Mercer, 1909; continuous kernel over compact set): For a continuous, symmetric, positive semi-definite function $K: X^2 \mapsto \mathbb{R}$, where $X \subset \mathbb{R}^n$ is compact, define an integral operator $T_K$ over $L^2(X)$, such that for $f \in L^2(X)$, $(T_K f)(x) = \int_X K(x, x') f(x')~d x'$. (1) There exists a total orthonormal subset $\{b_i(t)\}_{i \in \mathbb{N}}$ of $L^2(X)$ which consists of eigenfunctions of $T_K$ with nonnegative eigenvalues $\{\lambda_i\}_{i \in \mathbb{N}}$. (2) If an eigenvalue is positive, the associated eigenfunction is continuous. (3) The kernel can be represented as a series that converges absolutely and uniformly: $K(x, x') = \sum_{i=0}^\infty \lambda_i b_i(x) b_i(x')$.
Mercer's theorem (continuous kernel): Let $(X, \mathcal{T}, \mathcal{B(T)}, \mu)$ be a first-countable topological space endowed with a Borel complete measure $\mu$ such that every point has a finite-measure neighborhood. For a continuous kernel $K(x, x')$ whose diagonal $k(x) = K(x, x)$ is absolute-integrable, i.e. $k \in L^1_\mu(X)$, then (1-2) still hold and (3) holds on compact subsets of $X$.
Mercer's theorem ($L^2$ kernel): Let $(X, \Sigma, \mu)$ be a σ-finite measure space. For an $L^2$ kernel $K \in L^2_{\mu \otimes \mu} (X \times X)$, then (1) still holds and (3) holds with only $L^2$ convergence.
Karhunen–Loève Theorem (Karhunen 1947, Loève 1955; continuous covariance over compact interval): For a second-order random process $X(u, t)$ over $T = [a, b]$ with mean zero and continuous covariance function $K(t, t')$, there exists a total orthonormal subset $\{\phi_i(t)\}_{i \in \mathbb{N}}$ of $L^2(T)$ which consists of continuous eigenfunctions of $T_K$ with positive eigenvalues $\{\lambda_i\}_{i \in \mathbb{N}}$, such that the random process can be represented as a series with $L^2$ convergence: $X(u, t) = \sum_{i=0}^\infty Z_i(u) \phi_i(t)$, where (random projection coefficient) $Z_i(u) = \langle X(u, t), \phi_i(t) \rangle = \int_T X(u, t) \phi_i(t)~d t$. One can verify that $\{Z_i\}_{i \in \mathbb{N}}$ is a sequence of uncorrelated random variables each with mean zero and variance $\lambda_i$.
Karhunen–Loève Theorem (continuous separable covariance over a box): For a random field over a box $S = \prod_{i=1}^d I_i$, $I_i = [a_i, \phi_i]$, if the covariance function is separable $K(s, s') = \prod_{i=1}^d K(s_i, s_i')$, the 1d K-L theorem still holds, with $X(u, s) = \sum_{k \in \mathbb{N}^d} Z_k(u) \prod_{i=1}^d \phi_{k_i}^{(i)}(s_i)$ and $Z_k(u) = \int X(u, s) \prod_{i=1}^d \phi_{k_i}^{(i)}(s_i)~d s$. One can verify that $\{Z_k\}_{k \in \mathbb{N}^d}$ are uncorrelated random variables each with mean zero and variance $\prod_{i=1}^d \lambda_{k_i}^{(i)}$. The modes can be sorted by descendign variance, i.e. product of eigenvalues.
K-L expansion can be seen as a function version of principal component analysis (PCA), sometimes called the principal component transform. Proper orthogonal decomposition (POD) can be seen as an application of K-L expansion in computational fluid dynamics, where the time variable is treated as the "stochastic dimension".
The polynomial chaos expansion (PCE) method was first conceived by Wiener [@Wiener1938] as a means to integrate differential equation-type operators where differential Brownian motion, at the time viewed as chaotic, was an external forcing influence.
Polynomial Chaos Expansion is a generalized Fourier expansion on the $L^2(\Omega, \Sigma, P)$ space of random variables, distinguished by the stochastic bases used, to approximate a random variable.
PCE-based approximations have the form: $$X^{(n)} = \sum_{i=0}^{n} g_i \Gamma_i (\xi)$$
(Strictly speaking, it only applies to random variables that are transformations of the standard Gaussian.)
So we are essentially approximating distributions and assuming that the transformation from a base RV (e.g. the standard normal) to an arbitrarily distributed RV exists.
Note:
Strengths of PCE: Many of the fundamental conservation principles and governing equations of physical processes describe the behavior of field variables themselves but not their statistics or probability measures. PCE provide representations that are well suited for these mathematical models.
The Hermite polynomials can be defined as:
$$\Gamma_n(x)=(-1)^n e^{\frac{x^2}{2}}\tfrac{\mathrm{d}^n }{\mathrm{d}x^n} e^{-\frac{x^2}{2}} = \bigg (x-\frac{\mathrm{d}}{\mathrm{d}x} \bigg )^n\cdot 1, \quad n \in \mathbb{N}$$
The first four polynomials are $1, x, x^2-1, x^3 - 3x$.
Hermite polynomials is the set of complete orthogonal polynomials when the weighting function of the inner product on a Hilbert space is standard Gaussian PDF.
Norbert Wiener. The homogeneous chaos. American Journal of Mathematics, 1938. 60:897–936.