Numerical Analysis

Numerical error: truncation, round off, error propagation,

Numerical Linear Algebra

Linear equations

Direct methods: Gaussian elimination, LU decomposition; LDLt decomposition;
Iteration methods: Jacobi iteration, Gauss-Seidel iteration, successive over relaxation;
matrix norm, condition number, error estimation;

Notes: 用高斯消元法求解时，比较计算后和计算前的主对角元，比值越小、解的精度就少多少(?) (Using double precision floating point number) 系数矩阵条件数在百亿量级（1e10）时，解的精度大概只有1e-3或1e-4。——陈璞

Eigenvalues and eigenvectors

power method: acceleration, orthonormalization;
inverse iteration;
QR decomposition, Jacobi's method;

Root Finding

Solving Nonlinear Equations:

Newton method, etc.

Merit function for solving nonlinear equations is a scalar-valued function that indicates whether a new iterate is better or worse than the current iterate, in the sense of making progress toward a root. Merit functions are often obtained by combining the components of the vector field in some way, e.g. the sum of squares; all of which have some drawbacks.

Line search and trust-region techniques play an equally important role in optimization, but one can argue that trust-region algorithms have certain theoretical advantages in solving nonlinear equations.

Optimization

Optimization:

unconstrained optimization (for machine learning): quasi-Newton methods, etc.;
Convex Optimization;
smooth optimization;

Function Approximation

Interpolation

Interpolation & numerical differentiation

Fitting

References

Isaacson and Keller, 1966. Analysis of Numerical Methods.
Hamming, 1962, 1973. Numerical Methods for Scientists and Engineers.
Golub and Van Loan, 1983, 2013. Matrix Computations.
Dennis and Schnabel, 1983, 1996. Numerical Methods for Unconstrained Optimization and Nonlinear Equations.
Nocedal and Wright, 1999, 2006. Numerical Optimization.
Suli and Mayers, 2003. An Introduction to Numerical Analysis.

🏷 Category=Computation