For a matrix $M \in M_{m,n}$, a pseudoinverse or generalized inverse is a matrix
$M^+ \in M_{n,m}$ that has some properties analogous to the inverse of an invertible matrix.
Left inverse of an injective matrix is a generalized inverse
whose composition with the matrix is the identity map on the domain:
$\text{rank}(M) = n$, $\exists M^+ \in M_{n,m}$: $M^+ M = I_n$.
Right inverse of a surjective matrix is a generalized inverse
such that the composition of the matrix with it is the identity map on the codomain:
$\text{rank}(M) = m$, $\exists M^+ \in M_{n,m}$: $M M^+ = I_m$.
Left and right inverses are symmetric concepts:
if M is a left inverse of N, then N is a right inverse of M.
Left inverse and right inverse are not unique:
the set of left inverses of a matrix $M \in M^∗_{m,n}$, m < n, is
$[M^+] = \{(N^T M)^{-1} N^T : N \in M^∗_{m,n}, N^T M \in \text{GL}_n\}$;
notice that the element is invariant under $N \to N A$ where $A \in \text{GL}_n$;
$[M^+]$ is bijective to the Grassmann manifold $G_{n,m}$
minus the measure-zero set of subspaces with vectors orthogonal to M.
Moore-Penrose inverse
Moore-Penrose inverse of an m-by-n matrix [@Moore1920; @Bjerhammar1951; @Penrose1955]
is an n-by-m matrix that satisifes the following four Penrose equations [@Ben-Israel2003, Sec 1.1]:
(1) $M M^\dagger M = M$; (2) $M^\dagger M M^\dagger = M^\dagger$;
(3) $(M M^\dagger)^∗ = M M^\dagger$; and (4) $(M^\dagger M)^∗ = M^\dagger M$.
Or equivalently, one that satisifes the following two equations:
$M M^\dagger = P_{[M]}$ and $M^\dagger M = P_{[M^∗]}$,
where $P_{[M]}$ denotes the orthogonal projector onto the span of M.
Properties:
- $M^\dagger$ exists and is unique; or equivalently, $\dagger: M_{m,n} \cong M_{n,m}$.
- Involution: $(M^\dagger)^\dagger = M$.
- Fixed points: $M^\dagger = M$ if and only if M is an orthogonal projector;
or equivalently, $\dagger \cap \text{Id} = \mathcal{P}(k,n)$.
- Representation: let $M = V \Sigma W^∗$ be a compact SVD, then $M^\dagger = W \Sigma^{-1} V^∗$.
Relation with dual operator (Hermitian adjoint matrix):
- $M^\dagger$ equals $M^∗$ with nonzero singular values replaced by their reciprocals:
$M^\dagger = W \Sigma^{-1} V^∗$, $M^∗ = W \Sigma V^∗$.
- $M^\dagger$ and $M^∗$ have the same fundamental subspaces:
$\text{ker}(M^\dagger) = \text{ker}(M^∗)$, $\text{im}(M^\dagger) = \text{im}(M^∗)$.
Relation with matrix inverse:
- If M is invertible, then $M^\dagger = M^{-1}$; $\dagger|_{\text{GL}_n} = (\cdot)^{-1}$.
- If M has full column rank, then $M^\dagger = (M^∗ M)^{-1} M^∗$,
which is a left inverse: $M^\dagger M = I$.
- If M has full row rank, then $M^\dagger = M^∗ (M M^∗)^{-1}$,
which is a right inverse: $M M^\dagger = I$.
Relation with linear equations / model fitting:
- $X^\dagger y$ is the minimum Euclidean norm solution to $X \beta = y$, if $y \in \text{im}(X)$.
- $X^\dagger y$ is the least squares solution to $X \beta = y$, if $y \notin \text{im}(X)$.
Operation equalities (see [@Peterson-Minka] for more):
- It commutes with transposition and conjugation:
$(M^T)^\dagger = (M^\dagger)^T$, $\overline{M}^\dagger = \overline{M^\dagger}$.
- $(M N)^\dagger = (M^\dagger M N)^\dagger (M N N^\dagger)^\dagger$.
- Cases when $(M N)^\dagger = N^\dagger M^\dagger$:
(1) M and N have full rank; (2) $N = M^∗$; or (3) $N = M^T$.
- A, B, C have full rank $\not\Rightarrow (A B C)^\dagger = C^\dagger B^\dagger A^\dagger$.
For example, let $c = (c_1, c_2)$, then $(e_1 e_2^T c)^\dagger = c_2^\dagger e_1^T$
and $c^\dagger (e_2^T)^\dagger e_1^\dagger = (c_1^2+c_2^2)^\dagger c_2 e_1^T$,
which are not equal as long as $c_1, c_2 \ne 0$.
- $M^\dagger = (M^T M)^\dagger M^T$, $M^\dagger = M^T (M M^T)^\dagger$;
(can replace transpose with Hermitian conjugate).
- Let $P \in \mathcal{P}(k,m)$, then $(P M)^\dagger P = (P M)^\dagger$;
let $P \in \mathcal{P}(k,n)$, then $P (M P)^\dagger = (M P)^\dagger$.
🏷 Category=Algebra Category=Matrix Analysis