A **point estimator** is any function $W(\mathbf{X})$ of a sample.
That is, any statistic is a point estimator.

## Methods of Finding Estimators

### Substitution estimators

Method of Moments:

- Empirical Distribution Function
- Method of Moments Estimators

### Maximum likelihood estimators

Maximum likelihood:

- Likelihood Function
- Maximum Likelihood
- Properties of MLE
- Techniques for finding/verifying MLEs
- List of MLEs

### Bayesian estimation

Notes on Bayesian estimation

- The Bayesian approach to statistics
- Bayes estimators

In a broad sense, **MLE** and **Bayesian estimation** are both model selection methods, and they are really similar.
While the former is comparatively easier to implement, the latter is more robust to assumptions.

*MLE* calculates the likelihood function of a given sample, then takes the model with the maximum score.
With likelihood being the objective function, MLE chooses the model that best justifies your available observations, which is a really strong assumption.

*Bayesian estimation* only provides a probability distribution over a probabilistic model subspace, rather than specify a specific probabilistic model.
Bayesian estimation takes not only random sample as input, but also a prior distribution over the model subspace.
The output of Bayesian estimation called a posterior, the normalized product of prior and likelihood function.
The posterior typically provides sharper prediction than the prior, if the prior is close to the likelihood function.
The only issue with Bayesian method is how to justify your prior.

## Methods of Evaluating Estimators

- luck (Cor)
- solve (Cor)
- conditional (L-S)
- information inequality
- linear dependence

Def: unbiased

Def: UMVU

### Completeness & Sufficiency

Notes on Completeness & Sufficiency

Thm: (Rao-Blackwell)

Thm: (Lehmann-Scheffe)

Cor:

### Information Inequality (Cramer-Rao bound)

Notes on Information Inequality

Def: Score function

Def: Fisher information

Thm: (Information inequality)

## List of UMVU Estimators

🏷 Category=Statistics