Statistics has two aspects: algorithms and inference.

**Statistical inference** is a system of mathematical logic for guidance and correction (or justification).
Classical inference methodology: frequentist, Bayesian, and Fisherian.

Principles of Statistical Inference: Sufficiency principle, Conditionality principle, Likelihood principle.

The fundamental construct in probability is random variable; the fundamental construct in statistics is random sample.

**Random sample** is a sampling process from a hypothetical population.
Traditional statistics assumes "large $n$, small $p$" ($n$ for observations, $p$ for parameters measured.)
While in modern statistics, the problem typically is "small $n$, large $p$".

In statistics, a **model** is a probability distribution of one or more variables:
univariate models;
regression models;

Parametric and nonparametric methods do not have essential difference between or superiority to each other:
both are collections of models and take random samples as the sole input for estimation (frequentist).
**Parametric methods** are algorithms selecting one from a subspace of probability models,
which is indexed by model parameters.
**Nonparametric methods** are algorithms selecting one from another subspace of probability models,
only without an index.
Generally, nonparametric methods are non-mechanistic methods, which are statistical in essence.

Point Estimation: methods of finding and evaluating estimators, UMVU estimators;

Interval Estimation: confidence interval, tolerance interval;

Regression: Least-squares, lasso, ridge

Likelihood Ratio Test (LRT), Uniformly Most Powerful (UMP) Test

False discovery rate (FDR)

Asymptotic Analysis:

- Asymptotic Evaluation Criteria
- Local Asymptotic Normality
- Asymptotic order statistics
- Asymptotic Moment Estimators

**Statistical learning** is the attempt to explain techniques of learning from data
in a statistical framework.

prediction, explanation

Before Fisher, statisticians didn’t really understand estimation. The same can be said now about prediction. [@CASI2017]

- Statistical Tables: Normal, t, F, Chi-Squared
- Statistical Tables: Binomial
- Statistical Tables: Poisson
- Univariate Distribution Relationships

Notes on Intuitive Biostatistics [@Motulsky1995]

Table 1: Statistical Techniques

Purpose | Continuous Data | Count or Ranked Data | Arrival Time | Binary Data |
---|---|---|---|---|

(Examples) | (Height) | (Number of headaches in a week; Self-report score) | (Life expectancy of a patient; Minutes until REM sleep begins | Recurrence of infection) |

Describe one sample | Frequency distribution; Sample mean; Quantiles; Sample standard deviation | Frequency distribution; Quantiles; | Kaplan-Meier survival curve; Median survival curve; Five-year survival percentage | Proportion |

Distributional Test | Normality tests; Outlier tests | N/A | N/A | N/A |

Infer about one population | One-sample t test | Wilcoxon’s rank-sum test | Confidence bands around survival curve; CI of median survival | CI of proportion; Binomial test to compare observed distribution with a theoretical (expected) distribution |

Compare two unpaired groups | Unpaired t test | Mann-Whitney test | Log-rank test; Gehan-Breslow test; CI of ratio of median survival times; CI of hazard ratio | Fisher’s exact test; |

Compare two paired groups | Paired t test | Wilcoxon’s matched paires test | Conditional proportional hazards regression | McNemar’s test |

Compare three or more unpaired groups | One-way ANOVA followed by multiple comparison tests | Kruskal-Wallis test; Dunn’s posttest | Log-rank test; Gehan-Breslow test | Chi-squared test (for trend) |

Compare three or more paired groups | Repeated-measures ANOVA followed by multiple comparison tests | Friedman’s test; Dunn’s posttest | Conditional proportional hazards regression | Cochran’s Q |

Quantify association between two variables | Pearson’s correlation | Spearman’s correlation | N/A | N/A |

Predict one variable from one or several others | linear/nonlinear regression | N/A | Cox’s proportional hazards regression | Logistic regression |