Probability theory is nearly philosophical.

"Uncertainty concept in engineering" (and society) is not verified, esp. "small probability events", but is definitely a great topic for me to work on.

Concepts

Orthodox concepts:

  • probability: a measure of an event relative to a sample space
  • likelihood: probability of an information set conditioned on an event
  • confidence: pre-sampling probability

Alternative concepts:

  • risk
  • uncertainty
  • possibility

Probability triple: (measure space, events, probability assignments)

Three independent experiments: (specimens, measurement, environment)

Bayesian probability

Priori distribution, or the whole concept of Bayesian methods, is not a falsifiable concept. This is where theory separates from reality.

Two popular views on Bayesian probability:

  • objectivist view: The probability of a proposition corresponds to a reasonable belief everyone (even a "robot") sharing the same knowledge should share in accordance with the rules of Bayesian statistics, which can be justified by requirements of rationality and consistency. [Jaynes, E.T., 1986]
  • subjectivist view: Probability corresponds to a 'personal belief'; rationality and coherence constrain the probabilities a subject may have, but allow for substantial variation within those constraints. [de Finetti, B. (1974)]

My own practice is to use Bayesian analysis in the presence of genuine prior information; to use empirical Bayes methods in the parallel cases situation; and otherwise to be cautious when invoking uninformative priors. In the last case, Bayesian calculations cannot be uncritically accepted and should be checked by other methods, which usually means frequentistically.[^1]

Bayesian methods are easier to explain and understand than their frequentist counterparts.[^2]

For frequentists the prior must have a more objective foundation; ideally that is the relative frequency of events in repeatable, well-defined experiments.[^3]

Use and Interpretations of Probability

Use of Probability

If a problem can be solved by deductive reasoning, probability theory is not needed for it; thus our topic is the optimal processing of incomplete information.

Misuse of probability in disaster anticipation:

  • Space Shuttle Challenger disaster
  • earthquake forecast indicator(s)

Proper use of stochastic processes:

  • Einstein's theory of Brownian motion

Interpretations of probability

When we want to use probability theory for prediction (disasters) and decision making under uncertainty, we have to deal with the interpretation of probability.

  • subjective vs. axiomatic
  • Bayesian vs. frequentist
  • combinatorial

Notes

On Probability Assignment

Probability assignment can be skewed, i.e. not homogeneous.

For a certain universe \(\Omega\) and sigma-algebra \(\mathcal{F}\), design two probability assignments \(P_1\) and \(P_2\), such that under some random variable \(X: ( \Omega, \mathcal{F} ) \rightarrow ( [0,1], \mathcal{B} )\), we get two CDFs \(F_1 (x) = 1\); \(F_2 (x) = x^n\).

Both probability assignments satisfy the axioms for probability measure, but \(P_1\) assigns “homogenous” probability to the universe, under random variable \(X\), \(P_1\) assigns “skewed” probability to the universe, under the same random variable, with higher probability assigned to events with bigger realization.

Any two "admissible" probability assignments on a measurable space \(( \Omega, \mathcal{F} )\) are "indistinguishable".

  • "Admissible" probability assignment assigns non-degenerate probability density on the sigma-algebra.
  • Sigma-algebra resembles topology; "admissible probability assignments" resemble homeomorphism; the generalized result resembles the principle of relativity.

Relation with statistics

随机理论的流行并不意味着人们开始重视不确定性，而是统计方法带动了理论研究。 统计方法适用面广，同时计算性能发展迅速，面对大规模数据有待处理，各学科都被引向统计方法（化学——HDMR，生物——DNA测序，实验设计——Response surface，计算机科学——Bayesian network）。 统计的理论基础则是 Kolmogorov 的主流概率论和随机过程理论。 至于大家是否真的理解不确定性的本质，则很值得怀疑。 [In this case, random variable is useless, while distribution (joint/marginal) is all that make a difference.]

As a result, the imaginary distinction between "probability theory" and "statistical inference" disappears. And the field achieves not only logical unity and simplicity, but far greater technical power and flexibility in applications.

References

[^1]: Bradley Efron, Bayes' Theorem in the 21st Century. Science, 7 June 2013.

[^2]: Bradley Efron, A Statistically Significant Future for Bayes' Rule—Response. Science, 26 July 2013.

[^3]: John Allen Paulos, The Mathematics of Changing Your Mind. The New York Times Books Review.