Hypothesized relations between variables are what we call models. Models are at the core of prediction science.

This article reviews different types of models, compares vocabulary across disciplines and how to build and interpret models.

General Forms of Models

Most models are built as black boxes, while some are structural.

Models can also be categorized as deterministic or probabilistic.

Deterministic View

General black box model: (input - blackbox - output)

\[ x \rightarrow [f] \rightarrow y \]

Probabilistic View

General black box model (two parts): (Observables are stochastically related through a joint distribution.)

\[ X \sim f(X) \] \[ X \rightarrow f(Y|X) \rightarrow Y \]

Reduced form 1: Moment description

First-order description:

\[ X \rightarrow [f] \rightarrow E(Y|X) \]

Note: This is the view adopted in econometrics.

Second-order description: (Electrical engineering, w.s.s.)

Reduced form 2: (Bayesian) Network

\[ X ~ \prod_{i=1}^{n} f( X_i | \Pi_{X_i} ) \]

\( \Pi_{X_i} \) denotes the parent nodes of \( X_i \) in Bayesian network \( G={X,E} \).

Names of Models in Different Fields

Different fields and theories have different terminology for their models, but they share the same essence as outlined above. Collecting these concepts is the first step towards a general theory of prediction science. [Interestingly, this saying is rarely used by people other than my adviser.]

Black Box Models

  • function [mathematics]
  • operator [mathematics/analysis]
    • differentiation, integration, convolution, transform
  • transformation [stochastic analysis]
  • regression [statistics]
  • respond surface [experiment design]
  • diffusion map [machine learning]

  • constitutive relations [physics]

  • mapping

    • neural-behavioral maps

  • system [electrical engineering]

  • input-output [economics]

Other names for black box models:

  • phenomenological models [physics, mechanics, ecology]
  • empirical relationships
  • reduced form models [econometrics]

Structural Models

  • structural models [econometrics]

  • feedback loop [cybernetics]

  • system diagram [systems theory]
  • network [network science]

  • regulatory circuits [molecular biology]

  • idealized model [physics]

  • program [computer science]

Other names for structural models:

  • process-based models [ecology]
  • mechanistic models

The Modeling Process

Model Building

Point: the building block of mathematical theories.

Quantitative Theory Categories

Information used: prior knowledge about the system, measurements on variables, etc.

Impossibility principle of modeling building: It is impossible to build a model that provides meaningful output at a fine scale, with most observables at a coarser scale and only limited information that the fine scale.

Building structural models: hypothesization, abstraction, simplification.

普朗克推导黑体辐射定律时，做了能量子假设，使得导出的定律与全频谱的实验数据吻合得很好。相比而言，此前的维恩公式仅仅适用于低频部分。【经典建模实例】

Model Validation/Selection

Goodness-of-fit measures [statistics]

Occam's razor principle

Model Interpretation

Interpretation of parameters:

ceteris paribus: Should the traditional "all else being equal" be replaced by a more practical "all else left unknown", conclusions can become much more useful. This symbolizes a transition from deterministic differential viewpoint to statistical presentation.

References

  1. Black box
  2. For a review of models in molecular biology, see [Kim:2009]
  3. E Weinan. Principles of multiscale modeling. 2011

Topics in modeling:

  1. Forward model and inverse model (When variables have evident temporal, logical or even causal relations.)
  2. causality and correlation
  3. Power Laws

🏷 Category=Modeling