Relations between physical and mathematical
statistics
Armen Allahverdyan
(Yerevan Physics Institute)
-- Maximum aposteriori estimation and maximum-likelihood
-- Hidden Markov models
-- T=0 1d Ising model
-- outline
Math. statistics: data recovery from noise
precision, suffciency, algorithmic procedures,
machine learning, data mining
Phys. statistics: macroscopic matter (heat capacity, conductivity,...)
conservation laws, weak-coupling,
entropy and temperature
Math. and phys. statistics: introducing probabilistic description
greatly simplifies things
Direct relation between math. and phys. statistics
Maximum aposteriori estimation (MAP)
internal
observed
noise
Question: estimate x given y
MAP: maximize over x
(probabilities are known)
weak noise
MAP: minimize over x
MAP --> maximum likelihood
Minimum description length principle: the best hypothesis x for
data y minimizes the complexity of data encoded via x
+ the complexity of the hypothesis x
Structural hierarchy of random processes
independent process
Markov process:
one-step memory
p( x1 , x2 ,..., xn ) = p( x1 )p( x2 )...p( xn )
p( xn | xn −1 , xn − 2 , xn −3 ..., ) = p( xn | xn −1 )
Hidden Markov process:
noisy Markov process
No finite-memory processes
anymore
Simplest hidden
Markov process
Symmetric noise
y : hidden Markov process
Hamiltonian of 1d Ising model
in random fields
ferromagnetic
anti-ferromagnetic
+
−
+
−
frozen disorder
MAP-sequence --> the lowest energy state
Ferromagnetic 1d Ising
yk = 1, J > 0
+
<x>
+
h
low-temperature first-order
phase transition
gauge transformation
domain-wall structure
estimates = observations
z1 z 2
any number of + bonds define a wall
+
τ
odd number of -- bonds
non-unique outcome
anti-ferromagnetic clusters
even number of -- bonds
unique outcome
MAP
maximum aposteriori
MDL
minimum description length
equilibrium statistical
mechanics
Gibbs distribution
2d Ising model
image restoration
active estimation
HMM
hidden Markov processes
1d Ising model, ground-state
problem
finite-temperature physics
Unknown parameters (e.g. q)
non-equilibrium physics
relaxation problem
Law of large numbers
(self-averaging)