EMPIRICAL PREDICTION ALGORITHM
N.G. Z a g o r u i k o
I n s t i t u t e o f Mathematics
Academy of S c i e n c e s , USSR, S i b e r i a n Branch
N o v o s i b i r s k , 650090, USSR
Abstract
The f o l l o w i n g r e q u i r e m e n t s t o e m p i r i c a l
p r e d i c t i o n a l g o r i t h m s are i n v e s t i g a t e d :
u n i v e r s a l i t y , n o n - t r i v i a l i t y , and c o n s i s t e n c e . I t i s p o i n t e d out these r e q u i r e ments are t o o s t r o n g . Some v a r i a n t s of
algorithms in which the requirement of
"consistance" is replaced by that of "the
g r e a t e s t s i m p l i s i t y " are t e s t e d .
A n a l y s i s o f process o f problem s o l v i n g tasks by the systems, which i n v e s t i g a t o r s c a l l i n t e l l e c t u a l ones, shows t h a t
stage o f p r e d i c t i o n o f new f a c t s b y u s ing r e g u l a r i t i e s discovered on set of
well-known f a c t s , plays an important p a r t
i n t h i s process.
Thus, i n p a t t e r n r e c o g n i t i o n t a s k s a
b e l o n g i n g o f c o n t r o l r e a l i z a t i o n t o one
o r a n o t h e r image i s p r e d i c t e d o n t h e g r o u nds o f n a t u r a l r e g u l a r i t i e s observed o n
t r a i n i n g sequence between o b j e c t s ' p r o p e r t i e s and p a t t e r n name.
I n chess and d r a u g h t s programs, e t c .
a p r e d i c t i o n block is represented by p r o cedures o f t a k i n g a d e c i s i o n about t h e
most p r e f e r a b l e move. I n s o d o i n g t h e r e g u l a r i t i e s i n t r o d u c e d b y programmer o r
d i s c o v e r e d by computer are usedj t h e w i n n i n g move i s t h a t one when e s t i m a t i o n f u n c t i o n o f s u c h - a n d - s u c h k i n d reaches t h e
maximum v a l u e ; i n s u c h - a n d - s u c h s i t u a t i o n
from the l i s t p r e l i m i n a r i l y " l e a r n t b y
h e a r t " such-and-such move i s s u c c e s s f u l ,
etc.
I n programs i n t e n d e d f o r d e f i n i t i o n
of s t r u c t u r a l formulas of organic molecul e s b y t h e i r c h e m i c a l f o r m u l a and b y massspectrum, the mass-spectra of molecules
engendered by h y p o t h e s i s g e n e r a t o r are
c a l c u l a t e d w i t h the a i d o f p r e d i c t i o n
b l o c k . R e s u l t s o f t h i s p r e d i c t i o n are used
f o r s e l e c t i o n t h e s t r u c t u r e s which s h o u l d
have mass-spectrum n o t so much d i f f e r e n t
f r o m e x p e r i m e n t a l one.
A b i l i t y o f making s u c c e s s f u l p r e d i c t i o n s i s one o f t h e most i m p o r t a n t f e a t u r e s of any i n t e l l e c t u a l system. One can
a t t a i n a c e r t a i n o b j e c t o n l y i n case when
t h e r e i s a p o s s i b i l i t y t o foresee consequences of one or a n o t h e r a c t i o n s . On t h i s
s u b j e c t von F o e r s t e r w r i t e s t h a t t o s u r v i v e i s t o f o r e s e e c o r r e c t l y the e v e n t s
i n surroundings. Inductive inference i s
t h e l o g i c a l b a s i s o f f o r e s i g h t , i . e . method to search, given condition E, f o r
hypothesis h, which is confirmed by s u r r o u n d i n g s S and i s c o n v e n i e n t f o r a c e r t a i n
aim [ 1 ] .
I n t h e l a s t a n a l y s i s , purpose o f any
e m p i r i c a l science i s t o b r i n g t o l i g h t the
219
n a t u r a l r e g u l a r i t i e s between c h a r a c t e r i s t i c s o f o b s e r v a b l e phenomena, and t o f o r m u l a t e these r e g u l a r i t i e s i n t h e f o r m
t h a t c o u l d b e r e l i a b l e and c o n v e n i e n t means to f o r e s e e t h e new phenomena.
Pact o f f o r e s i g h t ( p r e d i c t i o n ) can
b e e s t a b l i s h e d o b j e c t i v e l y and d e f i n e t l y .
D e s c r i p t i o n o f t h e event t o b e p r e d i c t e d
is entered in p r o t o c o l ; correctness of
t h e p r e d i c t i o n i s t e s t e d b y course o f f u r t h e r events.
E s p e c i a l importance o f t h e stage
" p r e d i c t i o n " f o r i n t e l l e c t u a l systems and
the p o s s i b i l i t y o f c o n s t r u c t i v e d e f i n i t i o n o f the stage j u s t i f i e s c o n c e n t r a t i o n
o f e f f o r t s t o i t s s t u d y . T h i s paper c o n t a i n s r e v i e w of works on methodology and
o f a l g o r i t h m s o f p r e d i c t i o n o f f a c t s and
events i n e m p i r i c a l w o r l d .
P o s s i b i l i t y t o p r e d i c t events i s b a sed on acceptance of d e t e r m i n i s m c o n c e p t i o n . D e n i a l o f c a u s a l r e l a t i o n between
phenomena a u t o m a t i c a l l y e x c l u d e s such p o s s i b i l i t y . One can i l l u s t r a t e n e g a t i v e
p o s i t i o n r e l a t i v e t o any p r e d i c t i o n b y
W i t t g e n s t e i n ' s o p i n i o n [2] t h a t w e c a n n o t
p r e d i c t events of the f u t u r e on the b a s i s
o f the p t e s e n t . B e l i e f i n c a u s a l i t y i s
p r e j u d i c e . That t h e sun w i l l r i s e t o m o r row is a h y p o t h e s i s ; in o t h e r w o r d s , we
d o n o t know f i r m , whether i t w i l l r i s e .
In the f o l l o w i n g we s h a l l proceed
from the b e l i e f i n existence o f n a t u r a l
r e l a t i o n s between phenomena. P r e d i c t i o n s
w i l l b e made w i t h a p p l i c a t i o n o f r e g u l a r i t i e s t h a t are found i n e m p i r i c a l d a t a .
D i s c u s s i o n o f necessary r e q u i r e m e n t s f o r
methods ( s o m e t i m e s , a l g o r i t h m s ) o f p r e d i c t i o n w i l l b e t h e main aim o f our w o r k .
L e t us c o n s i d e r a number of c o n c e p t s
[ 5 ] . " E m p i r i c a l h y p o t h e s i s " w e s h a l l mean
as a s e t of f o r m a l i z e d n o t i o n s about c h a r a c t e r i s t i c s o f o b j e c t s o r phenomena u n der s t u d y . One can speak about p r o p e r t i e s
of the r e a l world only by f i x i n g i n s t r u ments (P) w h i c h measure these p r o p e r t i e s .
L e t a s e t of symbols O= {P 1 , R 2 ..., P n }
used f o r d e s i g n a t i o n o f i n s t r u m e n t s t n o r e
e x a c t l y , e m p i r i c a l r e l a t i o n s measured
w i t h t h e a i d o f these i n s t r u m e n t s ) b e s i gnature
of hypothesis.
Let empirical i n t e r p r e t a t i o n of r e l a t i o n s measured w i t h t h e a i d o f f i x e d
c o l l e c t i o n of instruments, designated by
symbols of 6 , be i n t e n s l o n a l b a s i s ( I n t )
of hypothesis.
an empirical hypothesis
affirming that
c e r t a i n protocols can be never obtained
if
is t r u e , and if the experiments are
made w i t h the given instruments over any
f i n i t e sets of objects (and not only over
the set
). The hypothesis
may be r e garded as a description of supposed char a c t e r i s t i c s of measuring instruments,
and the protocol Pr as a record of r e s u l t s
of measurements of elements of
carried
out w i t h these instruments* The hypothesis
is supposed to be such t h a t p r o t o col
, corresponding to the experiment
made over the set
9 is admissible under t h i s hypothesis ( i . e . the hypothesis
conforms to the protocol
). Otherwise we must state the hypothesis is r e futed by t h i s experiment and it must be
revised as a wrong i n i t i a l information.
It is precisely the case when our i n i t i a l
assumptions concerning c h a r a c t e r i s t i c s of
measuring instruments are wrong.
A d i s t i n c t act of p r e d i c t i o n , say
i s t h a t s t a r t i n g from
the i n i t i a l hypothesis
and using i n formation involved in the protocol
concerning elements of
t a new hypothesis
is pointed out, such t h a t :
(i)
is in a sense more (or at l e ast not less) informative than
;
(ii)
is admissible f o r the H a .
Let from the f i r s t we know only t h a t
current in c i r c u i t may have any value
from 0 to 100 amperage, and tension of
t h i s c i r c u i t is from 0 to 50 v o l t . Then
hypothesis
would regard any combination of mentioned values of current and t e nsion as possible one. As a r e s u l t of experiment over some part of e l e c t r i c c i r c u i t there have been obtained such combinations: 1. (U=3v, I=6a)t 2. (U=10v, 1=
20a); 3. (U=15v, I=30a); 4. (U=40v, I=
80a). Using these r e s u l t s one can constr u c t the hypothesis H 1 i n t e r d i c t i n g , in
contrast to
.such p a i r , f o r example,
as f o l l o w s : 5. (U=5v, I = 1 a ) | 6. (U=0v,
I=90a), e t c . One can express in f i g u r e
t h i s s i t u a t i o n . In such a case we s h a l l
consider the hypothesis
to be stronger
than the hypothesis E 0 • and w r i t e t h i s
The act of p r e d i c t i o n is considered
as successful (or true) t i l l a new set of
empirical objects, such that p r o t o c o l of
experiment of kind mentioned over t h i s
set is not admissible f o r
and is admissible f o r
, i s found. I t i s obvious,
t r i v i a l act of p r e d i c t i o n , i . e . any act
of type
, is always
successful in t h i s sense, being completely uninteresting.
220
An i n d i v i d u a l act of p r e d i c t i o n has
so f a r been concerned. As f o r method of
p r e d i c t i o n i t i s n a t u r a l t o consider i t
as a f u n c t i o n f of t y p e :
O f course, c e r t a i n r e s t r i c t i o n s t h a t a r i se from our i n t e n t i o n to impart c e r t a i n
d e s i r a b l e f e a t u r e s t o p r e d i c t i o n method
should be placed upon t h i s f u n c t i o n f.
I n [ 4 ] the f o l l o w i n g formulas o f
these requirements were g i v e n .
1. Universality.
the a l g o r i t h m would be a p p l i c a b l e to any
possible pair "protocol-hypothesis",
2. Non-triviality.
Hypothesis r e s u l t i n g from a l g o r i t h m work
would be, at l e a s t , no weaker than the
i n i t i a l one.
3. Consistency.
Let
be one-one computable t r a n s f o r m a t i o n of possible p a i r
i n t o possible pair
such t h a t hypotheses
and
are r e f u t e d or confirmed on the same sets of
o b j e c t s ( i . e . s i m u l t a n e o u s l y ) . I n such a
case we can regard
as e f f e c t i v e oneone t r a n s l a t i o n of hypothesis
and p r o tocol
i n t o another e q u i v a l e n t l a n g u age. I t i s n a t u r a l t o demand i n v a r i a n c e
o f r e s u l t s o f p r e d i c t i o n r e l a t i v e t o such
e f f e c t i v e one-one t r a n s l a t i o n s from one
language
i n t o another e q u i v a l e n t one; i . e .
if J
, and
then
would be r e f u t e d or confirmed simultaneously.
K.F. Samochvalov has proved the t h e orem 4 t h a t the only f u n c t i o n answering
these t h r e e requirements i s the f u n c t i o n
f•
c o n s t r u c t i n g the decoder, i . e . f u n c t i o n t h a t brings i n t o c o r r e l a t i o n p a i r
w i t h hypothesis H , i n t e r d i c t i n g a l l p r o t o c o l s except P r corresponding t o t r a i n i n g sequence. I t i s c l e a r ,
such a f u n c t i o n f* cannot be means of i n ductive generalization or t h a t of discovery of e m p i r i c a l laws. To o b t a i n u s e f u l
r e d i c t i o n method the requirements f o r m u ated above must be changed. Very l i k e l y ,
we have to reduce the t h i r d requirement
o f i n v a r i a n c e o f p r e d i c t i o n w i t h respect
t o f o r m a l l y e q u i v a l e n t ways o f i n i t i a l
data r e p r e s e n t a t i o n . For t h i s reason a t tempts t o c o n s t r u c t u n i v e r s a l , n o n - t r i v i a l , and u s e f u l method o f p r e d i c t i o n p r e suppose acceptance of "Goodman's approach"
[5] : one ought to p r e f e r some languages
t o another ones i f , i n s p i t e o f t h e i r e q u i v a l e n c e , they d i f f e r o n such c r i t e r i a
as " h a b i t u a l n e s s " and "frequency of usage"
of the terms.
The work [4] , very l i k e l y , may be
a j u s t i f i c a t i o n of widespread in science
p r i n c i p l e of " s i m p l i c i t y " [6,7] . According t o t h i s p r i n c i p l e , from two t h e o r i e s
e q u a l l y well-conformed to known f a c t one
ought to use more " s i m p l e " one to p r e d i c t
new events.
221
Along w i t h research of requirements
to e m p i r i c a l p r e d i c t i o n a l g o r i t h m s , some
v a r i a n t s o f a l g o r i t h m s i n which the r e q u irement of " c o n s i s t a n c e " is replaced by
t h a t o f " t h e g r e a t e s t s i m p l i s i t y " axe t e s t e d . One of much algorithms is as f o l i o i n g [8] :
1. Let
be the p r o t o c o l of experiment over the f i n i t e set
of e m p i r i cal objects.
Let
be the i n i t i a l h y p o t h e s i s .
Suppose we are i n t e r e s t e d in p r e d i c t i o n concerning the
new objects on the
grounds of more s t r o n g hypothesis H ; .
Such p r e d i c t i o n i s equivalent t o choice
of one p r o t o c o l P r , , having c a r d i n a l
from a l l p r o t o c o l s having the same c a r d i n a l , which conform t o hypothesis
We u s u a l l y know something about new
e m p i r i c a l o b j e c t s . We want to p r e d i c t the
values of some e m p i r i c a l r e l a t i o n s
L e t subprotocol d e s c r i b i n g the set
these r e l a t i o n s be E :
E =
2. To p r e d i c t subprotocol E we cons t r u c t (generate) a l l p r o t o c o l s
having cardinal
such t h a t :
a) each of them conforms to hypothesis
f o r each of them the set E is i t s
own subset.
3. We shorten obtained l i s t of p r o t o c o l s as f o l l o w s :
a) For every p r o t o c o l
from t h e
list
we f i n d the subprotocol pr
t h a t contains a l l non-isomorphic subprot o c o l s having c a r d i n a l K. (Number K is
equal to the number of c a l l i n g the p o i n t
3 , s o t h a t , f o r example, i n the f i r s t t i me K=1. in the second K=2, and so
b; We choose the subprotocols
having minimum c a r d i n a l , i . e .
c) We e l i m i n a t e from the
(and from f u r t h e r discussion) a l l those
p r o t o c o l s Pr
the subprotocols pr
of
which have c a r d i n a l more than n,ntn
4. If in c a r r y i n g out the step 2 we
e l i m i n a t e d from the l i s t i f only one p r o t o c o l , and i f remaining l i s t c o n s i s t s o f
more than one p r o t o c o l , we pass on to r e c a r r y i n g out o f step J .
5. The remaining set of p r o t o c o l s
together w i t h a l l isomorphic t o them c o r responds to the hypothesis H sought.
C l e a r l y , the step 3 provides the r e quirement o f n o n - t r i v i a l i t y f o r the a l g o r i t h m . At the same time it is the formal i z a t i o n o f the hypothesis o f " s i m p l i c i t y " i n accordance w i t h which from two r i v a l hypotheses the one t h a t is based on
r e g u l a r i t y , observable on the l e s s e r number of o b j e c t s , w i l l p r e d i c t more "successfully".
The examination of algorithms of such
types was c a r r i e d out on the tasks of d i scovering r e g u l a r i t i e s i n e m p i r i c a l t a b l e s of d i f f e r e n t k i n d s . The examples of
such problems f o l l o w .
"The Ohm's l a w " ,
t a i n s p r e d i c a t e symbol
. T h e i r i n t e r p r e t a t i o n is as f o l l o w s .
This hypothesis describes features of t a b l e 3 b r i e f l y enough and p r e c i s e l y ( w i t h
n o t g r e a t redundance o n l y ; .
T h e o r e t i c a l and e x p e r i m e n t a l i n v e s t i gations of empirical prediction algorithms
are s t i l l i n p r o g r e s s .
References.
1 . Von F o e r s t e r H . , I n s e l b e r g A . , Weston,
Memory and I n d u c t i v e i n f e r e n c e , i n :
C y b e r n e t i c Problems i n B i o n i c s , e d . b y
H.L. O e s t r e i c n e r ana D.R. Moore, Gordon a Breach S c i . P u b l . , New Y o r k - L o n d o n - P a r i s , 1968, p p . 3 1 .
2. Wittgenstein L., Tractatus L o g i c o - P h i l o s o p h i c u s - New Y o r k , 1963 p r o p o s i t i o n 5 1361.
222