Big Omega Versus the Wild Functions*
Paul MI L Vitanyi
Lambert Meertens
Cadre* Mathematics & ConO W Science (C WI.),
Miniskirt, 413, 1098 Si Amsterdam; The Ndhertands
ABSTRACT
The question of the desirable properties and proper definitions of the
Order-of-Magnitude symbols, in particular 0 and e, is addressed once
more. The definitions proposed are chosen for complementary mathematical properties, rather than for similarity of form.
The old order changeth, yielding place to new,
And God Urals himself in many ways,
Lest owegood custom should corrupt thc world-
L brmoDuCTioN
Tennyson,The 1 4
,
Ih i f
t h e
K i n g .
The issue of the proper definitions for the Order-of-Magnitude symbols would appear to
have been settled once and for all by Knuth in 111. At the end of an exhaustive discussion
the subject is, the author feels, about "beaten to death". The purpose of this communication is to point out that there is life in the old dog yeti' The dehlerations below Were
prompted by surprise that, while proving a lower bound where the precise definitions mattered, matters %
prove
something about the order of magnitude of a function we do not know, like the
P
worst-case
time of some algorithm, we can not assume that the function concerned
er e n orunning
t
does not oscillate o r
a s
-quotient of that function and the measuring function exists at all. I n such cases the order of
d e a r
magnitude
a s c aofna function may vary arbitrarily, depending on the precise definitions chosen.
-This encourages improper use, in particular of the symbol fa. A modification of the
b e
c u t
t h e
of this note appeared in the asildist e
a• A fint %train
s
ct Ston
a
s felt
eepeople
iteshould also be published on a forum more accusal* on the otherside of the Atbtgic, like
h
B
e
o SWAMnNews. m o a n
kA s s re c i a t i m
et Charlet Kingsley, Ike YewsAgo. 1857. "I fed his bead, diem is life in die old dog yet".
fi
m
or n o
m
i
tM oa nr i d i s /
g
a
qh
u
i
f
e
r
c&
j
a
m
t
f u n c t
a
s
proposal In (I), for the definitions of the Order-of-Magnitude symbols, appears to give a
more useful and manageable system.
2. HisrottY
The history of the Order-of-Magnitude symbols 0, 0, 8, o and to, is explored in [1]. Some
additional sources are as gallows. I n the dassic textbook on Analysis by Whittaker 8t Watson (2] the origin of the founding father 0 is given as: "This notation is due to Bachmann,
Zahlentheory (1894), p.401, and Landau, Prinuethlen, 1, (1909), p.61". The Entyldoptitie
derMa/hen:ass:schenWissenscheim contains, not surprisingly, occurrences of the Order-of.
Magnitude symbol 0 in a section on Analytical Number Theory by Bachmann himself
[3, p.6641, and also the equivalent, more ancient, relational symbols , a n d >-. These
symbols correspond, more or less, to the symbols 0, e , and eo, respectively; a [3, p. 751.
They are attributed to Du Bois-Reymond [4], and are said to hold between two functions,
only i f the lim it o f the quotient o f the functions exists. S o g ( n ) 2 i f
Ern,/ (n) / g(a) c 2 , 0 < c < o32, for s e o 2 . T h e author of this section o f [3],
Pringsheim, states that be uses the notation "—" also when the lirn sup and lim inf of the
quotient are distinct but still both finite and non-vanishing. So f (n) — e n) if
0 < c < lim
inf LU
a-•co g(n) n - e c o g ( n )
I <
l i z n
He adds the symbol
s•-•-•ufor p
L a
f (a) c e n), if bin" a It
i
•-.00 g(n)
<
Hardy used i n the
C (now standard) sense of Pringsheim's , or Du Bois-Reymond's
with c = I, and the<sign.tle. for Du B o i s
complete
the descendancy of the Order-of-Magnitude symbols as told in f I]: 0 is introReym
duced
in o[9,
nd
p.2251,
' s •8 •on-suggestions
•
of R E Tarjan and M.S. Paterson in [ I , Ix 20],
i
where
nthe to notation also appears first.
P rpotential
i n ginequality
s h eofithe
mlim' sup
s and the lin) inf of the quotient of two functions, even
The
i both
n are
t emonotonic,
r p ref. (5),
e tis both
a ta source
i
if
of discord and of the present note. I n queso
n
.
tions of classical Analysis this inequality was a troublesome matter. Therefore Hardy
T
o a well-behaved class of "tame" functions, the L -functions, which made
[6,7,8]
constructed
it possible to formulate the problems concerned precisely enough and to reason rigorously.
A similar class of functions was studied earlier by Liouville. Hardy [7] considers an L function
"essentially as the embodyment of an 'order of infinity', as ccpressing a certain rate of
increase or decrease or of approach to a limit; and fix this reason I consider only functions of a real variable which are real and one-valued and (as I shall show) ultimately
monotonic,excluding altogether oscillating functions suchas minx. These ideas do not
appear in Lionville; work at all. He was interestedsolely in problems of functional form:
sinx was for him ccacdy on the same footingas logx or ex ."
These L -functions will figure prominently in what follows. The departure of the definitions
presented here from the proposal in [1] basically rests on the difference in defining IL I n
(l) the course is taken to give all Order-of-Magnitude symbols the same definitional form
Yet the choice is also a matter of usefulness in expressing the things we want to express; of
how well it fits our mathematical intuition, in particular, whether it has elegant properties
that can be relied on and used to prove statements without meticulous reference to the original definition. W e shall strive for complementary mathematical properties. The use of
the symbols 0, 0 and 12 in classical mathematics, as in 16,7,11,9,101, is the same as ours (the
H-variant in Section 5). The corresponding definitions for e and co follow easily. T h e
Knuth proposals in t i t that is, (oz), (
alternative
notations for the established meaning of the well-known symbols -< , ,
,
0
and
›,
respectively.
x), ( e x ) ,
( Z i r )
a
n
d
3.USEFULNESS
(
w
g
)
To
i dot the Ps
n for my formal thinkers we detect some abuse of notation.
Notations/
convenient
abuse of notation is universally practised when
S
e
ccenVegi
t alL Some
i
o
dealing with the Order-of-Magnitude symbols. A n expression E depending on a varin able 11, such as n
2 to denote
5 used
, the finslion that m a n to E, A a 1E1 (s0 x
function).
The
This allows us to write 0(11
2 wm ha i yc h intended
d e n use
o will
t ebe cleart fromhthe context.
e
a ,rather
r 0(An
e
than
u s u a l
j
u
s
Suppose
l ywe can prove that the running time T(a) of some algorithm exceeds R
2t i nand
often
i tne
that
loyit is never less than a logs. W e would like to be able to say then that
d fi nealso
T(a) tC il(a
e s
that
2 T(n)
EEO
t
h kg a), which is not very informative. W e do not want to express that the
algorithm
). e
will always exceed a (rather weak) time limit; we want to express that it will
a limit for arbitrarily large instances of the data T o refresh the reader's
exceed
A c vcaotime
l
u
memory,
r d i n we repeat here the definition for 2 as proposed by Knuth in (11. Those for the
other
g esymbols can be found in Section 5.
o
t
(Qa)
fl a ( f (a ) ) denotes the set of all g(n) such that there exist positive constants 8 and a
o
o
E with g (a ) 8 / ( a ) for all a >ifito.
t
f
In
h the above example, the problem could be ascribed to our lack of knowledge about the
o 7•00. But the problem may also arise with functions that are fully known. Confunction
e
sider r
d ea
f mg
g(n) enP2(enP2ilog2log3a
i ti i
We
) < n for all n, and g(a ) = n for ,, of the form 2
o have
nv
non-decreasing.
(I t is easy to make a variant that is continuous and monotonic increas1s
e
.i A n
ing.)
Here
l s owe
, would
e like
n to) assert that
i goo
s ellt(a), but according to ( Q
m
"tame"
o
function
n
o
f(s)
t
such
o
n
that
k
g(a)
E
Il(f(s))
is f(s) = a*. Nonetheless, the least "tame"
xn v
function
)1, ta h f(n)
e such
l athat
r g(a)e
g e s )t ) is f(n) = a. (The concept of a "tame" function plays
a1 crucial
part here. T h e lagaritionice-exponential functions or L.-functions, introduced by
l
Hardy
u [6,7,81 to calibrate the orders of magnitude, constitutes an appropriate family of
1
e functions, cf Sections 4 and 5.) Whereas in giving an upper bound we generally
"tame"
t
want
h cto express that a function is in some sense confined by that upper bound, in stating a
t
e
,
o i
n s
l
lower bound we want to expras non-confinement. The problem would be remedied if in
definition ( e
x ) as all functions concerned are tame, this will make no difference.
long
w e
r e p l
4.
a b
c r reu m a N
"
f
For
o the
r function g(n) introduced above, we have, according to definition ( la
x
a
l
g(n) e wres) n 00),
)l ,
but,
n for all c >
(
e n) e E101
s
1
Such
consequence; from the definitions in [11 seem contrary to intuition. O n e of the
o
"
motivations
for [1] was to counter the improper use of the symbol 0 where 0 would (now)
m
1
be
of this seminal paper in 1976, the use of the Ordere appropriate_ Since the appearance
1 L i 0
of-Magnitude
symbols, both proper and improper, has become far more customary than
p
o
( 1
before.
For
that
very reason, juggling with Order-of-Magnitude symbols along the lines of
s
i
1 "By
way
of contradiction, suppose 1 (n) is not in IWO. 'Therefore, 1 (n) is in 000, and so ..."
t
i
t
)
has
mom attractive. This
is
so,
because the meanings of the Order-of-Magnitude
v become
e
.
symbols
On the tame functions are such that 0 corresponds with < , CAcorresponds with > ,
!
0
n corresponds with = , o corresponds with < and to corresponds with > . Since, in praco almost all functions am tame enough, one tends to forget that this correspondence does
tice,
" extend to all functions. However, in improper reasoning such as that above, the funcnot
tion
b f (n) under consideration may be unknown, and not known to be tame. Rather than
to
y rebuke authors who indulge in such practice (while referring to [1) for definitions), we
would
like to see a definition of the Order-of-Magnitude symbols that legitimizes this kind
"
of
f reasoning. Le t the ftmction ( n ) be fixed, so that we can simply write 0 for 0(1(11)),
o similarly for the other Order-of-Magnitude symbols. T h e meanings of • and 0 are
and
well
r established in mathematical practice and will not be disputed here. However, note
i
that
under the standard definition both 0 and 0 may contain negative functions, such as
n
—it. For the purpose of the discussion, it is convenient if we can restrict our attention to
the
f set of "non-negative functions", where a function is non-negative if it assumes no negative
m values for sufficiently large values of Ir. Denote this set by U. I f f
(i n ) ithen
function,
s each
a of the
n osets
n -0 ,n 1eand
g aea tisicontained
v e
in U under any reasonable
definition,
t
including that in (11. S o let us write (only here) • while meaning • tt U, and
e
similarly
for a The properties we want to have now are:
(l O
y
N
(m
w
)and
it
a
—
)n
o
=
y
,We
can use these desirable properties as definitions. I f one uses Knuth's definitions, the
—
2
first
0" two of these three properties are not assured in general (but do hold if restricted to
,.
A
s
e= ona
tame functions). The meanin,3 of 8 according to definition (EI
H
posed by way of remedy at the end of the previous section. I t is a consequence of the new
definitions
) i s t that
h e
s a m e
a
s
t
h
e
o
n
0 = e Uee n n d 12 = 9 uw.
p
r
o
The Order-of-Magnitude symbols 0, 0 , 0, ft and to) now have by and large the same general properties as the usual < , < , = , > and > . For example, just as we conclude x < y
from x <y and x Oy, we may conclude that g (n) G o(f (n)) from g(a) G 0 ( f OD and
g(n) % W(J i)). However, we may (analogously to y > x following from x <y ), conclude
f (n) E d4(g(x)) from g(n) E 0 (f (n)), but not vice versa. Similarly, if g (n) e e(f(n))2, we
may not, in general, conclude that./(n) e 0(g (R)). Thus, we have lost a pleasant property,
since these very conclusions were valid under the definitions in [I]. However, we feel that
the gain is worth the low in the practice of reasoning with these symbols, such a switch of
roles between the measured and the measuring function is rare. T h e stronger relation
g(n) e Ott (n)), in the sense of Knuth, may still be expressed, viz. as 0(g(4)) = W O D.
5. FORMALANALYSISOFPROPOSALS
The definitions proposed in (1) look as follows.
(
0
(OK) O U OD ' (g I 38,43.>01f. >.( it (n)1 4 al OA );
x
(e
)x
at 3
a>
)(ex)
i
fil(f (n)) = ( g0i 3
E
6
s
(caii)
to(f (n)) = ( g3I V g
x
,> 0
*.>
q
f
4
0 V here
i
The
pmposed
are:
3.
O
f definitions
r
3
.
.
>0
D
O
>
o(f
(n))
= (gI>V a > 0
V
,
,
,
I
g
=
s
0
3
... n ') ( geI 3
D
(OVOID
(
)>
>>o
l
o
=
g
ili(l(n))
= (g>eI 3
.3
1
g
.0
1
,
,
8
1
If
6
(V 3 8
(0
n
.(, , > oa V )ga >) >n• •
3
)
s
(i11(f
(n))( )n
= )tIg
]1 1n1
1oaI V3, &,
0
0
a
f.
<
>
0
I0
)do
4
U
OD
'
tz,f I v5>oviis>03.›.1g(s) > 8
3
.
„
1
1
.
.
(8
S
f
s( >
v
3
.)nO1
A
3 . proposals using the set of calibrating L -functions defined in 161.
We
.i shallf.. compare the0ntwo
)
.
;
>
) • are the smallest class of real one-valued functions of a real variDefinition.
The L )-functions
0
0
>
0
(g
1
>
(311n
)gthe
•.(1constant
able
n, containing
. . ). 1
t
functions and n, and closed under the arithmetic operations,
"
1
"
,
[
0
1
)
n)
)
)
) requirement of an L -function being real one-valued is satisfied if it is so
0
exp
, and();<log.n The
>;
>
for
all
values
;
of
a
greater
than some no.
i
3
>
a
a
ff
l
..
((
na
)
f
(
1
)
]
.g
(
)
n
.(
;
)
The fundamental theorem on L.-functions in [6,7,8] then is as follows.
Theorem. Arp lefiriwtion is uttimote0 continuous, of constantsign, and monotonk and tends, as
• ef the rekdions E o(g),, C O ( g ) Or J
a
.As
o ( gremarks,
)
h o
d sg are I. -functions then f /g is an L-fimction. Thus, the
r cHardy
if fl and
&
u ofxthe =
c , t part
second
theorem is a mere corollary of the first part; for it follows that f /g must
d
i
e
m
t
tend
to infinity, or to .zero or some other limit. I n the family of L-functions, therefore, the
o
K-or-H
choices of definitions for the Order-of-Magnitude symbols do not matter; the
theorem
is insensitive to the variations of definitions above. The theorem ensures that the
0
0 -functions are suitable for the purpose of calibrating the order of increase of functions,
L
,
since
they are totally ordered by the Order-of-Magnitude symbols. Tha t is, 0, 0, 0, Et and
o have precisely the same roles on the set L-functions as < , < , = , > and > have on the
6)
rrational%
t
o
g
n
a
z
l
g f )
ö
e
f
r
g e0(f)
•
g
o
n
g e e• ( f )
o
f
r
g
60
•
t
8
o
g Ecom(f
)
(1
s
g E k r ( f ))
o
•
m
eo lc( l)
e
o
t
Figure. The top line depicts die orderedset of L-funetions. The subsets of the set 01
h
functions induced by the different Order-of-Magnitude symbols, with respect to the wild
function g(*) = e x p A c t p
e
2have V i s a r % L i g
r
Llog
d
In
the
2 figure we sketch the meaning of the several proposals, couched in terms of the tame
e
log J
fZ.-functions
i.scripttthe symbols with H or K , to distinguish the different proposals referred to, whenever
,thef intended
on rn ,
meaning is not clear from the context. Thus, f E lax(g) it g , and so
n
cFor
o a
r tame
r e function
s p o g
n the wild middle gap (where f g and f g ) shrinks to zero
a
forth.
i
dv Ko
andi the
e andtH-definitions
coincide. The wilder a function, as compared to a tame class
tg
t
h
e
like the la-functions, the more the two defining methods will differ. The reader should conen
l
a
b
e
l
w
a picture for the function a n ) = expOr inn), or, more difficult, for a function
lstructlisuch
e
d
l
d
ig
l
i
n
fformer.
ue While
n
2
g and g cover a segment of the ordered set of L -functions, by being of
m
cirregular
t s i increase,
is
e
g g2 falls
m in a gap in the ordered set of L -functions; that is, although the
o
n
e
n
t
s
tu
g
c
.
(F
h
N
.
th
B
.
u
increase of the function does riot oscillate from that of one L -function to that of another,
there is no L.-function capable of measuring it [61. The question present; itself, whether the
family of L -functions is rich enough a class to calibrate the functions we meet and wish to
calibrate. According to Hardy [6, p. 32]
"._ it is possible, in a variety of ways, to construct functions whose inesesse cannot be
treasured by any L-funetion. I t is none the lem. true that no one has yet succeeded in
defining a mode o f incresse which is genuinely independent o f all logarithmicoexponential modes. ... No function has yet presented itself in analysis the laws of whose
increase, in so Ear as they can be stated at all, cannot be stated, so to say, in
logaridunico-exponential tarns. I t would Sc natural to expect that the arithmetical
functions which occur in the theory of numbers might give rise to genuinely new meow
of increase; but, so far as analysis has gone, the evidence is the other way."
Thus, really wild functions appear to have been a rare species. Seldom seen in the wild, they
had to be cultured under laboratory conditions. T h is state of affairs may be unchanged in
the realm of mathematical analysis. However, in computer science the area of algorithms
and computational complexity has enriched the taxonomy of "orders of infinity" with a
non
- tame enough, but that its rate of increase is far slower than is expressible by unbounded
is
L
L -functions.
REFERENCES
f u
Knuth, D.E., Big Omicron and Big Omega and Big Theta, &CACI' News, Volume 8, number 2,
n1c
1976, 18-24.
t i
Whittaker, P.. T., and G. N. Watson, A COMM in Modem Analysk University Press, Cambridge,
o2 n
England, 1902. (Fourth revised edition, 1927)
s
3 Ert9ickbridiedi!? MadronatisAut Winassuisaillut„ vol. J. Meyer, W.F., Editor, KG . Teubner, Leipzig,
l 1898-1904.
i 4 Du Bois-Raymond, P., Sur la grandeur relative des infmis des foncdons, Ann d i itlat. Para ed
k Applic. Ser. 114 (1871), p.339.
e5 Stolz, 8., Mark Ann. 14, 1879, p.232_
l 6 Hardy, G. H., Orekrs if Non* (The Afinitircalailt of Paid Du BoiF-Reyntund). Cambridge Tnters in
o Math. and Math. Physics, No. 12. University Press, Cambridge, England, 1910. (Second edition,
g 1924).
G .1
s7 Hazdy,
- ad Soeie0 (2) 10 (1912), 54-90.
,
81.,
Hardy, G. H., Oscillating Dirieblet's integrals (An essay in the Infmitiirea/eill' of Paul Du Boist Reymortd), QuarterOjegeraat •eMetakematies 44 (1912) 1 -40.
P r o p e r
h9t Hardy,
i e s G. H., and J. E. Litdewood, Some problems of Diophandoe approximation, Ada Mammaeo rica 37
f (1914), 155 -238,
"10l Titchmarsh,
o g a d E.C., The Theory of ihe &mum Zeia-Pion. Oxford, Clarendon Press, 1951.
ft h m i c
uo
nce x p o
tn e n t i
ia l
u
of
c
nn
t
i
a
n
lo
s
,
i
P
r