Carnegie Mellon University
Research Showcase @ CMU
Computer Science Department
School of Computer Science
1973
On computing reciprocals of power series
H. T. Kung
Carnegie Mellon University
Follow this and additional works at: http://repository.cmu.edu/compsci
This Technical Report is brought to you for free and open access by the School of Computer Science at Research Showcase @ CMU. It has been
accepted for inclusion in Computer Science Department by an authorized administrator of Research Showcase @ CMU. For more information, please
contact [email protected].
NOTICE WARNING CONCERNING COPYRIGHT RESTRICTIONS:
The copyright law o f the United States (title 17, U.S. Code) governs the making
o f photocopies or other reproductions o f copyrighted material. Any copying o f this
document without permission of its author may be prohibited by law.
ON COMPUTING RECIPROCALS OF POWER SERIES
H. T. Kung
Department of Computer Science
Carnegie-MelIon University
Pittsburgh, Pa.
September 1973
This research was supported in part by the National Science
Foundation under Grant GJ32111 and the Office of Naval Research under Contract N00014-67-A-0314-0010, NR 044-422.
ABSTRACT
Root-finding iterations are used to compute reciprocals of power series.
We show that Sieveking's algorithm is just Newton iteration applied in the
field of power series.
Let L^ denote the number of non-scalar multiplications
needed to compute the first rrH terms of the reciprocal.
n+1 £ L
n
£ 4n-log n.
z
0
We conjecture that
L
n
= 4n - lower order terms,
We show that
1.
INTRODUCTION
We consider the problem of computing reciprocals of power series.
This
problem is closely related to the problems of polynomial division, evaluation
and interpolation.
(For example, see Borodin (1973).)
Let L denote the
n
number of non-scalar multiplications needed to compute the first n+1 terms of
the reciprocal of a power series.
Recently Sieveking (1972) showed that
L ^ 5n-2,
n
In this paper, we show:
(i)
Root-finding iterations can be used in the field of power series,
1
Sieveking s algorithm is just Newton iteration applied to the
function f(x) = x ^-a, a ^ 0, in the field of power series.
(ii)
1
1
By modifying Sieveking s algorithm and analysis, Sieveking s
bound is improved to
L
(iii)
n
^ 4n-log2n.
We propose a new algorithm for computing reciprocals of power
f
series, which is competitive with Sieveking s algorithm, and
which is based on a third order iteration.
The bound in (ii)
can also be obtained by this new algorithm.
(iv)
L
^ n+1
for all n £ 0.
n
In Section 2 we define some basic notation and also prove results (i)
and (ii). Results (iii) and (iv) are proven in Sections 3 and 4, respectively.
-2-
In Section 5 we give a general family of algorithms for computing reciprocals
of power series.
Any algorithm in the family can compute the first n+1 terms
of the reciprocal of a power series in 0(n) non-scalar multiplications (and
can also compute them in 0(n log n) arithmetic operations if Fast Fourier
Transform is used for polynomial multiplication). We conjecture that Newton
iteration and the third order iteration are optimal among all algorithms in
the family.
-3-
2.
NEWTON ITERATION
We will use notation of Sieveking (1972) and Strassen (1973).
Let k be
an infinite field, a^, b^, i - 0,1,...,°°, indeterminates over k, A an extension field of k, and t an indeterminate over A.
Suppose that E and F are
finite subsets of A and that we do computations in the field A.
Let L(E mod F)
denote the number of multiplications or divisions by units which are necessary
to compute E starting from k U F not counting multiplications by scalars in k.
We shall prove the following theorem by using Newton iteration.
Theorem 2.1.
Suppose A = k(a^,a^,...), a^ is a unit in A and
(2.1)
n
.n
E a.t E b . t = 1
0
0
1
1
-
1
(t
n + 1
).
1
Then
L(bg,...,b mod
n
a^,...
9
a^)
£
4n-log2n for any n ^ 1.
We first use a technique of Strassen (1973) to prove the following
Lemma 2.1.
Suppose A = k(a ,b ,a ,b ,...) and
Q
Q
1
Eat
0
1
1
Sb.t
0
1
-
s
0
ct
+
2
I
1
1
c.t .
1
Then
L(e £,...,c
j
tt+TO
mod a ,... .a^.bg,... ,b ,c ,... ,c^_ ) £ n+m - jj+1
Q
for any n,m s 0 and Z such that n+m J I i 0.
m
Q
1
-1
(S is assumed to be zero.)
0
-4-
Proof of Lemma 2.1.
Let X., 1 £ j £ n+m - jif 1 , be any n-hn - ji+1 distinct nonzero elements
in k.
Observe that
(2.2)
n-hn-ji
2
c
0
.n
m
= X (S a x
S b X
i-0
i=0
X
J
J
i =
J
1-1
S c t )
0
J
c
for j = 1,...,n+m - &KI, and det(X^) £ 0.
Hence j^.^» 0 ^ i ^ n+m-j£, can
be obtained by solving the linear system (2.2).
L ( c
C
A'--" n4m
m
o
d
a
a
b
Therefore,
b
C
C
0"--' n' 0"*" m' 0 JH
)
n
.m
^ L( S a.X. 2 b.X., j = 1,...,nrhn - 4+1,mod a ,...,a ,b ,...,b )
i=0
i«0
n
1
J
1
3
n
n
m
= n-hn - jfrfl.
•
Proof of Theorem 2 J .
Denote S a.t by a and J b . t by b. Suppose that (2.1) holds for all n.
0
0
Then b is the reciprocal of a with respect to the field A(t).
1
1
Define the function f: A(t) - {0}
just the root of f.
A(t) by f(x) = x ^ - a .
Thus b
Applying Newton iteration to f, we obtain the iterate-;,
function
(2.3)
,
1
2
cp(x) = x - f (x)" f(x) = 2x-ax .
(In this paper,derivatives of f are defined by purely algebraic methods without employing any limit concept.
^65).)
(2.4)
For example, see van der Waerden (1953,
It follows from (2.3) that
2
co(x)-b = a(x-b) .
-5-
For notational simplicity, let L denote L(b_,...,b mod a......a ) .
*
n
0'
' n
O
n
To prove the theorem it suffices to show that
r
J
(2.5)
L„
£ L + 4n + 2
Zn+1
n
for n £ 0,
(2.6)
L„ £ L + 4n - 1
zn
n
fornil.
Suppose that bg,...,b
n
to be 2 b.t . Then
0
have already been computed.
In (2.4) let x be taken
1
tp(S b.t ) - b = 0
0
1
(t
2 l t f 2
),
1
or
n
.
2 2 b t 0
2n+1
. n
.
2
a.t (2b.t )
0
0
2n+l
2
o
2
1
(2.7)
1
1
1
1
=
1
.
b.t
1
.
(t
.
/ n + Z
).
1
Note that
2 at
0
1
1
s b.t H i
0
1
We define e
n+1
,..., e ^
(t
n + 1
),
1
by
2n+1
. n
.
3n+1
2 a t 2 b.t = 1 +
2 e.t\
0
0
n+1
1
(2.8)
1
1
1
Then by (2.7) and (2.8),
Therefore
( 2
*
9 )
L
L
2n+1 ^ n
+
L(e
+
e
n + 1 > •••' 2n+l
L ( b
b
m
n+l'"" 2n+1
o
d
m
o
V ' '
d
e
a
• ' 2n+l'V''' ' V
e
b
b
n+l--- 2n+T 0'--" n
)
-6-
By Lemma 2.1,
L
0
I
We have shown (2.5).
n
(2.10)
From (2.7) we also have
2n
2 S b.t
0
L
1
^ L + (2n+l) + (2n+l) = L + 4rrt-2,
n
n
n+1
. n
. ^
L
- Z a.t (S b . t V
o
L
2n
1
o
.
= E b.t
o
1
?n4-i
(t:
') .
1
(2.6) follows in the same as (2,5) by starting with (2.10) instead of (2.7).
One can easily check that the algorithm proposed by Sieveking (1972) is
just the Newton iteration stated above.
and careful estimation of L
However, because of (2.8), Lemma 2.1,
from (2.5) and (2.6) we are able to obtain
n
L
n
< 4n-log n
z
for n ^ 1
£ 5n-2
for n ^ 1
0
rather than
L
n
which is obtained by Sieveking (1972).
One should also note that the idea of
using Newton iteration to compute reciprocals has been known for a long
For example, Newton iteration is used to compute matrix inverses by
Schulz (1933), to compute reciprocals of real numbers by Rabinowitz ( 1 %
and to compute integer reciprocals by S. A. Cook (see Knuth (1969, ^4.33) ,
In this section, we have shown that Newton iteration can be used successfully
in the field of power series, and hence can compute reciprocals of power
series.
In fact, any root-finding iteration (Traub (1964)) can be used for
the problem of computing reciprocals (see Section 5 ) .
second order iteration.
Newton iteration is a
In the next section we propose a new algorithm for
computing reciprocals of power series, which is based on a third order itera1
tion, and which, is competitive with Sieveking s algorithm.
-7-
3.
ANOTHER ALGORITHM FOR RECIPROCALS
We use notation defined in the previous section.
Applying the third
order iteration (Traub (1964)),
-1
1
£(x) = x - f'(x) f(x) - ^ [ f ( x ) ' ]
3
2
f"(x)f(x) ,
to the function f(x) = x ^-a, we get
2
(3.1) ffi(x) = x(3-3ax + (ax) ).
It is easy to show that
2
(3.2)
3
i(x) - b = a ( x - b ) .
We shall now use 9 to prove Theorem 2.1 for n * 3.
L(b ,...,b mod a ,...,a ) as before.
Q
n
0
n
Note that L
Let L denote
n
1
£ 3 and L
2
<: 6.
It is
not difficult to check that it suffices to prove that for n £ 1,
(3.3)
L
3 n + 2
(3.4)
L
3 n + 1
(3.5)
L
3 n
£ L
n
SL +
^ L
n
n
+
8 n
+
5 ,
8 n + 1,
+ 8n - 3.
Suppose that b ,...,b have already been computed.
0
n
In (3.2) let x be taken
1
to be Ib.t then
0
1
n
*<E b.t ) - b = 0
0
1
(t
3 n + 3
),
1
or
n
(3.6)
«
-3
3
|
.
i t
i J
b t t
u
Note that
l
I J
+
o
n
, ™
t
n
(
*•
^
3
)
-8-
Sa.t
0
1
Sb.t
0
1
We define
e
e
n+
b
i>•••» 4 +2
1
— 1
(t
n + 1
).
1
y
n
3n+2
.n
4n+2
L e t .
n+1
±
S a.t s b . t - 1 +
1
1
0
1
1
0
1
Moreover, define d ,...,d by
Q
n
n
n
i 2
i
/ n+1,
Then
(3.7)
3n+2
S at
3-3
n
S b t
0
1
o
2 n + 1
S
3t
J
n + 1
d,f. .
t
±
i
n+1
t^ + 1 + 2 t
2
2 n + 1
J
n
e
n + 1 + i
2
+
3n+2
n
.
+ ( S a.t £ b . t )
0
u
n+l « £
f < f c
i
8
b
3n+3
„i j. , t + t
2n+2
v d t
J d t
t
1
(t
(t
3 n + 3
)
} >
1
0
Define S '"*' 3n+1
( t
e n + 1 + i
y
0
n
. 2n+1
,
3n+1
Sbt
I f t = Z g.t .
i
(3.8)
1
o
1
1
o
o
Then by (3.6), (3.7) and (3.8),
2n+1
,
2n+1
Sg.t - J b ^ . t
2n+2
(t
).
L
s
1
0
0
Therefore
L
3n+2
=
L
n
+
L(e
6
m
n+1'''*' 3n+2 °
D
+ L(d ,...,d mod e
Q
n
m
+ L(g .....g n+1 °
0
2
V
''' '
n + 1
d
e
a
3n+2'
,..•,e
n + V "
2 n + ]
e
3
b
0V
)
n
+ 2 , < i o ' - - "
d
-
,
b
o
^
-9-
By Lemma 2.1,
L
L
3n+2 * n
+
( 3 n + 2 )
+
( 2 n + 1 )
+
(
3 n + 2
>
= L + 8n + 5.
n
We have shown (3.3).
From (3.6) we also have
(3.4) and (3.5) follow in the same way as (3.3) by starting with (3.9) and
(3.10), respectively instead of (3.6).
-10-
4.
A LOWER BOUND
Under the hypotheses of Theorem 2.1, we show that
(4.1)
L(bg,...,t> mod a ^ j . . . ^ ^ ) ^ n+1.
n
Suppose that a^ = 0, i = 2,...,n.
i = 0,...,n.
Then it is clear that
* (a^a^^a^,
(4.1) follows from the following
Lemma 4.1.
Suppose A = k(a^,a^), a^ is unit in A and
t
i
-1 i -1
0 l^ 0
,
=
( a
x
a
a
9
1
A
=
n
°>--*> '
Then
L(bg,...,b
mod a^,a^) ^ n+1
Proof.
Consider an arbitrary algorithm which requires m non-scalar multiplications or divisions.
or divisions.
Let R^,...,R
denote the results of these multiplications
Then there exist p^
€ k, q^ 6 {k^aQ+k^a^ |k^k^ g k} U k,
i = 0,...,n, j = l,...,m such that
m
2 p. .R. + q. = b., i = 0,...,n.
j=l i»J J
1
Suppose that m < n+1.
Then there exist r
1
6 k, i « 0,,..,n, such that r
i
^ 0
for some i and
n
S r (b -q ) = 0,
l i i
Q
or
n
.
.
n
i n-i
I Vi o
a
(
,i
\ n+i
r
q
- * i i
) a
o
•
This clearly implies that r. = 0 for all i = 0....,n which is a contradiction.
-ii-
5.
A FAMILY OF ALGORITHMS FOR RECIPROCALS AND A CONJECTURE
00
i
i
Suppose a « E a.t and b • J b t and that ( 2 J ) is satisfied for all n.
0
0
Let A « k(a^,a^,...). For any nonnegative integers d, A g * ^ . . ( n o t all
oo
1
1
(k)
zero) we define an algorithm which generates the sequence of iterates {x
in A(t) by
^
t
x
( k + 1 )
(5.1)
1
= x ^ d - a x ^ ^ ) "
x
+
( k
, )
-
+ ...
k
i" (l.ax )J
j=0
4,-1
(k)
. . ^ ! ^ ^ ) ^
2
1
(l^< - >)\..(,^^)
I
*
)
o^Ck-Djj
j=0
k
d
; < " >
X
Vj1
(l-ax^V.
j-o
Then it can be shown that
(5.2)
x<
k + ,
k)
1
k
> . b - b(,-.K< ) °(,.. < -'>,''... ,.
1[
For all k define c
R
(
i
ai!
(^) d,
)
to be the greatest integer such that
Then from (5.2) it follows that
x
( k + 1 )
H b
(
t
V k
+
- "
+
V k - d
K
Hence
(5.3)
c
k + 1
^ j
o
V
k
_
t
.
Using (5.3) we can estimate the number of iteration step s necessary to
compute b , . . . , b from a ,...,a
Q
x
( k + 1
n
Q
^ by (5.1) we should use
n
for any given n.
^a.t
0
1
1
Note that in computing
1
for a(= f a t ) .
0
1
}
-12-
Example 5.1.
(i)
if d = 0 and &
= 2 we have
Q
x
(k
+ 1 )
=
x
(k)
[ 1
+
( 1
_
a x
(k)
) ]
2 x
=
( )
k
.
(w <k)
a x
x
#
This is the Newton iteration (see (2.5)).
(ii)
If d = 0 and &
x
= 3 we have
Q
(k
+ 1
)
= x
(k)
- x
( k )
[ 1 + ( 1
_
a x
(k)
[3 - 3ax
) + ( 1
( k )
.
(ax
+
a x
( k )
(k) 2
)
]
2
) ].
This is the third order iteration cp in (3.1).
(iii)
If d = 1 and JL = ^
= 1 we have
Q
x
(k 1) ^ ( k )
+
x
( 1
.
a x
(k-1)
)
+ x
(k-D
- - a x ^ x ^ + x ^
+
x
0
0
.
-1
One can check that this is the secant iteration applied to f(x) = x
-
In fact, (5.1) represents the algorithm which is obtained by a general
Hermite interpolatory iteration (Traub (1964)) applied to the function
f(x) = x ^ - a
0
A special case of (5.1) (i.e., d = 0) was pointed out before
by Rabinowitz (1961) for computing reciprocals of numbers.
By the same tech-
niques used in Sections 2 and 3 one could show that L(b^,...,b^ mod ag,...,a^)
is bounded by a linear function of n by using any algorithm defined by (5.1).
However, we believe that Newton iteration and the third order iteration are
optimal among all algorithms defined by (5.1).
following
More precisely, we state the
-13-
Conjecture.
L(b ,...,b
A
U
n
mod a.,...,a ) = 4n - lower order terms,
U
n
for large n.
If we use the Fast Fourier Transform for polynomial multiplication, then
it can be easily shown by techniques similar to those used in this paper that
any algorithm defined by (5.1) is able to compute the first nH-1 terms of the
reciprocal of a power series in 0(n log n) arithmetic operations,
ACKNOWLEDGMENT
I would like to thank Professor J. F. Traub for his comments on this
paper.
-14-
References
Boroxlin, A. (1973), "On the Number of Arithmetics Required to Computer Certain
Functions - Circa May 1973.
Appears in Complexity of Sequential and Parallel
Numerical Algorithms, edited by J. F. Traub, Academic Press, New York.
11
Knuth, D. E. (1969), The Art of Computer Programming. Vol. II, Seminumerical
Algorithms, Addison-Wesley, Reading, Mass.
Rabinowitz, P. (1961), "Multiple-Precision Division," Comm. ACM 4, p. 98.
Schulz, G. (1933), "Iterative Berechnung der reziproken Matrix," Z. Angew.
Math, and Mech., 13, pp. 57-59.
Sieveking, M. (1972), "An Algorithm for Division of Power Series," Computing
10, pp. 153-156.
Strassen, V. (1973), "Vermeidung von Divisionen."
To appear in J.Reine Angew.Math.
Traub, J. F. (1964), Iterative Methods for the Solution of Equations, PrenticeHall .
van der Waerden, B. L. (1953), Modern Algebra, Vol. 1, Frederick Ungar Publishing Co.