Chapter 2
Algorithms: Tiny Yet Powerful
Lenore Blum is a computer scientist who is now at Carnegie-Mellon University.
She is famous for her work with Steve Smale and Michael Shub on the “The P=NP
Question Over The Reals” [18]. This important work shows that there is a version
of P=NP for computations over the real numbers. Also she is one of the national
leaders in helping get more women more involved in Computer Science. Finally,
she is part of the most well known family in all of computer science: her husband,
Manny Blum, is a Turing Award winner, and her son, Avrim Blum, is one of best
researchers in the world in machine learning.
The work of hers that I am most interested in discussing here is work we did
together. We noticed that “algorithms are tiny.” Let me explain. Recall that the Gettysburg address is short, only 269 words, and was delivered in just over two minutes.
Short things can be very powerful. Here is a note we put together to explain the consequences of the fact that algorithms are tiny:
Algorithms: Tiny yet powerful and we can’t live without them.
The promise of information technology is intimately intertwined with advances in algorithms (methods for solving problems). Algorithms are different from software systems in
a fundamental way. They are tiny. An algorithm that solves an important problem can be
small enough to fit on a single piece of paper using normal sized handwriting. Microsoft’s
operating system, Vista, is a collection of many programs that in total has close to 100 million lines of code. That’s roughly the equivalent of 10,000 hardback books. Vista’s code
is a trade secret, but even if it wasn’t, its not clear a single person could ever understand
it all. Algorithms are at the opposite end of the spectrum. A postcard is often enough. As
such, algorithms can be shared, understood, analyzed & even proved correct. Algorithms
are based on ideas. New algorithms must be based on new ideas, new ways to approach the
problem, new thoughts. They often express a clear single thought that changes the way we
look at the world. Algorithms are like equations. Albert Einstein’s famous equation,
E = mc2
is a part of the popular culture. Few non-experts are able to explain the exact meaning of
this equation. But, when placed on a T-shirt, most recognize it. This equation is one of
the most profound and important discoveries of all time. For better or worse, the equation
created the Atomic Age. The plans and blueprints for building a modern atomic reactor
may be closer in size to Vista. But the equation that gives us the power to build the atomic
R.J. Lipton, The P=NP Question and Gödel’s Lost Letter,
DOI 10.1007/978-1-4419-7155-5_2, © Springer Science+Business Media, LLC 2010
9
10
2 Algorithms: Tiny Yet Powerful
reactor has just five symbols. Likewise, it is an algorithm that makes possible the modern
cryptographic systems that are critical for e-commerce. Prime numbers form the backbone
of these systems. The famous Miller and Rabin algorithm to test whether or not a number
is prime contains just 61 words and fits in a box [105, 115]:
Given input n, an odd integer greater than 1. Write n −1 as (2s )d by factoring powers
of 2 from n − 1. Pick a randomly in the range [1, n − 1]. If ad mod n = 1 or mod
n = −1 for some d in the range [0, s − 1], then return probably prime. Else return
composite.
Tiny, powerful algorithms have built the foundations for giant companies: The
airline industry, the shipping and logistics industries, and the financial sector all
rely on fast optimization, scheduling, and pricing algorithms. Medical imaging devices to detect heart valve damage would not be possible without the Fast Fourier
Transform. The list goes on and on. New algorithms will drive yet unknown industries. They will be essential for creating new kinds of computing: petascale,
nanoscale, quantum, bio, neuro, distributed, multicore. They speed up today’s technology while ushering in tomorrow’s. They will be key to facing the challenges
arising from the ubiquitous collections of mega sized data sets. In virtually any field
where computation is done, the long-run impact of algorithmic advances will, eventually, outrun even the impact or demise of Moore’s law. Natural processes such as
the regulation of proteins, the dynamics of social networks, and the strategic behavior of companies are all fundamentally algorithmic in nature. Algorithmic theory
provides powerful tools, as well as appropriate language and mindset to help understand these new domains. The monumental first mapping of the human genome was
accomplished by Gene Myers’ whole genome shotgun sequencing algorithm [110].
Exploring the potential of algorithms lies at the heart of the million dollar question:
Does P = NP?
While research to answer this question may seem esoteric, it lies behind our current ability to have private and secure electronic communications. Our ability to
maintain security in the future will hinge on research seeking answers to this question. Advances in our deep understanding of the nature of computation will underlie
and propel further advances in virtually every area of information technology.
2.1 A Challenge
Ironically, for secure communication what we desire is the absence of an algorithm.
There must not exist an efficient algorithm that would enable an eavesdropper to
decode our secret messages. Yet, the following simple challenge seems hopeless.
Prove there is no algorithm for SAT that runs in polynomial-time and can be written
down on a single piece of paper. You can make the paper 8 12 by 11, if you wish. The
writing has to be normal size. No tricks.
I claim that this challenge is completely hopeless. There is no way that anyone
will be able to prove that such an algorithm does not exists. Note, P=NP says much
2.3 Notes
11
more–it says that there is no algorithm at all of any length. The algorithm could be
a trillion pages in length, or it could be one page. What is so shocking is that the
conventional wisdom is confident, yet the one page challenge appears to be hopeless.
Does that bother you? I find it shocking that we cannot today rule out that there is a
one-page algorithm that solves SAT.
2.2 Open Problems
I have often wondered if there is some hope to prove that restricted algorithms cannot compute SAT. By restricted I mean algorithms that can only have very simple
“loop” structure for example. There may be a way to make this precise enough so
that we could make some progress. Yet, the full one page challenge is currently out
of reach.
2.3 Notes
This appeared as algorithms-tiny-yet-powerful in the blog.
http://www.springer.com/978-1-4419-7154-8