Issue No.04 - July/August (2004 vol.6)
Published by the IEEE Computer Society
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/MCSE.2004.14
In Response to a Recent Book Review
In his review of Stephen Wolfram's A New Kind of Science (vol. 4, no. 5, pp. 82–83), George Cybenko writes that the "scientific heavy lifting underlying this book's conclusions has been in development for many years by many scientists," and that "Wolfram offers the reader a specific worldview and justification in which to practice existing science, not new science, as claimed."
Cybenko, of course, writes a number of other things as well, many of which are positive. But this particular idea is one that has become somewhat of a modern intellectual urban legend—everyone seems to know someone who knows someone whose ideas Wolfram has taken without proper attribution. Yet the specifics of how the said idea relates to the book's core intellectual structure are inevitably lacking.
For many, the question of NKS's originality and significance revolves around its relationship to existing fields. If these fields construe the book as a manifesto for justifying the practical importance of computers, then it merely serves to reinforce their worldview. In reality, though, NKS seeks to introduce a basic, empirical science that investigates the behavior of very simple programs. Simplicity and discovery go hand in hand throughout its pages: simple systems can be enumerated and exhaustively searched, leaving no stone unturned. Simplicity furthermore allows effective visualization, which NKS strongly argues is essential for discovering previously unknown phenomena. Finally, although proving a theorem or stating an equation are appropriate outcomes in some cases, the book emphasizes basic, general questions whose answer may have a strong qualitative component.
Like Cybenko, we can use a simple yardstick to evaluate the idea of systematically exploring the computational world. Does an existing body of literature consist of, for example, papers enumerating the 2,048 two-state, two-color Turing machines and investigating the overall kinds of behavior they are capable of? Do any papers enumerate simple substitution systems, commenting on what kinds of common features emerge? Is there a group of researchers out there systematically and empirically investigating the behavior of simple computational systems for their own sake?
Outside of a few isolated examples, this generally doesn't happen: it's outside the jurisdiction of any existing field. The computational sciences, for example, aren't about computation, but how to use computation to solve problems in specific fields. Computational neuroscientists aren't about to start experimenting with cellular automata just for fun, and computer scientists are either proving theorems or engineering complex computer systems. Experimental mathematicians are focused on the existing constructs that have arisen in the developmental history of mathematics.
One would perhaps hope that complex systems research—the field Stephen Wolfram helped pioneer in the 1980s—would be a central figure in this sort of work. After all, simple computer programs are in a sense minimal examples of the phenomenon of complexity. Yet the paradigm in this field remains to study naturally occurring systems, not abstract computational ones. Without diminishing the work of others, one can objectively say that the notion of systematically exploring the computational universe is a relatively unexplored branch of science.
NKS argues that concepts such as complexity and randomness come from human perception. By providing a framework for understanding why we perceive complexity, NKS enables informed choices about what research directions are fruitful. For instance, if a single universal mechanism were at work in a complex-looking natural system, it would imply a limited number of ways that computational processes can achieve a given computation. Yet all the empirical evidence suggests that this is absolutely not the case.
The rest of the book's intellectual heavy lifting has to do with developing a methodology for addressing simple questions. The foundational observation that makes this enterprise at all possible is that even the simplest programs are capable of, and often achieve, arbitrarily sophisticated computations. The consequences of this are profound. First, there is essentially no need to go beyond simple programs to address general questions. An immediate corollary is that naturally occurring systems are disqualified as the objects of basic investigations. A crucial question, then, is this: Are experiments on abstract systems with no obvious natural origin of any consequence to traditional science? Put another way, do we expect that natural systems are governed by a set of laws not just different in detail but also different in character from abstract systems?
For the practicing scientist, the book provides a mass of basic knowledge about what's possible in principle. Just as the goals of education are to introduce a set of useful ideas that serve as building blocks to bend and adapt to new questions, the body of NKS provides material to all those interested in the basic question, "how do things work?"
By clearly defining its aims, justification, and relationship to the rest of science, Wolfram puts the study of simple programs on the map as a science of its own. Because the object of study requires an entirely different methodology, it's a kind of science. But is it fundamentally useful? Cybenko and others have expressed the feeling that NKS offers no real new tools or avenues of approach. Certainly, the question of whether the NKS approach is ultimately worthwhile can only be answered by its results.
Given the book's size and scope, one would expect Wolfram to present overwhelming evidence to support his conclusions. In terms of the basic computer experiments, there can be no doubt that simple programs exhibit great complexity that is unlikely to succumb to simple mathematical descriptions. So how does this intuition translate to other problems? Surely, any worthwhile new idea or method can nibble off a corner of science and present a few successes.
In physics, for example, Wolfram challenges the traditional notion of time and the fundamental separation of space and matter. He presents an approach based on network rewrites that is already capable of reproducing special and general relativity, and he argues that his deterministic model escapes conflicts with Bell's theorem because of its radically different setup. Furthermore, Wolfram explains the apparent validity of the second law of thermodynamics without resorting to the unproven assumption of ergodicity. In biology, Wolfram points out that natural selection must live within the same computational limits as all other processes. He presents models based on simple programs of phyllotaxis, leaf growth, and shell growth, showing that counter to popular wisdom, all reasonable variations do actually occur in nature.
Wolfram ultimately develops the Principle of Computational Equivalence, which successfully ties together many of this massive book's ideas. He goes on to apply it to several problems, arguing for instance that there is no abstract defining feature of the human condition. By any reasonable standard, this qualifies as an intellectual eruption, and it seems almost bizarre to claim that this approach will not achieve continued success. Many of the critics of NKS express vague doubts, but few make the claim that "rule 110 is an isolated case," "simple programs do not generate a diversity of interesting behavior," or "simple programs will not become important in modeling nature." Like Cybenko, they typically only express the feeling that their own personal toolbox will likely remain unchanged.
Given a specific problem, NKS might not seem immediately relevant: it picks off the low-hanging fruit of underexplored areas, whereas most of science is highly specialized to tackle questions with which its methods tend to succeed. Yet besides opening up a new basic science—which existing scientists understandably may not be interesting in participating in— NKS also opens up countless new approaches to existing fields.
The bottom line is this: NKS is a fundamental new science that is interesting in its own right, and it will eventually be a crucial feeder of ideas into the other sciences. It does not simply encourage scientists to continue to "model and simulate." It counsels them to become familiar with what kinds of behavior are abstractly possible, and it implores them to simplify their systems and use systematic methods that will allow their problems to be solved in unexpected ways. It challenges them to develop the appropriate abstractions rather than simply tackling the problems with a predefined set of methods. It reminds them to always ask the simplest questions and to continuously challenge their basic assumptions. And perhaps most of all, it forces a new basic self-awareness that the scientific process must also operate within the same universe that it studies. Although this implies that human thought is not above the universe, it also implies that it is not below it, either—with appropriate methods, the fundamental secrets of nature are ultimately accessible.
[Ed. note: This letter has been edited due to space constraints. If you'd like to see the original text in full, please contact Jenny Ferrero, firstname.lastname@example.org.]
George Cybenko's response:
Kovas Boguta raises many interesting issues but answers few of the challenges that I expressed in my original review. For example, in response to my challenge about what is new in NKS, Kovas writes:
"Does an existing body of literature consist of, for example, papers enumerating the 2,048 two-state, two-color Turing machines and investigating the overall kinds of behavior they are capable of? Are there papers enumerating simple substitution systems, commenting on what kinds of common features emerge?"
While I concede that these kinds of results from NKS appear to be new, we still have the nagging question of how such results empower readers to do or understand something they couldn't before. Put more simply, "so what?" It would have strengthened Kovas' thoughtful letter immensely if he could provide just one example of such empowerment drawn from any area of science that is studied and followed by more than a handful of devotees.