Symbolic Supercomputing
Symbolic modeling is central to capturing crucial phenomena of human cognition. People understand, make, compare and learn from complex arguments. These arguments are typically expressed in language, often aided by diagrams and maps. Recent evidence suggests that the effects of these kinds of representations go beyond communication: that language and diagram notation influences human mental representations. Moreover, people fluently apply complex processes, such as analogy, to use this information in learning and reasoning. The use of conceptual knowledge and symbolic systems such as language, maps, and diagrams are signature phenomena of human cognition. Symbolic modeling is an important method for exploring these phenomena.
It is important to distinguish this kind of modeling from neural modeling. Computational methodologies for studying the mind include both modeling neural systems and modeling conceptual processing. Modeling neural systems is an important source of insights for understanding how brains work. However, such techniques tend to be less appropriate for modeling higher-level cognition, e.g., how people reason and learn in complex situations, especially involving conceptual knowledge. It is still far from clear how neurons encode information: Is processing based on distributed representations, for example, or does it rely on temporal binding schemes? While these are important questions, one needn’t wait for them to be solved in order to tackle modeling of conceptual processing, just as chemistry made significant progress before quantum mechanics was worked out, and continues to be used as an important level of analysis. For building intelligent agents that can become partners with people, symbolic modeling is absolutely crucial, since it captures conceptual knowledge in ways that can be processed by software.
Human reasoning appears to derive much of its power from fluent access to large stores of knowledge (Newell & Simon, 1972). Reasoning and learning by analogy is a central process in operating over this knowledge. Research on human processing suggest that structure-mapping processes described by Gentner (1983) operate over a range of phenomena that runs from low-level visual similarity through categorization and problems solving to conceptual change. Our research is aimed at understanding how these processes work and how they can be exploited to create new kinds of intelligent systems. This commitment to large bodies of represented knowledge requires efficient processing techniques. In contrast to symbolic models that assume serial processing, our approach relies on parallel processing, both within fundamental processes (e.g., retrieval, parts of analogical matching) and in higher-level processes. We currently emulate parallelism on single-CPU machines, but our experiments would benefit greatly from the ability to use coarse-grained parallelism.
What does it take to speed up symbolic processing? Traditional supercomputers focus on floating-point computations in organized in regular structures (typically tessellations of some physical system), to maximize throughput in pipelines. Symbolic computing has very different demands. Symbolic structures are irregular. This makes the pattern of memory usage very different. Pointer operations and integer/string comparisons predominate. Floating-point pipelines are largely irrelevant, since what floating point operations do occur happen over sparse vectors (e.g., the content vector operations in our MAC/FAC retrieval model (Forbus, Gentner, & Law, 1995) and graph-structured representations.
Over many years of experimentation with various architectures for speeding up symbolic computation, the three ideas that seem to work the best across a broad range of reasoning problems are:
- Use the fastest CPUs available. In many graphics-intensive systems, the main CPU can be relatively slow by using a high-speed graphics coprocessor. Given the nature of symbolic processing, to date no one has found a subset of operations that lends itself to a similar specialization. An extremely fast general-purpose processor, with a large cache) still seems like the optimal choice.
- Distribute the processing in large, functionally-organized units.For some specialized computations fine-grained architectures may provide an advantage (e.g., data-parallel machines for fast marker-passing). But for building high-performance intelligent systems, the most successful architectures have been those which divide the processing along functional lines, replicating knowledge bases as required.
- Use large memories that are as fast as possible. Most symbolic processing consists of pointer operations. Optimal performance requires storing as much as possible in RAM (and cache) and avoiding virtual memory.
We created what we believe to be are the first symbolic supercomputers:
- Colossus was a five-node Linux Networx Evolocity cluster that supports the Companions project. Each node has two 2.8Ghz CPUs with 4GB of RAM and 120GB of disk. Five additional nodes were added to support the Learning Reader project. This cluster was funded by DARPA IPTO.
- Mk2 was a 67 node Linux Networx Evolocity cluster that became operational in August 2004, to support our ONR-sponsored AI and Cognitive Science research. Each node had two 3.2Ghz CPUs with 4GB of RAM and 80GB of disk. This cluster was funded by the Office of Naval Research through the DURIP program.
Both machines are shown in the picture below. The rack on the right is Colossus, the two racks on the left are Mk2.
Our current symbolic supercomputer is Captain Morgan, a 128 node HP cluster with 2,048 cores, 4 TB of RAM, and 64 TB of disk. This cluster was funded by the Air Force Office of Scientific Research, via the DURIP program.
Relevant Projects
Analogical Learning for Companion Cognitive Systems
Symbolic Supercomputing for Artificial Intelligence and Cognitive Science Research
Building and Using Large Common Sense Knowledge Bases