Ronald W. Ferguson
Statement of Research
Interests
I am interested in understanding the nature of visual
thinking. My research investigates the role of spatial relations in symmetry
detection and in other visual reasoning tasks.
This research has both cognitive and applied aspects. In my cognitive research, I have designed,
implemented, and explored the psychological implications of a new
computer-based model of symmetry detection, called MAGI. In my more applied research,
I have used both MAGI and a spatial reasoning engine called GeoRep to build
more flexible and effective computer reasoners for diagrams and sketches.
The MAGI model is based on a new framework that I have
proposed for symmetry and repetition detection. This framework utilizes some simple but powerful ideas. In this
framework, the visual system detects symmetry and repetition by aligning sets
of perceptual relations. This alignment uses a structure mapping
process, similar to that proposed by Dedre Gentner for analogy and similarity
comparisons. This means that symmetry
is detected not by mentally folding or rotating a figure about an axis, but by
aligning maximally interconnected systems of similar spatial relations within
the figure. For example, the symmetry
of a star-shaped figure is detected by aligning the corners and other visual
elements based on common concavity, placement, connectivity or other
qualitative characteristics.
The MAGI computer model implements this
framework. The MAGI program can take a
line-drawn figure, detect a set of qualitative spatial relations (using the
GeoRep system described below), and then determine the qualitative symmetry of
the figure by aligning systems of those relations. This model has significant implications: it provides a more
robust method for detecting inexact symmetry, and it more effectively explains
how humans detect complex structural symmetry.
More pragmatically, because MAGI can also align conceptual structure
(MAGI can detect symmetry in non-visual input, such as equations and story
narratives), it can explain the relationship between perceptual and conceptual
symmetry, which is a key ability in reasoning from diagrams.
I have collaborated with psychologists to test the
cognitive implications of the MAGI model.
Research conducted with Gentner and Alex Aminoff provides evidence that,
as MAGI predicts, human subjects are sensitive to qualitative visual structure
in symmetry judgments of polygons, even for extremely short display times (50
ms). MAGI has also been used to
simulate the results of classic symmetry experiments by Palmer & Hemenway
(1978). These experiments demonstrated
an interaction between symmetry orientation and ease of symmetry judgment. Our simulation showed that MAGI could explain
a key result of these experiments – why bilateral symmetries at some
orientations (such as vertically) are perceived more easily than at other
orientations.
I plan to continue my interdisciplinary collaboration
with psychologists to test other aspects of symmetry detection using the MAGI
model. MAGI’s strength in this research is its ability to use the same
line-drawn stimuli frequently given to human subjects, so that their results
can be compared with MAGI’s directly.
Symmetry detection occurs early both developmentally and in the visual
process, yet is still only partially understood. I believe that resolving open problems in this area – most
centrally the role of spatial relations – could provide key insights into
perception and cognition. I also plan to use MAGI in future diagrammatic
reasoners.
I have also built a system called GeoRep. GeoRep is a spatial representation system
that takes vector graphics files (i.e., line drawings) as input, and produces a
predicate calculus spatial representation.
It builds higher-level diagrammatic vocabularies (e.g., gates and wires
in a logic circuit diagram) in terms of lower-level spatial relations (e.g.,
corners, closed shapes and boundary characteristics, and intersections). GeoRep is not a model of drawing or sketch
recognition, but a diagrammatic reasoner that models how particular visual
relations support particular spatial inferences. Although its object-recognition ability is limited, once it has
identified an object it can reason extensively about its spatial
characteristics and how those characteristics support higher-level spatial
inferences.
GeoRep’s two-level architecture makes it easy to use
and extend. An initial stage builds a
low-level representation using a library of visual operations. A second stage builds a domain-specific
diagrammatic representation using a rule-based domain theory. Extending GeoRep to cover a new kind of diagram
often requires only the writing of a new domain theory. We have used GeoRep to handle diagrams for
logic circuits, military planning, and for simple physics diagrams.
More recently, we've explored the possibility of using
GeoRep with multimodal sketching systems.
In a prototype system called nuSketch (built in collaboration with Ken
Forbus and Jeff Usher), military planners combine speech and sketching gestures
to input information on a planning diagram.
For each drawn element, boundary and category information is passed on
to GeoRep, which then uses this information to answer qualitative geographic
queries (such as “What enemy obstacles are along this path?”).
In future research I hope to use GeoRep in new
diagrammatic domains, such as architecture or teaching diagrams, as well as
extend its basic capabilities. We are also investigating how GeoRep might be
used for reasoning with single-use diagrammatic conventions, such as
conventions invented in the course of drawing.
Current research issues include how to provide more flexible bottom-up
recognition of shapes (which may involve probabilistic or multi-scale
approaches to recognition), as well as how to better incorporate top-down
influences on recognition, such as domain knowledge or information from an
interactive dialogue with the user.