CogSketch: Sketch understanding for Cognitive Science Research and for Education

Part of the Spatial Intelligence and Learning Center, an NSF Sciences of Learning Center


Project Lead: Kenneth D. Forbus

Project Summary: Sketching is a powerful method of capturing and communicating spatial concepts. Teachers in STEM disciplines often use sketches in instruction, and believe that student's sketches are greatly revealing of their degree of understanding. Yet up to now, the great potential value of sketching in STEM instruction has not been realized. This is partly because scoring sketches is extremely time-consuming for instructors, but also because the time course of drawing is lost when pencil and paper are used to sketch. Recent studies show that the order in which students draw the parts of a diagram is highly revealing of their understanding (e.g., Jee et al, 2009). Moreover, there is ample evidence that intelligent tutoring systems and learning environments can provide significant benefits to learners. But such systems are rarely developed for spatial learning, in part because the science base needed to support sketching for interaction is missing. Consequently, one of SILC's strategic goals is the creation of that science base, by understanding the visual, spatial, and conceptual understanding that people bring to sketches, so that we can create software that sees sketches the way that we do.

Our research involves building and using a new sketch understanding system we have created, called CogSketch (Forbus et al 2011). CogSketch has two interlocking and synergistic purposes:
  1. CogSketch is a cognitive science research instrument.
    1. In laboratory experiments, it enables us to gather data more effectively to explore how sketching can be used in assessing and improving learning.
    2. In computational simulation experiments, it enables us to model spatial skills and learning processes.
  2. CogSketch is a platform for new kinds of sketch-based educational software. We are developing two types of educational software on top of it:
    1. Sketch Worksheets are like paper worksheets, except that students are able to get feedback on demand, immediately, as they work. Sketch Worksheets are domain-general: They use analogical matching to automatically generate advice, based on comparing a student's sketch with an expert sketch.
    2. The Design Coach is aimed at helping engineering students learn to communicate better via sketching. Students explain designs by drawing an interconnected set of sketches, a kind of comic graph to illustrate its intended behavior and using a structured interface to provide simplified English statements. Design Coach looks for gaps and inconsistencies in their explanations, using qualitative mechanical reasoning to give conceptual feedback.
These purposes are synergistic: The problems that arise in STEM education provide a source of research questions to explore, and the progress in the research extends the representations and reasoning capabilities of the educational software.

Selected Publications:


Back to Projects page | Back to QRG Home Page