CogSketch: Sketch understanding for Cognitive Science Research and for Education
Part of the Spatial Intelligence and Learning Center, an NSF Sciences of Learning Center
Project Lead: Kenneth D. Forbus
Project Summary: Sketching is a powerful method of capturing and communicating spatial concepts. Teachers in STEM disciplines often use sketches in instruction, and believe that student's sketches are greatly revealing of their degree of understanding. Yet up to now, the great potential value of sketching in STEM instruction has not been realized. This is partly because scoring sketches is extremely time-consuming for instructors, but also because the time course of drawing is lost when pencil and paper are used to sketch. Recent studies show that the order in which students draw the parts of a diagram is highly revealing of their understanding (e.g., Jee et al, 2009). Moreover, there is ample evidence that intelligent tutoring systems and learning environments can provide significant benefits to learners. But such systems are rarely developed for spatial learning, in part because the science base needed to support sketching for interaction is missing. Consequently, one of SILC's strategic goals is the creation of that science base, by understanding the visual, spatial, and conceptual understanding that people bring to sketches, so that we can create software that sees sketches the way that we do.Our research involves building and using a new sketch understanding system we have created, called CogSketch (Forbus et al 2011). CogSketch has two interlocking and synergistic purposes:
- CogSketch is a cognitive science research instrument.
- In laboratory experiments, it enables us to gather data more effectively to explore how sketching can be used in assessing and improving learning.
- In computational simulation experiments, it enables us to model spatial skills and learning processes.
- CogSketch is a platform for new kinds of sketch-based educational software. We are developing two types of educational software on top of it:
- Sketch Worksheets are like paper worksheets, except that students are able to get feedback on demand, immediately, as they work. Sketch Worksheets are domain-general: They use analogical matching to automatically generate advice, based on comparing a student's sketch with an expert sketch.
- The Design Coach is aimed at helping engineering students learn to communicate better via sketching. Students explain designs by drawing an interconnected set of sketches, a kind of comic graph to illustrate its intended behavior and using a structured interface to provide simplified English statements. Design Coach looks for gaps and inconsistencies in their explanations, using qualitative mechanical reasoning to give conceptual feedback.
- Sagi, E., Gentner, D. & Lovett, A. (2012). What difference reveals about similarity. Cognitive Science. doi: 10.1111/j.1551-6709.2012.01250.x
- Chang, M. & Forbus, K. (2012). Using quantitative information to improve analogical matching between sketches. Proceedings of the Innovative Applications of Artificial Intelligence Conference, July.
- Forbus, K., Usher, J., Lovett, A., Lockwood, K., & Wetzel, J. (2011). CogSketch: Sketch Understanding for Cognitive Science Research and For Education. Topics in Cognitive Science, 1-19. doi: 10.1111/j.1756-8765.2011.01149.x
- Lovett, A., & Forbus, K. (2011). Cultural commonalities and differences in spatial problem solving: A computational analysis. Cognition, 121, 281-287.
- Lovett, A., Forbus, K., & Usher, J. (2010). A structure-mapping model of Raven's Progressive Matrices. Proceedings of CogSci10.
- Yin, P., Forbus, K., Usher, J., Sageman, B. & Jee, B. (2010). Sketch Worksheets: A Sketch-based Educational Software System. Proceedings of the 22nd Annual Conference on Innovative Applications of Artificial Intelligence.
- Lovett, A., Tomai, E., Forbus, K. & Usher, J. (2009). Solving Geometric Analogy Problems through Two-Stage Analogical Mapping. Cognitive Science. 33(7), 1192-1231.
- Jee, B., Gentner, D., Forbus, K., Sageman, B. & Uttal, D. (2009). Drawing on experience: Use of sketching to evaluate knowledge of spatial scientific concepts. In Proceedings of the 31st Annual Conference of the Cognitive Science Society. Amsterdam, The Netherlands.
- Wetzel, J. & Forbus, K. (2009). Automated Critique of Sketched Mechanisms. In Proceedings of the 21st Innovative Applications of Artificial Intelligence Conference. Pasadena, California.