Common object representations for visual production and recognition

Abstract: Since the earliest etchings onto cave walls were made 40,000 years ago in modern-day Spain and Indonesia, people have devised many ways to render their thoughts in visual form, employing media ranging from stone and clay to paper and digital displays. The most basic and direct among these visualization techniques is drawing, in which a person produces marks that form an image. Drawn images predate symbolic writing systems, are pervasive in many cultures, and are produced prolifically by children from an early age. How do we convey concepts in visual form, and how does refining this skill, in turn, affect recognition? We developed a crowdsourcing platform for collecting large amounts of drawing and recognition data, and applied a deep neural network model of visual cortex to explore the hypothesis that drawing recruits the same abstract object representations that subserve visual recognition. Consistent with this hypothesis, we discovered that drawings contain the features most important for recognizing objects in photographs, and that learning to make more recognizable drawings of objects generalizes to enhanced recognition of those objects. These findings could explain why drawing is so effective for communicating visual concepts, they suggest novel approaches for evaluating and refining conceptual knowledge, and they highlight the potential of deep networks for understanding human learning.