AskUs: What’s going on in my head when I’m thinking of an image?

So one of our readers asked us this question the other day: What’s going on in my head when I’m thinking of an image?

ANSWER

In the 80’s there was a big debate about this called, appropriately, The Imagery Debate. There was even a (philosophy) book published with that name. I recommend checking out the Standford Encyclopedia of Philosophy (SEP; a great resource in general) article on mental imagery, particularly the section on the analog propositional debate.

Roughly, the two camps/ideas were this:

Analog View (Kosslyn): Mental images are imagistic or pictorial in nature and share many properties with an actual image / perception of the real world. This might include reactivation of cortical areas during mental imagery that are active during perception. It is like you are really seeing the thing (hence the term analog).

Propositional View (Pylyshyn): Although it may seem to us that we call up an image and examine it with our mind’s eye (whatever that may mean!) this is actually misleading and does not reflect the true nature of the representation and is also in danger of committing the Cartesian fallacy (who/what is looking at / analyzing this internal picture?). Instead, mental images are actually (symbolic and amodal) descriptions in a Language of Thought -like way. So, for example, when you imagine your room, you are not creating a picture in your head, but a series of relations between concepts like “to the left of (door, couch)” which means that the operator “to the left of” takes in two objects or arguments (door and couch) and describes the relation between them.

It would take too long to list all of the arguments for the two views, so here are a select few:

Evidence in favor of picture-like representations:

• Mental rotation: if you are comparing two 3D objects and trying to decide whether they are the same (just rotated versions) or not, the amount of time to answer is linearly related to the angle of rotation (Cooper and Shepard 1973. It is as if we are rotating a mental model just like we would a physical object to try to get them to match
• We represent empty space and preserve metric information: imagine a map of the US. Focus your attention on NY. Now move your attention to Maine. Go back to NY. Now shift your attention to California. Did it take you longer? Many people say yes. (The actual studies used fake maps that subjects had to memorize and different measures). This suggests that our mental images preserve metric properties: things that are far in the world are far in mental images, that just like for a picture, it takes time to scan our mental images, and that we represent the empty space between parts of our mental image/ focus of attention just like in a picture (Kosslyn, Ball, and Reiser 1978).
• We experience our mental images in different detail based on our imagined viewpoint /distance just like a real picture/ the world: imagine a rabbit next to an elephant. Does the rabbit have ears? Whiskers? Now imagine a rabbit next to a bee. Does it have those features now? Depending on the imagined “level of zoom” we imagine or can respond about features with varying degrees of resolution (Kosslyn 1975.
• Mental imagery activates early visual areas: Ganis, Thompson, and Kosslyn 2004, Slotnick, Thompson, and Kosslyn 2005, and many more. Although I would point out that there is still a question of exactly what is represented / what those activations mean and some of the evidence isn’t very convincing and vivid mental imagery can occur in the absence of primary visual cortex (Bridge et al. 2012, although see Stokes et al. 2009).

Evidence in favor of the propositional view:

• Our mental images and representations are typically fixed in a particular viewpoint/ representation and we can’t look at them a different way like we could with a picture. For example, in these Slezak figures, you can imagine rotating them 90 degrees counterclockwise and seeing a different animal. However, if subjects are shown these figures briefly and asked to memorize them, they do not notice the other interpretation, even though they can mentally rotate the figures (Slezak 1991; see also Chambers and Reisberg 1985 who showed that mental images cannot be ambiguous like pictures can). Nor can we detect the parts or patterns of a mental image like we could with a real image (Reed and Johnsen 1975). Rather, it is as if we have some sort of abstract, summary representation and not a picture.
• What we store and then try to imagine is greatly affected by the context and is not a veridical representation of what we have save / are trying to imagine. Carmichael, Hogan, and Waters 1932 (<– not a typo on the date!) had participants memorize and then later draw from memory a series of simple shapes. For example, they might be shown a circle with little lines sticking out of it on all sides. For one group of subjects, they told them it was a sun; for another group, they told them that it was a ship’s wheel. When subjects later drew the images from memory, their pictures were distorted based on the label that the image was given, even though the original image was the same for both groups. They argued that this showed that the images were not stored as pictures but as concepts which were susceptible to linguistic influence.
• As mentioned, the fMRI evidence is actually a bit mixed. In many of those studies, they find activation outside of early visual cortex in areas like LOC and in parietal cortex suggesting that representations are more diffuse and distributed and perhaps more abstract.

Resolution: The nature of mental image representations may be a combination of both types and may vary based on task. Some aspects are pictorial and some are propositional. See the dual-coding theory (the critical evidence in favor of this is probably Brooks 1968).