I. Introduction

“Every artist, as a creator, has something in him which calls for expression…”  -- Kandinsky

“Among artists one often hears the question, ‘How are you?’ answered gloomily by the words ‘Feeling very violet.’

“In the arts, the mind displays it passions most purely and applies the intelligence of it senses most fully.  Whoever can do justice to the psychology of the artists, their work, and their public is well on the way to giving an adequate account of human nature.”    -- Rudolf Arnheim

fMRI Image of Brain During Picture Perception

The Study of Art and the Mind-Brain
Research in cognitive science
Art and art history
Philosophy of Art

Four Main Questions for AMB
How do pictures represent?
Does art express emotion?
What is aesthetic experience?
Why does art have a history?

Some Recurrent Themes
What is innate and what is learned?
Do we need mental representations or not?
How isolated are perceptual and cognitive capacities?
How rational are we?

Matching Topics in Aesthetics and Science
  Pictorial representation ---> Perception
  Expression in art   ---> Emotion
  Aesthetic experience  --->   Attention
  Art history    --->   Memory

Can Science Tell Us Anything About Art? NO
Art is subjective, science is objective
Appreciating art involves norms; science deals in facts.
Art is a concept; understanding it depends on conceptual analysis, not empirical investigation.

Can Science Tell Us About Art? YES
True generalizations can be made about art.
There are principles of learning & plasticity.
Minimal psychological realism:  Norms should be related to human cognitive and perceptual abilities.
Some concepts are not like rules.  For them, a strict definition is not possible.

How Science, Art, and Philosophy Can Converge

fMRI Image of Brain During Picture Perception

Two Theories of the Mind-Brain
Identity Theory:  the human mind is identical to the human brain.
Functionalism:  the human mind is a set of powers that human brains, some non-human brains, and some computers can perform.
 
 

Ludwig Wittgenstein 1889-1951
Family Resemblance:
A continuous set of overlapping perceptible features that produce degrees of similarity among members of a family.

II. Pictorial Representation:  Convention

 Key Points in Goodman
All pictures classify or sort individuals (real or fictional) into types:  a ‘man-picture’ represents an individual as a man.
Some pictures also refer to an existing individual:  a ‘man-picture’ may denote Napoleon or Clinton.

      Conventionalism
Both sorting and standing-for (reference) in pictures are matters of convention.
Neither is determined by (a) resemblance or (b) a causal relation between the picture and the object it depicts.

  Special Learning
Cross-cultural evidence:  People cannot recognize pictures produced according to unfamiliar conventions.

Animals and young children are not able to perceive pictures like adults.

 Object Recognition
Infants can recognize objects in pictures.
Adults in non-pictorial societies can  recognize depicted objects right away.
Chimps, pigeons, and even spiders can recognize depicted objects.

 Spatial Layout
Children cannot perceive depth relations until age three.
Adults in non-pictorial societies seem unable to use depth cues to judge the size and distances of objects.
  Questions

Why can’t non-western adults perceive depicted space if western 3 year olds can?
The western 3 year olds had learned to use depth cues from exposure to pictures.  This shows need for special learning.
The experiments with non-western adults were flawed.
 

Flaws in Cross-cultural Studies
The pictures used were ambiguous: even some western adults were confused by them.
Non-western adults simply showed curiosity in turning the pictures over.
They may have had aesthetic preferences that interfered with their responses.
The testing situation was awkward for them.
They didn’t understand the test.

Do Children and Animals Have the Concept of a Picture?
No:  They can distinguish one picture from another and a picture of an object from the real object.  But they seem to view the picture as just a different real object (smelling, listening to it, etc.)
No:  Animals are reacting to diagnostic cues (e.g. a color), not representations.

III. Information - Gibson
Pictorial Representation
Information Theory

Review Key Points in Goodman
All pictures classify or sort individuals (real or fictional) into types:  a ‘man-picture’ represents an individual as a man.
Some pictures also refer to an existing individual:  a ‘man-picture’ may denote Napoleon or Clinton.
      Conventionalism
Both sorting and standing-for (reference) in pictures are matters of convention.
Neither is determined by (a) resemblance or (b) a causal relation between the picture and the object it depicts.

Pictures and Other Symbols
Interlude on the relation between
Sight
Sound

   Some Possible Connections
A chain of reference from one modality to another:  Kandinsky’s Décor for Mussorgsky’s ‘Pictures at an Exhibition’.
A case of musical reference

Does Kandinsky’s painting refer to Hartman’s paintings by way of Mussorgsky’s music?

 Cross-modal Effects
Does what you hear affect what you see, and vice versa?
If so, evidence for perceptual plasticity.

But also raises questions about how to carve up sensory systems:  Really the standard five?  Gibson:  no.

Argument for Special Learning
Cross-cultural evidence:  People cannot recognize pictures produced according to unfamiliar conventions.
Animals and young children are not able to perceive pictures like adults.

 Object Recognition
Infants can recognize objects in pictures.
Adults in non-pictorial societies can  recognize depicted objects right away.
Chimps, pigeons, and even spiders can recognize depicted objects.

 Spatial Layout
Children cannot perceive depth relations until age three.
Adults in non-pictorial societies seem unable to use depth cues to judge the size and distances of objects.
  Questions
Why can’t non-western adults perceive depicted space if western 3 year olds can?
The western 3 year olds had learned to use depth cues from exposure to pictures.  This shows need for special learning.  Or …
The experiments with non-western adults were flawed.
 

Flaws in Cross-cultural Studies
The pictures used were ambiguous: even some western adults were confused by them.
Non-western adults simply showed curiosity in turning the pictures over.
They may have had aesthetic preferences that interfered with their responses.
The testing situation was awkward for them.
They didn’t understand the test.
Do Children and Animals Have the Concept of a Picture?
No:  They can distinguish one picture from another and a picture of an object from the real object.  But they seem to view the picture as just a different real object (smelling, listening to it, etc.)

No:  Animals are reacting to diagnostic cues (e.g. a color), not representations.
 Aesthetic Significance?
If pictures are not seen as such, does pictorial representation lose aesthetic significance?  Or at least its significance for art?  To an animal, a picture is just a thing.

 Stages of Aesthetic        Development?
Children
Are born with the ability to recognize depicted objects
develop the ability to recognize spatial layout in pictures at age 3
develop the ability to appreciate composition and design at age 7.

 Stages of Aesthetic        Development?
Later stages are ‘higher’ or ‘more advanced’ than earlier ones.
 Stages of Aesthetic        Development?
Therefore, people who like figurative art are cases of ‘arrested development’.
 Stages of Aesthetic        Development?
And this can be seen in both individuals and in societies/historical periods.
So a developmental theory of art history is possible.
Right?

  Key Points in Gibson
Light has properties that ‘specify’ information, which we can directly detect.
The properties are higher-order invariants; i.e. they are constant across many different sensory inputs.
 

    Texture Gradients

 So What Is A Picture?
A picture represents an object or scene by having the “same kind of information” that the object or scene has.

But a picture is also itself an object with invariant surface properties of its own.

 The Duality of Pictures
There is “the space in which the picture lies and … the space in which the objects pictured lie.”
“They are not the same space.”

So How Do We Get Inside the        Picture?
“There is a relation of correspondence” between the invariant properties of the represented object and the invariant properties of the picture’s surface.
That relation itself must be invariant; a still-higher-order property.
 

  Direct Realism
Ordinarily, we detect perceptual invariants without our minds-brains having to derive (abstract, infer) them.

Different invariant properties cause different dynamic patterns in our perceptual systems.  Those cause different behaviors.
    Direct Picture Perception
If picture perception is direct, we must also be able to detect the still-higher-order relations among surface and object invariants.

It is fortunate that artists discovered various ways to tap into that natural capacity.
 

 What We Can Learn From Art

You usually attend to invariant properties = the naïve attitude.
But you can learn to attend to perspectives or momentary appearances as well = the perspective attitude.
 

  The Problem of       Misrepresentation
A picture should misrepresent if it presents invariant properties not found in the scene it depicts.
But how do we know which scene it depicts?  By the invariants it contains.
So a picture can never misrepresent.
 
 

IV. Computation

a.  Background

Brain Evidence
 Models of Picture Perception

Two Standard Models
Classic INFOPRO
Discrete stages
Feedforward only
Sensation first; knowledge later
Assembly line

 INTERACT(ive)
-- States, not stages
-- Feedback
-- Sensation and knowledge go hand-in-hand
Committee process
 

INFOPRO:  Artistic Styles Tap Different Stages
In Impressionism, “the retina predominates … because it tries to fix the most fugitive aspects of the external world”    (Gleizes & Metzinger, Cubism).
But Cubism “meant to penetrate to the essence of an object by representing it … the way it was …constituted in memory.”

Some Problems for the INFOPRO Account of Art
Artistic training can change the parts of the brain that are active in perception.
Parts of the brain with special functions can be destroyed, yet the functions can be restored in artists.
Why Is This The Case?
The visual brain is much more plastic than INFOPRO suggests
Some functions (e.g. object recognition) are just not very localized.
Some functions become automatic by training and are taken over by new parts of the brain.

The Interactive Model
Feedback from stage 3 ‘semantic analysis’ to stage 2 form analysis
Feedback from form analysis to basic edge detection
Suggests the possibility of a modification of basic edge detection by semantic analysis

Evidence
Psychological evidence for the context-dependence of perception
Neuroscientific evidence of numerous descending pathways and cross-connections in the brain
 

B.  ZEKI’s Model:  A Third Alternative
The visual system is both modular and plastic.
Special purpose units are more like way stations than stages.
There is both knowledge and consciousness at every station.

Some Evidence for ZEKI
Specialization of cells in the visual system.
the ‘what’ pathway and the ‘where’ pathway
color vision requires V4; motion perception requires V5.
“Artists have tapped this specialisation in their work” (63)

Are Zeki’s Modules Really Modular?
A ‘real’ module is
Localizable
Domain specific (responds only to certain inputs)
Cognitively impenetrable (unaffected by general knowledge)
Modularity and Consciousness
Strong modules are usually peripheral sensory systems.

They contrast with more central, higher-order processes that have access to all knowledge and are conscious.

Zeki’s Modules
Autonomous
Loci of knowledge and ‘understanding’
Sites of micro-consciousness

Zeki’s Argument
The other two theories of consciousness don’t work:
INFOPRO’s central executive (the cartesian theater):  no evidence for it.
The interactive model of synchronization of neuron firings:  evidence against it.

Conclusion
Consciousness and understanding are distributed across discrete modules.
Aesthetic properties are distributed across discrete modules.
There really is no unified aesthetic experience.  It’s an illusion.

What About Recovery After Brain Damage?
Some functions are widely distribution in the brain.  They are functionally, not physically, modular.

‘Modules’ are really processing systems.  They can include multiple areas.  Some areas are left intact after damage.

Problems With Zeki’s Account
‘Understanding’ implies more than applying knowledge from a limited domain.

Evidence for sharing of functions by modules.

No explanation of the illusion of unified aesthetic experience.

V. Danto
  Danto’s Philosophy of Art
The Role of Picture Perception

    Danto’s Positive Claims
Vision is modular in early stages.  This limits what we can see in pictures.
Pictures resemble what they denote.
But what a picture represents is determined by the perceiver’s beliefs and attitudes.

A Note on Terminology
For both Danto and Goodman, a picture denotes what it ‘stands for.’
Also, for both, being ‘pictorially competent’ is a matter of being able to recognize what a picture stands for or denotes.

Danto & Goodman
But for Danto, pictorial competence is natural and unlearned, contra Goodman.

And Danto does not clearly distinguish between ‘standing-for’ and ‘sorting-out.’ A picture can stand for a class or a fictional being, as well as a real individual:  hawks or the god Horus.
“Hawk pictures sometimes simply stand for hawks” (p. 2)
Even a weather-vane (which does not stand for any particular horse) resembles it denotatum, viz. a schematized horse or horse prototype.
But a picture may stand for more than what it resembles.
This is because we can interpret it.
So a picture might denote a reclining man, at the level of perceptual recognition (= a ‘reclining-man’ picture), yet stand for a dying general, at the level of our beliefs.
Interpretation endows a picture with meaning.
So, for Danto, denotation is not independent of meaning at the level of interpretation.
It is independent at the level of recognition, but reasons of neurology, not convention.
This is what matters.  Whatever we call it, Danto thinks we can recognize types of objects in pictures without special training.
And at a higher level, interpretation requires general historical knowledge, not the use of conventions.

The Role of Resemblance
It undercuts conventionalism.
It helps explain technical progress in picturing.
It plays a role in the argument for interpretation:  Resemblance is inadequate.

Anti-Conventionalism: The Universality of Picture Recognition
Western perceivers can recognize objects and layouts in European paintings.
Therefore, picture recognition is a universal ability.
 

     The Argument from Progress
There must be something constant in vision against which to measure progress in picturing.

The Inadequacy of Resemblance:  Danto’s Gallery of Indiscernibles
We can imagine a collection of pictures that are visually indistinguishable.  They resemble one another.
But the pictures have different contents, because they have different histories.

    Questions About Danto
On what does he base his claims about the role of resemblance?
Are his arguments/evidence convincing?
Does he really need resemblance?

    The Case for Resemblance
A picture will resemble what it denotes if the same basic perceptual processes and mental representations are used in recognizing both picture and its object.
Thus, knowing that one is a picture and the other isn’t (or any other knowledge) cannot cause a difference in our basic perceptions.

      Resemblance and Modularity
Therefore, basic perceptual recognition must be cognitively impenetrable, i.e. modular.
    The Case for Modularity
We do not have to learn to see pictures.
The same perceptual processes and neural pathways that are active we when recognize an object are active we when recognize a picture of that object.

Case for Modularity, Part II
Certain perceptual functions cannot be changed:  Depth perception is inevitable.
More Support for Modularity
The Persistence of Illusion

Module-based ‘resemblance’ is unconscious.  But perceptual theories of resemblance usually attribute the similarities to conscious experience.
There is evidence that the modularity thesis is false.

Is Resemblance Necessary?
Danto has two goals:
To provide a foundation for pictorial understanding
To show that pictures cannot be understood in terms of that foundation alone
For both goals, the modularity of vision will suffice.

 A Case for Plasticity Despite    Modularity
Picture recognition becomes automatic with training.  It’s taken on by a different part of the brain.
Part of the training involves scanning strategies, thus eye movements, thus motor control.

 A Note on Winner’s Evidence
It seems that there are two levels of perceptual training:  adding new concepts and then automatizing recognition.
Neither requires changing basic, modularized functions.
 Perceptual Strategies
There are reasons think that there are individual differences in the ways that perceivers deploy their modules.
Different modules can be employed to perform the same task:  shape from
shading
 vs. shape from edge and contour representation.

 The Importance of Such
Tasks performed habitually require little effort.
Motor skills and perceptual strategies can be used with different levels of expertise (speed, accuracy, ease).
So differences here will produce different pictorial experiences.

This suggests one possible answer to the question …
 
 
 

      Danto’s Theory of Pictorial       Content
What a picture represents is determined by its history.
History ‘supervenes’ on picture perception through the perceiver’s knowledge and beliefs.

Danto’s Account of Diversity
The variety of pictorial techniques in art is due to variations in the attitudes of cultures toward pictures.
But it’s also due to differences in skills and strategies.  These may be underwritten by internal mechanisms.

VI.  Schier's Recognition Theory
Pictorial Representation
Recognition Theories

Schier’s Recognition Theory
A picture is a symbol in an iconic system.
An iconic system is one that has the property of natural generativity.

Natural Generativity
Once you have learned to recognize the objects in one picture, you can then recognize the objects in any other picture.

Two conditions
The initial picture must have properties that will activate the object recognition functions of the visual system.
The identification of the depicted object must be caused by basic visual processes.

Note on Schier’s Terminology
Schier calls recognition or identification the “initial interpretation of a symbol” (p. 46).
But that’s not interpretation in Danto’s sense (which goes beyond basic pictorial competence).

What Is New In Schier’s Account?
First, Schier suggests that picture recogntion draws on visual modules:  “it must be possible to purify the interpretation [basic recognition] of all other influences … [and] other items of knowledge. (p. 50)

But, on his account, we don’t need to think of recognition as based on resemblance.

A Model for Basic Recognition
Biederman’s Recognition-By-Components:

Perceivers possess a basic visual vocabulary of 24 geons, out of which all object shapes can be constructed (and into which shapes can be decomposed).
Principle of Geon Sufficiency:  Any object can be recognized on the basis of 3 geons.
Relations among geons are
not dependent on viewpoint
are ‘non-accidental’ (i.e. usually correspond to real features of objects)
do not depend on familiarity with the objects
processed in stages

RBC is also constrained by certain innate assumptions:  that objects are generally rigid, that straight lines belong to straight edges, that object shapes tend to conform to gestalt principles.

Thus, while Biederman’s model allows for the possibility of top-down effects, they seem to be limited in how much they can change the basic modular mechanisms.

The model is a live, empirically-grounded candidate for Schier’s recognitional ability.
Recognition Without Resemblance
But RBC is essentially an unconscious process.

And geons need not themselves be like simple mental pictures; i.e. representing parts by resembling them.  They may be part descriptions (or even more abstract part-intersection descriptions).

Objects are ‘constructed’ from parts in the sense that object descriptions can be inferred from descriptions of parts and their relations.

Evidence for RBC
Subjects can recognize objects in pictures when two or three geons are visible.

But they have a harder time if the parts aren’t segmented in ways that RBC suggests.
How Does A Part-Representation Represent, If Not By Resemblance?

Schier:  “The causal relation of a perception to its object is … part of what it is for that perceptual experience to be of a particular object” (p. 50).

And recognition is an essential link in that causal chain.

This suggests that what an internal part-representation represents is determined by its typical cause.

A bar-detector is usually activated by bars.  An elongated cylinder representation is usually produced by elongated cylinders.
 

However, these may have to be identified in terms of evolutionary or teleological considerations.

These are default ‘assumptions’ about correlations between part shapes and object contours that have been acquired through natural selection.

The Relation of Recognition to Pictorial ‘Understanding’
Our ability to recognize objects enters into any further interpretations (beliefs, judgments) we may make

Understanding verbal interpretations presupposes the ability to recognize the objects to which the interpretation is directed.

It can’t be understood to be a picture of a dying general unless a reclining man is recognized in it. (= necessary condition)

But it does not follow that it will be understood to be a dying general if a reclining man is recognized in it. (Not a sufficient condition.)

Objections
Is recognition of anything in particular really a necessary condition for understanding it to be a picture of a certain thing?
Must we see a reclining man before we can understand that this is a picture of a dying general?
One problem is that RBC is only one way to recognize objects.  Spatial location, shading, etc. can also be used.
But what is seen in a picture depends, to some extent, on which strategy the perceiver uses.

If what is seen in a picture depends on the perceiver’s strategy, then we need an account of how strategies are deployed and which ones are likely to be selected under what condtions.

VII.  The Central Executive

Is There A Central Executive System In The Brain?
                     Some Evidence

Baddeley’s Model of Working Memory
Three components:
The central executive
An articulatory loop for processing verbal information
A visuo-spatial sketchpad for processing visual and spatial information

Evidence
Reasoning, articulation, and spatial tasks can be carried on simultaneously.

This implies that they are distinct, dissociable functions.

However, each component has limited resources.

So when either a verbal or a visuo-spatial task is very demanding, the executive can help carry it out.

This is usually accompanied by a sense of effort and focused attention.
 

Example
You are shown a stimulus:  AB
You are to press buttons for ‘true’ or ‘false’ in response to:  ‘A precedes B;’ ‘A is not preceded by B,’ etc.
= a reasoning task.
At the same time, you are to repeat aloud certain items; e.g. a list of 6 random numbers: ‘8, 2, 7, 1, 5, 9’
 = articulatory rehearsal for a recall task.

Results
Easy rehearsal and reasoning tasks can be performed at the same time. (= separate resource pools or functions)

Hard rehearsal tasks slow reasoning down (= drain executive resources)
Converging Evidence from Neuroscience
Posner and Raichle cite brain imaging studies to show an executive attention network.

It achieves goals, carries out instructions, organizes thoughts, guides behavior, chooses alternatives, creates new responses, and bring things to consciousness
Where Is It?
Primarily in the anterior cingulate gyrus.
Evidence
The anterior cingulate becomes active when people have to attend carefully and expend some effort at a task.

E.g. generating verbs to nouns, finding words in one category on a list (naming animals).
How Do We Know What the Anterior Cingulate is Doing?
Important clue:  reaction times increase the more target words there are on the list, but not when the total number of words gets larger.

All words are processed semantically.  Target words are processed in another way.
 

The Stroop Test
WHITE YELLOW  PURPLE
 BLUE GREEN

Executive attentional control inhibits the automatic response to read the word name.
Brain Damage
Lesions to the executive region produce akinetic mutism.

“It is as though they have no goals or no mechanisms to carry out goals” (P & R)

“Nothing ever came to mind.”

The Visuo-Spatial Sketchpad
Shepard’s image rotation studies

A kind of resemblance theory …
 But at a higher level.
 

Comparative Arts and Cognitive Science
Cross-modal plasticity
Synesthesia
Shepard’s universal invariants

VIII.  Review
What Is A Picture and How Does It Represent?
What distinguishes pictures from other types of symbol?

What distinguishes one picture from another?
What Distinguishes Pictures from Other Types of Symbol?
Goodman:  repleteness
Gibson:  invariant properties
Danto:  access to content through perceived similarities
Schier:  access to content through unlearned recognitional ability.
What Distinguishes One Picture from Another?
Its denotation or referent.

The type of picture it is.
What Determines Reference?
Goodman:  special conventions
Gibson:  specific information (but …)
Danto:  perceiver’s interpretation
Schier:  a causal relation
What Determines Picture Type?
Goodman:  how it is used to sort properties
Gibson:  type of information it contains
Danto:  what it resembles
Schier:  what can be recognized in it
Evidence, Arguments?
Goodman:  cross-cultural differences, varieties of perspective techniques, perceptual plasticity
Gibson:  texture gradients (studies of attention?)
Danto:  modularity (successful belief-attributions?)
Schier:  RBC
Aesthetic Issues
Can there be a unified aesthetics?  Based on common mechanisms for pictures, music, poetry?

What is style in pictures?  If two pictures have the same referent and are of the same type, how might they differ in style?

END as of 2/26/01