BEHAVIORAL AND NEURAL INVESTIGATIONS OF PERCEPTUAL [PDF]

BEHAVIORAL AND NEURAL INVESTIGATIONS OF PERCEPTUAL AFFECT

by

Edward Allen Vessel

A Dissertation Presented to the FACULTY OF THE GRADUATE SCHOOL UNIVERSITY OF SOUTHERN CALIFORNIA In Partial Fulfillment of the Requirements for the Degree DOCTOR OF PHILOSOPHY (NEUROSCIENCE)

May 2004

Copyright © 2004

Edward Allen Vessel

ii

ACKNOWLEDGMENTS First and foremost, I would like to thank my advisor, Irving Biederman, for his professional, financial, and personal support. He gave me this wonderful opportunity to work on this incredibly interesting idea, and the confidence to believe that I could do great things with it. He provided extensive guidance during the design, execution, and analysis of the experiments, and very efficiently gave me feedback on drafts of this document. Most important of all, he put up with my many idiosyncrasies and enthusiastically encouraged my successes.

Special thanks are also due to Mark Cohen of the UCLA Brain Mapping Center for his extensive guidance and friendship through all the phases of the brain imaging study, including design, running subjects, data analysis, and interpretation.

I would also like to thank the members of my dissertation committee. Zhonglin Lu gave me assistance with brain imaging analysis and group statistics in addition to general guidance & support. My discussions with Richard Thompson about learning and habituation proved very helpful, as was his general guidance and support. Bartlett Mel had many helpful thoughts on the modeling of preference. Mitchell Earleywine provided helpful

iii

comments on the factor analysis of preference, as well as personal support in becoming a better writer and researcher.

In addition, I am indebted to a number of other USC faculty have helped me with aspects of this work. Bosco Tjan provided an algorithm for creating the spectrally equivalent images, assisted with the computation of RMS contrast, provided scripts for the LO localizer scan, and gave extensive assistance in brain imaging analysis and Linux system administration. Alan Watts and Larry Swanson provided valuable insights into peptide systems, cortical output, anatomy, and emotion. The simulated annealing algorithm used to design imaging sequences was designed with the assistance of Laurent Itti, and Laura Baker provided assistance with the multiple regression analyses used in both the behavioral and imaging experiments.

This work would also have been quite difficult without the extensive input and support from my fellow graduate students, and I would like to thank them. In particular, Michael Mangini engaged me in many helpful discussions on this work, assisted with the multiple regression analysis, and provided some much needed distractions. My officemate Marissa Nederhouser provided assistance with statistical analyses, good music, and interesting conversation over the years. Several undergraduates also assisted in running subjects

iv

and creating stimuli, including Michelle Greene, Henry Nguyen, Tiger Nguyen, Viet Nguyen and Ali Narayan.

This work would also not have been possible without extensive help from a number of researchers around the world, and they deserve my heartfelt thanks. Moshe Bar (currently at Massachusetts General Hospital) provided many helpful discussions on preference, priming, and scene perception, and assisted in my selection of imaging parameters and fMRI sequence design. David Glahn (currently at UTHSCSA) assisted with the imaging study design and analysis, particularly in the use of the AFNI software. At UCLA, graduate student Richard Albistegui-Dubois provided extensive instruction on and assistance with operating the magnet and data analysis, and both Zrinka Bilusic and Anne Firestine provided analysis help and assistance with the magnet. Craig Stark (JHU) provided assistance with the AFNI software and the region of interest (ROI) analysis. Geoff Boynton (UCSD) aided extensively in the design and analysis of the imaging experiment. Russ Poldrack (UCLA) provided assistance with impulse response functions and general analysis. Bob Cox (NIMH) provided help with the AFNI software, especially as it pertains to the general linear model, deconvolutions, and motion correction. Douglas Ward (Medical College of Wisconsin) assisted with the deconvolution of my imaging data using the program 3dDeconvolve. Stephen Smith and the rest of FMRIB group (Oxford University Center for

v

Functional MRI of the Brain) were a tremendous help with the analysis of my imaging data using the FSL software. Randy Buckner (WUSTL), also provided helpful analysis information. I would also like to thank Jean-Marc Fellous (UCSD Salk Institute) for bringing the opioid gradient to the attention of Dr. Biederman.

I would like to my heartfelt gratitude to my many friends who put up with me while working on these experiments, and for keeping me sane. In particular, my housemates Jason, Trystan, Ian, and Adam deserve thanks for allowing me to bring my work home with me, as do Roxanne, Ann, and Walid for interesting discussions and being great friends. Lastly, Shari Cha deserves special thanks for providing a peaceful place to write, helping me relax, keeping me motivated, and cheering my spirits!

The Dartmouth Summer Institute deserves acknowledgment for giving me a first chance to get some practical experience with imaging and analysis and for introducing me to a number of people that have been very helpful.

While this work was being completed, I was supported by a variety of sources, including an NIH Predoctoral Cognitive and Computational Training Grant to USC (T32 MH20003-05), and grants awarded to Irving Biederman from the Human Frontiers Science Program Organization (RG0035/2000B),

vi

The Army Research Office (MURI ARO DAAG55-98-1-0293), the National Science Foundation (IMSC NSF EEC-9529152), and the James S. McDonnell Foundation (99-53).

vii

TABLE OF CONTENTS

ACKNOWLEDGMENTS……………………………………………………………ii LIST OF TABLES………………………………………………...………………... x LIST OF FIGURES………………………………………………………………... xi ABBREVIATIONS……………………………………………………..………….xiv ABSTRACT……………………………………………..………………………. xvi I. INTRODUCTION………………………………………………………...……….1 II. A THEORY OF COGNITIVE AND PERCEPTUAL AFFECT………………. 9 Positive Affect in the Ventral Visual Pathway……………………………9 A Neurochemical Index of Interpretability…..…………………. 12 Behavioral Effects of Opioid Antagonists……………… 16 Opioid Systems in the Brain…………………………..… 17 Novelty, Priming, and Competitive Learning………………...…26 Perceptual Affect in a Systems Perspective…………………... 32 Predictions and Extensions………………………………………………37 III. VISUAL PREFERENCE………………………………………………………43 Background……………………………………………………………..… 43 Complexity vs. Interpretability……………………………………43 Effects of Previous Exposure…………………………………… 46 Arousal and Emotional Valence……………………………...….54 A Stimulus Set for Testing Positive Perceptual Affect………………...57 Calibration Experiment……………………………………...…… 57 Stimuli……………………………………......................... 58 Subjects…………………………………………...............58 Method………………………………………………..…… 59 Results………………………………………………….…. 61 IV. REPETITION……………………………………………………………….….65 Experiment 1: Effects of Repetition, Between-Subjects Design…….. 66 Methods………………………………………………………….... 66 Subjects………………………………………………….... 66

viii

Stimuli……………………………………………………... 67 Procedure…………………………………………….…… 67 Results…………………………………………………………..… 68 Conclusions from Experiment 1……………………………....... 73 Experiment 2: Selection of Image Sets for FMRI…………………...…75 Methods………………………………………………………….... 76 Subjects………………………………………………….... 76 Stimuli……………………………………………………... 76 Procedure…………………………………………………. 79 Results………………………………………………………….…. 80 Conclusions from Experiment 2………………………………....88 V. ENVIRONMENTAL UTILITY……………………………………………….... 90 An Evolutionary Account of Scene Preference……………………….. 90 Experiment 3: Predicting Scene Preference from Environmental Utility……………………………………………………. 99 Methods………………………………………………………….... 99 Stimuli………………………………………………………99 Subjects………………………………………………..… 100 Procedure……………………………………………...…101 Results………………………………………………………...….106 Individual Factor Results……………………………..…106 Consistency Results………………………………….… 113 Multiple Regression Results…………………………....114 Conclusions & Discussion………………………………………117 Biophilia and the Preference for Natural Environments…………………………………………...120 VI. FMRI INVESTIGATION OF SCENE PREFERENCE……….………….. 125 Background……………………………………………………………… 125 Experiment 4: Neural Correlates of Perceptual Affect……………… 131 Methods………………………………………………………….. 132 Subjects………………………………………………..… 132 Apparatus………………………………………………... 132 Imaging Protocol………………………………………... 133 Preference Experiment………………...………………. 135 Procedures…………………………………….… 135 Trial Sequence………………………………….. 137 LOC Localizer…………………………………………… 139 Procedures………………………………….…… 139 Analysis and Results……………………………………...……. 140

ix

Single Subject Analysis Procedure………………...…. 140 Group Analysis: Whole Brain Voxelwise ANOVA…… 143 Group Analysis: Cluster-Level Significance……..……150 Selection of a Subset of Subjects……………...158 Group Analysis: Regions of Interest ANOVA…………163 Individual Differences in Activation Patterns………….169 Repetition “Rebound”……………………………….... 172 Image-Based Correlation Analysis……………………. 173 Conclusions ……………………………………………………. 181 VII. CONTRIBUTIONS AND FUTURE DIRECTIONS…………………..….. 184 Contributions……………………………………………………………..184 Future Directions………………………………………………………...186 Behavioral Studies……………………………………………....187 Eyetracking……………………………………………………….188 Individual Differences in the Neural Correlates of Preference…..………………………...……………………….. 189 Pharmacology…………………………………………...……….190 Natural vs. Urban: Distribution of Image Factors in the Ventral Visual Pathway……...…………………………..…….190 Learning Effects…………………………………………….……192 Cross-Modal and Conceptual Priming………………………...193 Direct Measures of Preference………………………………... 194 BIBLIOGRAPHY………………………………………………………………... 195

x

LIST OF TABLES

Table 4.1. Correlation of Average Preference Ratings for Experiments 2a and 2b………………………………………………………………………..…86 Table 5.1. Factor Summary Statistics……………………………………...... 107 Table 5.2. Consistency of Factor Ratings…………………………………… 113 Table 5.3. Correlates of Preference: Multiple Regression Results……….. 114 Table 5.4. Correlates of Preference: Part Correlations for Each Model…..116 Table 6.1. Summary of BOLD Effects: Preference and Repetition……..…162 Table 6.2. Correlation with Preference by ROI……………………………... 179

xi

LIST OF FIGURES

Figure 2.1. The Ventral Visual Pathway and Object Recognition Speed…..11 Figure 2.2. A Gradient of µ Opioid Receptors in the Ventral Visual Pathway…………………………………………………………………… 13 Figure 2.3. Cortical Areas Comprising the Ventral Visual Pathway……...…14 Figure 2.4. Pathways from Perceptual Affect to Behavior…………………...35 Figure 2.5. Predictions of a Theory of Cognitive and Perceptual Affect……38 Figure 3.1. Droodles: An Illustration of Rich Interpretability………………....46 Figure 3.2. Effects of Mere Exposure on Preference Judgments………….. 48 Figure 3.3. Berlyne’s Evidence for Habituation of Preference……………....51 Figure 3.4. Distribution of Average Preference Scores……………………... 62 Figure 3.5. Normal P-P Plot of Average Preference Ratings.…………….... 62 Figure 3.6. Most and Least Preferred Scenes………………………………...63 Figure 4.1. Exp. 1: Trial Sequence……………………………………………..68 Figure 4.2. Exp. 1: Preference vs. Trial Number……………………………...69 Figure 4.3. Exp. 1: Average Preference vs. Exposure………………………. 70 Figure 4.4. Exp. 1, Block 1: Average Preference Score vs. Exposure, Grouped by Initial Preference…………….…………………………………….. 70 Figure 4.5. Exp. 1: Results for Practice Block………………………………... 71 Figure 4.6. Exp. 1: Results for Block 2…………………………………………72 Figure 4.7. Exp. 2a: Distribution of Preference Ratings by Image Category……………………………………………………………………..……. 77

xii

Figure 4.8. Exp. 2b: Distribution of Preference Ratings by Image Category………………………………………………………………………..…. 78 Figure 4.9. Exp. 2: Change in Average Preference Over Trial Blocks…..… 81 Figure 4.10 Exp. 2a: Preference vs. Exposure for Images Categorized by A Priori Preference……………………………………………………………….82 Figure 4.11. Exp. 2a: Data from Eight Individual Subjects………………….. 84 Figure 4.12. Exp. 2b: Preference vs. Exposure for Images Categorized by A Priori Preference…………………………………………………………..……85 Figure 4.13. Exp. 2a: Sex Differences in Preference vs. Exposure………...86 Figure 4.14. Exp. 2b: Sex Differences in Preference vs. Exposure………...87 Figure 5.1. Komar & Melamid’s Most and Least Wanted Paintings……...…98 Figure 5.2. Examples of Spectrally Equivalent, Phase Scrambled Images…………………………………………………………………………… 105 Figure 5.3. Scatterplots of Preference vs. Factors…………………………. 107 Figure 5.4. Examples of Scenes rated High and Low on Each of the Six Image Factors………………………………………………………………. 110 Figure 6.1. A General Linear Model Used to Analyze Single Subject Data…………………………………………………………………………….....142 Figure 6.2. Average Brain Activation Across All Conditions (Compared to No Image)…………………………………………………..…...146 Figure 6.3. Effects of Preference…………………………………………...…148 Figure 6.4. Effects of Repetition…………………………………………….... 149 Figure 6.5. Low Preference Minus High Preference for All Sixteen Subjects………………………………………………………………………..… 152 Figure 6.6. First Presentation Minus Fifth Presentation for All Sixteen Subjects………………………………………………………………………….. 153

xiii

Figure 6.7. Rebound of Activity in Early Visual Areas for Later Repetitions………………………………………………………………………. 155 Figure 6.8. The Effects of Repetition and Preference in Parietal Cortex……………………………………………………………………………..156 Figure 6.9. Effect of Repetition in the Right Superior Frontal Gyrus……... 158 Figure 6.10. Voxelwise Overlap of Subject’s Preference Effects…………. 160 Figure 6.11. High Preference Minus Low Preference for Nine Subjects….161 Figure 6.12. Region of Interest (ROI) Analysis for Nine Subjects………… 166 Figure 6.13. Left Parahippocampus ROI: Canonical HRF Model Fit……...169 Figure 6.14. Preference Effects for Two Individual Subjects…….………...170 Figure 6.15. Preference Effects for a Third Individual Subject………….… 171 Figure 6.16. Rebound Effects are Not Due to Stimulus Timing…….…….. 173 Figure 6.17. Image-Based Correlation Analysis for Four ROI’s…………... 180

xiv

ABBREVIATIONS AC

Anterior Commissure

AFNI

Analysis of Functional NeuroImages software package

ANOVA

Analysis of Variance

BA

Brodmann’s Area

BOLD

Blood Oxygen Level Dependent

DAGO

[D-Alanine-N-Methyl-Phenylalanine, Glycinol] Enkephalin

EM

Endomorphin

EMG

Electromyography

EPI

Echo-Planar Imaging

ER-FMRI

Event-Related FMRI

FILM

FMRIB’s Improved Linear Model

FLIRT

FMRIB’s Linear Image Registration Tool

FMR-A

FMRI Adaptation

FMRI

Functional Magnetic Resonance Imaging

FMRIB

Oxford Center for Functional Magnetic Resonance Imaging of the Brain

FSL

FMRIB Software Library

FWHM

Full Width at Half-Maximum

GABA

Gamma-Amino-n-Butyric Acid

GLM

General Linear Model

HRF

Hemodynamic Response Function

xv

IAPS

International Affective Picture System

IRF

Impulse Response Function

IT

Inferotemporal Cortex

LCD

Liquid Crystal Display

LO

Lateral Occipital – The posterior section of the LOC

LOC

Lateral Occipital Complex

LSF

Least-Squares Fitting

MNI

Montreal Neurological Institute

ms

millisecond(s)

mRNA

messenger Ribonucleic Acid

PC

Posterior Commissure

PET

Positron Emission Tomography

PF

Posterior Fusiform – The anterior section of the LOC

PHG

Parahippocampal Gyrus

RMS

Root Mean Squared (contrast)

ROI

Region of Interest

RSVP

Rapid Serial Visual Presentation

SPM99

Statistical Parametric Mapping 99 software package

TE

The anterior section of inferotemporal cortex

TEO

The posterior section of inferotemporal cortex

TR

Repetition Time

VTA

Ventral Tegmental Area

xvi

ABSTRACT

People prefer some perceptual inputs to others, an effect readily manifested in visual fixations during free viewing. This preference may be based on the activity of a gradient of µ opioid receptors (ligand = endomorphin) that, surprisingly, is found in the ventral cortical pathway for visual recognition. This gradient, discovered in the macaque, is sparse in V1 and increases in density through V2, V4, TEO, IT and the parahippocampal cortex. The magnitude of endomorphin activity would determine perceptual and cognitive preference, resulting in a preference for patterns that are both richly interpretable (because they activate many associations in the opioid rich anterior regions of the ventral pathway) and novel. Repetition of a scene would result in less activity because of competitive interactions. Using a set of full color scenes calibrated for their a priori preference and presented for one second, repetition led to decreases in preference for all levels of a priori preference. In addition, the environmental utility afforded by a scene (such as its naturalness, or the presence of a good view) was a good predictor of a priori preference, suggesting that preferences may, in part, reflect an evolutionarily conserved mechanism for the rapid assessment of one’s environment. Using ER-fMRI, we recorded brain activity as subjects attended to one second presentations of scenes that varied in their rated preference and were repeated five times. Highly preferred and novel scenes produced

xvii

greater BOLD activity than less preferred and repeated scenes in more anterior areas of the ventral pathway, such as the parahippocampal cortex, consistent with the endomorphin hypothesis. In contrast, activity in the lateral occipital complex (LOC) was inversely related to preference, and early visual areas generally showed an inverted-U shaped function with repetition, likely reflecting an active reexamination of the visual scenes. Surprisingly, frontoparietal networks associated with voluntary attention were more active for later repetitions and low preference images, suggesting that endogenous attentional effort is required to attend to stimuli which do not, on their own, provide a novel, richly interpretable experience.

1

CHAPTER 1: INTRODUCTION

A casual survey of what a person spends their day doing reveals a very large role for what are commonly referred to as “play” or “exploratory” behaviors that have little or no immediate survival value. While most evident in human behavior, exploratory behaviors which are not in the service of other drive states have also been documented in nonhuman primates and, to a lesser extent, rodents. For example, Harlow describes the development of what he calls “curiosity motivation” in four stages starting immediately after birth, beginning with reflexive orienting and leading to aggressive object manipulation and interaction with other animals (Harlow, 1986, 182).

Clearly we engage in behaviors that can be seen as motivated by drives for thirst, hunger, and the avoidance of harm. Yet such commonplace pastimes as going to a movie, reading a book, engaging in a conversation, visiting a museum, listening to music, surfing the internet, or driving down a scenic coast highway hardly seem, at first glance, to be understandable within the context of a drive state. Even when one considers activities which do fit within the standard drive reduction paradigm, which restaurant we eat at, who we choose to court, and where we go to enjoy a beverage with that chosen someone are decisions we make tens of times a day in the absence of any discriminable survival value for one choice over another.

2

On a shorter timescale, the moment to moment shifts in where we choose to look about our visual world are also not predictable from standard drive states, yet are not random either. Clearly, certain parts of a visual scene, and more broadly speaking, certain types of information, are preferentially selected over others. For example, a person walking into a room in which he or she has never been does not stare at blank sections of walls, but instead fixates regions of “interest.” Previous attempts to understand how regions of a visual scene are spontaneously selected have focused either on the poorly defined concept of complexity (e.g. Berlyne, 1958) or on saliency (e.g. Itti, Koch & Niebur, 1998). Yet neither complexity, which lacks generalizability across multiple stimulus types, nor feature based saliency, which fails to incorporate higher level relationships among the elements of a scene, can account for human spontaneous visual selection (Kaplan, 1992; Oliva et al., 2003; Heidenreich & Turano, 2003).

We propose that spontaneous visual selection is an example of another, very basic drive separate from those commonly defined in the study of motivated behaviors: information foraging. More specifically, humans engage in many behaviors in order to maximize the rate of acquisition of novel, yet richly interpretable information: we are “infovores.” This drive, which likely operates continuously in the absence of more pressing biological needs

3

(such as harm avoidance or hunger), is unlike the traditional drives in that information foraging behavior has no clear end point: one does not simply become “satiated” with information and stop foraging. Instead, it is likely that an information seeking system would be wired in such a way that the acquisition of information itself acts as a reward, thus leading to the “winshift” behavior typical of information seeking behaviors.

The importance of an information foraging system stems from the fact that it is often impossible to know what aspects of our environment contain potentially useful information for future decisions. Humans have a vast capacity for remembering specific episodes of experience, as well as the names and appearance of objects, people, and places we encounter. The decisions we make at any moment are informed by a combination of sensory cues collected at that moment and a vast repository of experience. By selectively attending to those aspects of an environment that strike a balance between novelty and interpretability (e.g. relatability to previous memories), an information foraging system allows for the preferential encoding of information which is maximally beneficial for making life saving decisions at a later time. Such a system likely evolved alongside the dramatic expansion of the mammalian brain as survival strategies shifted focus away from largely instinctual reactions towards solutions which depended on general problem solving abilities and the observation of information patterns over long periods

4

of time. As an example, consider a situation in which an early hominid might casually observe a friendly interaction between two other individuals. While that bit of information may have no bearing on his current situation, it would become relevant if he were to consider a fight with one of the individuals later. Not only would he have to contend with one person, but he now knows that he might potentially have to contend with the second person as well. In the words of Stephen Kaplan, “when drives are satisfied, when nothing else is going on would be precisely the time when the individual could be obtaining information about the environment for use at a later time.” (Kaplan, 1992, pp. 584)

In the next chapter, we present a theory for how an information foraging system could be built into the architecture of the brain’s perceptual systems. A key element to understanding information foraging as a drive state is that attaining new information is experienced as pleasurable: every act of perception, by virtue of its information content, gives rise to an affective response. We suggest that this affective response may be mediated by a gradient of µ opioid receptors in perceptual pathways. In this way, the activity in perceptual systems mediating the recognition of interpretable information can influence a wide range of behaviors and learning processes, such as moment to moment eye movements, spontaneous preferences for certain scenes over others, the formation of long term aesthetic dispositions,

5

and the identification of likely sources of information in one’s environment (such as TV, urban centers, movies, and other people).

Stephen Pinker, in his 1997 book “How the Mind Works,” provides another poignant example for how built-in information foraging biases and links to preference would have been critical for the survival of early hominids. Quoting from Tooby & Cosmides (1992):

Imagine that you are on a camping trip that lasts a lifetime. Having to carry water from a stream and firewood from the trees, one quickly learns to appreciate the advantages of some campsites over others. Dealing with exposure on a daily basis quickly gives one an appreciation for sheltered sites, out of the wind, snow, or rain. For hunter-gatherers, there is no escape from this way of life: no opportunities to pick up food at the grocery store, no telephones, no emergency services, no artificial water supplies, no fuel deliveries, no cages, guards, or animal control officers to protect one from the predatory animals. In these circumstances, one’s life depends on the operation of mechanisms that cause one to prefer habitats that provide sufficient food, water, shelter, information, and safety to support human life, and that cause one to avoid those that do not.

Pinker makes explicit the direct analogy between human preferences, as studied within the field of environmental aesthetics, and the biological study of habitat selection: people enjoy being in some places more than others (Pinker, 1997, Ch. 6).

6

As a means of assessing this system, we have chosen to investigate visual preference behavior and its neural correlates. While there exists only a minimal literature on direct investigations of visual preference, a variety of different disciplines have touched on related issues of information foraging or emotional responses to stimuli. In Chapter Three, we discuss some of the wider influences on visual preference that can be gleaned from some of these fields. Early studies of preference and orienting behavior by Berlyne and his colleagues and a large number of studies documenting the effects of previous exposure on affective responses to stimuli provide the most relevant background within cognitive psychology. Preferential looking has been used as a method to assess the development of visual abilities in infants for decades with a very limited understanding of why infants show such preferences in the first place. More recently, studies of visual attention and high level scene perception have investigated the contributions of low level features, scene gist and informativeness to spontaneous visual selection, yet without any reference to the affective consequences of these processes. A majority of the relevant findings on visual preference are to be found in the field of environmental psychology and landscape assessment. Stephen Kaplan’s influential work has provided a bridge between the practical need to develop an understanding of landscape management and the deeper theoretical implications of observed preferences for certain types of landscapes over others. Finally, the effects of stimulus valence and arousal

7

on positive and negative affect have been studied in the context of scenes and faces. Yet the main focus of this body of research has been on the induction or perception of negative emotions and the role of the amygdala in these responses. Very few of these studies even address positive affect, or have done so using inadequate stimulus sets or with a focus on highly arousing stimuli (such as erotica). Unlike these approaches, we have chosen to focus on low valence, mildly positive affect as a method for studying the properties of information foraging separate from other drive states responsible for avoidance of harm or sexual excitement.

Within these various domains, affective responses to visual stimuli have been assessed using eye movements, physiological measures, and rated preferences, yet none of these fields have provided an adequate exploration of the links between perception and affective preference, nor an explanation for the brain mechanisms underlying the affective response to potentially interesting aspects of the environment. In Chapters Four and Five, we present a series of behavioral experiments which explore human visual preference using full color scenes. We demonstrate that preferences for visual scenes are highly predictable across subjects and that increased exposure produces a reliable decrease in scene preference. In addition, we show that the interpretability of a scene, and hence its preference, may be largely understood within the context of the evolutionary value of certain

8

scene characteristics such as the presence of natural elements, the degree of mystery, and the presence of an expansive view. Such “built-in” preferences are those which would have been critical to our early hominid ancestors for successful habitat selection. In Chapter Six, we present imaging data supporting the claim that differential activity distributions in the ventral visual pathway give rise to differences in stimulus preference, a result which is in agreement with the hypothesis that µ opioid receptors may mediate the link between perception of highly interpretable information and positive affective responses. Finally, in Chapter Seven, we outline several directions for future research and summarize our contributions within the field of perceptual affect.

9

CHAPTER 2: A THEORY OF COGNITIVE AND PERCEPTUAL AFFECT

When you walk down a busy street, hike along a mountain path, or walk into a room in which you have never been before, where do you look and what mechanisms determine this? We propose a neurocomputational theory of cognitive and perceptual affect based upon a gradient of µ opioid receptors in the primate visual pathway mediating recognition. This theory provides an explanation for how the perception of novel, richly interpretable information can provide a signal used for both the spontaneous visual selection of “regions of interest” and the accompanying positive affective reaction elicited by information foraging.

Positive Affect in the Ventral Visual Pathway

What scenes we choose to look at and where we look within a scene are decidedly nonrandom activities. On the order of three times a second, we make saccades around our visual world which are clearly not random. We do not stare at blank walls nor uninterpretable masses, are fascinated by art and computer animation, and gaze longingly at scenery. That is, we don’t look at regions where nothing is happening nor do we look at regions of uninterpretable variation. Instead we look at “regions of interest.” But how can we understand “interest?” In addition, we generally show a preference

10

for novelty, preferring a new book, movie, or conversation to one experienced previously. Once a particular source of information has been adequately explored, there is very little benefit from revisiting that same source in the near future. How is this “win-shift” behavior implemented in the brain? The challenge posed by these questions is to understand how a motivational system can be built into a real time perceptual-cognitive system that allows for the exploration of new aspects of one’s environment that are not so foreign as to be uninterpretable.

A major constraint imposed on the possible instantiations of such a system is the ability to control real-time eye movements based upon an affective evaluation of the regions of one’s visual environment. Feedforward activity in the macaque ventral visual pathway is estimated to reach ventral temporal cortex on the order of 80 to 100 milliseconds, with eye movement planning and execution occurring within 250 milliseconds (Thorpe & Fabre-Thorpe, 2001; see Figure 2.1). These estimates are slightly longer for humans, where differential EEG activity for categorization tasks has been observed at 150 milliseconds, with behavioral responses typically occurring on the order of 400 milliseconds (and as fast as 250 ms) after stimulus onset (Thorpe, Fize & Marlot, 1996; Thorpe & Fabre-Thorpe, 2001). Therefore, this component of the affective response to a perceptual stimulus cannot rely on lengthy cognitive operations nor feedback from limbic structures, but instead

11

involves signals arising directly from perceptual processing in a feedforward manner. It is not an “afterthought” or attributional process, but an integral aspect of perception, which by virtue of its immediate nature and links to the brain’s effector systems, guides spontaneous visual selection about one’s environment and produces other measurable indices of positive affect (Winkielman & Caccioppo, 2001; Winkielman et al., 1997; Zajonc, 1980).

Figure 2.1. The ventral visual pathway is able to extract the form of visual objects and scenes in under 150 ms. Reprinted with permission from Thorpe & Fabre-Thorpe, Science, 291:260-63 (2001). Illustration: Carin Cain. Copyright 2001 American Association of the Advancement of Science.

12

A Neurochemical Index of Interpretability

We posit a specific function for the gradient of µ opioid receptors discovered by Lewis and his colleagues in the ventral visual pathway of the macaque monkey as mediators of this positive perceptual affect (Lewis et al., 1981; Wise & Herkenham, 1982). Using radioactively labeled naloxone ([3H]naloxone), a potent opiate antagonist, they quantified the relative amounts of the µ opioid receptor in different cortical and subcortical regions of a rhesus (Macaca mulatta) monkey. As can be seen in Figure 2.2, the amount of naloxone binding was found to increase monotonically from cortical areas V1, V2, V4, TEO, IT, fusiform cortex (TF), parahippocampal cortex (TH), and periamygdalar cortex (A) on both the medial and lateral surfaces. Note that the labels for the cortical fields used by the authors differ from the currently used monkey designations, with OC corresponding to V1, OB roughly to V2, and OA roughly to V4. In contrast, the distribution of the δ opioid receptor, as assessed by binding of tritiated Leu-enkephalin was found to be uniformly distributed across cortical regions, indicating that the µ receptor might be involved in a specific aspect of perceptual processing.

13

Figure 2.2. The density of µ-like opioid receptors as labeled by tritiated naloxone is relatively low in early cortical visual areas (OC = striate cortex, OB = peristriate, OA = preoccipital) and gradually increases on both the lateral and medial surfaces of the ventral visual pathway. Reprinted with permission from Lewis et al., Science 211:1166-1169 (1981). Illustration: ??. Copyright 1981 American Association for the Advancement of Science.

In humans, the ventral visual pathway is greatly expanded over that of the macaque, and includes a large swath of tissue extending from the occipital pole along the ventral, lateral, and medial aspects of the brain into the temporal pole. This includes the lingual and parahippocampal gyri on the medial aspect, the fusiform gyrus, and the inferotemporal gyrus on the lateral aspect. Figure 2.3 shows a ventral view of the human brain with the cerebellum removed and several of these structures labeled.

14

Figure 2.3. The ventral aspect of a human brain with the cerebellum removed. The ventral visual pathway includes the lingual (22) and parahippocampal gyri (17) medially, the fusiform gyrus (18) along the center, and the inferior occipital (23) and inferior temporal gyri (19) on the lateral aspect. Adapted from Bergman et al., 2001.

Why would opioid activity be associated with a perceptual pathway? The exact role of these receptors is not clear, but the demonstration that endogenous opioids often play an important role in motivation, reinforcement, and pleasure raises an interesting possibility. Noting a possible parallel with the gating function of opioids in the spinal cord, Lewis et al. speculate that “opiatergic neurons may be involved in the filtering of sensory stimuli at the cortical level, and thus play a role in selective attention.” (Lewis et al., 1981, 1168). The crucial aspect of this discovery is the direct match between the density of µ opioid receptors and the sequence of processing stages of the

15

ventral visual stream underlying the perception of visual form, often referred to as the “what” pathway (Ungerleider & Mishkin, 1982; Macko et al., 1982; Goodale & Milner, 1992). The assumption is that as the activity from earlier visual areas is passed on to later cortical fields, this activity leads to the release of endogenous opioids and subsequent binding to µ opioid receptors. The significance of finding a gradient that is most dense in association cortex is profound: this pattern suggests that maximal µ opioid activity (and hence, reinforcement) would result from visual experiences that lead to a high degree of interpretation. Stimuli which are merely “busy” but not richly interpretable would provide little pleasure. Our proposal is that richly interpretable perceptual inputs would lead initially to the most neural activity – and hence endogenous opioid activity – in the anterior stages of the ventral visual pathway (IT and parahippocampal cortices). High activity would then be associated with high preference (or interest).

As will be described below, once this activity is unleashed in the endomorphin-rich association areas, competitive learning sets in, resulting in a diminution of the activity when the input is repeated resulting in reduced preference. The net effect is that the strongest preference will be manifested for novel, richly interpretable stimuli.

16

A similar gradient was discovered from primary to secondary auditory cortex, and some evidence of such a gradient was seen in somatosensory areas as well. Such gradients may provide a neurochemical index of interpretability by which activity in perceptual pathways can influence selection mechanisms and lead to affective responses in an immediate and bottom-up manner.

Behavioral Effects of Opioid Antagonists

Investigations of µ opioids in humans using the antagonist naloxone are consistent with a possible role for this system in perceptual affect. Reports from subjects that have been administered naloxone both in research and clinical settings indicate no pervasive effects on mood, though there are occasional reports of anhedonia (flattened affect) in which the world seems devoid of much of its normal excitement, and dysphoria (unhappiness) particularly in situations in which the subject is required to remain passive (Arnsten et al., 1984). Naltrexone, a related, milder opiate antagonist, is currently in use as a treatment for severe alcoholism because of its tendency to reduce the rewarding aspects of alcohol that are responsible for sending an alcoholic on a binge (Volpicelli et al., 1992). It appears as if the opiate antagonist slightly reduces the subject’s initial desire to have a drink, but also strips the alcoholic experience of its pleasing aspects which, in turn, reduces the desire.

17

Naloxone has also been investigated with regards to its effects on auditory stimuli. Goldstein found that, in some subjects which often experience “chills” in response to music, the administration of naloxone attenuated those chills (Goldstein, 1980).

Opioid Systems in the Brain

There likely exist many different functional brain systems which are influenced by opioid mediated signaling, and while a majority of the work on these systems to date has been on the subcortical pathways involved in nociception, reward, and stress, the presence of opioid receptors and endogenous opioid ligands in the cortex is well established (Lewis et al., 1981; Lewis et al., 1983). Three main opioid receptor types, µ, δ, and κ, have been identified across different mammalian species and their cortical and laminar distribution studied using radioreceptor assays, receptor autoradiography, and more recently, in situ hybridization studies of receptor mRNA (Peckys & Landwehrmeyer, 1999; Hiller & Fan, 1996; Mansour et al, 1995; Lewis et al, 1983; Wise & Herkenham, 1982; Lewis et al, 1981). The subcortical and cortical distribution of the three receptor types, which all belong to the superfamily of G-protein-coupled receptors (Peckys & Landwehrmeyer, 1999), are quite different, implicating them in a variety of

18

systems with widely ranging function. Most opioid systems investigated to date have found the receptors to be coupled to inhibitory G proteins that ultimately produce a decrease in neuronal excitability through some combination of activation of inwardly rectifying potassium channels and decreases in calcium channel conductance (Peckys & Landwehrmeyer, 1999; Hille, 1992, 186-7).

The initial discovery of the µ opioid receptor in nervous tissue in 1973 marked a revolution in the understanding of how drugs have their effect on the nervous system, and began the race for the discovery of the natural, endogenous ligands for the receptors (Pert & Snyder, 1973; Zadina et al., 1999). A number of nonspecific endogenous ligands, including β-endorphin (which binds to µ and δ), and ligands with higher affinity and selectivity for the δ and κ receptors (such as Met-enkephalin, Leu-enkephalin and dynorphin, which show weak µ activity but primarily bind to δ, δ, and κ respectively) were discovered in the seventies (Khachaturian et al., 1985; for a review, see Corbett et al., 1993). But it was not until 1997 that two potent, selective endogenous agonists for the µ receptor were discovered in the human brain (Zadina et al., 1997; Hackler et al., 1997; Zadina et al., 1999; Zadina, 2002). Termed endomorphins (short for “endogenous morphine”), these tetrapeptides show distributions in the central nervous system consistent with the localization of µ opioid receptors and a role in a diversity

19

of functions including pain and stress gating, reward, and other autonomic and endocrine functions (Martin-Schild et al., 1999; Zadina et al., 1999). Below, I will briefly review the known cortical distributions of µ opioid receptors and their possible ligands in rats, monkeys, and humans.

Much of the early work on opioid receptors sought to understand the systems underlying morphine’s analgesic ability, and therefore focused on elucidating the spinal and brainstem opiatergic systems. Yet clinical observations of patients on morphine clearly indicate that it has effects on “higher” processes such as attention, mood, and motivation (Buchsbaum et al., 1982). These observations led Michael Lewis and his colleagues at NIMH to reexamine the cortical distribution of opioid receptors in light of known functional cortical systems. As discussed above, the most striking aspect of their findings was the graded increase in naloxone binding from primary sensory areas to higher sensory and association areas. These gradients peaked in periamygdaloid cortex on the medial surface of the temporal lobe (Lewis et al., 1981). Differences in naloxone binding were also observed in the frontal cortex. The lowest amount of binding was seen in the motor strip, while orbitofrontal cortex showed a high degree of binding. The olfactory tubercle, was also associated with strong naloxone binding.

20

The following year, another group at NIMH reexamined naloxone binding in the macaque monkey with an interest in characterizing the laminar patterns of opioid receptor distributions, a feat not possible in the earlier study, which used brain homogenates (Wise & Herkenham, 1982). They reported that the ventral temporal and orbital frontal fields had among highest levels of binding in the cortex, with “progressively higher levels of opiate binding were observed with increasing proximity to the most primitive type of cortex.” (Ibid., 218). The majority of cortical binding was strongest in layer V; in the striate and peristriate cortex, they saw enriched naloxone binding in layers V and VI, as well as some binding in layer IV as well. Conversely, the frontal eye fields had the strongest binding in layer III. Since layer V is the predominant source of corticofugal projections out of cortex, the authors suggested that a main effect of cortical µ opioid receptor activity is to influence the outflow of cortical fields, with an emphasis on polymodal information procession and limbic function. Studies in rat show rough agreement with the distribution of µ opioid receptors across cortical fields, though there are species differences in the lamination patterns (Lewis et al., 1983).

The distribution of opioid receptors in humans have since been investigated using both receptor radiography and mRNA immunohistochemistry, with somewhat conflicting results. A radiographic study using tritiated DAGO, a potent µ agonist with greater selectivity than naloxone, found µ receptors to

21

predominate in layer III (especially IIIa), with a second peak of density in either layer V (frontal) or IV (temporal and occipital; Hiller & Fan, 1996). On the other hand, mRNA for the µ receptor was found to predominately label pyramidal cells in layer V (Peckys & Landwehrmeyer, 1999), though this study did not look at as many different cortical divisions as the radiographic study. The authors of the latter study suggest that while the overall amount of opioid mRNA appeared to be lower in human than rat, the relative amounts in cortex and hippocampus appear much higher in human.

While an earlier study of DAGO binding in human cortex reported that µ receptors are most concentrated in superficial layers (I and II), both of the more detailed DAGO studies are in good agreement on the issue of the µ receptor distributions in the ventral visual pathway. Referring to the gradients reported by Lewis et al. (1981 and 1983), Quirion & Palapil state that “similar organizations may exist in human brain cortex.” (Quiron & Palapil, 1991, 104) These studies found virtually no evidence for µ receptors in striate cortex, with little or no binding seen in area 18 (referred to as lateral occipital cortex by Hiller & Fan) either. The lateral occipitotemporal gyrus (BA 37) showed an intermediate degree of binding and the inferior temporal gyrus (BA 20) showed one of the highest degrees of µ opioid receptor labeling. The parahippocampal gyrus also showed a high degree of DAGO binding (highest in layer IV followed by layer III), though not as high as the

22

inferior temporal or medial temporal (BA 21) gyri (Hiller & Fan, 1996; Quiron & Palapil, 1991). The hippocampus proper and dentate gyrus contain very few µ opioid receptors, a finding that is quite contrary to the animal literature, while high levels are seen in all divisions of the amygdala, cerebellum, striatum, globus pallidus, nucleus accumbens, and other nuclei of the basal forebrain (Quiron & Palapil, 1991).

In comparison to the level of detail available on the distribution of opioid receptors, the current evidence available on the distribution of the endogenous ligands for these receptors is quite sparse. This is especially true for the µ receptor given the recency of the discovery of potent, selective endogenous ligands. Reports on the cortical distribution of EM1 and EM2 (two different forms of endomorphin) in the human brain are lacking, and studies in rat have focused primarily on subcortical structures. What is known is that there are some clear differences in the distribution of EM1 and EM2; EM2 is more prevalent in the spinal cord, while EM1 is more widely distributed in areas associated with reward, arousal, limbic and cognitive functions (Martin-Schild et al., 1999).

In the rat, fibers containing EM1 have been visualized in frontal and limbic cortices, all divisions of the amygdala, nucleus accumbens, septal nuclei, the diagonal band of Broca, bed nucleus of the stria terminalis, midline thalamic

23

nuclei, locus coeruleus, and the striatum (Zadina et al., 1999; Zadina, 2002). The highest densities of EM1 immunoreactive fibers are in the posterior hypothalamus (dorsomedial, ventrolateral, and arcuate nuclei) and the nucleus of the solitary tract, which are also the only observed locus of cell bodies containing EM1 immunoreactivity (Martin-Schild et al., 1999). However, the authors stress that visualization of immunoreactive cell bodies depends on the administration of colchicine to prevent axonal transport. Their administration of colchicine into the lateral ventricles may not have been sufficient to allow for the visualization of EM containing cell bodies in parts of the brain not immediately surrounding the ventricles, making it likely that additional cells containing endomorphin will be identified in future studies. In addition, there are likely to be species differences in the distribution of EM containing cells. As an example of a clear inadequacy in our understanding of µ opioid systems, the authors point to the mismatch between the presence of µ opioid receptors in the striasomes of the rat striatum, yet an absence of any endomorphin-like immunoreactivity (Ibid.). It is therefore plausible that some µ opioid systems, including those in the cortex, may interact with some of the mixed affinity enkephalins1 or as of yet unidentified endomorphins.

1

These enkephalins have been shown to be synthesized in rat cortex (Khachaturian et al., 1985; Mansour et al., 1995) and to exist in human occipital and ventral temporal cortex (Hurd, 1996).

24

Given the presence of both µ and δ opioid receptors in cortex in differing distributions, it remains an interesting possibility that the release of enkephalin compounds from cortical and/or subcortical sources with varying µ/δ affinity ratios could provide a rich diversity of neurochemical modulation of sensory signals. Significant levels of proenkephalin mRNA (the precursor gene for Met- and Leu- enkephalin) have been observed in all regions of cortex, and their laminar distribution matches relatively well the distribution of human µ binding sites (Hurd, 1996; Quiron & Palapil, 1991). Moderate to high levels were observed in temporal, occipitotemporal, occipital, frontal and motor cortices and the olfactory gyrus. In general, the ventral visual pathway contained a quite high amount of proenkephalin mRNA, with an especially high concentration in the lingual gyrus. In contrast, the limbic cortices showed very little proenkephalin mRNA expression (Hurd, 1996).

The proposed role of µ opioids as mediators of increased perceptual pleasure is somewhat counterintuitive given the fact that most opioid signaling mechanisms are linked to inhibitory G proteins and therefore act to inhibit their targets. The available evidence on the distributions of opioid receptors within neurons indicates that opioids may act as neuromodulators in some cases and as direct neurotransmitters in other systems (Simon & Hiller, 1994), and it is likely that in many cases the cortical influence of opioid systems is through the presynaptic inhibition of neurotransmitter release

25

(Hiller & Fan, 1996). One possibility is that the co-release of endomorphin by a glutamatergic cortical neuron binds to µ opioid receptors on the presynaptic terminals of nearby GABAergic neurons. Activation of these neurons would then serve to reduce the tonic level of inhibition impinging back on the glutamatergic neurons in that part of the cortex. This arrangement is analogous to the disinhibitory role of opioid receptors in other brain systems that have been studied. For example, opioid receptors are found on GABAergic neurons in the rat hippocampus (Morris & Johnston, 1995; Cohen, Doze & Madison, 1992), and in the brain’s reward circuits, where the euphoric effects of morphine are thought to result from an inhibition of striatal GABAergic neurons and a subsequent disinhibition of their dopamine releasing target cells in the substantia nigra (Peckys & Landwehrmeyer, 1999; Mansour et al., 1995). Finally, dopamine releasing neurons in the VTA projecting to the nucleus accumbens are also thought to be released from GABAergic inhibition as a result of the activation of µ receptors on the GABAergic neurons, implicating a role for endogenous opioids in sexual behavior as well (L. M. Coolen, colloquium. USC, January 15, 2003).

Consequently, endomorphin activity in perceptual pathways likely results in a selective reduction of local inhibition, allowing greater activation of areas associated with reward such as the amygdala, orbitofrontal cortex, and subcortical dopaminergic pathways. This local increase might also potentiate

26

synaptic plasticity and play a key role in the network dynamics underlying competitive interactions.

Novelty, Priming, and Competitive Learning

As will be discussed more extensively in the next chapter, stimuli are often less preferred the second time they are experienced. While there may be some comfort associated with familiarity in the face of uncertainty, interest is much more often associated with novel environments, ideas, sounds, and pictures. Even in familiar settings, we still seek to assimilate novel information in picking up a book or turning on the TV or going to a movie or desiring a good conversation. The familiar setting may offer fewer distractions (i.e. competing sources of novel information) to these information rich sources. In addition to providing a basis for a preference of highly interpretable stimuli, our theory can also account for the general preference for novelty. Given the assumption that increased activity in the more anterior stages of the ventral visual pathway leads to increased preference and visual interest, there exists a wide base of theoretical and experimental support for the proposition that stimulus repetition decreases this activity.

Behavioral studies typically show faster recognition times and increased recognition performance with repetition of the stimuli (see Schacter, Chiu &

27

Ochsner, 1993 for a review). This repetition priming occurs for stimuli which are both explicitly recognized as well as stimuli which are presented too briefly to be recognized on a single exposure (Bar & Biederman, 1998). At longer presentation times, the magnitude of the priming is robust to changes in retinal position, mirror reflection, size, and as long as an object’s parts remain readily discernible, rotation in depth (Biederman & Cooper, 1991; Fiser & Biederman, 2001; Biederman & Gerhardstein, 1993; Biederman & Bar, 1999).

Priming is generally believed to be a product of the reactivation of the same cellular networks by a repeated stimulus, which occurs with the strengthening of the connections between a neuron responding to some stimulus aspect and the inputs that cell receives from earlier stages of processing. Importantly, the increase in local connection weights is accompanied by an increase in inhibition from the most active unit(s) to units less strongly activated by the initial presentation. Priming involving stimuli which change in their low level properties (such as object position or orientation in depth) presumably occurs as a result of overlapping cellular activation at a higher level of processing in a manner which reflects the level of representation at which the two stimuli are equivalent. For example, the degree of translation invariance observed in supraliminal priming implicate the human homologue

28

of monkey TE as the locus for the neural changes underlying the facilitation of recognition (Bar & Biederman, 1999).

Neurophysiological studies of stimulus repetition in the rhesus monkey indicate that priming may be the result of the formation of a sparser code for representing that stimulus. Using complex, multicolored pictures (such as faces, objects, patterns, and textures) which were familiar to the animal, Miller, Li, & Desimone (1993) found that 48% of the cells they recorded from in the anterior, ventral portion of IT (TE) discriminated between a matching (repeated) and a non-matching stimulus, even when a number of stimuli intervened between the sample and the match. Of these, 92% showed decreased firing upon repetition of a stimulus. In an earlier study, the same group found that many IT neurons developed stimulus-specific suppression lasting long periods when they were tested with novel stimuli repeatedly until they became familiar (Miller, Li, & Desimone, 1991b). They propose the existence of a passive, suppressive mechanism as well as an active mechanism for tagging a particular stimulus as a target. Another group, also demonstrated what they term stimulus specific adaptation (SSA) in macaque IT cells, and report that such adaptation only occurred for those neurons with an excitatory response to the stimulus (Sobotka & Ringo, 1994).

29

It is important to note that in addition to the widespread suppression observed in these studies, a very few cells, likely those which come to represent the stimulus, increase their firing over repeated exposure. This evidence is consistent with what would be expected if the cells that respond best to the initially novel stimulus act to suppress the firing of the general population of less active cells. Cellular networks in higher order visual areas mediating recognition (TE, TEO, and perhaps V4 and other earlier areas to a lesser degree) undergo competitive interactions, whereby cells mutually inhibit each other and readjust their connection strength from earlier layers in a process that maximizes individual cell firing. This refinement of initially broad and noisy activity leads to the development of a robust, sparse representation for a novel stimulus as it becomes familiar.

The global effect of this process would be an overall decrease in the amount of neuronal activity for a stimulus which has been seen previously. Brain imaging studies in humans confirm that stimulus repetition leads to both short term adaptation and longer term reorganization of cortical circuits, seen as reduced FMRI activity in the human extrastriate and inferotemporal cortex (Buckner et al., 1998; James et al., 2000, van Turrenout, Ellmore & Martin, 2000). This reduction in visual areas tends to occur to a greater extent in the later stages of processing, and is sometimes absent from early visual areas completely (Buckner et al., 1998). A study of repetition priming by

30

Vuilleumier et al. (2002) reported that the decreases observed in area LO and the posterior inferior temporal cortex (including the fusiform) were invariant to changes in the retinal size of the images, and some fusiform adaptation was observed for changes in viewpoint as well. Finally, the presentation of a different exemplar with the same name as a previously presented object did not produce changes in any visual areas, but did affect left inferior frontal cortex.

There have been some reports of increases of activity to repetition of stimuli, particularly when the stimuli are unfamiliar (Henson, Shallice & Dolan, 2000) or shown for only very brief durations (James et al., 2000). Conflicting reports indicate that both familiar (nameable) and unfamiliar (nonsense) objects lead to decreases (and no observable increases) in left and right inferior occipital gyri and fusiform gyri (van Turrenout, Ellmore & Martin, 2000; Vuilleumier et al., 2002), though the authors do note that the locus of regions showing decreased activity was more variable across subjects for nonsense objects. A more recent study has reported that while lateral occipital and posterior inferotemporal cortex show repetition suppression for both nameable and novel objects, the fusiform gyrus only shows decreases for nameable objects (and overall lower activity for nameable objects). The likely resolution for these conflicting accounts lies in whether or not the repetition of a stimulus leads to an increase in interpretability, as would be

31

expected if its activity propagated forward (more anterior) on a second presentation, or merely activates the same representation a second time, leading to a decrease in activity. Stimuli which are presented too briefly to be fully perceived, or are so uninterpretable as to require longer viewing for understanding would therefore lead to increases on a second presentation. In agreement with this idea, an imaging study by Bar et al. in 2001 demonstrated that subjective ratings of increased interpretability for an initially unrecognized stimulus led to an anterior shift in the focus of activity in the temporal cortex for that stimulus. These ideas will be explored in greater detail in the next chapter when we consider the behavioral effects of subliminal exposures on preference judgments.

For repetition of a stimulus which does not lead to an increase in interpretability, then, there is a decrease in the overall amount of neural activity in the ventral visual pathway, resulting in less pleasure from viewing that stimulus a second time.

There are some apparent exceptions to preferences for novelty. Yet further investigation of these cases reveal that they do not run counter to our theory, but are instead cases where interpretability is lacking and additional processing is needed to achieve understanding. A commonly cited example is the mere exposure effect, in which the “mere” exposure of a stimulus leads

32

to a preference for that stimulus over novel items (Zajonc, 1968). We will review the evidence for and against this effect in greater detail in Chapter Three. It is likely that this effect arises when a stimulus is presented in such a manner that a single exposure is well masked or very brief, preventing an understanding on a first exposure. A second example is the preference of a child to have the same bedtime story read to him or her on many occasions. Again, it is likely that the child does not fully understand the story, even though they may be able to recite the story verbatim. Each subsequent exposure provides an opportunity for the child to extract something new, such as an association between a word and a picture or a deeper understanding of the story being told. In an analogous manner, the effort required of an adult to achieve mastery of a new subject (such as the struggle to understand a new theorem) can be frustrating before the moment of comprehension, but the knowledge of that impending moment and its accompanying pleasure provides motivation to continue struggling.

Perceptual Affect in a Systems Perspective

While we have focused on understanding how activity in perceptual streams may be modulated by differences in the novelty and interpretability of a stimulus, it is important to understand how these differences affect other systems in the brain that ultimately produce the behavioral effects of

33

selection, pleasure, and recall. An extensive overview of these systems is beyond the scope of this theory, but it is helpful to think about how activity in the anterior stages of the ventral visual pathway may exert influence on other brain areas.

The anterior portion of the inferotemporal cortex (area TE) and the parahippocampal cortices are heavily connected to the hippocampus (via the perirhinal and entorhinal cortices; Saleem & Tanaka, 1996; Suzuki & Amaral, 1994a), the lateral, basal, and basal accessory nuclei of the amygdalar complex (Turner, Mishkin & Knapp, 1980; Iwai & Yukie, 1987; Cheng, Saleem & Tanaka, 1997), and a variety of subcortical structures, including the striatum (including the ventral striatum, e.g., nucleus accumbens, both directly from TE and via the amygdala and hippocampus; Cheng, Saleem & Tanaka, 1997; Friedman et al., 2002), the superior colliculus (though this connection appears to be stronger in infants than adult monkeys; Webster, Bachevalier & Ungerleider, 1993; Webster, Bachevalier & Ungerleider, 1995), the pulvinar (Webster, Bachevalier & Ungerleider, 1993) and the claustrum (Cheng, Saleem & Tanaka, 1997). Cortical targets include the prefrontal cortex (including the dorsolateral prefrontal cortex and frontal eye fields, BA 8 and 45, and the orbitofrontal cortex, BA 12, 11, 13; Webster, Bachevalier & Ungerleider, 1994; Bullier, Schall & Morel, 1996; Lavenex et al., 2002), the superior temporal sulcus (Saleem et al., 2000), the insular,

34

cingulate, and retrosplenial cortices (Lavenex et al., 2002), and regions in the parietal cortex (such as LIP and 7a; Lavenex et al., 2002; Webster, Bachevalier & Ungerleider, 1993). In addition, the multiple regions in the anterior portion of the temporal lobe are highly interconnected and give rise to large feedback projections to earlier visual areas. Interestingly, a large number of studies have recently shown that the anterior portion of area TE appears to be functionally and anatomically divided into a ventral aspect (TEav) and a dorsal aspect (TEad), with these divisions giving rise to different connectivity patterns in many target structures (Saleem & Tanaka, 1996; Cheng, Saleem & Tanaka, 1997; Saleem et al., 2000; Yoshida et al., 2003).

Therefore, information about object form influences a large number of brain systems, such as those involved in decision making and movement planning, the formation of episodic and factual memories, the formation of stimulusreward associations, and emotional responses such as fear, pleasure, disgust, etc. As it relates to information foraging, the systems which are of importance to us are those involved in spontaneous visual selection and the subjective experience of pleasure. Through its projections to the frontal eye fields (FEF) and the superior colliculus, activity distributions in the anterior inferotemporal cortex could directly guide eye movements during free viewing to areas of high interest. Conversely, the affective component of

35

information foraging behavior, i.e., the “thrill” elicited from the acquisition of novel and highly interpretable information, is likely a result of activity in the nigrostriatal system (dopaminergic afferents from the ventral tegmental area, nucleus accumbens, the substantia nigra, and their targets in hypothalamic and brainstem motor nuclei; Robbins & Everitt, 1996). As is illustrated in Fig. 2.4, the ventral visual pathway can exert influence on this system either directly (Cheng, Saleem & Tanaka, 1997), or via activity in the hippocampus, amygdalar nuclei and prefrontal areas (Friedman, Aggleton & Saunders, 2002). It is likely that affective preference is mediated by the direct projections, which provide a mechanism by which information acquisition itself can act as a rewarding stimulus in the absence of any external reinforcers.

Figure 2.4. The affective component of information foraging behavior is likely mediated by direct projections from sensory cortices to the striatum (from Rolls, 1999, pp. 257).

36

An important question raised by the divergent projections arising from the anterior aspect of the ventral visual stream is how activity distributions in these areas can differentially affect mnemonic and affective systems. Stimuli which are recognized as being familiar do not necessarily produce strong positive affective responses, and vice versa. An interesting possibility is suggested by the previously mentioned process of “sparsification” implicated in the neural studies of stimulus repetition. Stimuli which are both novel and richly interpretable have not yet come to be represented in a sparse manner, and will lead to widespread activity distributions in area TE and the parahippocampal cortex (though individual cells may not fire at their maximal rate). As a person becomes familiar with a stimulus, competitive interactions lead to a change in the representation of that stimulus, as a very few cells which are initially most activated adjust their connection strength in a way which leads to even greater activation by the same stimulus in the future, while the majority of weakly activated cells adjust their connection strength in a manner which decreases their firing to the stimulus. It is possible that the projections from area TE and the parahippocampal cortex to brain pathways mediating perceptual affect and spontaneous visual selection integrate over large populations of cells in order to be most sensitive to wide, shallow activity distributions. On the other hand, connections to mnemonic systems are likely to be more specific in nature.

37

Predictions and Extensions

This theory has broad implications for many aspects of human behavior. Below, we will outline some predictions as they pertain to visual preference, learning likely sources of new information, the moment to moment control of eye movements, and the possible basis for individual differences in information foraging.

In the next chapter, we will introduce visual preference as a method for testing this theory and present a review of known factors influencing visual preference. Figure 2.5 presents a series of predictions based upon hypothetical activity distributions in a µ opioid gradient. Blank walls are not preferred because they result in little activity in this pathway. Uninterpretable inputs (such as random-appearing masses) are not preferred because they result in little activity in the endomorphin-rich anterior areas such as TE. Novel inputs that result in extensive interpretation and/or association maximize endomorphin release, and repetition of stimuli will result in less activity in areas high in endomorphin receptors.

38

Figure 2.5. Hypothetical activity distributions along the gradient of endomorphin receptors in the ventral visual pathway for visual scenes of increasing interpretability.

If information acquisition itself can act as a reward, it seems reasonable that the dopamine brain systems underlying stimulus-reward associations(See Tremblay & Schultz, 1999; Schultz, Tremblay & Hollerman, 2000) should be able to learn that certain uninteresting behaviors or objects themselves can lead to large payoffs in information. For example, the act of waiting in line at a movie theater may not be pleasurable in itself, but is done in the service of

39

the likely payoff produced by seeing the movie. Similarly, people know that television, radio, the internet, cultural functions, other people, and cities are all likely sources of rich information, and may develop associations and habits around these things similar to those seen with items associated with food or drug reward. While these activities represent a deviation from the maximization of novel and richly interpretable information in the short term, they are performed in the service of this drive. A full characterization of the effect of the information foraging drive on moment to moment behavior will have to integrate the probability of reward over multiple time scales, though it should be noted that in many cases both long term payoffs and short term interest can be served simultaneously.

A major aspect of information foraging is the real time control of eye movements about one’s visual environment. Unlike a camera, only the central region of an image on the retina is processed in detail. By saccading to regions of interest, the foveal region of the retina can maximize the amount of information gathered on each fixation (Barlow, 1961). The demonstration that µ opioids often act via synaptic transmission (Simon & Hiller, 1994) allows that activation of an opioid gradient could have the temporal resolution necessary to influence selection mechanisms in real time. Perceptually driven affective responses would allow eye movements to track novelty and interpretability in a fast, feedforward manner that is not

40

dependent on extensive feedback from limbic areas nor lengthy cognitive deliberations.

Several recent studies demonstrate that spontaneous eye movements are sensitive to this type of information. Using a gaze-contingent display in which a section of a scene could be moderately blurred as a fixation was released, Loschky, McConkie, Yang, and Miller (2001) showed that saccades in free viewing would avoid the blurred regions, even when they were in the periphery. This result could reflect the “calculation” that it would be difficult to obtain good information from such regions. Other recent studies of eye movements while looking at scenes and artwork confirm that image factors in addition to salience influence the position of fixations (Oliva et al., 2003; Heidenreich & Turano, 2003).

A key element for a successful information foraging strategy is the interpretability of a stimulus. A stimulus’ interpretability is heavily dependent on past experience and can differ from person to person. By providing a link between ventral visual activity and visual preference, our theory provides a basis for studying individual differences in preference. If a particular picture were to lead to more activity in endomorphin-rich areas in one subject than in the general population (for example when an art connoisseur views modern

41

art), we would expect that subject to show higher than average preference for that image

A second source of individual variability in information foraging may be related to the rate at which an individual habituates to novel information. Some individuals are natural “sensation seekers” who get bored easily and are always seeking out new experiences.(Zuckerman, 1995). Others, according to Zuckerman, take a much more conservative approach to novelty, and are often quite content with that which they are familiar with. Zuckerman has proposed that sensation seeking is related to the activity of monoamine systems (particularly dopamine and norepinephrine; Zuckerman, 1995), suggesting that individual differences in sensation seeking could be the result of differing levels of monoamines, receptors, inputs to these systems, or enzymes responsible for their breakdown (such as monoamine oxidase, MAO). An individual who gets “bored” easily may have a hypoactive dopamine system or alternatively, a reduced input to the nigrostriatal dopaminergic system as a result of a decrease in the strength of cortical µ opioid gradients. Tests of the link between perceptual affect and sensation seeking may have important consequences for the clinical treatment of anhedonia and depression.

42

The neurocomputational theory outlined above has the potential of accounting for how interest can be expressed in real time in a perceptual pathway and why habituation occurs so that, as stimuli become familiar, once they are fully understood, there is less pleasure in perceiving them again. This theory assumes only that greater neural activity in perceptual pathways results in greater endomorphin release, and that the greater the endomorphin release, the more pleasurable the experience. We predict that high levels of neural activity in association areas should be experienced as pleasurable. In addition to providing an input to the brain systems mediating the qualitative experience of pleasure, this activity would serve to inform frontal and subcortical mechanisms guiding spontaneous visual selection. The magnitude of this endomorphin activity would subserve perceptual and cognitive affect, resulting in a preference for patterns which are both novel (because they have yet to undergo competitive interactions), and richly interpretable (as such patterns would initially activate a rich set of associations).

43

CHAPTER 3: VISUAL PREFERENCE

Our theory predicts that novel, highly interpretable stimuli produce high activity in temporal lobe structures and thus high endomorphin release, resulting in high visual preference. We have developed a paradigm to measure the functioning of this system in which we measure the rated preferences of individuals as they view pictures of various degrees of initial interest.

Background

The literature on visual preference, while relatively sparse, indicates that visual preference may possibly be influenced by a number of factors, including stimulus complexity, previous exposure, arousal, emotional valence, and the evolutionary significance of a scene.

Complexity vs. Interpretability

A large number of studies from approximately 1950 to the present by Daniel Berlyne and others have investigated the role that “collative” stimulus properties such as novelty, “complexity,” “surprisingness,” and “incongruity” have on a variety of different measures, such as subjective novelty ratings,

44

pleasingness, interestingness, and galvanic skin response (Berlyne, 1958, 1963; Aitken, 1974). This extensive series of studies used a large number of different stimulus types, including irregular polygons (such as those studied by Attneave), groupings of lines and dots, simple line drawings, and occasionally black and white versions of paintings. While Berlyne was quite successful in showing that these properties exhibited robust effects on a number of different measures, efforts to pin down exactly what determined stimulus “complexity” were unsuccessful. As a result, effects of complexity on preference were generally not achieved. For a relatively simple class of objects, such as irregular polygons, it appears that on average subjects find figures of low and high complexity, defined by number of sides, least preferable, while those somewhere in the middle are most preferred (Aitken, 1974). Typically, ratings of “interestingness” show a similar pattern, with a peak at slightly higher levels (Ibid). However, this inverted-U shaped function does not generalize to all types of stimuli, especially since Berlyne’s theory does not adequately explain what comprises “complexity” across different stimulus sets.

From the perspective of opioid-mediated perceptual preference, we can see that it is not complexity per se that underlies preference but the extent of opioid activity. Very simple polygons might not allow many novel interpretations; highly complex ones may merely resemble texture masses

45

that also do not allow much interpretation. One can imagine that polygons with an intermediate number of sides might suggest some object or interpretation, thereby resulting in increased associative activity and, consequently, increased opioid activity. But as examples with droodles presented below will illustrate, it is not the complexity of the pattern that matters but the interpretability of that pattern.

In general, the approach we have taken differs from that of Berlyne in that we do not attempt to define the “complexity” of our stimuli but seek instead to relate a stimulus’ preference to its interpretability. A stimulus which is highly interpretable is one which makes many contacts with memory. The sheer number of elements in a stimulus is not a direct measure of its interpretability.

As an example of what is meant by interpretability, consider the cartoons, or “droodles,” illustrated in Figure 3.1. As a test for yourself, look at the drawings alone before consulting the caption for the interpretations. By themselves, these simple drawings are easily understood (and most definitely not visually complex) but not very interesting either. However, when a novel and meaningful interpretation is paired with the droodle, the experience is pleasurable.

46

Figure 3.1. These “droodles” illustrate the principle that rich interpretability (in the absence of visual “complexity”) is pleasurable. a) A soldier and his dog walking behind a wall. b) A close-up of a pig reading book titles in a library.

Effects of Previous Exposure

In 1968 Zajonc first reported that moderately complex visual stimuli which had been seen for very brief durations were preferred over equally complex stimuli that had not been seen at all. Many studies since that time (e.g. Seamon et al., 1984; Bornstein, 1992) have expanded on his findings to include a wide range of stimuli, and have demonstrated that the effect is robust (and perhaps even stronger) when subjects have no memory of having seen the stimuli. However, an equally large number of researchers have failed to find a robust effect of previous exposure under a large range of

47

conditions, including ourselves (Vessel & Biederman, unpublished data), Bornstein (1989), and Berlyne (1970).

We attempted to replicate the effect of mere exposure on several classes of stimuli, including line drawings of objects, two-dimensional Fourier descriptors (smooth, closed outlines) created by Shepard & Cermak (1973), and, like Zajonc, irregular octagons generated using an algorithm described by Attneave & Arnoult (1956). In only one experiment out of 5 did our results show any evidence of better than chance performance on an affective choice task (i.e. which stimulus do you prefer?). Irregular octagons which were preexposed ten times for 13.3 ms and followed by a 40 ms mask produced 57% liking and 55% recognition, collapsed over judgment order (95% confidence interval for liking judgment: 50.3 – 63.8). We did not observe a benefit for affect over recognition, nor was affect significantly better than chance, for images pre-exposed one or four times. In addition, the results for affect were heavily dependent on both the order in which the subjects made their judgments (affect first or recognition first) and on the sex of the subject.

48

Figure 3.2. Results from a mere exposure experiment in which subjects indicated in a 2 alternative-forced choice paradigm which of two stimuli they recognized or preferred. As indicated by the blue lines, when both the recognition and the affect judgments occurred on the first block, ten exposures produced a preference for the previously seen image 58% of the time and recognition only 53% of the time. Notice, however, that the order in which subjects made the judgment interacted significantly with the effect: Block 2 recognition was slightly better than Block 2 affect at all three exposure levels.

Importantly, the effects of exposure in these types of experiments are miniscule in comparison to the effects of a priori image preference. In a previous experiment, we observed a correlation of 0.87 between the average rank preference ordering given to the irregular octagons by a group of five subjects and the number of times each stimulus was chosen by a different group of twenty subjects in a mere exposure paradigm using forced choice preference judgments (regardless of the number of previous exposures). In the experimental results reported above, these rank orderings were used to create stimulus pairs which were matched on a priori preference.

49

In an interesting 1970 paper entitled “Novelty, complexity, and hedonic value,” Berlyne showed subjects a large number of sequences of repeating stimuli of differing complexity, and found that all the sequences showed a highly significant general trend towards a preference for novel stimuli, counter to Zajonc’s mere exposure effect. The data from the first two of these experiments is shown in Figure 3.3. Condition four, which used differently shaped and colored stimuli on every trial, did not show this decrease. Only in a third experiment, in which a wide variety of very complex, different stimuli were used on each trial did Berlyne observe any increase in preference throughout the sequence. Finally, in a fourth and fifth experiment, Berlyne repeatedly assessed subjects preference for two “simple” stimuli (one nonrepresentational, one a portrait) and two “complex” stimuli (one nonrepresentational, one a reproduction of a crowded painting with many elements), with intervening presentations between each judgment. He found that while preferences for the simple stimuli declined for each judgment, preferences for more complex stimuli initially rose and then fell in an inverted U shaped function (Berlyne, 1970). Berlyne interpreted these results as indicating that very complex stimuli, when they do not have to be rated each trial, are less susceptible to what he referred to as a “tedium” factor.

50

In 1990, Bornstein et al. published a similar observation in a paper entitled “Boredom as a limiting condition on the mere exposure effect,” highlighting the fact that mere exposure effects only seem to work with short experiments, using very few stimuli, shown for very brief exposures, with happy, well adjusted subjects. The overwhelming amount of evidence, including decades of preference habituation studies in infants, indicates that under most circumstances people show a preference for novelty.

51

Figure 3.3. Judgments of A) preference and B) interestingness decline with repeated presentation with a variety of stimulus types and presentation conditions. The only exception are experiments in which novel patterns are used on each trial (bottom row of A and B). Individual data points showing a discontinuous jump in preference are from trials where a different pattern was presented. From Berlyne, 1970.

52

Interestingly, recent EEG and FMRI work in visual priming may indicate that on a very short timescale, in the period before an object or scene has been recognized, priming might actually result in an increase in neural activity (James et al., 2000). If this is the case, then it is likely that mere exposure effects are the result of this transitory, early increase in visual activity during the first hundreds of milliseconds before a person recognizes an object. During this period, brief exposure increases the interpretability of stimuli, akin to Seamon’s concept of perceptual fluency (Seamon, Marsh & Brody, 1984), as neural activity propagates further through the ventral visual stream, but is not yet subject to competitive interactions. Within the FMRI literature, Bar et al. have shown that the focus of visual activity moves progressively anterior in the ventral pathway as subjects report increased knowledge regarding stimulus identity (Bar et al., 2001). More recently, Zago & Bar have shown that priming a 500 ms presentation of an object with a presentation of that same object leads to a change in the BOLD signal across multiple areas of the ventral visual stream that is dependent on the duration of the prime (Zago & Bar, 2003). Both behavioral priming and the observed suppression of the BOLD signal peaked between 150 to 250ms, with both shorter (40 ms) and longer (350, 500, or 1900 ms) primes resulting in less of a reaction time savings and less BOLD signal reduction than these peak times. If preference is coupled directly to activity in this pathway through the action of

53

endomorphin receptors, these studies would provide an explanation for why, with exposures which are not long enough to allow a subject to completely recognize a stimulus, preference for subsequent exposures increases as activity propagates further through the ventral visual pathway allowing for increased interpretability. Bar & Biederman (1999) proposed that the human homologue to monkey V4 or TEO is the locus of subliminal priming; the presence of moderate levels of endomorphin receptors in these areas could mediate the mild increase in affect observed for subliminal primes. Longer pre-exposure durations that allow for a stimulus to be recognized in a single presentation reliably produce decreases in preference on subsequent presentation.

Using EMG as a physiological marker of positive affect, Winkielman & Cacioppo (2001) showed that increases in the degree of processing as a result of increased presentation time or the presentation of an outline prime elicit automatic, positive affective reactions. Once again, these effects are found to be rather weak. Due to the fragile nature of these biases, such priming based effects would be largely washed away by the later effects of preference habituation as a stimulus becomes familiar. Returning to Zajonc’s work, it appears as if the basic finding of increased affect for previously seen stimuli only operates within a relatively narrow domain; the major lesson to be learned from this body of work appears to be that preferences are

54

automatic, requiring no inferences or attributional operations (Zajonc, 1980, Winkielman et al., 1997). In the context of the opioid gradient account, it could be that the positive effects of brief exposures may be a consequence of their being processed to a greater extent in the ventral pathway than the control stimuli, but not to the extent that competitive learning would reduce their advantage.

Arousal and Emotional Valence

The next two factors shown to influence visual preference, arousal and emotional valence, have largely been studied in the context of fear conditioning and reactions to aversive stimuli. Cahill et al. (1996; O’Carroll et al., 1999) has explored in depth the likely involvement of the amygdala and the influence of its beta-adrenergic projections on a subject’s ability to recall highly arousing, aversive scenes. Patients with damage to the amygdala do not show enhanced memory for highly arousing, aversive stimuli, and PET activity in the amygdala correlates with long-term, free recall (Cahill et al., 1996). In addition, blockade of beta-adrenergic activity at encoding also led to a reduction in recall for highly arousing, aversive stimuli (O’Carroll et al., 1999). More recently, it has been claimed that amygdala damage may lead to a deficit in remembering the “gist” of highly arousing negative scenes, but not for remembering visual details (Adolphs et al., 2001). In addition to

55

certain amygdala nuclei, the orbitofrontal cortex, the temporal pole, and the superior frontal gyrus are also likely to play a role in the reaction to emotionally valent stimuli (Royet et al., 2000).

It is likely that this system, which has a very high evolutionary value, is separate from a putative information foraging system. While the amygdala has been shown to be involved in positively reinforcing behaviors such as food reward (Rolls, 1999), it is unclear what role it may play in information foraging. For these reasons, we have chosen to use stimuli which do not depict highly arousing negative scenes nor erotica, such as those in the International Affective Picture System (IAPS) developed by P. Lang at the Center for the Study of Emotion and Attention (e.g. Lang et al., 1997a). Our focus also differs significantly from that of the IAPS stimulus set in that we aim to investigate visual preference within the larger context of information foraging across many scene types, and not just as it pertains to particular objects with high emotional content such as happy babies or mutilated bodies.

While it may be the case that visual orienting responses and certain physiological responses to these stimuli still obey the general framework of our theory of perceptual affect (Lang et al., 1993), as noted previously, other motivational systems would be expected to override that of information

56

foraging. Support for a dissociation of information foraging and the fear response comes from a recent study of the rate of memory formation for scenes with positive and negative affective content, which reported separable contributions of valence and arousal (Maljkovic & Martini, 2003). Using rapid serial visual presentation (RSVP) of scenes covering a range of emotional valence, they found that the rate of memory acquisition (as measured by the performance on an old/new recognition test for sequences of varying exposure duration) was related to valence. Highly arousing scenes of either valence were better remembered overall, but negative scenes were poorly remembered at the shortest presentation rates and increased steeply, while positive and neutral scenes showed constant increase in memory performance with increasing exposure. The authors concluded that highly negative scenes may lead to an initial freezing response with later accelerated processing. These conclusions agree with our proposal that highly informative scenes, regardless of their emotional valence, will initially lead to a greater positive affective response, though this initial response may then be superceded if the stimulus engages another drive state such as avoidance or sexual interest.

Of greater interest to our work, however, is the likelihood that evolution has shaped our preferences for certain visual scenes within the more neutral

57

domains of navigation, monitoring, and habitat selection. Chapter Five will deal extensively with this topic.

A Stimulus Set for Testing Positive Perceptual Affect

As a method to test the functioning of an information foraging system and its links to perceptual affect, we have developed a stimulus set of full color scenes for use in visual preference experiments.

Given the wide range of interpretability of different classes of stimuli, we chose to focus our efforts on understanding how preference varied within a set of stimuli which were all of relatively high interpretability. We used full color, natural scenes likely to activate the entire ventral visual pathway in an attempt to maximize the range of any preference effect.

Calibration Experiment

Based upon previous reports of the reliability and intuitive nature of preference judgments (Berlyne, 1970; Zajonc, 1980), we selected a simple preference rating task as our basic behavioral measure. Our first step was to gather baseline preference data on all of the images in our stimulus set. In

58

addition, we chose an exposure duration of one second, which was clearly long enough to allow an inspection of each scene sufficient for a rich interpretation, if one was to be had.1

Stimuli

Using a combination of images collected using a digital camera and the MasterClips Premium Image Collection from IMSI2 we selected a set of 200 full color scenes which varied widely on a number of dimensions, such as natural vs. urban/man-made, close-up vs. scenic, animate vs. inanimate, and cluttered vs. simple. All images were cropped to 640x480 pixels using Adobe Photoshop and saved as high quality rgb jpeg images.3

Subjects

Twenty subjects participated in this experiment either for credit in an undergraduate psychology course or as a paid volunteer. Informed consent was obtained in accordance with the guidelines established by the University

1

At 100 ms, subjects can readily determine the setting of a scene and the interactions and roles of its major entities (Potter, 1976; Biederman, 1981; Biederman, Mezzanotte & Rabinowitz, 1982). 2 IMSI MasterClips© and MasterPhotos™ Premium Image Collection, 1895Fransisco Blvd., East, San Rafael, CA 94901-5506, USA. 3 The full stimulus set is available online at http://geon.usc.edu.

59

of Southern California Institutional Review Board (IRB). The subjects consisted of ten males and ten females, all of which were right handed. The mean age was 20.3 years, and all subjects had normal or corrected to normal vision.

Method

Subjects were instructed that they were taking part in an experiment designed to investigate how people make judgments about scenes. They were shown each scene in our full image set for one second each and asked to indicate how much they liked the scene on a seven point scale. They were to give the scene a six or seven if they found it very visually interesting and enjoyed looking at it and to give the scene a one or two if they found the scene boring, uninteresting, or if they didn’t like looking at the scene. If they had no strong feeling either way, they were to give the scene a middle rating. The instructions stressed that a good way to think about this judgment was to ask themselves “how much would I like to see this picture again?”

The images were shown to the subject on a Sony Trinitron display controlled by a Macintosh G3 computer running Matlab and the Psychophysics Toolbox (Brainard, 1997; Pelli, 1997). The scenes were shown at a resolution of 82 pixels per inch, creating images which were 7.75 x 5.875 inches on the

60

screen. At a distance of 32 inches, these scenes subtended approximately 14 degrees of visual angle horizontally.

Following an adaptation period during which the subject’s eyes were allowed to adjust to the dark room, each trial began with a fixation point followed by a scene for one second, displayed on a mid gray background. The order of presentation of the scenes was randomized across subjects, with the even numbered subjects seeing the reverse sequence of the odd numbered subjects, resulting in the same average serial position across subjects for all scenes. After the scene disappeared, a prompt appeared on the screen which read “Enter Preference: 1 (don’t like it) Å---Æ 4 Å---Æ 7 (really like it).” The subject responded using the numeric keypad on the computer keyboard, and was given unlimited time to respond. Subjects were instructed to rate each image based on their preference for that image at that moment, regardless of the ratings they had given on other trials, and to try and use the entire seven point scale. When the subject responded, the prompt disappeared, the screen remained blank for 500 milliseconds, and the next trial began. Subjects were given the opportunity to take a break after every 40 trials.

61

Results

Any errors or omissions on the part of the subject were recorded by the experimenter and corrected before analysis. Average preference ratings were computed for all 200 scenes. The average preference rating across subjects was 3.97 + 0.62, and the average variance in ratings across subjects was 2.62. Figure 3.4 shows the distribution of preference scores for the whole image set, and the normal probability plot in Figure 3.5 indicates that this distribution is roughly normal across the rating scale. The six most and six least preferred scenes are shown in Figure 3.6 along with their average rating.

62

Figure 3.4. The distribution of average preference ratings across twenty subjects for the full 200 scene image set.

Figure 3.5. A plot of the expected cumulative probability versus observed cumulative probability for the average preference ratings of all 200 images, showing only slight deviations from normality.

63

Figure 3.6. The a) six most and b) six least preferred scenes in the calibration experiment.

64

It is clear that the preferred and non-preferred scenes differ in a number of ways. Perhaps the most obvious from the examples shown in Figure 3.6 is the prevalence of natural vistas in the set of preferred scenes and the number of more man-made, urban elements in the low preferred scenes. In addition, the low preferred scenes commonly contained a large amount of clutter (e.g. the pile of garbage) or texture, often as the result of many repeated elements (e.g. the pile of bricks or rows of chairs).

A measure of cross-subject consistency was calculated by correlating a single subject’s preference ratings on all 200 images with the mean of the other 19 subjects. The mean of the square of these correlations gives an indication of the overall amount of variance in the ratings that can be attributed to the stimulus set: on average, 31% of the variance in an individual subject’s preference scores can be predicted by the group mean for the stimulus being shown.

In a set of subsequent experiments, we used these ratings to explore 1) what aspects of the images might account for these differences in initial preference, 2) how these preferences change with repeated presentation, and 3) how the ventral visual stream differentially responds to images of varying preference.

65

CHAPTER 4: REPETITION

The theory we presented in Chapter Two predicts that novel, highly interpretable stimuli should be preferred as they lead to the most activity in the anterior regions of the ventral visual pathway. Here we present two experiments which tested the effects of stimulus repetition on rated preferences. Our results provide support for the hypothesis that repetition decreases preference, and does so in a consistent manner for scenes of varying initial interest. In addition, these experiments allowed us to develop a stimulus set and experimental parameters for use in brain imaging experiments.

A major hurdle to overcome in the experimental design was to control for the local novelty environment such that scene novelty was not confounded with time (or trial number) in the experiment. That is, it should not be the case that novel scenes occur early in the session and repeated scenes later. In each of the experiments reported below and our imaging experiment, measures were taken to deconfound repetition and trial number.

Given the demonstration by Berlyne that the preference for a novel image presented in the context of repeated images is increased above what would

66

be expected amongst a sequence of other novel images (Berlyne, 1970), we also sought to control the “preference context” of our images.

Experiment 1: Effects of Repetition, Between-Subjects Design

In this first experiment, our goal was to assess the effects of multiple repetitions on stimulus preference over the entire set of 200 stimuli. In order to assess the limits of repetition effects, we chose a design which employed a large number of repetitions but only allowed for each subject to see a subset of the total set of images.

Methods

Subjects

Forty volunteers participated in this experiment either for payment or credit in psychology courses. The procedures for informed consent were followed in this and all subsequent experiments unless noted otherwise. The sample consisted of 11 men, 29 women, with a mean age of 19.8. Information on handedness was not collected. None of the subjects had previously seen any of the images, and all subjects had normal or corrected-to normal-vision.

67

Stimuli

Stimuli consisted of 200 full color scenes selected to vary on a wide number of dimensions and independently rated for a priori preference (see the Calibration Experiment in Chapter Three for details on the stimuli).

Procedure

Each subject saw a subset of thirty images with a full range of a priori preference values (based on the pretest ratings), grouped into three sets of ten images. Each of these lists covered the range of preference ratings from high to low. The first set was used as “practice” stimuli at the beginning of the experiment and were subsequently used as “spacers” throughout the test trials. The second set was used for “Block 1,” during which a new image was introduced within each group of ten trials and then repeated in each subsequent group of trials up to ten repetitions. The third set was used for “Block 2,” which continued to introduce new images at a controlled rate, but did not result in a full ten repetitions for each image (see Fig. 4.1). This trial sequence controlled for the average serial position of the scenes, the local novelty context (novelty was not confounded with trial number for the Block 1 images), and the preference context (based on the pretest results). Over the set of forty subjects, the design allowed each image to be evaluated as a

68

Block 1 image by two subjects. The instructions, trial timing, and the subject’s task were identical to that of the calibration experiment.

Figure 4.1. The trial sequence was designed in such a way as to maintain the local novelty context while achieving ten full repetitions of the Block One images. Images labeled as P were part of the practice list, B1 the Block One list, and B2 the Block Two list.

Results

The mean preference rating was 4.22 + 0.12. No significant change over the course of the test block occurred, indicating that the design of the trial sequences adequately controlled for the novelty context (see Fig. 4.2). As can be seen in Figure 4.2, the control for novelty context was very much needed insofar as there was a decline in preference over the “Practice” Block (first three trial blocks), where novelty was confounded with trial block. For

69

the Test Blocks (Fig. 4.3), the average rating given for the first exposure of Block 1 images was 5.21, which declined steadily over the ten exposures of those same images to 3.6. It is evident from Figure 4.4 that there is a consistent decline with repetition for all preference levels, except for a floor effect with the least preferred images. On the other hand, highly rated scenes may be asymptoting well above floor.

Figure 4.2. The average preference ratings over the course of the test block remained constant.

70

Figure 4.3. Average preference declined over repeated presentations.

Figure 4.4. Notwithstanding floor effects, average preference declined over repeated presentation for all images regardless of initial preference.

71

The pattern of results for the Practice images and Block 2 images was similar, though with a bit more variability due to the unequal number of presentations of each image in Block 2 and less than optimal balancing for the Practice images (Fig. 4.5 & 4.6).

Figure 4.5. Preferences for the Practice and Block Two images showed a similar pattern to that of the more controlled Block One data.

72

Figure 4.6. Preferences for the Block Two images showed a similar pattern to that of the more controlled Block One data and the Practice block. The Block Two data does not contain error bars since the images were used to fill in the end of the experiment, resulting in fewer data points as one progresses to later exposures.

In order to evaluate the consistency of this decline over individual images, we computed the linear and quadratic trends for all 200 images. The two sets of ratings obtained for each image as a Block 1 stimulus were first averaged before being submitted to a hierarchical regression analysis and tested for significance using a two-tailed F test. Out of the 200 images, sixty-two did not show a significant trend at the 0.05 level. 128 images showed a significant negative linear trend, while only a single image showed a

73

significant positive trend. Thirty three images showed a significant positive quadratic trend (falling then rising) and ten had a significant negative quadratic trend over and above any linear trend. These data indicate that the overwhelming effect is for images to be less preferred as they are repeated.

We also performed a trend analysis on individual subjects to evaluate the universality of the observed decreases in preference. Out of the forty subjects, eight did not show a significant change in preference with repetition. The remaining thirty-two subjects all showed a significant negative linear trend. The data from twelve of these subjects also showed an additional quadratic component, all but one of which was positive.

Conclusions from Experiment 1

From this experiment we can conclude that under these experimental conditions, preference declines with repeated presentation. This habituation occurs for images with a wide range of a priori preference. While the data seem to indicate that preferences decline at a relatively consistent rate for all levels of initial rated preference, we cannot draw any quantitative conclusions regarding the rate of habituation as a function of initial preference because we cannot assume that the preference judgments are made on an interval or ratio scale. A physiological measure of preference such as eye fixations or

74

the monitoring of facial muscles using EMG would allow for one to quantitatively assess the effects of repetition and a priori preference, though the problem remains that these measures may not be linearly related to the underlying psychological variable of preference. In Chapter Seven, we outline how this “win-shift” behavior might be explored within a preferential looking paradigm to avoid the use of a subjective rating scale.

From the practice blocks of the trial sequence data (Fig. 4.2), it is clear that preference is influenced by the local novelty context, which we term a “contrast” effect. Additionally, the subsequent rebound and leveling of preference during the trial blocks indicates that an intermixed design such as the one used here is an adequate control for such effects.

There are also several aspects of these results that are important when considering possible designs for an FMRI experiment. First, a significant habituation effect was seen in as few as five exposures, allowing us to drastically reduce the number of individual image presentations. Second, a priori preferences were relatively stable across subjects, but not perfectly predictable. In Experiment 2 we overcame this by selecting a set of images with low variance in their cross-subject preference scores.

75

Furthermore, reducing the number of preference levels from seven to three, a low, medium and high preference group, allows us to have good control over a subject’s preference for an image without having to collect behavioral responses during an imaging session. Such a design allows us both to scan our subjects in a “naive” state (one in which they are not making explicit preference judgments) and to use a fully factorial experimental design.

Experiment 2: Selection of image sets for FMRI

In two preliminary experiments, we created three groups of images with consistent a priori preferences across subjects to test the predictability of preference ratings from a priori image preference and repetition. In addition, the use of a fully factorial design allowed us to test for the presence of an interaction in the ratings between a priori preference and repetition related changes. As noted previously, because we could not assume an interval or ratio scale, no inference could be made as to any processes responsible for the interaction or lack thereof.

76

Methods

Subjects

Two groups of twenty volunteers participated in this experiment either for payment or credit in psychology courses. The first sample consisted of 6 men and 14 women, 4 of whom were left handed, and 16 right handed. The mean age of the sample was 21.1 + 2.9. The second sample consisted of 4 men and 16 women, all of whom were right handed, with a mean age of 19.3 + 1.5. None of the subjects had previously seen any of the images, and all subjects had normal or corrected to normal vision.

Stimuli

Using the preference ratings from the calibration experiment, the full set of two hundred images was ranked according to preference and variance. Importantly, these rankings were based upon the data from only the first twelve of the twenty subjects in the calibration experiment, as the final eight were collected at a later time. Therefore, there were slight differences between the preference ordering used for this first pass image selection and the final values used as a priori preferences. A subset of twenty images was selected from each of three ranges: 5 to 7, 3 to 5, and 1 to 3. In addition to

77

the mean preference score and low variance constraint, images were selected to minimize the semantic and visual similarity of the entire image set so, for example, there was only one parking lot scene and one bedroom. An additional twenty four images were selected to serve as buffers at the beginning and end of the experiment. Figure 4.7 shows the distribution of these three image groups across the range of a priori preferences computed from the full set of twenty subjects in the calibration experiment.

Figure 4.7. Distribution of a priori image preferences (based on all twenty subjects from the calibration experiment in Chapter Three) for the stimuli used in Experiment 2a. The bars show the number of images at each preference level for the three image categories, while the lines show fitted normal curves for each of the three distributions and the buffer images. Note that the images were selected based on the average ratings from only twelve of the subjects in the calibration experiment.

78

Following Experiment 2a, 13 of the stimuli were replaced following an analysis of their preference ratings in order to achieve an image set with more consistent ratings. This new set of 60 stimuli were used in Experiment 2b. Figure 4.8 shows the distribution of a priori preference ratings for this set of images based upon the calibration experiment. The decision to keep certain images that overlapped in a priori preference with images of another category was based upon the image ratings from Experiments 1 and 2a.

Figure 4.8. Distribution of a priori image preferences (based on all twenty subjects from the calibration experiment in Chapter Three) for the stimuli used in Experiment 2b. The bars show the number of images at each preference level for the three image categories, while the lines show fitted normal curves for each of the three distributions and the buffer images. Note that the images were selected based on the average ratings from only twelve of the subjects in the calibration experiment.

79

Procedure

The instructions and trial timing were identical to that used in the calibration experiment and Experiment 1. Unlike Experiment 1, however, all subjects saw all images in a within subjects, full factorial design. The experiment began with the presentation of a sequence of buffer images designed to preexpose the subject to three images once, a different three images twice, and other sets three and four times. This sequence of thirty images (3 + 6 + 9 + 12) was followed by twenty four blocks of fifteen images. Each block of fifteen images contained one image in each of the fifteen conditions: three preference levels by five exposures. For the first four of these blocks, the images comprising the latter repetitions were taken from the set of preexposed buffers. During each block of fifteen trials, a new high, medium, and low image were introduced and subsequently repeated on successive blocks until all twenty stimuli in each preference category had been shown five times. The last four blocks of trials also required the interjection of buffer images to allow for the experimental images introduced last to be shown the second, third, fourth, and fifth times. This design was necessary to maintain a constant novelty and preference context throughout the entire experiment and deconfound trial number with repetition number. Within each block of fifteen trials, the condition order was counterbalanced and verified to prevent the repetition of an image at the borders between blocks. The order of image

80

introduction was different for each subject, with one subject receiving the images in the opposite direction as another to control for the overall position of each image over the entire set of subjects.

Results

To evaluate the effectiveness of our experimental design on controlling the local novelty environment, we evaluated the change in preference over the course of the experiment averaged over blocks of fifteen trials (Fig. 4.9). An omnibus F-test for any differences among the trial blocks revealed a significant difference for Experiment 2a, F(25,475) = 2.65, p < 10-4. This test may be too sensitive to random fluctuations within a the sequence due to the extremely high degrees of freedom in the error term. A more appropriate test of any significant linear trend over the course of the experiment found a significant slope of -0.18 (t = -4.22, p < 10-4. Therefore, there did appear to be some overall decrease in preference over the course of this experiment. However, it is not likely that this decrease was responsible for the decrease in preference over repetition, as the magnitude is much less. Over five blocks in the experiment, the overall preference went down by 0.13 units on average, while the change as a result of image repetition was 1.1 units over five repetitions (collapsed across image preference).

81

The F-test for block differences in Experiment 2b (which contained the actual stimuli used in the imaging experiment) was not significant, F(25,475) = 1.02, p = 0.44.

Figure 4.9. No overall change in average preference scores was observed in either a) Experiment 2a, nor b) Experiment 2b. The dashed lines above and below the datapoints indicate the standard error of the subjects’ mean preference rating for each block of fifteen trials.

82

Figure 4.10 shows the results from Experiment 2a. Preference declined with repeated presentation for images at all levels of a priori preference. In addition, the average preference of the high, medium, and low image groups showed good separation in the group data at all repetition levels. A two-way analysis of variance (ANOVA) revealed main effects of preference (F = 127.7, p 1), and summed across subjects (Fig. 6.10). The highest degree of overlap was observed in the posterior part of the left parahippocampal gyrus, with nine out of sixteen subjects showing overlap at seven voxels, and eight of those showing an overlap at twenty-eight voxels.

We then performed an analysis for group effects within this subset of nine subjects. Again, we were interested in asking whether any voxels within that region showing activity to visual stimulation (i.e., stimulus greater than no stimulus) showed greater activity to the image set with High a priori preference over the image set with Low a priori preference. Within this subset of subjects, we found a single cluster of voxels in the left parahippocampal gyrus which was significant at the group level (Fig. 6.11).

160

Figure 6.10. Voxelwise overlap of preference effects. A) The number of subjects showing greater activity for High than for Low preference images at each voxel in standard space. This image was produced by thresholding each subject’s High minus Low contrast at z > 1 and adding these images together. These nine axial slices progress from the most inferior (upper left) to most superior (lower right). B) This coronal section shows the location of the 9 slices shown in A, which are spaced 4mm apart.

161

Figure 6.11. High minus Low preference for nine subjects. Within those voxels which showed a significant effect of image presentation versus rest (with a cluster level significance of p < 0.015), we first thresholded the High minus Low contrast at a z of 3.8 to produce clusters, and then tested for the significance of those clusters at p < 0.05. One cluster in the left parahippocampal gyrus survives, which is shown in a A) coronal view, B) saggital view, and C) axial view.

Table 6.1 presents a summary of the effects we found for both preference and repetition. Overall, the effects of repetition were several orders of magnitude greater than those of a priori image preference, though this is not very surprising given the established upper limit for the ability of a priori image preference for this stimulus set to predict preferences for individual subjects as noted earlier (and in Chapter Four). It is perhaps surprising that the largest effects of a priori image preference were negative: activity in the lateral occipital complex (both a lateral, more superior locus on the right side and more inferior bilateral loci on the borders of the fusiform gyri) was greatest for low and medium preferred images.

162

Summary of BOLD Effects: Preference and Repetition Size Structure

(Voxels)

Max Z

Max Location

Signif. log10(p)

x

y

z

7.79

18

-40

-10

16.2

R parahippocampal g. (36) 896

7.01

-25

-42

-8

12.9

R cuneus (19)

395

6.17

30

-84

25

7.69

L. Lingual g. (17)

137

4.73

-17

-87

4

4.09

L Cuneus (17)

82

4.54

-5

-79

10

3.07

g. (18)

70

4.94

-43

-77

1

2.82

L Middle Occipital g. (19)

29

4.75

-35

-87

20

1.83

L parahippocampal g./subiculum(27/30)

21

4.31

-17

-32

-2

1.59

R parahippocampal g. /perirhinal (35)

14

4.19

20

-19

-13

1.36

(16 Subjects) Repetition: 1> 5 L parahippocampal g. (36)

1291

L middle occipital

Table 6.1. Summary of significant effects. Brodmann areas or functional designations are in parentheses. The location of the voxel with the maximum Z within the cluster is reported in Talairach coordinates. g., gyrus; post., posterior; LOC, lateral occipital complex; PF, posterior fusiform, signif., significance; max, maximum. All effects were computed by thresholding the contrast’s Z image at 3.8 and then testing for significance at p < 0.05, **except for the right supramarginal gyrus – this activation was extensively connected to other clusters and so was tested only at the voxelwise level.

163

Size

Max Location

Signif.

(Voxels)

Max Z

x

y

L precuneus (7)

476

5.76

-5

-66

35

7.68

R superior frontal g. (10)

90

5.09

30

57

15

2.28

R inferior parietal lobule (40)

46

4.79

66

-30

30

1.33

R middle occipital g. (LOC)

175

5.18

46

-69

7

4.71

L post. Fusiform (PF)

36

4.3

-41

-56

-11

2.02

R post. Fusiform (PF)

26

4.52

34

-54

-11

1.74

R supramarginal g.

**

3.11

42

-44

35

**

-14

-34

-14

1.31

Structure

z

log10(p)

Repetition: 5 > 1

Preference: L > H

(9 Subjects) Preference: H > L L parahippocampal g. (36)

15

4.47

Table 6.1 (continued).

Group Analysis: Regions of Interest ANOVA

Given the problems with subject registration and anatomical variability in the whole brain group analysis, we sought to create a set of more constrained statistical hypotheses using a region of interest analysis. By selecting a set

164

of voxels in each subject’s individual data space and averaging the timecourses over those voxels, one can eliminate the drastic reduction in statistical power one suffers when performing a whole brain analysis. In our region of interest analysis, we focused on the effects of preference and repetition within a region of the parahippocampal gyrus, LO, early visual cortex, and the lingual gyrus.

Regions of interest were drawn based on either anatomical scans (for the parahippocampal gyrus, lingual gyrus, and early visual cortex) or separate localizer scans (for LO) in each individual subject. For the LO localizer scan, we examined a contrast of Intact > Scrambled, collapsed across all image types (novel, familiar, line drawing, and grayscale object). This contrast was then superimposed on the subject’s high resolution EPI scan and used to select a contiguous region of voxels in the lateral occipital cortex, primarily in the middle and inferior occipital gyri. No attempt was made to differentially identify a more posterior (LO) versus anterior (PF) locus, though it is likely that the majority of ROI’s drawn reflect the more posterior and superior (LO) part of the lateral occipital complex.

These ROIs were then used to extract the original waveform from the individual subject’s timeseries data, which was analyzed using a GLM-based averaging procedure implemented within AFNI (Analysis of Functional

165

NeuroImages; Cox, 1996). Importantly, instead of using a canonical impulse response function (IRF) to fit the data, this procedure estimates the impulse response function from the data itself.

The data from the nine subjects selected in the previous analysis are shown in Fig. 6.12. However, the data for both left and right LO are from only seven of those subjects, as independent localizer scans were not available for two of the nine subjects. High preferred images led to a greater response than either Medium or Low preferred images in the left parahippocampal ROI (Fig. 6.12a), and a greater response than Low preferred images in the right parahippocampal ROI (Fig. 6.12b). LO showed less activity for High preferred images compared to Low preferred, while the Medium image group showed the greatest response in these ROI’s (Fig. 6.12 c & d). This result is in agreement with the finding that the Medium image group contained, on average, more objects than the Low image group, which in turn contained more objects than the high image group. Striate cortex showed no difference in the three preference groups (Fig. 6.12e), though it did show much higher average activity.

166

Figure 6.12. Region of Interest (ROI) analysis for the nine subjects subselected in the previous section. The average hemodynamic impulse response functions for High, Medium, and Low Preference are plotted for ROI’s in A) left parahippocampal cortex, B) right parahippocampal cortex, C) left lateral occipital cortex, D) right lateral occipital cortex, E) and striate cortex. The data for the LO ROI’s does not include two subjects for which an independent localizer scan was not available. Dashed lines indicate the standard errors of the subject means.

167

Figure 6.12 (continued).

168

Figure 6.12 (continued).

Using the canonical hemodynamic response function provided with the SPM99 software (Friston et al., 1995), a fit was calculated by multiplying this model function with the hemodynamic response functions estimated for each condition in each ROI (Fig. 6.13). Within the left parahippocampal ROI, High preferred images produced greater activity than Low preferred images for the first three exposure levels, with the difference disappearing on the fourth and fifth presentation. The Medium image category, however, showed a somewhat different profile. On average, activation in this condition fell between that for the High and Low image categories (Fig. 6.12a), though its activity on the first presentation in the left parahippocampal ROI was less than that of the Low preference images (Fig. 6.13). In addition, it appears as

169

if the slight rebound seen on later repetitions in this area was largely driven by the Medium image category.

Figure 6.13. Using the canonical hemodynamic impulse response function (HRF or IRF) from SPM99 as a model, a fit was derived for each condition and plotted. Dashed lines indicate the standard errors of the subject means.

Individual Differences in Activation Patterns

While the effects of repetition appear to be quite robust across individual subjects, the effects of preference are much more variable. Overall, it appears as if a majority of subjects show a positive effect of a priori image

170

preference in a posterior region of the parahippocampal gyrus, though to varying degrees. Given the variability in the statistical reliability and spatial locus of effects in individual subjects, it is difficult to define a single quantitative measure capturing the presence or absence of this effect in all subjects. However, it appears as if approximately twelve of the sixteen subjects show at least some degree of preference related activity in an area in or near the parahippocampal cortical locus we have identified. As shown previously, nine of these show overlap of these effects on a small subset of voxels. At the individual subject level, however, these effects can be quite robust (Fig. 6.14).

Figure 6.14. Two individual subjects showing strong effects of preference in the parahippocampal gyrus. The High > Low contrast was thresholded at 3.1 for the subject in A and 2.5 for the subject in B to produce well defined clusters. These clusters were then tested for significance at the 0.05 level. These statistical images have been overlaid on the subject’s own high resolution EPI image.

171

In addition to this characteristic activity pattern, several subjects showed effects of preference scattered throughout the ventral visual pathway. Figure 6.15 shows an example of a subject with preference effects at several loci in the frontal cortex and early visual areas, in addition to an activation in the right lingual gyrus (just posterior to the locus observed in the group average, but in the opposite hemisphere).

Figure 6.15. An individual subject showing preference effects (High preference greater than Low preference images) at a number of locations within the ventral visual pathway and frontal cortex.

172

Future studies which collect preference ratings in the magnet for each individual subject may reduce the amount of variability in the individual preference results, and allow for a more meaningful investigation of the differences between subjects.

Repetition “Rebound”

The group data show a rebound in large sections of the ventral visual pathway for the later repetitions (most evident on the fifth repetition but present on earlier repetitions as well). This effect is also evident at the single subject level. To rule out the possibility that this effect may have been due to slight differences in the mean position of an image’s first repetition and its last repetition (Fig. 6.16a), we examined the effects of repetition for a different general linear model which did not differ in the mean position of the repetition conditions. The same effects are present in this image-based correlation analysis, which included all of the “buffer” images in the parameter estimates for the five repetitions, eliminating any difference in the mean trial position (Fig. 6.16b). In addition, as can be seen from the parahippocampal ROI data (Fig. 6.13), this rebound was not identical for all three preference levels, while there were no differences in the average serial position of the images in the different preference categories.

173

Figure 6.16. The rebound seen on the fifth presentation of an image is not due to differences in stimulus timing. A) Input vectors for First (top, red) and Fifth (bottom, green) repetitions for the High Preference images in the fifteen parameter factorial model. The fifth presentation of each image came, on average, 80 TR later than the first presentation. The vectors for Medium and Low conditions are similarly staggered. B) Input vectors for the First (top, red) and Fifth (bottom, green) repetitions in the six parameter image-based correlation analysis (this analysis was analyzed separately for the two sessions). Note that for this model there is no overall temporal shift in the input vectors for First and Fifth presentations, as the buffer images were included. However, a rebound in signal on the fifth presentation was still observed. C) Representative data from one voxel analyzed using the parameters in A or B.

Image-Based Correlation Analysis

The grouping of our image set into high, medium, and low preference categories allowed us to design a fully factorial, mixed design experiment. However, these groupings were based on the mean ratings of a separate set

174

of subjects, allowing us to perform an image-based correlation analysis which capitalized on the variation within each preference level.

We used this analysis technique to create a summary of how preference correlated with activity at various stages within the ventral visual stream. We sought to assess the plausibility of a graded change in preference-related activity as would be predicted by the involvement of µ opioid receptors as mediators of visual preference.

A major problem with this analysis is the lack of easily defined anatomical “stages” in the ventral visual pathway: the subject variability in both the physical placement of a particular anatomically defined cortical field and the mapping between anatomy and functionality make it very difficult to define a set of processing stages in all subjects. We did not perform independent localizers for faces, places and retinotopic cortex in all of our subjects, which would have allowed us to select such cortical fields based on the functional response of each voxel. Instead, we overcame this variability by choosing a set of four cortical regions defined by a combination of individual anatomy, independent functional localizers, and stimulus factor deconvolutions of our scene preference experiment.

175

The first ROI chosen was striate cortex, defined anatomically in the subject’s data space as described above. The second ROI chosen was defined as object responsive cortex. For this ROI, we used both the LO ROI defined above using an independent localizer scan as well as an ROI defined from the scene preference experiment using the “objectness” factor described in Chapter Five. The extraction of this and the next two ROI’s will be described in detail below. Our third ROI was defined as “face” responsive cortex, and our fourth ROI as those voxels responding to our “place” factor. While these four ROI’s did not cover the entirety of the ventral visual pathway, they allowed us to look at the correlation of a priori image preference and the BOLD signal at four points along the ventral visual pathway defined in individual subjects.

The latter three ROI’s were defined using a stimulus based deconvolution of the data from our scene preference experiment. One subject was excluded from this analysis completely due to a lack of data for some of the sixty images used. For the remaining fifteen subjects, four regressors were created by taking the image sequence used for that subject and specifying each image’s rating on a subset of the factors reported in Chapter Five. For this analysis, we used RMS contrast, objectness, the presence of faces, and the degree to which each image depicted a place. Each of these factors was scaled to vary between zero and one, and we also included a fifth factor

176

specifying the exposure number of each image, also scaled to vary between zero and one. These five factors were then used in a general linear model similar to that described above for the fully factorial analysis (Fig. 6.1), including each factor plus its temporal derivative as parameters in FILM. RMS contrast and exposure number were included to help make the model a more complete description of the signal components we expected to see and remove any contribution of contrast or repetition from the estimates of the other factors. The computed parameter estimates were then used to derive contrasts and t tests for each factor.

Unlike other work which has used pure stimuli to nominate stimulus selective cortical fields, we did not create contrasts which, for example, nominated voxels which responded greater to places than to objects and faces. Since our basic method of analyzing each factor’s contribution to a voxel’s activity is through correlation, any voxel which responds strongly to more than one factor will already have a lower weight than one which responds more selectively. We therefore used contrasts which weighed the factor of interest as one and all other factors as zero. In order to produce masks with well defined regions, we then thresholded these contrasts (t-statistic images) at a voxelwise t of 2.58 (corresponding to p < 0.01 uncorrected), and subsequently eliminated all clusters which were not significant at the cluster level threshold of 0.3 (using Gaussian Random Field Theory as described

177

previously). This very relaxed threshold produced factor related clusters for most subjects and most factors. Each of the masks was then edited to remove any clusters which were not in the ventral visual pathway, loosely defined as the occipital lobe plus the inferior temporal, fusiform, and parahippocampal gyri.

While this method allows us to draw “stimulus specific” regions of interest, there is a potential drawback. These regions were not selected using “pure” stimuli, but only a specification of when each image contained the specified stimulus type. Therefore, any spurious correlations between different stimuli present in the images will hinder our ability to define category specific voxels. While the correlations between most of the factors were low enough to not make this a concern, the correlation of 0.43 between the presence of objects and faces may have led to some difficulty in defining separate clusters of voxels. In the present analysis, we allowed voxels which were nominated strongly by more than one factor to be part of both ROI’s.

The surviving voxels for each covariate were then used as a mask to select voxels from each subject’s scene preference experiment. The timecourses of each set of voxels were averaged together to form single timecourses for object responsive voxels, face responsive voxels and place responsive voxels. A similar timecourse was constructed by averaging the voxelwise

178

timecourses of the anatomically defined striate ROI and the LO ROI on each side defined by the independent localizer scan. Note that unlike this ROI, the ROI’s defined by the factor covariates did not distinguish between the left and right hemisphere, and in some cases included sets of non-contiguous voxels. For example, it should be noted that the “face responsive voxels” include two non-contiguous areas on each side of the brain, one in a more ventral location corresponding to the fusiform face area and a second on the lateral surface of the occipital lobe.

These timecourses were then used to derive IRF’s for each of the sixty images used in the experiment for each ROI in each subject. This IRF was then averaged over the set of subjects with data for that ROI, and each timepoint of this average IRF was then correlated with a priori image preference for each ROI individually. As can be seen in the ROI plots in figure 6.12, the second timepoint is the maximum of the IRF. We therefore report the correlation between preference and average activity of the second timepoint for each factor. These correlations are reported in Table 6.2. Figure 6.17 shows the average location of each of these 4 ROI’s superimposed on an average anatomical brain with the corresponding correlations.

179

Correlation with Preference by ROI Correlation of Region of Interest Activity and Preference Striate Cortex -0.053 Object Responsive Cortex -0.295 (from preference scan) Left LOC (from LOC localizer) -0.101 Right LOC (from LOC localizer) -0.350 Face Responsive Cortex -0.091 Place Responsive Cortex 0.213

Number of Subjects 15 6 11 11 11 9

Table 6.2. Correlations of activity and preference for four regions of interest. Activity was defined as the maximum height of the average hemodynamic impulse response function over subjects for which each ROI could be established.

While the correlations do not progress in a monotonic fashion from early to late visual areas, these results do indicate an increase in the importance of more anterior cortex for stimulus preference. It is likely that the increased complexity of the human ventral visual pathway over that of the macaque may have led to some departure from a strictly linear increase in the importance of anterior cortical fields for preference. It is also possible that the inclusion of non-ventral voxels in the “face selective” ROI may have weakened the correlation between activity in this ROI and preference. However, as was shown in Chapter Five, the presence of a face in a stimulus is not a good predictor of preference.

180

Figure 6.17. Correlation of image preference and maximal activity in four ROI’s within the ventral visual pathway. The correlation is near zero in striate cortex, is negative in object responsive voxels, approaches zero again in face responsive voxels, and is positive for place responsive voxels. These correlations were computed by taking the average waveform in the preference experiment over all voxels of the ROI (disregarding laterality), computing the IRF for each image, and correlating this with that image’s a priori preference. A) The regions of interest are displayed on a series of axial slices of the MNI standard atlas spaced 8mm apart, starting with the most inferior slice in the upper left (Talairach z = -19) to the most superior in the lower right (Talairach z = 16). B) This rendering shows the position of the four ROI’s superimposed on the MNI standard atlas with the inferior and posterior sections of brain remaining. The position of the brain is as if it were being viewed from above and slightly to the left.

181

Conclusions

Scenes that received high preference ratings produced greater activity in the parahippocampal gyrus than scenes that received low preference ratings, an effect consistent with the endomorphin hypothesis. However, individual subjects appear to show large variation in the anatomical locus of preference related activity. The fact that several individual subjects show very strong effects of preference, despite their lack of anatomical overlap, may be an indication that: a) the spatial distribution of endomorphin receptors is variable across individuals, and/or b) the neural loci of the cells sensitive to variables that affect scene preference are variable across individuals. While it is likely that human opioid gradients would follow a general increase from early to late areas, either or both of these factors could be the reason for two people seeing the world in the same way but with preferences for differing aspects of the same stimulus.

This positive effect of preference was generally not apparent in the earlier stages of the ventral pathway and negative in area LO. The negative relationship between preference and the LO response provides strong evidence that overall activity in the ventral pathway is not what leads to preference for a stimulus, but only activity in the more anterior regions that

182

are likely activated by rich interpretations and are associated with high levels of endomorphin receptors.

Repetition produced a consistent decline in activity in the parahippocampal gyrus which tracked the preference ratings but produced a U-shaped function in more posterior areas. The increase in activity over repetitions 3 through 5 likely reflected an increase in active processing of the scenes. It is interesting to note that repetition, which accounted for less of the variance than a priori preference in Experiment 2 (9.4% and 78% respectively) led to much greater activity differences in the resultant BOLD signal. This suggests that the activation of a very few cells in the most anterior, endomorphin-rich areas of the temporal lobes can outweigh the activity from a larger population of cells in a more posterior area. In a sense, the neural activity subserving preference effects may retain the mapping observed in the ventral pathway in general whereby there is a many-to-one mapping in that receptive fields in anterior temporal lobe (which in TE can be larger than 70 degrees) are dramatically larger than those in V1 (0.5-2.0 degrees).

Our finding of preference and repetition effects in a posterior portion of the parahippocampal gyrus have interesting implications for a growing body of research implicating a similar, if not overlapping region in the representation of places (Epstein et al., 1999; Talairach coordinates 20, -39, -5 and -28, -39,

183

-6) and the analysis of visual context (Bar & Aminoff, 2003; Talairach coordinates -24, -41, -4). Our preference related activation was just anterior and inferior to this activation (-14, -34, -14), while our repetition related decreases were virtually overlapping (18, -40, -10 and -25, -42, -8). From an adaptation standpoint, it seems reasonable that the repetition of an image set comprised primarily of full color scenes would lead to repetition related decreases in an area which is involved in the representation of place. In addition, highly contextual objects are more relatable to memory and lead to richer interpretations. What our work demonstrates is that objects and scenes rich in contextual, interpretable information are also pleasurable to look at -- at least the first few times.

Our results provide some confirmation of the theoretical account of information foraging behavior: high activity in the opioid receptor rich areas of the ventral temporal cortex provides a “win” signal to the brain systems mediating response selection and preference. Neural adaptation of this signal through competitive learning leads to a decrease, triggering a “shift” response to other stimuli (or aspects of the scene) which is likely to result in additional novel associations. Under circumstances in which there is no alternative to a boring stimulus yet there is a motivation to maintain attention (what else are you going to do in the magnet?), voluntary attentional effort is required.

184

CHAPTER 7: CONTRIBUTIONS AND FUTURE DIRECTIONS

The goal of this thesis was to develop a theory of cognitive and perceptual pleasure by investigating the behavioral properties of visual preference and its relation to the known organization of the ventral visual pathway. Below I will outline the specific contributions this work has made to our understanding of positive perceptual affect. In addition, I will outline several future directions suggested by the work. Several of these questions will be addressed as part of my postdoctoral research efforts.

Contributions

A major contribution of this work is the recognition of a need for a theory of perceptual affect. Understanding how the brain analyzes the visual world does not answer the question of motivation: why are some things processed preferentially, and how does the brain choose what gets ignored and what gets processed? Although issues of stimulus selection have been studied under the guise of visual attention, the need for links to stimulus interpretability and the pleasurable aspects of obtaining information had not been recognized.

185

The behavioral data presented here have contributed significantly to our understanding of perceptual affect. Using a well defined, naturalistic stimulus set and rigorous testing situations, we replicated previously shown effects of stimulus repetition on preference, and showed that these effects hold over a wide range of initial preferences. Second, we replicated previous reports in the landscape assessment literature on the predictability of preference from mystery, naturalness, and the presence of a good view (vista). We showed that for full color scenes, these types of factors account for a larger proportion of stimulus preference than factors such as the presence of objects, faces, and places. In addition, we also provided a unifying framework for understanding the effects of novelty, familiarity, and stimulus interpretability on preference.

Third, our brain imaging experiments have had a significant impact on our understanding of the physiological basis of perceptual affect. Using BOLD fMRI, we found that highly preferred images and novel images lead to greater activity in parts of the parahippocampal cortex, and that these same areas showed the highest correlation with image preference. This finding of preference-related activity in a perceptual pathway provides evidence for the possible involvement of opioid gradients present in the human and nonhuman primate ventral visual pathway. A major aspect of this contribution was the development of a balanced, event-related fMRI paradigm for testing

186

the effects of repetition in a mixed design that also controlled for effects of preference context. Our imaging results also show that these effects observed in later visual areas are not merely feedforward effects from earlier areas, as preference was negatively correlated with activity in area LO. This data underlines the importance of a differential weighting of activity like that which would be produced by a neurochemical gradient. Finally, our results provide a novel interpretation of voluntary attention, in that we found evidence for the activation of brain networks thought to be involved in the deployment of voluntary attentional resources when stimuli are non-preferred either due to less intrinsic interest (interpretability) or extensive repetition.

Future Directions

The theory we have proposed raises a large number of questions which we have only begun to address in the series of experiments presented here. Below, I will outline several directions in which this work could be expanded in order to test specific claims of the theory and its generality to other stimulus domains.

187

Behavioral Studies

The most basic extension of the work we have done is to test whether the decrease in repetition with repeated exposure we observed in Experiments 1 and 2 are solely a function of the number of times an image has been seen, or whether the observed effect is modulated by presentation time. In all of our experiments, the images were shown for a full second. As was discussed in Chapter Three, the effects of preference may be linked with the phenomenon of priming. Zago & Bar (2003) showed that a prime on the order of 250 ms is most effective, while both shorter and longer durations are less effective as primes. Due to the hypothesized importance of visual preference for information foraging, it is possible that the peak of this function at 250 ms may be related to the time of an average fixation. A 250 ms exposure of an image may be sufficient to produce the repetition related decreases in preference we observed in our experiment. Similarly, it is possible that shorter exposure durations may not lead to repetition related decreases, and may even reflect an inverted U shaped function as a result of initial “mere exposure” effects and subsequent habituation. By testing several shorter exposure durations, we can test how tightly linked visual preference is to priming, and whether the mere exposure effect and repetition related decreases in preference are two ends of a continuum.

188

In addition to timing effects, we also plan to investigate whether there are Differences in preferences when ratings are collected on every presentation versus passive viewing. This will allow us to address whether the act of making a preference judgment influences the degree of preference habituation.

Eyetracking

Although preference judgments are quite stable within an individual, it is unclear whether such explicit judgments provide an accurate measure of perceptual affect. The subject’s expectations, social norms and changing criterion may all adversely affect the link between a person’s perceptually driven affective reaction and their rated preference. Since a major functionality of an information foraging system is to guide spontaneous selection, we plan to measure how well eye movement patterns can be predicted from preference judgments. As a simple test, we plan on using a preferential looking paradigm like that used in infants, in which two pictures are shown to a subject side-by-side on the screen while eye movements are monitored. On any one trial, the subject will be shown two images side by side on a computer monitor which differ in their a priori preference (high, medium, or low), the number of times they have been seen before, or both. In a similar task, Berlyne (1958) and found that total fixation time (but not first

189

fixation) decreased monotonically to a repeated picture, though he did not assess effects of preference. We predict that total fixation time and/or first fixations will be greater for images of high a priori preference and images which are novel.

Individual Differences in the Neural Correlates of Preference

Our ability to detect preference related changes in Experiment 4 was limited by the fact that group preference ratings are able to account for an average of only 46% of the variance in individual preferences. This tradeoff allowed us to observe preference related changes in the absence of explicit preference judgments. To complement our findings, we plan to perform an FMRI experiment in which preference ratings are collected for all 200 images as the subject views them in the magnet. This design, while it does not dissociate between activity related to affective preference and that related to the act of making preference judgments, will allow us to capitalize on the individual differences in preference and better localize preference related neural activity. In addition to the activity we observed in the ventral visual pathway and parietal regions in Experiment 4, we would predict that frontal networks shown to be involved in decision making and executive functions should also be active (Fuster, 2000).

190

Pharmacology

As a direct test of the involvement of the µ opioid receptor gradient in positive perceptual affect, normal subjects could be administered µ opioid agonists (such as fentanyl) or antagonists (such as naloxone or naltrexone) while performing a visual preference task. In order to demonstrate a selective effect of a drug such as naloxone on visual preference, we would require the subject to perform several visual tasks in addition to the preference task, such as a visual recognition test and a fear cueing test. These same tasks could potentially be studied in a separate set of subjects on a GABAergic or cholinergic [antagonist] (such as benzodiazapene and scopolamine, respectively) to rule out any nonspecific effects of inhibition or memory impairment on visual preference (e.g. Robbins et al., 1997).

Natural vs. Urban: Distribution of Image Factors in the Ventral Visual Pathway

The high correlation of preference with the natural vs. urban dimension points to a potentially interesting prediction for how this factor is represented in higher order visual areas. Unlike objects, a landscape viewed from a distance does not require much rotational invariance and contains very few properties which would aid in the decomposition of the landscape into shape

191

primitives. Instead, classification of landscapes may be mediated by color, texture, and global form. Light blue above darker blue abutting beige may be a reliable cue combination for a beach, while a solid mass of green may signal forest or park. In a study comparing the naming speeds of “colordiagnostic” and “color-nondiagnostic” scenes, Oliva & Schyns found that reversing the colors of a color-diagnostic scene (either a canyon, forest, coastline, or desert) increased subjects’ reaction times above that of pure luminance (black and white) versions of the scenes, which were in turn significantly worse than the originals (Oliva & Schyns, 2000). This was in contrast to color-nondiagnostic scenes (which included the categories city, shopping area, road, and room), which showed no cost for either the luminance only or color reversed versions. The authors claim that this difference comes from the fact that in color space, the color-diagnostic scenes are largely non-overlapping, while the nondiagnostic (urban) scenes significantly overlapped. An alternative, but complimentary explanation might be that the urban scenes show no cost due to the relative ease of extracting non-accidental properties and object independent geon structural descriptions. Variation in the natural vs. urban dimension is therefore likely to modulate the balance between structural and textural information and produce differences in the distribution of activity in the form perception pathway. It is possible that this dimension corresponds to a medial / lateral distinction in processing, with urban scenes leading to greater activity in the

192

lateral aspect of this pathway (leading out of LO) and natural scenes producing greater activity in the medial aspect (leading out of the lingual gyrus and into the parahippocampal gyrus). An fMRI study could be designed to map such a distribution. A set of such maps would then allow for a more complete picture of whether the activity distribution across the ventral visual pathway can be used to predict stimulus preference, as would be the case if a gradient of µ opioid receptors mediates between neural activity and preference.

Learning Effects

We plan to investigate how preference interacts with the learning of novel objects and object classes. Preference may affect how well an object is learned, and the context in which an object is learned may in turn affect its subsequent preference. Such relationships likely have important consequences for how particular stimuli are represented in the brain. We plan to study these questions by varying the interpretability of the context in which an object is learned, and by studying the development of visual expertise for object classes of differing preference. One possible experiment along these lines would utilize a paradigm similar to that of Beauchamp et al. (2001) in which small tokens are shown moving in patterns which are either random, passive (like billiard balls), or mimicking human movement. By

193

varying the context in which a token is encountered, we can test whether its association with a rich interpretation leads to a preference for that token by itself. A second set of experiments would involve the development of a training program for a class of visual experts. Performance on an expert discrimination task would be measured within groups of subjects which are learning both a preferred visual discrimination and a nonpreferred discrimination. Such an experiment would allow us to asses whether a person’s preferences for certain types of expert discriminations affect their learning rate for the discriminations and the underlying neural representations of the new object class. Alternatively, a simpler experiment would involve the presentation of images and a preference rating during one session, followed several days (or weeks) later by a recognition test to see if preferred stimuli are better recognized than nonpreferred stimuli. Recognition memory performance and preference could also be related to the amount of neural activity in medial temporal lobe structures previously observed to predict recognition memory (Fernandez et al., 1999).

Cross-Modal and Conceptual Priming

The theory we presented in Chapter Two can be generalized to perceptual modalities other than vision, and even to the combination of cues from multiple modalities. In addition to testing whether novelty and interpretability

194

can be used to predict preference for auditory, haptic, or taste stimuli, it is important to understand how preference is influenced by the degree to which information from two modalities is integrated

Direct Measures of Preference

A major drawback of the research we have done so far is the lack of a more objective measure of positive affect. While measuring eye movement patterns moves us a bit closer towards a quantitative measure of preference, they are not a pure measure of the contribution of information foraging and may reflect other mechanisms (such as a response to sudden onsets or motivated searches) under certain circumstances.

As a means of measuring positive affect more directly, it may be useful to explore the use of EMG. Winkielman & Cacioppo (2001) found that an electromyographic signal from the zygomatic muscle provides a reliable signal of positive perceptual affect even during subliminal exposures. While this technique has not been explored within the context of full color scenes nor under repetition of the same stimuli, it is possible that measuring the activity from the “smile muscles” during perceptual tasks would allow for a quantitative study of the effects of interpretability and habituation on positive perceptual affect.

195

BIBLIOGRAPHY Adolphs, R., Denburg, N. L., & Tranel, D. (2001). The amygdala's role in long-term declarative memory for gist and detail. Behavioral Neuroscience, 115(5), 983-992. Aitken, P. P. (1974). Judgments of pleasingness and interestingness as functions of visual complexity. Journal of Experimental Psychology, 103(2), 240-244. Appleton, J. (1975). The Experience of Landscape. London: Wiley. Appleton, J. (1988). Prospects and refuges revisited. In J. L. Nasar (Ed.), Environmental Aesthetics (pp. 27-44). New York: Cambridge University Press. Arnsten, A. F. T., Neville, H. J., Hillyard, S. A., Janowsky, D. S., & Segal, D. S. (1984). Naloxone Increases Electrophysiological Measures of Selective Information-Processing in Humans. Journal of Neuroscience, 4(12), 2912-2919. Attneave, F., & Arnoult, M. D. (1956). The quantitative study of shape and pattern perception. Psychological Bulletin, 53, 452-471. Bar, M., & Biederman, I. (1998). Subliminal visual priming. Psychological Science, 9(6), 464-469. Bar, M., & Biederman, I. (1999). Localizing the cortical region mediating visual awareness of object identity. Proc. Natl. Acad. Sci. USA, 96, 1790-1793. Bar, M., Tootell, R. B., Schacter, D. L., Greve, D. N., Fischl, B., Mendola, J. D., Rosen, B. R., & Dale, A. M. (2001). Cortical mechanisms specific to explicit visual object recognition. Neuron, 29(2), 529-35. Bar, M., & Aminoff, E. (2003). Cortical analysis of visual context. Neuron, 38, 347-358. Barlow, H. B. (1961). Possible principles underlying the transformation of sensory images. In W. A. Rosenblith (Ed.), Sensory Communication (pp. 217-234). Cambridge, MA: MIT Press.

196

Beckmann, C. F., Smith, S. M., & Jenkinson, M. (2001). General multi-level linear modeling for group analysis in FMRI (Technical Report TR01CB1). Oxford, UK: Oxford Centre for Functional Magnetic Resonance Imaging of the Brain (FMRIB). Bergman, R., Williams, T. H., Gluhbegovic, N., & Jew, J. (2001). Atlas of Human Anatomy in Cross Section, [Web document]. The University of Iowa: Virtual Hospital. Available: http://www.vh.org/Providers/Textbooks/HumanAnatomy/CrossSection Atlas.html. Berlyne, D. E. (1958). The influence of complexity and novelty in visual figures on orienting responses. Journal of Experimental Psychology, 55(3), 289-296. Berlyne, D. E. (1963). Novelty, complexity, incongruity, extrinsic motivation, and the GSR. Journal of Experimental Psychology, 66(6), 560-567. Berlyne, D. E. (1970). Novelty, complexity, and hedonic value. Perception & Psychophysics, 8(5A), 279-286. Biederman, I., & Cooper, E. E. (1991). Priming contour-deleted images: evidence for intermediate representations in visual object recognition. Cognitive Psychology, 23(3), 393-419. Biederman, I., & Gerhardstein, P. C. (1993). Recognizing depth-rotated objects: evidence and conditions for three-dimensional viewpoint invariance. Journal of Experimental Psychology: Human Perception and Performance, 19(6), 1162-1182. Bornstein, R. F. (1989). Exposure and Affect - Overview and Meta-Analysis of Research, 1968-1987. Psychological Bulletin, 106(2), 265-289. Bornstein, R. F., Kale, A. R., & Cornell, K. R. (1990). Boredom as a limiting condition on the mere exposure effect. Journal of Personality and Social Psychology, 58, 791-800. Bornstein, R. F. (1992). Subliminal mere exposure effects. In R. F. Bornstein & T. S. Pittman (Eds.), Perception Without Awareness: Cognitive, Clinical, and Social Perspectives (pp. 191-211). New York: The Guilford Press. Brainard, D. H. (1997). The psychophysics toolbox. Spatial Vision, 10, 443446.

197

Buchsbaum, M. S., Reus, V. I., Davis, G. C., Holcomb, H. H., Cappelletti, J., & Silberman, E. (1982). Role of opioid peptides in disorders of attention in psychopathology. Annals of the New York Academy of Sciences, 398, 352-365. Buckner, R. L., Goodman, J., Burock, M., Rotte, M., Koutstaal, W., Schacter, D., Rosen, B., & Dale, A. M. (1998). Functional-anatomic correlates of object priming in humans revealed by rapid presentation event-related fMRI. Neuron, 20(2), 285-296. Buckner, R. L., & Braver, T. S. (1999). Event-related functional MRI. In C. T. W. M. P. A. Bandettini (Ed.), Functional MRI . Bullier, J., Schall, J. D., & Morel, A. (1996). Functional streams in occipitofrontal connections in the monkey. Behavioural Brain Research, 76(12), 89-97. Burock, M. A., Buckner, R. L., Woldorff, M. G., Rosen, B. R., & Dale, A. M. (1998). Randomized event-related experimental designs allow for extremely rapid presentation rates using functional MRI. Neuroreport, 9(16), 3735-3739. Cahill, L., Haier, R. J., Fallon, J., Alkire, M. T., Tang, C., Keator, D., Wu, J., & McGaugh, J. L. (1996). Amygdala activity at encoding correlated with long-term, free recall of emotional information. Proceedings of the National Academy of Sciences of the United States of America, 93(15), 8016-8021. Cheng, K., Saleem, K. S., & Tanaka, K. (1997). Organization of corticostriatal and corticoamygdalar projections arising from the anterior inferotemporal area TE of the macaque monkey: A Phaseolus vulgaris leucoagglutinin study. Journal of Neuroscience, 17(20), 7902-7925. Cohen, G. A., Doze, V. A., & Madison, D. V. (1992). Opioid inhibition of GABA release from presynaptic terminals of rat hippocampal interneurons. Neuron, 9, 325-335. Corbett, A. D., Paterson, S. J., & Kosterlitz, H. W. (1993). Selectivity of ligands for opioid receptors. In A. Hertz (Ed.), Opioids I (1st ed., Vol. 104, pp. 645-673). Berlin: Springer. Corbetta, M., & Shulman, G. L. (2002). Control of goal-directed and stimulusdriven attention in the brain. Nature Reviews Neuroscience, 3, 201215.

198

Cox, R. W. (1996). Software for analysis and visualization of functional magnetic resonance neuroimages. Computers and Biomedical Research, 29, 162-173. Dale, A. M., & Buckner, R. L. (1997). Selective averaging of rapidly presented individual trials using fMRI. Human Brain Mapping, 5, 329340. Donaldson, D. L., & Buckner, R. L. (2001). Effective paradigm design. In P. Jezzard, Matthews, P.M., Smith, S.M. (Ed.), Functional MRI: An Introduction to Methods (pp. 177-195). Elliott, R., & Dolan, R. J. (1998). Neural response during preference and memory judgments for subliminally presented stimuli: a functional neuroimaging study. Journal of Neuroscience, 18(12 (June 15)), 46974704. Epstein, R., Harris, A., Stanley, D., & Kanwisher, N. (1999). The parahippocampal place area: Recognition, navigation, or encoding? Neuron, 23(1), 115-125. Fernandez, G., Brewer, J. B., Zhao, Z., Glover, G. H., & Gabrieli, J. D. (1999). Level of sustained entorhinal activity at study correlates with subsequent cued-recall performance: A functional magnetic resonance imaging study with high acquisition rate. Hippocampus, 9, 35-44. Fiser, J., & Biederman, I. (2001). Invariance of long-term visual priming to scale, reflection, translation, and hemisphere. Vision Research, 41, 221-234. Friedman, D. P., Aggleton, J. P., & Saunders, R. C. (2002). Comparison of hippocampal, amygdala, and perirhinal projections to the nucleus accumbens: Combined anterograde and retrograde tracing study in the macaque brain. Journal of Comparative Neurology, 450(4), 345365. Friston, K. J., Holmes, A. P., Worsley, K. J., Poline, J. B., Frith, C. D., & Frackowiak, R. S. J. (1995). Statistical parametric maps in functional imaging: a general linear approach. Human Brain Mapping, 2, 189210.

199

Fuster, Joaquin M. (2000). Executive frontal functions. Experimental Brain Research, 133(1), 66-70. Goldstein, A. (1980). Thrills in Response to Music and Other Stimuli. Physiological Psychology, 8(1), 126-129. Goodale, M. A., & Milner, A. D. (1992). Separate visual pathways for perception and action. Trends in Neurosciences, 15(1), 20-25. Grill-Spector, K., Kushnir, T., Hendler, T., Edelman, S., Itzchak, Y., & Malach, R. (1998). A sequence of object-processing stages revealed by fMRI in the human occipital lobe. Hum Brain Mapp, 6(4), 316-28. Grill-Spector, K., Kushnir, T., Edelman, S., Avidan, G., Itzchak, Y., & Malach, R. (1999). Differential processing of objects under various viewing conditions in the human lateral occipital complex. Neuron, 24(1), 187203. Grill-Spector, K., Kourtzi, Z., & Kanwisher, N. (2001). The lateral occipital complex and its role in object recognition. Vision Research, 41, 14091422. Hackler, L., Zadina, J. E., Ge, L. J., & Kastin, A. J. (1997). Isolation of relatively large amounts of endomorphin-1 and endomorphin-2 from human brain cortex. Peptides, 18(10), 1635-1639. Harlow, H. F. (1986). From Learning to Love: The Selected Papers of H.F. Harlow. New York: Praeger Publishers. Haxby, J. V., Gobbini, M. I., Furey, M. L., Ishai, A., Schouten, J. L., & Pietrini, P. (2001). Distributed and overlapping representations of faces and objects in ventral temporal cortex. Science, 293(5539), 2425-2430. Heerwagen, J. H., & Orians, G. H. (1993). Humans, Habitats, and Aesthetics. In K. S.L. & E. O. Wilson (Eds.), The Biophilia Hypothesis (pp. 138172). Covelo, California: Island Press. Heindenreich, S. M., & Turano, K. A. (2003). What predicts where one will look when viewing artwork? (Poster Presented at the 3rd Annual Meeting of the Vision Sciences Society in Sarasota, Florida. MO80). Henson, R., Shallice, T., & Dolan, R. (2000). Neuroimaging evidence for dissociable forms of repetition priming. Science, 287(5456), 12691272.

200

Hille, B. (1992). Ionic channels of excitable membranes (2nd ed.). Sunderland, Mass.: Sinauer Associates, Inc. Hiller, J. M., & Fan, L.-Q. (1996). Laminar distribution of the multiple opioid receptors in the human cerebral cortex. Neurochemical Research, 21(11), 1333-1345. Hurd, Y. L. (1996). Differential messenger RNA expression of prodynorphin and proenkephalin in the human brain (vol 72, pg 767, 1996). Neuroscience, 74(1), 293-293. Ishai, A., Ungerleider, L. G., Martin, A., Schouten, J. L., & Haxby, J. V. (1999). Distributed representation of objects in the human ventral visual pathway. Proceedings of the National Academy of Science, USA, 96, 9379-9384. Itti, L., Koch, C., & Niebur, E. (1998). A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(11), 1254-1259. Iwai, E., & Yukie, M. (1987). Amygdalofugal and amygdalopetal connections with modality-specific visual cortical areas in macaques (macaca fuscata, M. mulatta, and M. fascicularis). The Journal of Comparative Neurology, 261, 362-387. James, T. W., Humphrey, G. K., Gati, J. S., Menon, R. S., & Goodale, M. A. (2000). The effects of visual object priming on brain activation before and after recognition. Curr Biol, 10(17), 1017-24. Jenkinson, M., Bannister, P., Brady, M., & Smith, S. (2002a). Improved optimisation for the robust and accurate linear registration and motion correction of brain images. NeuroImage, 17(2), 825-841. Kanwisher, N., McDermott, J., & Chun, M. M. (1997). The fusiform face area: a module in human extrastriate cortex specialized for face perception. Journal of Neuroscience, 17(11), 4302-4311. Kaplan, S., Kaplan, R., & Wendt, J. S. (1972). Rated preference and complexity for natural and urban visual material. Perception & Psychophysics, 12(4), 354-356.

201

Kaplan, S. (1992). Environmental preference in a knowledge-seeking, knowledge-using organism. In J. H. Barkow, Cosmides, L. & Tooby, J. (Ed.), The Adapted Mind: Evolutionary Psychology and the Generation of Culture (pp. 581-598). New York: Oxford. Kaplan, R., & Kaplan, S. (1995). The Experience of Nature: A Psychological Perspective. Ann Arbor, Michigan: Ulrich's Bookstore. Khachaturian, H., Lewis, M. E., Schafer, M. K. H., & Watson, S. J. (1985). Anatomy of the CNS Opioid Systems. Trends in Neurosciences, 8(3), 111-119 . Kirkpatrick, S., Gelatt, C. D., & Vecchi, M. P. (1983). Optimization by simulated annealing. Science, 220, 671-680. Komar, V., & Melamid, A. (1993). United States: Most Wanted Painting : Dia Art Foundation (http://www.diacenter.org/km). Kourtzi, Z., & Kanwisher, N. (2000). Cortical regions involved in perceiving object shape. J. Neuroscience, 20(9), 3310-3318. Lang, P. J., Greenwald, M. K., Bradley, M. M., & Hamm, A. O. (1993). Looking At Pictures - Affective, Facial, Visceral, and Behavioral Reactions. Psychophysiology, 30(3), 261-273. Lang, P. J., Bradley, M. M., & Cuthbert, B. N. (1997a). International affective picture system (IAPS): Technical manual and affective ratings. (Technical Report ). Gainesville: The Center for Research in Psychophysiology, University of Florida. Lang, P. J., Bradley, M. M., Fitzsimmons, J. R., Cuthbert, B. N., Scott, J. D., Moulder, B., & Nangia, V. (1998). Emotional arousal and activation of the visual cortex: An fMRI analysis. Psychophysiology, 35(2), 199210. Lavenex, P., Suzuki, W. A., & Amaral, D. G. (2002). Perirhinal and parahippocampal cortices of the macaque monkey: Projections to the neocortex. Journal of Comparative Neurology, 447(4), 394-420. Lewis, M., Mishkin, M., Bragin, E., Brown, R. M., Pert, C. B., & Pert, A. (1981). Opiate receptor gradients in monkey cerebral cortex: correspondence with sensory processing hierarchies. Science, 211(13, March), 1166-1169.

202

Lewis, M. E., Pert, A., Pert, C. B., & Herkenham, M. (1983). Opiate Receptor Localization in Rat Cerebral-Cortex. Journal of Comparative Neurology, 216(3), 339-358. Loschky, L. C., McConkie, G. W., Yang, J., & Miller, M. E. (2001). Perceptual effects of a gaze-contingent multi-resolution display based on a model of visual sensitivity. Paper presented at the ARL Federated Laboratory 5th Annual Symposium - ADID Consortium Proceedings. Macko, K. A., Jarvis, C. D., Kennedy, C., Miyaoka, M., Shinohara, M., Sokoloff, L., & Mishkin, M. (1982). Mapping the primate visual system with [2-14C]Deoxyglucose. Science, 218(4570), 394-397. Maljkovic, V., & Martini, P. (2003). Different rates of memory formation for scenes with positive and negative affective content (Presentation given at the Vision Sciences Society 3rd Annual Meeting. TA5). Sarasota, FL. Mansour, A., Fox, C. A., Akil, H., & Watson, S. J. (1995). Opioid-receptor mRNA expression in the rat CNS: anatomical and functional implications. Trends in Neurosciences, 18(1), 22-29. Martin, A., & Weisberg, J. (2003). Neural foundations for understanding social and mechanical concepts. Cognitive Neuropsychology, 20, 575587. Martin-Schild, S., Gerall, A. A., Kastin, A. J., & Zadina, J. E. (1999). Differential distribution of endomorphin 1- and endomorphin 2-like immunoreactivities in the CNS of the rodent. The Journal of Comparative Neurology, 405, 450-471. Menon, R. S., & Goodyear, B. G. (2001). Spatial and temporal resolution in fMRI. In P. Jezzard, Matthews, P.M., Smith, S.M. (Ed.), Functional MRI: An Introduction to Methods (pp. 145-158). Miller, E. K., Li, L., & Desimone, R. (1991b). A Neural Mechanism For Working and Recognition Memory in Inferior Temporal Cortex. Science, 254(5036), 1377-1379. Miller, E. K., Li, L., & Desimone, R. (1993). Activity of neurons in the anterior inferior temporal cortex during a short term memory task. Journal of Neuroscience, 1460-1478.

203

Morris, B. J., & Johnston, H. M. (1995). A role for hippocampal opioids in long-term functional plasticity. Trends in Neuroscience, 18, 350-355. Nasar, J. L. (1988). Visual preferences in urban street scenes: a crosscultural comparison between Japan and the United States. In J. Nasar (Ed.), Environmental Aesthetics: Theory, Research, and Applications (pp. 260-274). New York: Cambridge. O'Carroll, R. E., Drysdale, E., Cahill, L., Shajahan, P., & Ebmeier, K. P. (1999). Stimulation of the noradrenergic system enhances and blockade reduces memory for emotional material in man. Psychological Medicine, 29(5), 1083-1088. Oliva, A., & Schyns, P. G. (2000). Diagnostic colors mediate scene recognition. Cognitive Psychology, 41(2), 176-210. Oliva, A., & Torralba, A. (2001). Modeling the shape of the scene: A holistic representation of the spatial envelope. International Journal of Computer Vision, 42(3), 145-175. Oliva, A., Torralba, A., Castelhano, M. S., & Henderson, J. M. (2003). Topdown control of visual attention in object detection. Paper presented at the IEEE International Conference on Image Processing, September 14-17, Barcelona, Spain. Peckys, D., & Landwehrmeyer, G. B. (1999). Expression of mu, kappa, and delta opioid receptor messenger RNA in the human CNS: A P-33 in situ hybridization study. Neuroscience, 88(4), 1093-1135. Pelli, D. G. (1997). The videotoolbox software for visual psychophysics: transforming numbers into movies. Spatial Vision, 10, 437-442. Pert, C. B., & Snyder, S. (1973). Opiate receptor: demonstration in nervous tissue. Science, 179, 1011-1014. Pinker, S. (1997). How the Mind Works. New York: W. W. Norton & Company. Pinker, S. (1997). How the Mind Works. New York: W.W. Norton & Company. Quirion, R., & Pilapil, C. (1991). Distribution of multiple opioid receptors in the human brain, Receptors in the Human Nervous System (pp. 103-121): Academic Press.

204

Robbins, T. W., & Everitt, B. J. (1996). Neurobehavioral mechanisms of reward and motivation. Current Opinion in Neurobiology, 6, 228-236. Robbins, T. W., Semple, J., Kumar, R., Truman, M. I., Shorter, J., Ferraro, A., Fox, B., McKay, G., & Matthews, K. (1997). Effects of scopolamine on delayed-matching-to-sample and paired associates tests of visual memory and learning in human subjects: comparison with diazepam and implications for dementia. Psychopharmacology, 134(1), 95-106. Rolls, E. T. (1999). The Brain and Emotion. New York: Oxford. Royet, J. P., Zald, D., Versace, M., Costes, N., Lavenne, F., Koenig, O., & Gervais, R. (2000). Emotional responses to pleasant and unpleasant olfactory, visual, and auditory stimuli: a positron emission tomography study. Journal of Neuroscience, 20(20), 7752-7759. Saleem, K. S., & Tanaka, K. (1996). Divergent projections from the anterior inferotemporal area TE to the perirhinal and entorhinal cortices in the macaque monkey. Journal of Neuroscience, 16(15), 4757-4775. Saleem, K. S., Suzuki, W., Tanaka, K., & Hashikawa, T. (2000). Connections between anterior inferotemporal cortex and superior temporal sulcus regions in the macaque monkey. Journal of Neuroscience, 20(13), 5083-5101. Schacter, D. S., Chiu, P., & Ochsner, K. N. (1993). Implicit memory: a selective review. Annual Review of Neuroscience, 16, 159-182. Schultz, W., Tremblay, L., & Hollerman, J. R. (2000). Reward processing in primate orbitofrontal cortex and basal ganglia. Cereb Cortex, 10(3), 272-84. Seamon, J. G., Marsh, R. L., & Brody, N. (1984). Critical importance of exposure duration for affective discrimination of stimuli that are not recognized. Journal of Experimental Psychology: Learning, Memory, and Cognition, 10(3), 465-469. Shepard, R. N., & Cermak, G. W. (1973). Perceptual-cognitive explorations of a toroidal set of free-form stimuli. Cognitive Psychology, 4(3), 351377.

205

Simon, E. J., & Hiller, J. M. (1994). Opioid peptides and opioid receptors. In G. J. S. e. al. (Ed.), Basic Neurochemistry: Molecular, Cellular, and Medical Aspects (5th ed., pp. 321-339). New York: Raven Press, Ltd. Smith, S. (2002). Fast robust automated brain extraction. Human Brain Mapping, 17(3), 143-155. Sobotka, S., & Ringo, J. L. (1994). Stimulus specific adaptation in excited but not in inhibited cells in inferotemporal cortex of macaque. Brain Research, 646, 95-99. Suzuki, W. A., & Amaral, D. G. (1994). Perirhinal and Parahippocampal Cortices of the Macaque Monkey - Cortical Afferents. Journal of Comparative Neurology, 350(4), 497-533. Tanaka, K. (1997). Mechanisms of visual object recognition: monkey and human studies. Current Opinion in Neurobiology, 7(4), 523-529. Thorpe, S., Fize, D., & Marlot, C. (1996). Speed of processing in the human visual system. Nature, 381(June 6), 520-522. Thorpe, S. J., & Fabre-Thorpe, M. (2001). Seeking categories in the brain. Science, 291(5502), 260-263. Tong, F., Nakayama, K., Moscovitch, M., Weinrib, O., & Kanwisher, N. (2000). Response properties of the human fusiform face area. Cognitive Neuropsychology, 17(1-3), 257-279. Tooby, J., & Cosmides, L. (1992). The psychological foundations of culture. In J. H. Barkow, L. Cosmides, & J. Tooby (Eds.), The Adapted Mind: Evolutionary Psychology and the Generation of Culture (pp. 19-136). London: Oxford University Press. Tremblay, L., & Schultz, W. (1999). Relative reward preference in primate orbitofrontal cortex. Nature, 398(6729), 704-8. Turner, B., Mishkin, M., & Knapp, M. (1980). Organization of the amygdalopetal projections from modality-specific cortical association areas in the monkey. The Journal of Comparative Neurology, 191, 515-543. Ulrich, R. S. (1974). Scenery and the Shopping Trip: The Roadside Environment as a Factor in Route Choice. Unpublished Doctoral dissertation, University of Michigan, Ann Arbor.

206

Ulrich, R. S. (1993). Biophilia, biophobia, and natural landscapes. In S. L. Kellert & E. O. Wilson (Eds.), The Biophilia Hypothesis (pp. 73-137). Covelo, California: Island Press. Ungerleider, L. G., & Mishkin, M. (1982). Two cortical visual systems. In M. A. G. D.J. Ingle, R.J.W. Mansfield (Ed.), Analysis of Visual Behavior (pp. 549-586). Cambridge, MA: MIT Press. van Turennout, M., Ellmore, T., & Martin, A. (2000). Long-lasting cortical plasticity in the object naming system. Nat Neurosci, 3(12), 1329-34. Vitz, P. C. (1966). Preferences for different amounts of visual complexity. Behavioral Science, 11, 105-114. Volpicelli, J. R., Alterman, A. I., Hayashida, M., & O'Brien, C. P. (1992). Naltrexone in the treatment of alcohol dependence. Arch Gen Psychiatry, 49(11), 876-880. Vuilleumier, P., Henson, R. N., Driver, J., & Dolan, R. J. (2002). Multiple levels of visual object constancy revealed by event- related fMRI of repetition priming. Nature Neuroscience, 5(5), 491-499. Webster, M. J., Bachevalier, J., & Ungerleider, L. G. (1993). Subcortical Connections of Inferior Temporal Areas Te and Teo in Macaque Monkeys. Journal of Comparative Neurology, 335(1), 73-91. Webster, M. J., Bachevalier, J., & Ungerleider, L. G. (1994). Connections of Inferior Temporal Areas Teo and Te With Parietal and Frontal-Cortex in Macaque Monkeys. Cerebral Cortex, 4(5), 470-483. Webster, M. J., Bachevalier, J., & Ungerleider, L. G. (1995). Transient Subcortical Connections of Inferior Temporal Areas Te and Teo in Infant Macaque Monkeys. Journal of Comparative Neurology, 352(2), 213-226. Wilson, E. O. (1984). Biophilia: The Human Bond with Other Species. Cambridge, MA: Harvard University Press. Wilson, E. O. (1993). Biophilia and the conservation ethic. In S. L. Kellert & E. O. Wilson (Eds.), The Biophilia Hypothesis (pp. 31-41). Covelo, California: Island Press.

207

Winkielman, P., Zajonc, R. B., & Schwarz, N. (1997). Subliminal affective priming resists attributional interventions. Cognition and Emotion, 11(4), 433-465. Winkielman, P., & Cacioppo, J. T. (2001). Mind at ease puts a smile on the face: Psychophysiological evidence that processing facilitation elicits positive affect. Journal of Personality and Social Psychology, 81(6), 989-1000. Wise, S. P., & Herkenham, M. (1982). Opiate receptor distribution in the cerebral cortex of the rhesus monkey. Science, 218(22), 387-389. Woods, R. P., Grafton, S. T., Watson, J. D. G., Sicotte, N. L., & Mazziotta, J. C. (1998). Automated image registration II: Intersubject validation of linear and nonlinear models. Journal of Computer Assisted Tomography, 22, 153-165. Woolrich, M., Ripley, B., Brady, J., & Smith, S. (2001b). Temporal autocorrelation in univariate linear modeling of FMRI data. NeuroImage, 14(6), 1370-1386. Worsley, K. J. (2001). Statistical analysis of activation images. In M. Jezzard, & Smith (Ed.), Functional MRI: An Introduction to Methods . Oxford, UK: Oxford University Press. Yantis, S., & Serences, J. T. (2003). Cortical mechanisms of space-based and object-based attentional control. Current Opinion in Neurobiology, 13, 187-193. Yoshida, M., Naya, Y., & Miyashita, Y. (2003). Anatomical organization of forward fiber projections from area TE to perirhinal neurons representing visual long-term memory in monkeys. Proceedings of the National Academy of Sciences of the United States of America, 100(7), 4257-4262. Zadina, J. E., Hackler, L., Ge, L.-J., & Kastin, A. J. (1997). A potent and selective endogenous agonist for the mu-opiate receptor. Nature, 386, 499-502. Zadina, J. E., Martin-Schild, S., Gerall, A. A., Kastin, A. J., Hackler, L., Ge, L. J., & Zhang, X. (1999). Endomorphins: Novel endogenous mu-opiate receptor agonists in regions of high mu-opiate receptor density, Neuropeptides: Structure and Function in Biology and Behavior (Vol. 897, pp. 136-144).

208

Zadina, J. E. (2002). Isolation and distribution of endomorphins in the central nervous system. Japanese Journal of Pharmacology, 89, 203-208. Zago, L., & Bar, M. (2003). The rise and fall of visual priming (Presentation given at the Vision Sciences Society 3rd Annual Meeting. TA192). Sarasota, FL. Zajonc, R. B. (1968). Attitudinal effects of mere exposure. Journal of Personality and Social Psychology, 9(Monograph supplement No. 2, Part 2). Zajonc, R. B. (1980). Feeling and thinking. American Psychologist, 35(2), 151-175. Zuckerman, M. (1995). Good and Bad Humors: Biochemical Bases of Personality and Its Disorders. Psychological Science, 6(November (6)), 325-332.