Review of

Affective and Non-affective Meaning in Words and Pictures

Vicky Tzuyin Lai, Peter Hagoort, Daniel Casasanto

Cognitive Science Society 2011

Introduction

When people see a spider, their brains process both non-affective information (a spider is an animal) and affective information (the spider is dangerous). Affective information refers to the emotional aspect and non-affective side refers to the semantic aspect of any phenomenon. The question as to which of these two processing occur first has been rampant for several years now. The Affective Primacy Hypothesis (affective evaluation is quicker and automatic) and the Cognitive Primacy Hypothesis (people need to understand what they are seeing before they can feel anything about it) have attributed temporal primacy to affective and non-affective processing respectively. The paper under discussion here proposes that the relative speed with which affective/non-affective processing occurs in response to stimuli is context-dependent. As a small example, they have pointed out that, for a person hiking in a tropical jungle, the affective information gets priority as he is constantly reminded of the possibly dangerous animals lurking around him. However, for a person classifying animals as part of a timed biology test, the non affective details (various physical features of animals) are far more important than if some of them are dangerous or not.

First of all, the authors of this paper have analyzed studies done in the past that subscribe to the Affective or Cognitive Primacy theories (propositions), and pointed out fallacies in their arguments (opposition).

Primacy of Affective Processing : three main pillars of support

1. Subliminal stimuli ¹

Proposition: Past researchers had contended that affective reactions are evoked with minimum stimulus and no non-affective processing. Their work involved priming under suboptimal (very brief) and optimal (sufficiently long) times of viewing. At suboptimal exposures, affect-laden primes (faces carrying different emotional expressions) can determine the affective judgments of subsequent novel stimuli (Chinese ideographs), whereas non-affective primes (differently sized geometric shapes) cannot determine the non-affective evaluation of the unseen stimuli (size of object represented by the ideographs). This led to the researchers to figure that without conscious awareness, affect elicited is diffuse and nonspecific in nature, and its non-affective counterparts (origin and address) are not processed. On the other hand, at optimal times of viewing, the results obtained were in the reverse pattern i.e. mainly the non-affective (hence semantic) primes influenced non-affective judgments to subsequent novel stimuli (like doctor: nurse). The researchers hence argued that affect is elicited first, with minimal stimulation, and before the processing of its non-affective counterpart.

Opposition: The experiment does show an early and fast processing of affect. But the contention that affect is processed necessarily before non-affect is not well supported. The non-affective response in the subliminal priming could have been elicited very early if the category of primes (size) were made important to the test-taker (like finding the correct sized block for building something in a contest). Also, other non-affective dimensions (e.g. color) could have been processed early where this one dimension (size) is not. Moreover, the affective and the non-affective processing assessed in this experiment were occurring in a more or less independent manner, and not in direct competition with one another.

2. Affective priming ²

Proposition: Here experimenters showed that targets of a positive or negative flavor (like sunshine) can be primed when items of congruent flavor (like love) precede them, whereas primes of incongruent flavor (like death) failed to be effective. It was also noted that priming with affective items was regardless of the nature of the task (affective - pleasant/unpleasant, or category - people/animal judgment), where non-affective priming was not so. Also, it was noted that affective priming occurs even without overt attention (concurrent verbal task), thus proposing the automatic nature of affective evaluation.

Opposition: A direct counter-evidence for this contention is the automaticity of non-affective (lexical-semantic) processing that is determined by the classic Stroop task. Just as people do not suppress affective judgments while doing another verbal task, similarly, people also cannot suppress the non-affective processing of the word red although their task is to state verbally the color (blue) of the word red printed on the screen. If the two kinds of information in question are both viewed as a combined feature in the semantic network of our mind, then the question of primacy of activation becomes pointless.

3. Neuropsychological evidence ³

Proposition: fMRI data has shown that when exposed to emotional stimuli, a neural system is activated that learns the affective value of the stimuli. This system is present to provide for a shortcut that by-passes the neural system responsible for identification of objects and people and is capable of producing a fast reaction to stimuli that are potentially life-threatening.

Opposition: There is no evidence that such a neural network prevents a parallel processing of the neural network involved in the non-affective aspects of the stimuli. Hence, the temporal order of affective and non-affective processing cannot be commented upon from these observations.

Primacy of Non-affective Processing : two main pillars of support

1. Lack of affective priming ⁴

Proposition: In five studies, the experimenters varied both the affective and semantic connections between the used primes and targets (in the form of words and pictures). In four of the studies, non-affective priming was found effective and affective priming was not. One study did show the affective priming as effective, but only when the words used came from the same semantic category. Thus they concluded that in general affective processing is fragile in nature, and when people are made to deal with only one dimension (non-affective), they rely on it, regardless of its nature.

Opposition: Similar to the experiments based on subliminal priming, this method does not strongly rule out the possible occurring of affective priming as the style of testing does not compare them directly. Also the one experiment where affective priming was found hints more to the direction that it is the context that plays a big role here.

2. Eye-tracking experiments ⁵

Proposition: Here, test-takers were made to view, extra-foveally, paired neutral and emotional scenes having animals or humans. Participants were instructed to shift gaze in order to categorize the scene as pleasant/unpleasant or animal/human. It was noted that although both kind of categorizations (affective and non-affective) were fast, semantic categorization took less time than affective categorization. When the style of exposure was changed to foveal, it was seen that affective categorization slowed down more than non-affective categorization. Thus the researcherrs argued that since the threshold exposure required for accurate semantic processing is less than that required for affective processing, so it seems that the non-affective processing is precedent to and required for affective processing.

Opposition: This conclusion has been contested by the authors of this paper on the basis that in the above experiments, the particular nature of stimuli used can bias the test-taker to one side of the results. They have reasoned that the photographs shown carry non-affective information on a more coarse-grained level (animal/human) whereas the affective component is at a much more fine-grained level (facial expression). It is then obvious that judgments of non-affective nature (easier to perceive) are made faster than the affective ones. Here lower-level visual factors pose a problem that could be solved by using a different kind of stimuli.

Ad Hoc Cognition : context dependent processing

If concepts, categories, and word meanings are stable, how can people use them so flexibly? Here we explore a possible answer: maybe this stability is an illusion.

- Casasanto, D. & Lupyan, G. (2011). Ad Hoc Cognition

The authors have leaned towards the proposition that maybe all categorizations, concepts and semantics are constructed ad hoc, whenever we use them. And the details of this construction are determined by the manner in which retrieval cues relate to the physical, linguistic and social context they are in. A pair of direct examples of such a system comes from the fact that the emotional tone of words heard can determine the nature of subsequent processing of the words, and that studies have shown that affective processing is often goal-dependent or task-relevant. ⁶

Experimental approach

Experiments done by the authors of this paper are aimed at participants processing the same stimuli in different contexts. In order to provide for contexts, they took inspiration from Allport and Wylie (2000) Task Set Inertia paradigm which proposes that that readiness for a given task (i.e. task set) involuntarily tends to persist over a number of successive trials, kind of like a high-order of priming. Here the authors have used target trials (stimuli such as words of affective/non-affective dimension) and filler trials (stimuli such as scenes of affective/non-affective dimension). The theory is that evaluation during the filler trials will persist and influence evaluation during the target trials. So, the filler trials are serving as a context that can orient the test-taker towards a certain aspect of the target trial stimuli.

Experiment 1

Here context-dependency was tested in the processing of word meaning. Target trials included non-affective (animal/human) and affective (positive/negative) words. Filler trials included non-affective (indoor/outdoor) and affective (pleasant/unpleasant) scenes. Faster processing time was predicted for targets having congruency in context.

Experiment 2

Here, to avoid biasness by nature of stimuli, context-dependency of processing pictures was tested. The target and the filler trials were swapped from Experiment 1, and the same test was tried on a different set of test-takers. Here, the scenes were the targets, and the words were the fillers. Here as well, congruency with context was predicted to produce less processing time.

Method and Materials

1. The test takers were selected from among native Dutch undergraduates and randomly divided into either the affective context group or the non-affective context group.

2. The target stimuli (filler for 2^nd experiment) consisted of equal number of items belonging to 4 categories : positively flavored humans (princess), negatively flavored humans (murderer), positively flavored animals (bunny), and negatively flavored animals (cockroach).

3. The filler stimuli (target for 2^nd experiment) consisted of equal numbers of items belonging to the 4 categories : pleasant outdoor, pleasant indoor, unpleasant outdoor and unpleasant indoor.

4. The degree to which the stimuli have positive or negative flavor was determined by a pre-test in which randomly selected individuals were asked to assign a value from 1 (most negative) to 10 (most positive) to each of the items used as stimulus. The responses were normalized, and items were selected having an average positivity value of around 6.8 and negativity value of around 2.8.

5. The response to stimuli was to be shown by the movement to a switch (left to right and right to left), and another small pre-test was done on each test-taker to affix which movement signifies which response.

6. The reaction time (RT) was recorded for all the target trials and not the filler trials. The percentage accuracy was calculated, and extremely long reaction times (>5 seconds) and low accuracy data (<80%) were expunged from the analysis.

Results and Inferences

For Experiment 1:

1. With affective contexts, the affective judgments were made faster than the non-affective ones. And with non-affective contexts, the non-affective judgments were made faster than the affective ones.

2. Within the affective evaluations, the processing does not seem to be significantly faster in the affective context, than in the non-affective context. Within the non-affective evaluation, the processing does not seem to be significantly faster in the non-affective context, than in the affective context.

3. A regression analysis was done by the authors of 2 context types versus 2 judgment types, and a strong correlation was found among congruent types.

For Experiment 2:

1. With affective contexts, the affective judgments were made faster than the non-affective ones. And with non-affective contexts, the affective judgments were still made faster than the affective ones.

2. Within the affective evaluations, the processing does not seem to be significantly faster in the affective context, than in the non-affective context. Within the non-affective evaluation, the processing is indeed significantly faster in the non-affective context, than in the affective context.

3. A regression analysis was done by the authors of 2 context types versus 2 judgment types, and a strong correlation was found among congruent types.

4. A possible explanation for the unprecedented results for one half of this 2^nd experiment is that for the picture targets, the nature of the stimuli poses a constraint for the retrieval cues. For example, the word puppy can refer to any kind of puppy whereas a detailed color photograph of a puppy biases the test-taker to that specific puppy. This biasness may be the reason why affective responses are elicited faster here, no matter what the context says.

Conclusion

It seems from both the experiments, that processing (hence reaction time) of any information in the brain works according to cues-in-context available at the time. Thus any broad generalizations regarding the temporal primacy of a particular kind of evaluation seems baseless. The authors of this paper rather support the framework of Ad Hoc Cognition which advocates that words and scenes stimulate a unique neuro-cognitive processing every time they are encountered. And the context in which the stimuli are present plays a very important role in determining the details of such a processing.

References

1. Murphy, S.T. & Zajonc, R.B. Affect, cognition, and awareness: affective priming with optimal and suboptimal stimulus exposures. J Pers Soc Psychol 64, 723-739 (1993).

2. Calvo, M.G. & Nummenmaa, L. Processing of unattended emotional visual scenes. J Exp Psychol Gen 136, 347-369 (2007).

3. Whalen, P.J., et al. Masked presentations of emotional facial expressions modulate amygdala activity without explicit knowledge. J Neurosci 18, 411-418 (1998).

4. Storbeck, J. & Robinson, M.D. Preferences and inferences in encoding visual objects: a systematic comparison of semantic and affective priming. Pers Soc Psychol Bull 30, 81-93 (2004).

5. Nummenmaa, L., Hyona, J. & Calvo, M.G. Semantic categorization precedes affective evaluation of visual scenes. J Exp Psychol Gen 139, 222-246.

6. Elman, J.L. An alternative view of the mental lexicon. Trends Cogn Sci 8, 301-306 (2004).