Brain mechanisms for changes in processing of conditioned stimuli in Pavlovian conditioning: Implications for behavior theory

This article is a review of the results of a series of experiments designed to identify brain systems involved in appetitive conditioning of rats. It discusses some of their implications for behavioral theories of learning, especially those that concern changes in processing of conditioned stimuli (CSs). Evidence is presented which suggests that separable brain circuits are involved in (1) the production of CS-dependent conditioned orienting responses, (2) the enhancement of CS associability produced when expectancies about upcoming events are violated, (3) the reduction of CS associability produced when stimuli are consistent predictors of other events or are presented without consequence, and (4) the abilities of CSs to serve as reinforcers for second-order conditioning and to be sensitive to postconditioning changes in the value of the unconditioned stimulus (US). Finally, none of these circuits seems critical for normal acquisition of the most common indicator of Pavlovian conditioning, US-dependent conditioned responses (CRs). Although the independence of brain pathways does not demand independence of behavioral function, clustering of behavioral phenomena on anatomical grounds may provide useful guides for constructing behavior theories.     

Modulatory mechanisms in Pavlovian conditioning

Pavlovian conditioning has traditionally been thought to involve the acquisition of excitatory and inhibitory associations between a conditioned stimulus (CS) and an unconditioned stimulus (US). Recent research, however, has encouraged the view that Pavlovian learning may also encompass a higher order modulatory mechanism, in which animals use information about another stimulus to control responding to the CS. Positive modulators signal a positive relationship between the CS and the US, whereas negative modulators signal that the CS-US relationship is not in force. In both cases, the modulatory control appears to be orthogonal to the modulator’s direct associations with the US. This article reviews and evaluates the literature on this Pavlovian modulatory mechanism.     

Secondary extinction in Pavlovian fear conditioning

Pavlov (1927/1960) reported that following the conditioning of several stimuli, extinction of one conditioned stimulus (CS) attenuated responding to others that had not undergone direct extinction. However, this secondary extinction effect has not been widely replicated in the contemporary literature. In three conditioned suppression experiments with rats, we further explored the phenomenon. In Experiment 1, we asked whether secondary extinction is more likely to occur with target CSs that have themselves undergone some prior extinction. A robust secondary extinction effect was obtained with a nonextinguished target CS. Experiment 2 showed that extinction of one CS was sufficient to reduce renewal of a second CS when it was tested in a neutral (nonextinction) context. In Experiment 3, secondary extinction was observed in groups that initially received intermixed conditioning trials with the target and nontarget CSs, but not in groups that received conditioning of the two CSs in separate sessions. The results are consistent with the hypothesis that CSs must be associated with a common temporal context during conditioning for secondary extinction to occur.     

Pavlovian conditioning in multiple contexts: Competition between contexts for comparator status

Water-deprived rats were used to investigate the effects of training a CS in more than one context on conditioned lick suppression. In each experiment, partial reinforcement of the CS was intermingled with unsignaled presentations of the US. In Experiment 1, subjects were either trained in one context alone, trained consecutively in two contexts (such that all training in one context occurred prior to any training in the second context), or trained alternately in two contexts. Following training, the first context, the second context, or neither context was extinguished. Testing of the CS occurred in a third (neutral) context. To the extent that either training context became established as a comparator stimulus for the CS, the comparator hypothesis (Miller & Matzel, 1988) predicts an increase in excitatory responding to the CS following extinction of that context. Subjects trained in a single context exhibited appreciable fear of the CS only when the CS’s training context had been extinguished. Additionally, subjects trained consecutively in the two contexts showed increased fear of the CS following extinction of the second, but not the first training context (i.e., a recency effect). Subjects trained alternately in the two contexts showed no increased fear of the CS as a result of either context alone being extinguished. In Experiment 2, subjects trained alternately in two contexts showed increased fear of the CS only when both training contexts were extinguished, suggesting that both training contexts had become comparator stimuli. These data indicate that multiple training contexts can either compete or act synergistic-ally in modulating responding to a Pavlovian trained CS as a function of the order of training in the different contexts.     

Overexpectation in appetitive Pavlovian and instrumental conditioning

In three experiments that used appetitive preparations with rats, we examined the effects of reinforcing a compound consisting of two previously reinforced stimuli on subsequent responding to those stimuli. Experiment 1 showed that a Pavlovian conditioned stimulus given this treatment evoked fewer magazine entries when presented alone than did a reinforced stimulus that did not receive the compound treatment. Experiment 2 examined inhibition of delay and generalization decrement accounts for the results of Experiment 1. Experiment 3 extended this finding to an instrumental learning paradigm.     

Pavlovian appetitive conditioning under curare in rats

A CS was paired with rewarding intracranial stimulation while Ss were immobilized by curare. Tests of the activity response to the CS were carried out before and after conditioning. With either simple or discriminative conditioning, activity to the rewarded CS increased after training. These results are comparable to those in studies using food reward and support the hypothesis that a central motivational state is conditioned to stimuli preceding reward.     

Signaling and affective functions in Pavlovian conditioning

The present study employed a Pavlovian-instrumental-transfer paradigm to investigate the role of conditioned fear in appetitive discrimination learning. Each of three Pavlovian training procedures was used to establish a conditioned fear excitor (CS+), a “neutral” CS (CSo), and a conditioned fear inhibitor (CS?). Then, the CSs were administered to rats in the three groups contingent upon the rewarded response in a difficult visual discrimination. In addition, half of each group received shock punishment for each incorrect response. Relative to CSo, CS+ facilitated performance in contrast to the usual interfering effect of conditioned suppressors; conversely, CS? retarded performance even when its reinforcing action (fear inhibition) was potentiated by punishment for the incorrect response. These results, together with other findings showing a reversed outcome when the CSs are administered for the incorrect response, indicate that Pavlovian conditioning comprises both general signaling and affective functions, the former reflecting a basic “expectancy” or nominal type of cognitive processing in the rat.     

Outcome-specific conditioned inhibition in Pavlovian backward conditioning

In the present experiments, the outcome specificity of learning was explored in an appetitive Pavlovian backward conditioning procedure with rats. The rats initially were administered Pavlovian backward training with two qualitatively different unconditioned stimulus conditioned-stimulus (US-CS) pairs of stimuli (e.g., pellet --> noise or sucrose --> light), and then the effects of this training were assessed in Pavlovian-to-instrumental transfer (Experiment 1) and retardation-of-learning (Experiment 2) tests. In the transfer test, it was shown that during the last 10-sec interval, the CSs selectively reduced the rate of the instrumental responses with which they shared a US, relative to the instrumental responses with which they did not share a US. The opposite result was obtained when the USs (in the absence of the CSs) were presented noncontingently. In the retardation test, conditioned magazine approach, responding to the CSs was acquired more slowly when the stimulus-outcome combinations in the backward and the forward conditioning phases were the same, as compared with when they were reversed. These results are collectively in accord with the view that Pavlovian backward conditioning can result in the formation of outcome-specific inhibitory associations. Alternative views of backward conditioning are also examined.     

Encoding of the unconditioned stimulus in Pavlovian conditioning

Three experiments, using rats, demonstrated the encoding of a food unconditioned stimulus (US) in a simple Pavlovian conditioning paradigm. In all three studies, one stimulus was used to signal the delivery of pellets and a different stimulus was used to signal the delivery of sucrose. In Experiment 1, postconditioning devaluation of one of the food USs selectively reduced the frequency of conditioned magazine-directed behavior during the stimulus trained with that US. In Experiment 2, transfer of the stimuli to instrumental responses resulted in selective depression of the response trained with a different outcome. In Experiment 3, acquisition of stimulus-outcome learning was impaired by unsignaled intertrial presentations of the same outcome but not of a different outcome. These results indicate that a detailed representation of the outcome is encoded in the normal course of Pavlovian conditioning.     

Temporal integration in Pavlovian appetitive conditioning in rats

We used an appetitive sensory preconditioning procedure to investigate temporal integration in rats in two experiments. In Phase 1, rats were presented with simultaneous compound trials on which a 10-sec conditioned stimulus (CS) X was embedded within a 60-sec CS A. In Group Early, CS X occurred during the early portion of CS A, whereas in Group Late, CS X occurred during the latter portion of CS A. In Phase 2, CS X was paired simultaneously with sucrose. On a subsequent test with CS A, the rate of magazine entries peaked during the early portions of the stimulus in Group Early and during the latter portions of the stimulus in Group Late (Experiments 1 and 2). Similar response peaks were not observed on tests with a control stimulus that had been presented in compound with a stimulus that did not signal reward (Experiment 2).     

Devaluation of Pavlovian conditioning in the 10-day-old rat

This experiment established that for the infant rat, 10 days postpartum, a preference conditioned to an olfactory stimulus (conditioned stimulus) could be substantially-decreased-by subsequently lowering the value of the unconditioned stimulus (heat). This devaluation effect disappeared when a sufficiently long interval elapsed between the devaluation treatment and the test, despite maintained retention of the original conditioned preference over this same interval. This suggests that devaluation in infant rats does not permanently change the animal’s original representation of the unconditioned stimulus, but instead may replace it temporarily with a conflicting representation.     

Contextual associations in trace conditioning

Rats were given tone-footshock pairings with a 0-, 10-, or 30-sec trace interval between tone offset and shock onset. Half the rats within each trace interval were tested for their conditioned fear of the tone through a lick suppression procedure; the remaining rats were evaluated for their fear of the background or contextual cues through their avoidance of the compartment in which conditioning had occurred. Less conditioning was observed to the tone with increasing trace intervals. However, conditioned fear of the context increased with increases in the trace duration. The ability of the more predictive stimulus, the tone, to overshadow the contextual cues was determined by the tone’s temporal contiguity with the footshock. The need to incorporate temporal parameters within current theories of conditioning is discussed.     

Evolution of an elemental theory of Pavlovian conditioning

Kenneth Spence (1936, 1937) formalized a quantitative, elemental approach to association theory that has had a broad and dominating influence on learning theory for many years. A set of challenges to the basic approach has spurred the subsequent evolution of elemental theory in various ways. Four of the challenges and some resulting theoretical accommodations are described in the context of Pavlovian conditioning. The evolution involves departures from important specifics of Spence’s theory, but is viewed as demonstrating the utility of the basic, elemental approach that is one of his legacies.     

Enhanced Pavlovian conditioning with a change in appetitive reinforcer

In three experiments, rats received appetitive conditioning with two unconditioned stimuli (US₊ and US*) that supported a similar conditioned response. Stimulus A was first paired with US₊, and then, for a second stage of training, the compound AB was paired with US*. Subsequent test trials with A revealed an abnormally strong conditioned response. The findings from additional groups indicated that such enhanced conditioning with A depended on the presence of B in the second stage and on the use of US₊ and US* for the two training stages. A modification to a configural theory of conditioning is proposed in order to explain the results.     

A Pavlovian analysis of food-attraction conditioning in the snailHelix aspersa

Following brief pairing of an odor with a feeding experience (food-attraction conditioning), snails will lower their tentacles when subsequently presented with that odor alone. Three experiments investigated the possible behavioral mechanism mediating food-attraction conditioning in the snailHelix aspersa. It is suggested that food-attraction conditioning is an example of Pavlovian conditioning. In this case, the odor (conditioned stimulus) is paired with oral stimulation (unconditioned stimulus), which elicits lowering of the tentacles (unconditioned response). Following conditioning, the odor comes to elicit lowering of the tentacles (conditioned response). Experiment 1 ruled out nonassociative effects, such as habituation and sensitization, using an unpaired control group. Experiment 2 provided further evidence against a role for habituation of neophobia, through the demonstration of extinction following conditioning. In Experiment 3, an omission procedure was used to rule out the possible role of instrumental contingencies.     

Taste potentiates color-sickness associations in pigeons and quail

The present experiments assessed cue utilization in pigeons and quail on similar tests of poison-based aversion learning. In Experiments 1 and 2, three groups of pigeons were given colored water, flavored water, or colored flavored water prior to induction of sickness; these experiments differed only as to the specific colors and flavors used as stimuli. In both experiments, the birds trained with flavored water exhibited reliable taste aversions when tested with uncolored flavored water. Similar degrees of aversion were observed whether the flavored water had been colored or uncolored during training, suggesting that the color cue had little or no effect on the conditioning of the flavor cue. In contrast, the flavor cue had a pronounced effect on the conditioning of the color cue. When tested with unflavored colored water, the birds trained with colored flavored water exhibited significantly stronger color aversions than those trained with unflavored colored water. That is, the flavor cue enhanced or potentiated the conditioning of the color cue. In a third experiment, quail were trained in the same way as the pigeons with virtually the same result. The pattern of cue utilization observed in the present experiments with pigeons and quail differs markedly from that proposed by Wilcoxon, Dragoin, and Kral (1971) for quail. However, a reexamination of the results obtained by Wilcoxon et al. suggested that they are susceptible to an alternative interpretation consistent with the present results.     

Differences in taste-potentiated odor aversions with O+/OT+ versus OT+/O+ conditioning: Implications for configural associations

The present research demonstrates a conditioning order effect difference: Odor-aversion conditioning is stronger following OT+/O+ conditioning than following O+/OT+ conditioning with specific odor (O) and taste (T) cues. When a weak odor cue was used in Experiments 1A and 1B, OT+/O+ conditioning produced significantly stronger odor aversions than did either O+/OT+ or O+/O+ conditioning, which did not differ. The same design was used in Experiment 2 with a strong odor, but the order effect difference was not replicated, suggesting that the order effect difference is specific to conditions that produce taste-potentiated odor aversions. The interpretation that O+/OT+ conditioning is weaker because of the absence of a taste-odor within-compound association was not supported in Experiments 3 and 4. Notably, using stimuli that supported potentiation in earlier experiments, in Experiment 4, we found evidence of a taste-odor within-compound association in the absence of potentiation. These results confirm that previous theories of potentiation (within-compound association model, sensory-and-gate channeling model) are not sufficient to produce potentiation. Instead, these results are interpreted in terms of taste-odor configural associations.     

Transfer testing after serial feature-ambiguous discrimination in Pavlovian keypeck conditioning

Four experiments were performed to explore modulatory transfer after serial feature-ambiguous (F-A) discrimination (X→A+, A?, X→B?, B+) in Pavlovian keypeck conditioning (autoshaping) with pigeons. Diffuse features were used in the first three experiments, and no modulatory transfer was found: (1) There was no modulatory transfer between two concurrently trained F-A tasks; (2) modulatory transfer to conditioned and then extinguished stimuli was not observed; and (3) responding to the targets (A and B) after their contingencies were reversed was not affected by presentation of their original feature stimulus (X). Transfer between two serial F-A tasks was obtained in the last experiment, in which keylights were used as features, but other evidence indicated that this was due to stimulus generalization between the features. Together, the results strongly suggest that specific target-food occasion setting or configural learning is the underlying mechanism of F-A discrimination.     

Proactive effects in pigeons’ timing behavior: Implications for an internal-clock model

We have found proactive effects in pigeons’ timing behavior, a finding inconsistent with internal-clock models of timing that assume a resetable working-memory component. Six pigeons were trained to discriminate between 2- and 10-sec illuminations of a white light; choice of a red pecking key was correct and rewarded after presentation of the short stimulus whereas choice of a green key was correct and rewarded after presentation of the long stimulus. During training sessions, there were 60 trials separated by a 20-sec intertriai interval; short and long light occurred in a randomized order and correct choices were reinforced with 5-sec access to grain on a partial (75%) schedule. During test sessions, there were 120 trials separated by a 2-sec intertrial inter val. Light presentations occurred in a fixed order throughout these sessions: 2, 6, 10, 10, 6, 2 2, 6, 10 sec, and so forth. Choice of either red or green after 6 sec was not reinforced. However, red continued to be correct after 2 sec and green continued to be correct after 10 sec. Of central interest was how the subjects classified 6 sec of light in ascending (2, 6, 10) and descending (10. 6, 2) sequences of durations: Subjects chose the short alternative on 42% of the 6-sec trials in ascending series but only 29% in descending series, a result most plausibly interpreted as show ing that duration information from a preceding trial affects duration classifications on the cur rent trial. Such proactive effects should not occur according to working-memory models that as sume that stored information is cleared at the end of a trial.     

Integration of cardiac responses to serial stimuli after Pavlovian conditioning in rats

The expression of cardiac responses to sequences of two sounds was studied in restrained rats following discriminative trace or delay conditioning. Stimuli paired with a tail shock 10 sec later (CS1) elicited conditioned bradycardia. Unpaired or neutral stimuli (CS0) elicited mostly tachycardia. Rats did not learn to suppress responding to nonreinforced sequences with an interval of 6 sec between sounds. Responses to the second stimulus were significantly augmented following a CS1 stimulus, but not following a CS0 stimulus. Real-time summation of simple responses provided a more complete and quantitative prediction of dual responses than did resetting or facilitation. These results extend the time range over which summation may be observed from less than 2 sec to at least 16 sec. They appear to be inconsistent with models involving competition between unitary representations of stimuli in short-term memory and suggest the existence of multiple stimulus traces with independent time courses.