Acquisition and extinction of differential responses to signals paired with shock or shock omission in goldfish: Evidence for truly discriminated avoidance learning

Goldfish trained to discriminate between signals paired with shock (S?) and signals paired with shock omission (S+) with a linear presentation procedure, originally learned (OL) to control the signal state of a shuttle box and showed a decided preference for the S+ signal. In Experiment 1, following OL, groups had one OL signal replaced (S+ or S?), both signals replaced (S+ and S?), or the OL signals reversed (S+ and S? reversed) and were then tested in a transfer training procedure. In transfer, groups with one signal replaced maintained discriminated performance at OL levels; the S+ replaced group was slightly superior to the S? replaced group on the first day of transfer. With both OL signals replaced, discrimination dropped to chance performance levels, whereas, with OL signal shock pairing reversed, discrimination performance dropped below chance levels. In Experiment 2, following OL, extinction procedures consisted of turning off the shocker (0% shock) or of shocking 100% or a random 25% of the trials. A fourth extinction procedure (R,) retained the trial start response-dependent shock-omission contingency, but shock differentiating the S+ and S? signals was eliminated entirely. Extinction of the S+/S? discrimination was measured both during extinction training per se and with reversal retraining of the S+/S? discrimination later. Groups for which the OL S+ was paired with shock during extinction extinguished on both measures, but groups for which the OL S? was paired with shock omission did not extinguish, especially as shown by the reversal test procedure. Theoretical implications and the implications for several conditioning procedures are discussed.     

Replacement of event-generated memories of nonreinforcement with signal-generated memories of reinforcement during partial reinforcement training: Effect on resistance to extinction

Event-generated memory refers to the memory of a reinforcement (R) or nonreinforcement (N) event from an immediately preceding trial;signal-generated memory refers to the memory of a temporally remote R or N, retrieval of which is generated by presentation of a signal with which the memory is associated (Haggbloom, 1988). In each of three experiments, Group Signal-R received runway discrimination training in Phase 1 to establish a stimulus as a signal for R, and partial reinforcement training in Phase 2. An extinction test measured learning about the memory of nonreward (S^N)—learning that occurs when S^N is retrieved on R trials that follow N trials. In Group Signal-H, those R trials were accompanied by the signal for R, a treatment we hypothesized would generate retrieval of the memory of reinforcement (S^R) so that signal-generated S^R would replace event-generated S^N as the operative memory, thereby eliminating the increased resistance to extinction normally produced by PRF training. In each experiment, Group Signal-R was less resistant to extinction than was a control group conditioned to respond to-event-generated S^N. Extinction was as rapid in Group Signal-R as it was in a consistent reinforcement control group (Experiment 1) and in a group given intertrial reinforcements to interfere with learning about S^N (Experiment 3). Experiment 2 tested two alternative interpretations of the failure to learn about S^N in Group Signal-R. Those alternatives were found to be less viable than the hypothesis that the signal for R actively recruited retrieval of a competing memory.     

Signaling the omission of a response-contingent outcome reduces discriminative control

The effect of training a positive discriminative stimulus (S+ ) as a signal for the nonreinforcement of an instrumental response (S?) on the ability of that stimulus to evoke its original instrumental response was examined in three experiments using rats. In all three experiments, two different stimuli were established as S+s for different response-outcome relations. In Experiment 1, an S+ was less effective in controlling its original response after it had undergone training as an S? for a new response that earned the same outcome than it was after training as an S? for a response that earned a different outcome. Experiment 2 established that this effect was not mediated by Pavlovian inhibitory conditioning produced by the negative correlation between the S+ and the outcome during S? training. Simply arranging a negative correlation between S+ and the outcome whose occurrence it had previously signaled did not impair the ability of that S+ to elicit its original response. In Experiment 3, the response-evoking properties of an S+ were found to be undermined by using the S+ as a signal for the simple extinction of a new response trained with the same outcome, but not with a different outcome. These results suggest that positive discriminative stimuli use their associations with the outcomes earned in their presence to control the responses that earned those outcomes.     

Pigeons’ short-term memories for surprising vs. expected reinforcement and nonreinforcement

Pigeons performed a delayed matching-to-sample task in which they matched red and green disks as comparison stimuli to samples of food and no food. The birds were also taught a discrimination between two lines: vertical (S+) followed by food and horizontal (S?) followed by no food. The two kinds of trials were then chained in infrequent probes such that (a) S+ and S? preceded samples of food and no food, (b) a longer than usual delay occurred, and then, (c) the comparison stimuli were presented. Therefore, in probes when S+ preceded food and S? preceded no food, the samples were “expected. ” But in probes when S+ signaled no food and S? signaled food, the samples were “surprising. ” Matching to surprising samples was more accurate than matching to expected samples. This result completes a pattern of findings implying that surprising reinforcers enhance learning and also persist (are longer rehearsed) in short-term memory.     

Memories of reward events and expectancies of reward events may work in tandem

A common assumption is that expectancies of reward events in instrumental tasks are established on the basis of Pavlovian conditioning. According to the tandem hypothesis, tested in the four runway investigations reported here employing rats, memories of reward events may serve as the conditioned stimuli eliciting expectancies. In Experiments 1–3, rats were trained under a schedule of partial reward (P), which did not produce increased resistance to extinction, and subsequently shifted to consistent reward (C). According to the tandem hypothesis, the shift to the C schedule should result in increased resistance to extinction if, as hypothesized, under the P schedule the memory of reward, S^R, came to elicit the expectancy of nonreward,_EN. This hypothesis was confirmed under a variety of conditions. It was shown that increased resistance to extinction could not be attributed to the P schedule alone, to the rats receiving two schedules, P and C, to stimuli other than S^R eliciting E_N, or to the rats forgetting reward-produced memories when expecting nonreward (Experiment 4). It was shown that the tandem hypothesis could explain the divergent findings obtained in prior studies employing a shift from P to C as well as in the present study.     

Gradient form and sequential effects during generalization testing in extinction

Four naive pigeons were given six generalization tests in extinction after periods of pretraining in which S+ appeared with food reinforcement and S? appeared in extinction. An analysis of sequential effects among presentations of test stimuli showed that the overall gradient was influenced differently by stimuli at the extremes of the continuum of test stimuli and by S+ and adjacent stimuli. Gradients consisting of responding in each stimulus when it was preceded by an extreme stimulus tended to peak at S+, while gradients produced when each stimulus was preceded by S+ or an adjacent stimulus tended to show a peak shift. This was true whether the overall gradient showed a peak shift or not. Two naive subjects were added and four additional tests were given after pretraining in which unequal frequencies of reinforcement accompanied both S+ and S?. Results of all 10 tests show that sequential effects occur during generalization testing in extinction and that these “local dimensional effects” are unlike local contrast. These stimulus-specific sequential effects may greatly influence overall gradient form.     

Attention as an acquisition and performance variable (AAPV)

This article introduces a new model of Pavlovian conditioning, attention as an acquisition and performance variable (AAPV), which, like several other so-called attentional models, emphasizes the role of variation of cue salience, together with associative strength, in accounting for conditioning phenomena. AAPV is primarily (but not exclusively) a performance-focused model in that it assumes not only that both the saliences and associative strengths of cue representations change during acquisition, but also that they are both influential at the time of test in determining responding. Different weights are given to the representations’ associative strengths according to the representations’ respective saliences at test. The model also treats the representation of a stimulus that is directly activated by presentation of that stimulus as distinct from the representation of the same stimulus that is activated by presenting a companion of the stimulus. Additionally, extinction is viewed as resulting from a decrease in the salience of the cue’s representation, rather than a decrease in associative strength. Simulations of several Pavlovian phenomena are presented in order to illustrate the model and assess its robustness.     

Effects of stimulus alternations on extinguishing a discrimination

Rats were trained in a light ON vs light OFF discrimination in operant chambers with food reinforcement. Following acquisition, extinction under conditions of no alternations between S+ and S? and under conditions of numerous alternations between S+ and S? were examined. In Experiment I, extinction in S+ or S? alone produced less responding to S+ and more responding to S? than extinction with regular alternations between S+ and S?. In Experiment II, 9, 39, or 79 alternations in extinction between S+ and S? produced no differences in responding. These results indicate that during extinction of a discrimination there is (a) sharpening of differential performance, (b) a difference between multiple- and single-stimulus procedures, and (c) little effect of different numbers of alternations.     

Resistance to discrimination and subsequent resistance to extinction as a function of the sequence of partial S+ reward in differential conditioning

The effects of transitions from nonrewarded (N) to rewarded (R) trials (N-R transitions) on discriminative behavior in differential conditioning and subsequent resistance to extinction were investigated in two experiments. In Experiment 1, groups given N-R transitions within S+ were more resistant to discrimination (ran fast in S?) and extinction than were groups given a partial reinforcement (PRF) schedule in S+ devoid of N-R transitions. Experiment 2 indicated that N-R transitions that occur when an N trial in S? is followed by an R trial in S+ are as effective in increasing resistance to discrimination, but not resistance to extinction, as are N-R transitions that occur within S+. The sequential effects obtained here were highly similar to those in conventional PRF and support the view that differential conditioning and PRF are highly interrelated phenomena. The results are discussed in terms of the extension of sequential theory to differential conditioning and the importance of internal reward-produced cues in discrimination learning.     

Food as a contextual cue in counterconditioning experiments: Is there a counterconditioning process?

In three experiments, counterconditioning was found to reduce fear less effectively than extinction. In Experiments 1 and 2, the resistance to extinction of avoidance was greater if food was given during extinction of fear to the CS than if no food was given, even when exposure to the CS and numbers of food and no food confinement trials were equated. It is suggested that these results could be attributed to contextual control of fear extinction by the food cue and/or to frustration produced by removing food for the counterconditioning group. Experiment 3 also found counterconditioning to be less effective than extinction and provided evidence that this difference occurs because of contextual control of fear extinction by the food cue. Measuring conditioned suppression of licking, in a test with no food present, less fear was shown if no food had been present during fear extinction, and greater fear was shown if no food had been present during fear conditioning. These results indicate that food is an important part of the context controlling fear and fear extinction. It is suggested that there may be no unique counterconditioning process. Rather, when counterconditioning procedures are employed, rules governing interference paradigms in general may apply. Thus, in a test for fear following counterconditioning, fear will be shown to the extent the test situation is similar to that in which fear conditioning occurred rather than that in which fear reduction occurred.     

An investigation of ambiguous-cue learning in pigeons

Two experiments demonstrated that pigeons can solve a simultaneous discrimination in which on half the trials the positive and an ambiguous cue (A) are presented and on half the trials choice is between A and the negative stimulus. In Experiment 1, where a relatively nondistinctive A cue was used, performance on the former type of trial was superior to that shown on the latter. In Experiment 2, where a distinctive A cue was provided, this pattern of results was reversed. These findings are interpreted in terms of an approach-avoidance explanation first proposed by Leary (1958). Experiment 3 tested and confirmed a central prediction of this explanation by showing that in an orthodox simultaneous discrimination, occasional reinforcements of the negative stimulus produce less accurate performance than do nonreinforcements of the positive.     

Taste quality and extinction of a conditioned taste aversion in rats

Rats (Rattus norvegicus) that received a taste cue (saccharin, saline, quinine, or sucrose) paired with a lithium chloride (LiCl) injection displayed a robust decrease in consumption of that taste, relative to controls that had the taste unpaired with LiCl. Consumption of the paired taste increased with each nonreinforced presentation (i.e., extinction). After asymptotic extinction, rats that had had a 0.1% saccharin cue paired with LiCl consumed less of the saccharin solution than did controls. A similar data pattern was observed with a 10% sucrose solution. These results are consistent with the view that some aspect of the excitatory CS-US association remains after extinction. On the other hand, rats that had a bitter (0.005% or 0.001% quinine) or salty (1% or 0.5% saline) solution paired with LiCl drank similar amounts of the fluid as controls after asymptotic extinction treatment. Together, these experiments suggest that a taste that is either sweet or preferred is required in order to demonstrate the chronic decrease in fluid consumption after extinction treatment. The data suggest that the conditioning experience prevents the later development of a preference for the sweet taste, rather than there being a retained aversion that suppresses fluid consumption.     

Reward sequence and reinforcement level as determinants of S− behavior in differential conditioning

In two differential conditioning experiments, groups of 10 rats each differed with respect to average reward and schedule of reward received in S+. Nonreward (N) occurred on all S? trials. In both experiments, extinction of responding to S? (resistance to discrimination) was extensively regulated by reward sequence and was largely independent of average reward. In Experiment 1, resistance to discrimination was a function of transitions from N to rewarded (R) trials (N-R transitions). In Experiment 2, resistance to discrimination was increased by large reward on the R trial of N-R transitions and decreased by large reward on the R trial of R-N transitions. These schedule effects on resistance to discrimination parallel the effects of comparable schedules on resistance to extinction following partial reinforcement. The results are discussed in terms of sequential theory, reinforcement level theory, and their implications for various schedule manipulations that have previously shown S? behavior to be inversely related to average reward in S+.     

Further evidence that conditioned inhibition is not the mechanism of an extinction cue’s effect: A reinforced cue prevents spontaneous recovery

Previous research has demonstrated that discrete cues presented during extinction of a conditioned response (extinction cues) subsequently reduce spontaneous recovery and the renewal effect. In order to assess whether an extinction cue’s effect is due to conditioned inhibition, the effect of pairing an extinction cue with food on the cue’s capacity to modulate spontaneous recovery was investigated in an appetitive conditioning experiment with rats. Conditioned inhibitors paired with the unconditioned stimulus lose their potential to inhibit responding. The food-paired extinction cue did not lose its potential to reduce spontaneous recovery. In fact, it reduced spontaneous recovery more than did an extinction cue that was not paired with food. The results indicate that extinction cues attenuate postextinction recovery of conditioned responding through a mechanism other than conditioned inhibition. The cue’s action appears to be similar to that of serial negative occasion setters. Theories of spontaneous recovery and the relationship of extinction cues to other modulatory stimuli are discussed.     

Protection from extinction provided by a conditioned inhibitor

Three conditioned suppression experiments with rats as subjects investigated the influence of higher order associations in determining the response potential of a target stimulus. In these experiments, a Pavlovian conditioned inhibitor was compounded with the target cue during extinction treatment. In Experiment 1, strong suppression was observed to the target cue that was given extinction treatment in the presence of a conditioned inhibitor, relative to a target that was extinguished with an associatively neutral cue or was extinguished alone, suggestive of enhanced protection from extinction provided by a conditioned inhibitor. This effect was replicated in a sensory preconditioning preparation in Experiment 2; in Experiment 3, in a sensory preconditioning preparation, this protection effect was retroactively attenuated when the conditioned excitor used to train the conditioned inhibitor was extinguished following extinction of the target. This provides evidence that, at least in a sensory preconditioning preparation, stimuli that are only indirectly associated with the target cue can contribute to the response potential of that target.     

Recovery of extinction responding in rats following discontinuation of reinforcement of alternative behavior: A test of two explanations

One procedure which has been used to supplement extinction in order to produce faster and more complete response suppression is to provide reinforcement for some alternative response which is incompatible with the response undergoing extinction. When reinforcement for the alternative behavior is discontinued, however, substantial recovery of the original response has often been observed. The present set of experiments demonstrated that such recovery is best accounted for by a “response prevention” hypothesis rather than by a “discriminative cue” hypothesis. High-frequency reinforcement of alternative behavior during the first half of an extinction phase seems similar in effect to procedures which physically prevent rats from emitting the response programmed for extinction.     

A comparison of signaled vs. unsignaled free-operant avoidance in wild and domesticated rats

Three experiments compared the performance of domesticated hooded rats and nondomesticated black rats under signaled and unsignaled free-operant leverpress avoidance. There was no difference between groups in asymptotic shock rates under unsignaled avoidance; however, the black rats avoided much more successfully when the signal was present, while the hooded rats showed little or no improvement. When a longer signal was employed (10 vs. 5 sec), the effects were essentially the same. The black rats generally had higher response rates, and this difference was most pronounced in extinction, where the hooded rats made very few responses. The domesticated rats received a disproportionately large number of shocks early in the session (warm-up) under unsignaled avoidance, but this tendency was much less pronounced for the black rats. However, both groups showed appreciable amounts of warm-up during signalled avoidance. The findings are discussed in terms of differences in levels of activation/arousal between domesticated and nondomesticated animals.     

Enhancement and reduction of associative retroactive cue interference by training in multiple contexts

Retroactive cue interference refers to situations in which a target cue X is paired with an outcome in phase 1 and a nontarget cue Z is paired with the same outcome in phase 2, with less subsequent responding to X being seen as a result of the phase 2 training. Two conditioned suppression experiments with rats were conducted to determine whether retroactive cue interference is similarly modulated by a manipulation that influences retroactive outcome interference (e.g., extinction). Both experiments used an ABC renewal-like design in which phase 1 training, phase 2 training, and testing each occurred in different contexts. Experiment 1 found that training the target association in multiple contexts without altering the number of training trials during phase 1 decreased retroactive cue interference (i.e., increased responding consistent with the target association). Experiment 2 found that training the interfering association in multiple contexts without altering the number of interference trials during phase 2 increased retroactive cue interference (i.e., decreased responding consistent with the target association). The possibility of similar mechanisms underlying cue interference and outcome interference is discussed.     

Spontaneous recovery and ABC renewal from retroactive cue interference

Two conditioned suppression experiments with rats were conducted to determine whether the spontaneous recovery and renewal that are commonly observed in retroactive outcome interference (e.g., extinction) also occur in retroactive cue interference. Experiment 1 showed that a long delay between Phase 2 (the interfering phase) and testing produces a recovery from the cue interference (i.e., the delay enhanced responding to the target cue trained in Phase 1), which is analogous to the spontaneous recovery effect observed in extinction and other retroactive outcome interference procedures. Experiment 2 showed that, when target and interfering cues are trained in separate contexts and testing occurs in a different but familiar context, a recovery from the cue interference is also observed (i.e., the context shift enhanced responding to the target), which is analogous to ABC renewal from extinction. The results are discussed in terms of the possibility that similar associative mechanisms underlie cue and outcome interference.     

Extended exposure to environmental cues,but not to sucrose,reduces sucrose cue reactivity in rats

In the present study, we examined the effects of extinction of sucrose-predictive contextual cues and/or sucrose satiation on the expression of sucrose cue reactivity in a rat model of relapse. Context extinction was imposed by housing rats in their home cage or in the operant conditioning chamber for 17 h prior to testing. For sucrose satiation, rats were allowed unlimited access to water or sucrose for 17 h prior to testing. Cue reactivity was assessed after either one (Day 1) or 30 (Day 30) days of forced abstinence from sucrose self-administration. An abstinence-dependent increase in sucrose cue reactivity was observed in all conditions (“incubation of craving”). Context extinction dramatically reduced lever responding on both Day 1 and Day 30. Sucrose satiation had no significant effect on cue reactivity in any condition. These results demonstrate that the context in which self-administration occurs maintains a powerful influence over cue reactivity, even after extended forced abstinence. In contrast, the primary reinforcer has little control over cue reactivity. These findings highlight the important role of conditioned contextual cues in driving relapse behavior.