Resurgence of instrumental behavior after an abstinence contingency

In resurgence, an extinguished instrumental behavior (R1) recovers when a behavior that has replaced it (R2) is also extinguished. The phenomenon may be relevant to understanding relapse that can occur after the termination of “contingency management” treatments, in which an unwanted behavior (e.g., substance abuse) is reduced by reinforcing an alternative behavior. When reinforcement is discontinued, the unwanted behavior might resurge. However, unlike most resurgence experiments, contingency management treatments also introduce a negative contingency, in which reinforcers are not delivered unless the client has abstained from the unwanted behavior. In two experiments with rats, we therefore examined the effects of adding a negative “abstinence” contingency to the resurgence design. During response elimination, R2 was not reinforced unless R1 had not been emitted for a minimum period of time (45, 90, or 135 s). In both experiments, adding such a contingency to simple R1 extinction reduced, but did not eliminate, resurgence. In Experiment 2, we found the same effect in a yoked group that could earn reinforcers for R2 at the same points in time as the negative-contingency group, but without the requirement to abstain from R1. Thus, the negative contingency per se did not contribute to the reduction in resurgence. These results suggest that the contingency reduced resurgence by making reinforcers more difficult to earn and more widely spaced in time. This could have allowed the animal to learn that R1 was extinguished in the “context” of infrequent reinforcement—a context more like that of resurgence testing. The results are thus consistent with a contextual (renewal) account of resurgence. The method might provide a better model of relapse after termination of a contingency management treatment.     

Quantitative analysis of local-level resurgence

Resurgence is the recurrence of a previously reinforced and then extinguished behavior induced by the extinction of another more recently reinforced behavior. Resurgence provides insight into behavioral processes relevant to treatment relapse of a range of problem behaviors. Resurgence is typically studied across three phases: (1) reinforcement of a target response, (2) extinction of the target and concurrent reinforcement of an alternative response, and (3) extinction of the alternative response, resulting in the recurrence of target responding. Because each phase typically occurs successively and spans multiple sessions, extended time frames separate the training and resurgence of target responding. This study assessed resurgence more dynamically and throughout ongoing training in 6 pigeons. Baseline entailed 50-s trials of a free-operant psychophysical procedure, resembling Phases 1 and 2 of typical resurgence procedures. During the first 25 s, we reinforced target (left-key) responding but not alternative (right-key) responding; contingencies reversed during the second 25 s. Target and alternative responding followed the baseline reinforcement contingencies, with alternative responding replacing target responding across the 50 s. We observed resurgence of target responding during signaled and unsignaled probes that extended trial durations an additional 100 s in extinction. Furthermore, resurgence was greater and/or sooner when probes were signaled, suggesting an important role of discriminating transitions to extinction in resurgence. The data were well described by an extension of a stimulus-control model of discrimination that assumes resurgence is the result of generalization of obtained reinforcers across space and time. Therefore, the present findings introduce novel methods and quantitative analyses for assessing behavioral processes underlying resurgence.     

Hurdle-jump responding in the rat as a function of conspecific odor of reward and nonreward

A hurdle-jump escape response was employed to assess the laboratory rat’s aversion or attraction to different types of conspecific odor. Odorant donor subjects received 112 runway acquisition trials on a continuous reward schedule followed by 32 extinction trials, 112 acquisition trials on a 50% schedule of reward and nonreward followed by 32 extinction trials, or 144 “neutral” trials with no reward in the alley. Different groups of test subjects escaped from odor excreted by odorant subjects on (a) nonrewarded acquisition and extinction trials, (b) rewarded trials during continuous reinforcement, (c) rewarded trials during partial reinforcement, or (d) neutral trials; others escaped from a clean box. The principal findings were: (1) significant aversion to “odor of nonreward” appeared after the donor odorants had received 12 exposures to reward; (2) production of odor of nonreward by odorant subjects changed as a function of training experience with reward; (3) after repeated exposure to odor of nonreward, the escape response habituated; (4) greater or different odor excretion in extinction resulted from subjects trained on a continuous reward schedule than on a partial reward schedule. Relationships of the data to frustration theory were discussed, assuming that inferred differences in production of odor reflect differences in frustration reaction.     

Development of preference and spontaneous recovery in choice behavior with concurrent variable-interval schedules

Pigeons pecked on two response keys that delivered reinforcers on a variable-interval schedule. The proportion of reinforcers delivered by one key was constant for a few sessions and then changed, and subjects’ choice responses were recorded during these periods of transition. In Experiment 1, response proportions approached a new asymptote slightly more slowly when the switch in reinforcement proportions was more extreme. In Experiment 2, slightly faster transitions were found with higher overall rates of reinforcement. The results from the first session, after a switch in the reinforcement proportions, were generally consistent with a mathematical model that assumes that the strength of each response is increased by reinforcement and decreased by nonreinforcement. However, neither this model nor other similar models predicted the “spontaneous recovery” observed in later sessions: At the start of these sessions, response proportions reverted toward their preswitch levels. Computer simulations could mimic the spontaneous recovery by assuming that subjects store separate representations of response strength for each session, which are averaged at the start of each new session.     

Effects of reinforcement-paired stimuli on general activity

Two experiments were performed to explore the mechanisms responsible for the increase in activity that occurs in response to stimuli which have been paired with reinforcement (S^R. The first experiment showed a sharp increase in activity to S^R-paired stimuli under conditions in which subjects were not required to perform any instrumental response to obtain the S^R. This result seemed to rule out reinforcement of instrumental “food getting” behavior as the mechanism responsible for the learned activity increases. A second experiment used an “omission training” procedure to further explore the mechanisms underlying the activity increase. In this experiment, S^R was omitted on those trials on which activity increases were present during the stimulus. In this condition, no increases in activity were observed during the stimulus. There was, however, the characteristic increase in activity in a yoked-control group which received the same number and distribution of stimulus-paired S^Rs. The results of the second experiment open the possibility that increases in activity to S^R -paired stimuli could be due to the adventitious reinforcement of motor behavior rather than the Pavlovian conditioning of a motivational state.     

The effects of extrinsic reinforcement on intrinsic motivation

In this paper, the role of extrinsic reinforcement in intrinsic motivation is discussed in terms of Deci's cognitive attribution theory. The competing response hypothesis and the frustration hypothesis are presented as alternative modes of analysis. “Undermining” is the phenomenon proposed by the cognitive attribution theorists that accounts for reported decrements in intrinsically motivated behaviors following external rewards. The literature regarding undermining in token economy research is critically evaluated. The author concludes that: (a) Cognitive attribution theory lacks parsimony, in that extant reinforcement analysis can account for undermining with equal facility. (b) Undermining is of little significance as a concept, due to its elusive and transient impact on operant behavior.     

Nonreward anticipated: Effects on extinction runway performance in the rat

Following a 47-day extinction procedure, the reinstatement of the cue previously associated with reward produced an immediate improvement in performance. The cue was the opportunity to traverse the alley following an “anticipated” nonrewarded runway trial. Moreover, the animals trained in this matter exhibited daily increments in performance during the initial phase of extinction testing. The results were interpreted as consistent with the notion that the difference in extinction performance of one group as compared to another does not necessarily reflect the relative “strengths” of the instrumentally acquired habits. Instead, it probably indicates the degree of similarity of the extinction testing procedure to the acquisition training condition previously associated with reinforcement for the two individual groups.     

Effects of prior response-contingent reinforcement on superstitious behavior

Three pigeons were exposed to fixed-time (FT) 15 sec, fixed-interval (FI) 15 sec for performing an arbitrary response, a reversal back to FT 15 sec, and then extinction (no reinforcement). During each phase, a computer-controlled tracking system continuously recorded the position of the bird’s head as it moved freely in the experimental chamber. During the first exposure to FT 15 sec, all 3 birds developed a pattern of feeder-wall-directed behavior with occasional circular excursions from the feeder immediately following reinforcement. During FI 15 sec, all birds performed the arbitrary operant, which consisted of contacting a virtual target sphere near the rear of the chamber, and did not engage in feeder-wall-directed behavior. During the reversal back to FT 15 sec, the birds developed a behavior sequence consisting of moving in the direction of the target sphere after reinforcement, followed by feeder-wall-directed behavior prior to the next reinforcement. During extinction, either moves toward the target sphere or wall-directed behavior occurred separately, interspersed with reappearance of the two as a sequence, followed by cessation of both members of the behavior sequence. These findings indicate that prior reinforcement of an arbitrary response can affect the location and form of superstitious behavior that develops near the beginning of the interreinforcement interval, but that other factors (e.g., immediacy of reinforcement) affect the location and form of the behavior near the end of the interval. The findings can be interpreted in the context of superstitious chaining.     

Behavioral momentum and relapse of extinguished operant responding

Previous experiments on behavioral momentum have shown that relative resistance to extinction of operant behavior in the presence of a stimulus depends on the rate of reinforcement associated with that stimulus, even if some of those reinforcers occur independently of the behavior. We present three experiments examining whether the rate of reinforcement in the presence of a stimulus similarly modulates the relative relapse of operant behavior produced by reinstatement, resurgence, and renewal paradigms. During baseline conditions, pigeons responded for food reinforcement on variable-interval 120-sec schedules in alternating periods of exposure to two stimuli arranged by a multiple schedule. Additional response-independent food presentations were also delivered in the presence of one of the multiple-schedule stimuli. Consistent with previous research, baseline response rates were lower in the presence of the stimulus with the added response-independent reinforcement, and relative resistance to extinction was greater in the presence of that stimulus. In addition, following extinction, the relative relapse of responding produced by reinstatement, resurgence, and renewal paradigms was greater in the presence of the stimulus associated with the higher rate of reinforcement. We suggest that a model of extinction from behavioral momentum theory may be useful for understanding these results.     

Configural conditioning in the CER: A possible artifact

Gray and Lethbridge (1976) have reported a demonstration of configural conditioning in the CER in rats. With repeated reinforcement of a compound CS, suppression to the elements diminished considerably, while suppression to the compound remained substantially intact. The basic form of the Gray and Lethbridge findings has been replicated in the present study. We observed, however, that suppression to the compound itself diminished with repeated reinforcement of the compound. Further, we also tested the compound after deliberate extinction of the elements. This test of the excitatory effect of the “configurai cue” indicated that it also had diminished with repeated reinforcement of the compound. The observed greater suppression to the compound than to the elements can be parsimoniously interpreted as a simple summation effect, and the attenuation of suppression with repeated reinforcement is common in CER studies. Thus, the data require no appeal to configural conditioning.     

Partial reinforcement in autoshaping with pigeons

The acquisition, maintenance, and extinction of autoshaped responding in pigeons were studied under partial and continuous reinforcement. Five values of probability of reinforcement, ranging from .1 to 1.0, were combined factorially with five values of intertrial interval ranging from 15 to 250 sec for different groups. The number of trials required before autoshaped responding emerged varied inversely with the duration of the intertriai interval and probability of reinforcment, but partial reinforcement did not increase the number of reinforcers before acquisition. During maintained training, partial reinforcement increased the overall rate of responding. A temporal gradient of accelerated responding over the trial duration emerged during maintenance training for partial reinforcement groups, and was evident for all groups in extinction. Partial reinforcement groups responded more than continuous reinforcement groups over an equivalent number of trials in extinction. However, this partial-reinforcment extinction effect disappeared when examined in terms of the omission of “expected” reinforcers.     

The “disinhibition” of extinguished operant behavior in pigeons: Trial-tempo shifts and novel stimulus effects

After conditioning and extinction of keypecking with 25-see intertrial intervals between key illuminations, an immediate change to 5-sec intertrial intervals reinstated pecking during trials. Brief illuminations of the chamber during intertrial intervals also temporarily restored extinguished keypecking. The same manipulations of trial tempo and chamber illumination usually weakened well established, still reinforced keypecking. No recovery of extinguished behavior occurred when the intertrial interval was shifted upward from 5 sec to 25 sec. These and other behavioral findings were examined in relation to (a) Pavlov’s and Skinner’s research and views on “disinhibition” and “external inhibition” and to (b) analogous phenomena in physiological studies of habituation and dishabituation. The reliable evidence of disinhibition obtained in the present experiments suggests the involvement of an inhibitory process in extinction.     

On the organization of stereotyped response sequences

When pigeons are required to peck each of two keys in any order for reinforcement, stereotyped response sequences develop that are resistant to disruption by extinction, schedules of reinforcement, or contingencies requiring sequence variability. To test the hypothesis that stereotyped response sequences become integrated behavioral units, two experiments introduced within-sequence temporal delays of varying duration. Experiment 1 found that when a delay followed each peck in a sequence, there was substantial disruption of sequence performance that was independent of delay duration. However, such disruption was only temporary. Experiment 2 found that when the location of a delay within a sequence was varied, sequence disruption was a function of when, in a sequence, the delay occurred. Delays that occurred within sequence subunits had large effects, whereas delays that occurred between such subunits had small effects. The data indicate that pigeons can learn to bridge within-sequence delays, and suggest that response sequences are organized into “phrases.”     

Training reinforcement rates,resistance to extinction,and the role of context in reinstatement

Behavior reduced as a consequence of extinction or intervention can relapse. According to behavioral momentum theory, the extent to which behavior persists and relapses once it has been eliminated depends on the relative training reinforcement rate among discriminative stimuli. In addition, studies of context renewal reveal that relapse depends on the similarity between the training stimulus context and the test stimulus context following disruption by extinction. In the present experiments with pigeons, we arranged different reinforcement rates in the presence of distinct discriminative stimuli across components of a multiple schedule. Following extinction, we attempted to reinstate responding in the presence of those target components with response-independent food presentations. Importantly, we arranged the reinstating food presentations either within the target components or in separate components, either paired with extinction (Experiment 1) or reinforcement (Experiment 2) during baseline. Reinstatement increased with greater training reinforcement rates when the reinstating food presentations were arranged in the target components and the separate components paired with reinforcement during training. Reinstatement was smaller and was not systematically related to training reinforcement rates in the target components when reinstating food presentation occurred in separate components paired with extinction. These findings suggest that relapse depends on the history of reinforcement associated with the discriminative stimuli in which the relapse-inducing event occurs.     

Feedback functions for reinforcement: A paradigmatic experiment

A reinforcement schedule states a rule for obtaining reinforcement as a function of behavior actually emitted and perhaps as a function of additional variables. These functions are here called “feedback functions.” Behavior actually emitted is, in turn, a function of obtained reinforcement. This reciprocal interdependency was quantified for an experiment in which pigeons chose among either three or two alternatives. Shifting from three to two alternatives, and vice versa, produced changes in the distribution of responding which were approximately accounted for by equations that combined the feedback functions with the matching law for reinforced responding. These equations predicted, among other things, a violation of the constant-ratio rule of formal choice theory and an absence of simultaneous contrast effects between response alternatives reinforced on variable-interval schedules. Both predictions were approximately confirmed.     

Behavioral contrast and type of reward: Role of elicited response topography

Groups of pigeons were exposed to multiple variable-interval variable-interval and multiple variable-interval extinction schedules of either food or water reinforcement for keypecking. Discriminative stimuli associated with component schedules were located either on the operant key or on a second “signal” key. When the stimuli were projected on the operant key, positive contrast appeared during discrimination conditions with either food or water as the reinforcer. When the stimuli were projected on the signal key, overall responding to the operant and signal keys showed contrast with food, but negative induction with water as the reinforcer. In the latter condition, the signal for the variable-interval shcedule of water reinforcement elicited a variety of water-related behavior, only some of which was directed at the signal. Thus, the type of reward and location of discriminative stimuli interacted to determine the presence or absence of behavioral contrast effects. In large part, these results support and extend the autoshaping view of contrast.     

Extinction of operant behavior: An analysis based on foraging considerations

In two experiments, the frequency of food reinforcement provided by variable interval (VI) schedules prior to extinction was varied. In the first experiment, two-component multiple schedules resulted in a greater number of responses in extinction in the presence of the stimulus previously associated with the richer of the two component schedules than that previously associated with the leaner schedule. In the second experiment, different groups of animals were trained on different VI schedules. Responding in extinction was analyzed into bouts of responding showing that the number of response bouts increased and the number of responses per bout decreased with decreasing frequency of reinforcement during training. These data are compatible with an analysis of operant behavior based on an analogy to processes that presumably-occur-in naturalistic foraging situations. According to this analogy, behavior associated with search for a food source (i.e., number of response bouts) and that of procurement of food from a source (i.e., responses per bout) represent aspects of behavior that are differentially strengthened by different VI schedules. Extinction serves to reveal this differential strengthening.     

Gradient form and sequential effects during generalization testing in extinction

Four naive pigeons were given six generalization tests in extinction after periods of pretraining in which S+ appeared with food reinforcement and S? appeared in extinction. An analysis of sequential effects among presentations of test stimuli showed that the overall gradient was influenced differently by stimuli at the extremes of the continuum of test stimuli and by S+ and adjacent stimuli. Gradients consisting of responding in each stimulus when it was preceded by an extreme stimulus tended to peak at S+, while gradients produced when each stimulus was preceded by S+ or an adjacent stimulus tended to show a peak shift. This was true whether the overall gradient showed a peak shift or not. Two naive subjects were added and four additional tests were given after pretraining in which unequal frequencies of reinforcement accompanied both S+ and S?. Results of all 10 tests show that sequential effects occur during generalization testing in extinction and that these “local dimensional effects” are unlike local contrast. These stimulus-specific sequential effects may greatly influence overall gradient form.     

What is meant by the term “schedule-induced,” and how general is schedule induction?

The term “schedule-induced” implies that the overall frequency of a behavior is greater in the presence of an intermittent schedule of reinforcement than in the absence of such a schedule. Consequently, the occurrence of interreinforcement behavior is not in itself sufficient evidence of schedule induction: a test of induction requires comparison between an intermittent-schedule condition and a nonschedule baseline. The relative merits of different types of nonschedule baseline are examined, and it is concluded that the best test of schedule-induction involves both an extinction and a massed-reinforcer baseline. A working definition of schedule-induction is suggested on this basis. Studies purporting to show schedule induction of activities other than drinking are critically reviewed, and it is concluded that schedule induction may be less general than is usually supposed. It may therefore be more fruitful to seek an explanation of schedule-induced drinking which focuses specifically on the interaction between food and water ingestion in the rat, rather than an explanation involving concepts such as stress, frustration, or arousal.