Eliminating behavior with omission and extinction after varying amounts of training

After rats were trained to leverpress for 1, 3, 9, or 27 days on a variable interval reinforcement schedule, omission training was compared with extinction in effectiveness of response elimination. Extinction produced faster response elimination than omission, although both procedures eventually led to equal response elimination. Resistance to response elimination increased with length of baseline training, although this effect did not interact with omission vs extinction. A test of the durability of elimination effects followed, using a response-independent variable time reinforcement schedule. After extinction, resumption of responding in the durability test increased with length of baseline training, but there was little response resumption following omission regardless of the length of the baseline training. These results amplify and extend previous findings which show omission to be an effective and durable response elimination method.     

Inhibitory effects of omission training

In Experiment 1, pigeons were trained to peck red or blue keys for food reinforcement at variable intervals, while food was contingent on withholding key pecks in the presence of a vertical line (omission training). When the line was briefly superimposed on red or blue in a compound test, responding was reduced. When the orientation of the line was varied during extinction, generalization gradients were variable but often had most responding at or near vertical. In Experiment 2, pigeons were trained in a discrete trials procedure that made food contingent upon pecking in the presence of triangle, and upon the absence of pecking in the presence of red (omission training). Food was never given on green-key trials (extinction). When red or green backgrounds were presented with the triangle in a compound test, responding was reduced similarly in the presence of both key colors. Subsequent resistance to auto-shaping was also similar for red and green. These data, taken together with reports in the literature, suggest that the inhibitory effects of omission training are quite similar to those of extinction. Thus, the crucial condition for obtaining inhibitory effects is not a negative stimulus-reinforcer correlation, as in extinction, but simply the establishment of low rates of responding to the inhibitory stimulus.     

Stimulus control in multiple variable-ratio schedules of reinforcement

One component of a multiple variable-ratio 150 variable-ratio 150 schedule remained unchanged while the reinforcement schedule of the other component was periodically changed to extinction and then returned to variable ratio 150. When the reinforcement schedule of the changed component was an unmodified variable ratio 150 schedule, the magnitude of negative contrast during baseline recovery was equal to the magnitude of positive contrast observed during the previous change from multiple variable-ratio 150 variable-ratio 150 to multiple variable-ratio 150 extinction. When the schedule of the changed component was modified during baseline recovery so that responses terminating interresponse times greater than the average baseline interresponse time in the unchanged component were not reinforced, the magnitude of the unchanged-component response rate decrease was reduced. The magnitude of negative contrast was attenuated even though response and reinforcement rates in the variable component increased to levels at or above their prior multiple variable-ratio 150 variable-ratio 150 baseline levels. The results support a general theory that negative contrast results from both (1) the removal of certain positive-contrast-producing operations and (2) changes in the interresponse time-reinforcer relations that occur as a byproduct of schedule manipulations.     

Training reinforcement rates,resistance to extinction,and the role of context in reinstatement

Behavior reduced as a consequence of extinction or intervention can relapse. According to behavioral momentum theory, the extent to which behavior persists and relapses once it has been eliminated depends on the relative training reinforcement rate among discriminative stimuli. In addition, studies of context renewal reveal that relapse depends on the similarity between the training stimulus context and the test stimulus context following disruption by extinction. In the present experiments with pigeons, we arranged different reinforcement rates in the presence of distinct discriminative stimuli across components of a multiple schedule. Following extinction, we attempted to reinstate responding in the presence of those target components with response-independent food presentations. Importantly, we arranged the reinstating food presentations either within the target components or in separate components, either paired with extinction (Experiment 1) or reinforcement (Experiment 2) during baseline. Reinstatement increased with greater training reinforcement rates when the reinstating food presentations were arranged in the target components and the separate components paired with reinforcement during training. Reinstatement was smaller and was not systematically related to training reinforcement rates in the target components when reinstating food presentation occurred in separate components paired with extinction. These findings suggest that relapse depends on the history of reinforcement associated with the discriminative stimuli in which the relapse-inducing event occurs.     

Partial reinforcement in autoshaping with pigeons

The acquisition, maintenance, and extinction of autoshaped responding in pigeons were studied under partial and continuous reinforcement. Five values of probability of reinforcement, ranging from .1 to 1.0, were combined factorially with five values of intertrial interval ranging from 15 to 250 sec for different groups. The number of trials required before autoshaped responding emerged varied inversely with the duration of the intertriai interval and probability of reinforcment, but partial reinforcement did not increase the number of reinforcers before acquisition. During maintained training, partial reinforcement increased the overall rate of responding. A temporal gradient of accelerated responding over the trial duration emerged during maintenance training for partial reinforcement groups, and was evident for all groups in extinction. Partial reinforcement groups responded more than continuous reinforcement groups over an equivalent number of trials in extinction. However, this partial-reinforcment extinction effect disappeared when examined in terms of the omission of “expected” reinforcers.     

Analysis of autoshaping in goldfish

Stimulus-reinforcer and response-reinforcer control of target-striking in goldfish were studied in a series of discrete-trials experiments. Evidence of control by adventitious response-reinforcer contiguity was provided by the first experiment, which showed less response in animals given omission training than in yoked animals with the same stimulus-reinforcer experience. In the next three experiments, evidence of control by stimulus-reinforcer contiguity apart from response-reinforcer contiguity was sought in within-subjects comparisons of omission and extinction. Only the last of these experiments, in which the CS duration was short and the ITI long, showed greater response in omission. A subsidiary finding is that autoshaped goldfish respond very little, either to the CS target or to the feeder, when target and feeder are spatially discontiguous.     

Resurgence of behavior during extinction depends on previous rate of response

In two experiments, food-deprived rat subjects leverpressed for food in three successive training phases. In the first phase of both experiments, rats were exposed to a multiple schedule, one component of which produced a high rate of response, and the other of which produced a lower rate of response (multiple random ratio [RR], random interval [RI] in Experiment 1, and multiple differential reinforcement of high rate, differential reinforcement of low rate in Experiment 2). Rats were then transferred to a multiple fixed interval (FI; 60-sec, 60-sec) schedule, until the effects of the first phase on response rate were no longer apparent and their response rates did not differ from those of rats responding on a multiple FI 60-sec, FI 60-sec schedule without previously experiencing a multiple RR, RI schedule. During the third stage oftraining, all rats were placed into extinction. During extinction, rates of responding were higher in the component previously associated with the high rate of responding in Phase 1, and they were lower in the component previously associated with low rates of responding in Phase 1. These results suggest that resurgence effects, like other history effects, are controlled by previous rates of responding.     

Procedure for preventing response strain on random interval schedules with a linear feedback loop

An experiment examined the impact of a procedure designed to prevent response or extinction strain occurring on random interval schedules with a linear feedback loop (i.e., an RI+ schedule). Rats lever-pressed for food reinforcement on either a RI+ or a random interval (RI) schedule that was matched to the RI+ schedule in terms of reinforcement rate. Two groups of rats responded on an RI+ and two on an RI schedule matched for rate of reinforcement. One group on each schedule also received response-independent food if there had been no response for 60 s, and response-independent food continued to be delivered on an RT-60 schedule until a response was made. Rats on the RI and RI+ obtained similar rates of reinforcement and had similar reinforced inter-response times to one another. On the schedules without response-independent food, rats had similar rates of response to one another. However, while the delivery of response-independent food reduced rates of response on an RI schedule, they enhanced response rates on an RI+ schedule. These results suggest that rats can display sensitivity to the molar aspects of the free-operant contingency, when procedures are implemented to reduce the impact of factors such as extinction-strain.     

Positive behavioral contrast,autoshaping, and omission responding in the goldfish (Carassius auratus)

Two experiments with goldfish were performed to investigate the role of stimulus-reinforcer vs. response-reinforcer relationships in omission training and the role of stimulus localizability in a positive behavioral contrast paradigm. The directed behavior of fish, like that of pigeons and rats in other studies, was greatly influenced by positive stimulus-reinforcer correlations, as evidenced by maintained contacts of a signal for food, even though such responses terminated the signal and cancelled reinforcement delivery. Goldfish exhibited positive behavioral contrast when the signals for reinforcement and nonreinforcement were displayed directly on the response key, but no contrast was observed when variations in a diffuse houselight stimulus were used as signals for reinforcement or nonreinforcement. Analysis of sequential-trial data yielded effects analogous to Pavlovian positive and negative induction. Theoretical and methodological problems were briefly considered.     

Additive summation by stimulus compounding irrespective of behavioral contrast during discrimination training: An investigation with positive reinforcement and avoidance schedules

The similarity in the discrimination training leading to behavioral contrast and that preceding tests producing response enhancement to combined discriminative stimuli suggested that the two phenomena might be related. This was investigated by determining if contrast indiscrimination training was necessary for this outcome of stimulus compounding. Responding to tone, light, and to the simultaneous absence of tone and light (T + L) was maintained during baseline training by food reinforcement in Experiment I and by shock avoidance in Experiment II. During subsequent discrimination training, responding was reduced in T + L by programming nonreinforcement in Experiment I and safety or response-punishment in Experiment II. In the first experiment, one rat exhibited positive behavioral contrast, i.e., tone and light rates increased while his T + L rate decreased. In Experiment II, rats punished in T + L showed contrast in tone and light, this being the first demonstration of punishment contrast on an avoidance baseline with rats. The discrimination acquisition data are discussed in the light of current explanations of contrast by Gamzu and Schwartz (1973) and Terrace (1972). During stimulus compounding tests, all subjects in both experiments emitted more responses to tone-plus-light than to tone or light (additive summation). An analysis of the terminal training baselines suggests that the factors producing these test results seem unrelated to whether or not contrast occurred during discrimination training. It was concluded that the stimulus compounding test reveals the operation of the terminal baseline response associations and reinforcement associations conditioned on these multicomponent free-operant schedules of reinforcement.

     

Multiple-schedule interactions and discrimination

Pigeons received variable-interval (VI) reinforcement for keypecking during randomized presentations of seven line-orientation stimuli forming a continuum ranging from horizontal (0 deg) to vertical (90 deg). Each line presentation lasted for 30 sec and was preceded and followed by 30-sec time-outs. After responding stabilized, only responding in the two extreme stimuli (0 and 90 deg) was reinforced. As discrimination training proceeded, strong behavioral contrast and dimensional contrast effects appeared. However, only marginal local effects (local contrast and local dimensional effects), exerted by one line-orientation component upon a second, appeared, indicating that behavioral and dimensional contrast may be independent of parallel local effects. An attempt was made to apply Blough’s (1975) quantitative model of operant generalization and discrimination to the present discrimination procedure. However, this model did not predict the generalization gradient shape that was experimentally obtained. This experiment also yielded two serendipitous findings: (1) Positive behavioral contrast appeared in an extinction-related stimulus (time-out) when other stimuli were switched from reinforcement to extinction (hitherto, positive behavioral contrast had been observed only in responding to a reinforcementrelated stimulus when other stimuli were switched from reinforcement to extinction), and (2) final responding was higher in the presence of an extinction stimulus that had always been an extinction stimulus than it was in the presence of other extinction stimuli that had previously been paired with VI reinforcement.     

The effects of signaled reward on sign tracking and response rate

If, while responding on a variable interval schedule, rats are given a brief cue prior to reward, their response rate is markedly lower than the rate for yoked partners who receive the cue randomly with respect to reward. This signaled-reward phenomenon has been explained in terms of sign tracking. Two experiments reported here replicated the phenomenon and examined sign tracking directly through visual inspection of the animals’ behavior. Although sign tracking did, indeed, occur more in the signaled reward condition, it did not fully account for the difference in response rates.     

Behavioral contrast and reinforcement value

Behavioral contrast was produced in two target components of a four-component multiple schedule by having two target stimuli followed either by a higher rate of reinforcement or by extinction. Response rate was higher in the target followed by extinction. Periodic probe trials were then presented in which the two target stimuli were presented together. Choice on these probe trials was in favor of the stimulus followed by the higher rate of reinforcement during regular training. Experiment 2 replicated this finding but with probe trials presented throughout training. Here, preference for the stimulus followed by the higher rate of reinforcement was evident early in training, substantially before the contrast effects developed. The results challenge interpretations of contrast based on the concept of relative value.     

Acquisition and extinction of differential responses to signals paired with shock or shock omission in goldfish: Evidence for truly discriminated avoidance learning

Goldfish trained to discriminate between signals paired with shock (S?) and signals paired with shock omission (S+) with a linear presentation procedure, originally learned (OL) to control the signal state of a shuttle box and showed a decided preference for the S+ signal. In Experiment 1, following OL, groups had one OL signal replaced (S+ or S?), both signals replaced (S+ and S?), or the OL signals reversed (S+ and S? reversed) and were then tested in a transfer training procedure. In transfer, groups with one signal replaced maintained discriminated performance at OL levels; the S+ replaced group was slightly superior to the S? replaced group on the first day of transfer. With both OL signals replaced, discrimination dropped to chance performance levels, whereas, with OL signal shock pairing reversed, discrimination performance dropped below chance levels. In Experiment 2, following OL, extinction procedures consisted of turning off the shocker (0% shock) or of shocking 100% or a random 25% of the trials. A fourth extinction procedure (R,) retained the trial start response-dependent shock-omission contingency, but shock differentiating the S+ and S? signals was eliminated entirely. Extinction of the S+/S? discrimination was measured both during extinction training per se and with reversal retraining of the S+/S? discrimination later. Groups for which the OL S+ was paired with shock during extinction extinguished on both measures, but groups for which the OL S? was paired with shock omission did not extinguish, especially as shown by the reversal test procedure. Theoretical implications and the implications for several conditioning procedures are discussed.     

Rate of reinforcement and session duration as determinants of within-session patterns of responding

Rats pressed levers for Noyes pellets or keys for sweetened condensed milk reinforcers delivered by multiple schedules. Session length and baseline rates of reinforcement were varied in two experiments. Rates of responding increased during the early part of the session and then decreased for both responses and reinforcers, as well as for all subjects and values of the independent variables. Changes in response rates across the session sometimes exceeded 500%. Respoiise rates peaked approximately 20 min after the beginning of the session, regardless of session duration, when subjects responded on a multiple variable interval 1-min variable interval 1-min schedule. The function was flatter for longer sessions than it was for shorter sessions. The function was flatter, more symmetrical, and peaked later for lower rates of reinforcement than for higher rates of reinforcement. The function appeared early in training, and further experience moved and reduced its peak. Variables related to reinforcement exerted more control over some aspects of this function than did variables related to responding. These within-session patterns of responding may have fundamental implications for experimental design and theorizing.     

Behavioral momentum and relapse of extinguished operant responding

Previous experiments on behavioral momentum have shown that relative resistance to extinction of operant behavior in the presence of a stimulus depends on the rate of reinforcement associated with that stimulus, even if some of those reinforcers occur independently of the behavior. We present three experiments examining whether the rate of reinforcement in the presence of a stimulus similarly modulates the relative relapse of operant behavior produced by reinstatement, resurgence, and renewal paradigms. During baseline conditions, pigeons responded for food reinforcement on variable-interval 120-sec schedules in alternating periods of exposure to two stimuli arranged by a multiple schedule. Additional response-independent food presentations were also delivered in the presence of one of the multiple-schedule stimuli. Consistent with previous research, baseline response rates were lower in the presence of the stimulus with the added response-independent reinforcement, and relative resistance to extinction was greater in the presence of that stimulus. In addition, following extinction, the relative relapse of responding produced by reinstatement, resurgence, and renewal paradigms was greater in the presence of the stimulus associated with the higher rate of reinforcement. We suggest that a model of extinction from behavioral momentum theory may be useful for understanding these results.     

Preference for the interval schedule following multiple variable-ratio yoked-variable-interval schedule training

Pigeons were studied on multiple variable-ratio yoked-variable-interval schedules in which components had equal rates of food reinforcement and appeared equally often on each of two keys. Interpolated between component changes on the final multiple schedule were 10-sec probes in which both schedule stimuli were present, one on each key. During multiple schedule training, variable-ratio response rates were greater than yoked-variable-interval rates; however, response rate differences in the components were not a function of the mean ratio value for the 40-to-320-ratio range studied. During the choice probes, subjects responded more to the stimulus associated with the interval schedule than to the one associated with the ratio schedule. It was concluded that pigeons prefer interval schedules over equal reinforcement rate ratio schedules, because the former generate fewer responses per reinforcement.     

Response-cost punishment with pigeons: Further evidence of response suppression via token loss

Four pigeons responded on a two-component multiple token-reinforcement schedule, in which tokens were produced according to a random-interval 30-sec schedule and exchanged according to a variable-ratio 4 schedule in both components. To assess the effects of contingent token loss, tokens were removed after every second response (i.e., fixed-ratio 2 loss) in one of the components. Response rates were selectively lower in the loss components relative to baseline (no-loss) conditions, as well as to the within-condition no-loss components. Response rates were decreased to a greater degree in the presence of tokens than in their absence. To control for the effects of changes in the density of token and food reinforcement, two parts consisted of additional conditions where food density and token loss were yoked to those in a previous loss condition. In the yoked-food condition, tokens were produced as usual in both components, but the overall density of food reinforcement in one of the components was yoked to that obtained during a previous token-loss condition. In the yoked token-loss condition, tokens were removed during one component of the multiple schedule at a rate that approximately matched the obtained rate of loss from a previous token-loss condition. Response rates in these yoked components were less affected than those in comparable loss components, despite similar densities of token, exchange, and food reinforcement. On the whole, the results support the conclusion that contingent token loss serves as an effective punisher with pigeons.     

Antecedent reinforcement schedule training and operant response reinstatement in rats

Rats were trained on reinforcement schedules which generated high or low response rates. After extinguishing responding by eliminating food-reinforcement delivery, response-independent food presentations reinstated responding. Higher response rates occurred if the schedule preceding extinction controlled high response rates, suggesting that discriminative stimulus properties of the reinforcer were a function of antecedent training schedules.     

Resistance to extinction: contingency termination and generalization decrement

We present an algebraic model of resistance to extinction that is consistent with research on resistance to change. The model assumes that response strength is a power function of reinforcer rate and that extinction involves two additive, decremental processes: (1) the termination of the reinforcement contingency and (2) generalization decrement resulting from reinforcer omission. The model was supported by three experiments. In Experiment 1, 4 pigeons were trained on two-component multiple variable-interval (VI) 60-sec, VI 240-sec schedules. In two conditions, resistance to change was tested by terminating the response-reinforcer contingency and presenting response-independent reinforcers at the same rate as in training. In two further conditions, resistance to change was tested by prefeeding and by extinction. In Experiment 2, 6 pigeons were trained on two-component multiple VI 150-sec schedules with 8-sec or 2-sec reinforcers, and resistance to change was tested by terminating the response-reinforcer contingency in three conditions. In two of those conditions, brief delays were interposed between responses and response-independent reinforcers. In both Experiments 1 and 2, response rate was more resistant to change in the richer component, except for extinction in Experiment 1. In Experiment 3, 8 pigeons were trained on multiple VI 30-sec, VI 120-sec schedules. During extinction, half of the presentations of each component were accompanied by a novel stimulus to produce generalization decrement. The extinction data of Experiments 1 and 3 were well described by our model. The value of the exponent relating response strength and reinforcement was similar in all three experiments.