Ontogeny of persistence: Immediate extinction effects in preweanling and weanling rats

In Experiment I rats were trained for 21÷2 days under partial (PRF) or continuous reinforcement (CRF) conditions starting at 18, 22, 28, or 36 days of age and were then subjected to immediate extinction. At all ages there was a strong partial reinforcement extinction effect (PREE), and absolute size of PREE was greatest in the youngest rats. Rate of extinction increased as a function of age following both CRF and PRF. In Experiment II the youngest and oldest age groups of Experiment I were run under the two reward conditions of Experiment I and in a third condition, PRF with number of rewards rather than trials equated to CRF (PRF-R). The PRF-R and PRF groups were not different in extinction, and both were more persistent than CRF. The youngest rats were again more persistent than the oldest, particularly after PRF training. In Experiment III it was shown that the well-known paradoxical effect, greater reward in CRF acquisition leads to faster extinction, operates in our youngest and oldest animals, but is more pronounced in the oldest. The results are discussed in terms of whether they require different explanations than those often applied to extinction data from adult rats.     

Instrumental escape learning in the rat at 24-hour intertriai interval: Effects of magnitude and schedule of reinforcement

Three experiments investigated the effects of magnitude and schedule of reinforcement and level of training in instrumental escape learning at a 24-h intertriai interval. In Experiment I, two magnitudes of reinforcement were factorially combined with two schedules of reinforcement (CRF and PRF). Under PRF, large reward produced greater resistance to extinction than did small reward, while the reverse was true under CRF. In Experiment II, two levels of acquisition training were factorially combined with three schedules of reinforcement (CRF, single-alternation, and nonalternated PRF). Patterned running was observed late in acquisition in the single-alternation extended-training condition. Resistance to extinction was greater for the nonalternated PRF condition than for the single-alternation condition following extended acquisition, and the reverse was true following limited acquisition. Experiment III confirmed the extinction findings of Experiment II. The results of all three experiments supported an analysis of escape learning at spaced trials in terms of Capaldi’s (1967) sequential theory.     

Prolonged,unsignaled, inescapable shocks increase persistence in subsequent appetitive instrumental learning

In the first experiment, a prolonged period of intermittent, unsignaled shocks preceded appetitive runway acquisition, under either continuous (CRF) or partial reinforcement (PRF) and extinction. In the second experiment, the shock treatment came between CRF or PRF acquisition and extinction; and in the third experiment, the shocks intervened between appetitive CRF acquisition and shock-punishment extinction. The main finding was that compared with an unshocked control, shock facilitated acquisition in Experiment 1, and led to increased resistance to extinction and/or punishment in all experiments. In Experiment 1, the shock effect in appetitive extinction was seen mainly in the CRF group; in Experiment 2, the effect was to increase persistence in both the CRF and PRF groups; and in Experiment 3, shock treatment produced stronger resistance to punished extinction. The discussion is in terms of habituation and a general theory of persistence, and the concept of helplessness.     

A behavioral field approach to instrumental learning in the rat: I. Partial reinforcement effects and sex differences

The behavioral field approach employs naturalistic observation and simultaneous, multiple recordings of ecologically relevant aspects of behavior-environment interactions. Applying this approach to runway learning in the rat, the inferior acquisition speed of partial reinforcement (PRF) subjects as compared to continuous reinforcement (CRF) subjects was found to result from greater response variability, more sniffing and goal-avoidance behavior, and slower dropping out of collateral behaviors (e.g., drinking and sand-digging). Extinction first produced an increase in exploratory behavior, then displacement activities (e.g., grooming and biting) and goal-avoidance. CRF subjects showed greater response persistence as measured by number of extinction trials to disrupt an established, favored path. PRF subjects showed greater goal persistence as measured by trials to retrace from goalbox. In extinction, while CRF subjects were more inclined to engage in drinking and sand-digging in the startbox, PRF subjects exhibited more biting behavior in the goalbox. The only sex-related differences were superior speeds by female CRF subjects, inferior goal speeds by female PRF subjects during acquisition, and superior goalbox escape learning by females in extinction.     

Training reinforcement rates,resistance to extinction,and the role of context in reinstatement

Behavior reduced as a consequence of extinction or intervention can relapse. According to behavioral momentum theory, the extent to which behavior persists and relapses once it has been eliminated depends on the relative training reinforcement rate among discriminative stimuli. In addition, studies of context renewal reveal that relapse depends on the similarity between the training stimulus context and the test stimulus context following disruption by extinction. In the present experiments with pigeons, we arranged different reinforcement rates in the presence of distinct discriminative stimuli across components of a multiple schedule. Following extinction, we attempted to reinstate responding in the presence of those target components with response-independent food presentations. Importantly, we arranged the reinstating food presentations either within the target components or in separate components, either paired with extinction (Experiment 1) or reinforcement (Experiment 2) during baseline. Reinstatement increased with greater training reinforcement rates when the reinstating food presentations were arranged in the target components and the separate components paired with reinforcement during training. Reinstatement was smaller and was not systematically related to training reinforcement rates in the target components when reinstating food presentation occurred in separate components paired with extinction. These findings suggest that relapse depends on the history of reinforcement associated with the discriminative stimuli in which the relapse-inducing event occurs.     

Schedule-induced wood-chewing in rats and its dependence on body weight

In Experiment 1, 12 rats were exposed to an FT 60 schedule of food reinforcement, followed either by extinction or by a massed-food control condition, in the presence of a wood block. In 9 rats, wood-chewing behavior increased systematically during the FT 60 condition and declined again during extinction or massed food, while the other 3 rats showed virtually no chewing behavior at any stage of the experiment. In Experiment 2, frequency and bout duration of wood-chewing under an FT 60 schedule of food reinforcement declined as body weight increased, in 7 rats. We conclude that wood-chewing qualifies as a schedule-induced behavior, and that it resembles schedule-induced drinking in its dependence on body weight. Unlike drinking, however, induced chewing occupied the middle region of the 60-sec interreinforcement interval, declined markedly within the session, and showed considerable within- and between-subject variability.     

Resurgence of behavior during extinction depends on previous rate of response

In two experiments, food-deprived rat subjects leverpressed for food in three successive training phases. In the first phase of both experiments, rats were exposed to a multiple schedule, one component of which produced a high rate of response, and the other of which produced a lower rate of response (multiple random ratio [RR], random interval [RI] in Experiment 1, and multiple differential reinforcement of high rate, differential reinforcement of low rate in Experiment 2). Rats were then transferred to a multiple fixed interval (FI; 60-sec, 60-sec) schedule, until the effects of the first phase on response rate were no longer apparent and their response rates did not differ from those of rats responding on a multiple FI 60-sec, FI 60-sec schedule without previously experiencing a multiple RR, RI schedule. During the third stage oftraining, all rats were placed into extinction. During extinction, rates of responding were higher in the component previously associated with the high rate of responding in Phase 1, and they were lower in the component previously associated with low rates of responding in Phase 1. These results suggest that resurgence effects, like other history effects, are controlled by previous rates of responding.     

Response strength and temporal control in fixed-interval schedules

Pigeons responded in a two-component peak procedure in which the components differed in terms of reinforcement magnitude (Experiment 1), immediacy (Experiment 2), or probability (Experiment 3). The prediction of behavioral momentum theory that responding in the relatively richer component should be more resistant to change was tested by (1) presenting response-independent food in the intervals between components according to a variable-time (VT) schedule, (2) prefeeding, and (3) extinction. In all the experiments, peak location in baseline occurred earlier, relative to the schedule value in the richer component. Peak response rate was more resistant to change in the richer component during the VT and prefeeding tests, and change in peak rate was more sensitive to differential reinforcement than change in overall response rate. Changes in measures of performance on peak trials during the disruptor tests were partially consistent with predictions of the behavioral theory of timing. The results suggest that peak response rate provides a more sensitive index of resistance to change for fixed-interval schedules than does overall response rate and that reinforcement strengthens both peak responding and temporal control.     

Resistance to extinction as a function of percentage of reward: A reinforcement-level interpretation

Three experiments investigated the effects of percentage of reinforcement on the resistance to extinction of an instrumental running response. In Experiment 1, with N-length held constant, 47% reinforcement during acquisition generated greater resistance to extinction (R_n) than did 77%. In Experiment 2, this result was replicated with both functional N-length and number of N-R transitions held constant. In Experiment 3, R_n was shown to be a function of both N-length and percentage of reinforcement. The results of all three experiments were discussed in terms of Capaldi’s reinforcement level theory and possible alternative explanations.     

The effect of drug habituation before and after taste aversion learning in rats

In Experiment I, rats received eight habituation injections of either lithium chloride (LiCl) or sodium chloride (NaCl), then two aversion training trials in which access to saccharin solution was followed by LiCl injections, and finally eight extinction trials with saccharin but no injections. The rats habituated to LiCl showed less aversion to saccharin during training and extinction. In Experiment II, rats received two aversion training trials, then eight habituation trials to either LiCl or NaCl, then eight extinction trials, four more aversion training trials, and eight more extinction trials. The rats habituated to LiCl did not differ during the first extinction period from those habituated to NaCl, but showed less aversion to saccharin during the second training and extinction periods. Consequently, habituation to LiCl reduces the learning of an aversion to saccharin but does not reduce the performance of a previously learned aversion.     

Integration of reinforcement effects over time

Two accounts of how density of reinforcement affects steady-state performance on probabilistic schedules were compared: the real-time linear operator (RTLO) model and a temporal control model (in which response strength is determined by reinforcement probability as a function of postreinforcement time). In Experiment 1, the probability of reinforcement repeatedly cycled between extinction and a random-ratio 10 schedule. Response-rate gain and phase did not change with period of the cycle as predicted by the RTLO model, nor did either model predict the differences in response rate following reinforcement at different points in the cycle. In Experiment 2, the probability of reinforcement was elevated immediately following a reinforcement but fell after a few seconds. Previous reinforcements had no effect upon responding. An extension of the temporal control model, the cumulative impulse model, allowed for the summing of response strength over successive reinforcements and was consistent with the data of both experiments.     

Partial reinforcement effects on learning and extinction of place preferences in the water maze

In two experiments, two groups of rats were trained in a navigation task according to either a continuous or a partial schedule of reinforcement. In Experiment 1, animals that were given continuous reinforcement extinguished the spatial response of approaching the goal location more readily than animals given partial reinforcement—a partial reinforcement extinction effect. In Experiment 2, after partially or continuously reinforced training, animals were trained in a new task that made use of the same reinforcer according to a continuous reinforcement schedule. Animals initially given partial reinforcement performed better in the novel task than did rats initially given continuous reinforcement. These results replicate, in the spatial domain, well-known partial reinforcement phenomena typically observed in the context of Pavlovian and instrumental conditioning, suggesting that similar principles govern spatial and associative learning. The results reported support the notion that salience modulation processes play a key role in determining partial reinforcement effects.     

Spontaneous recovery varies inversely with the training-extinction interval

Four experiments found the magnitude of spontaneous recovery after extinction to be greater with a shorter interval between initial conditioning and extinction. Experiments 1 and 2 used a Pavlovian magazine approach procedure with rat subjects, Experiment 3 used an instrumental training procedure with rats, and Experiment 4 used a sign-tracking procedure with pigeons. These results are not anticipated by many accounts of spontaneous recovery that attribute it to the fading of learning that occurred during extinction.     

Within-session changes in responding during concurrent schedules that employ two different operanda

Five rats responded on several concurrent schedules in which pressing a key produced reinforcers in one component and pressing a lever produced reinforcers in the other component (Experiment 1). Four pigeons responded on several concurrent keypeck treadlepress schedules (Experiment 2). The programmed rates of reinforcement varied from 15 to 240 reinforcers per hour in different conditions. Rates of responding usually changed systematically within experimental sessions, and the changes were similar for the two components of a concurrent schedule. These results imply that within-session changes in responding may not confound the predictions of theories that describe the ratio of the rates of responding during the two components of concurrent schedules. Instead, within-session changes may be controlled by a mechanism that integrates the reinforcers obtained from the two components.     

Competing sources of stimulus value in anticipatory contrast

In previous studies of anticipatory contrast, identical target components (A and B) preceded either a lower (extinction) or a richer schedule. Higher response rates occurred during the target preceding the lower rate of reinforcement, whereas preference was in favor of the target preceding the richer schedule. In Experiment 1, the response and preference measures were positively related when additional stimuli, with no reinforcement of their own, preceded the target components. The effect of these additional stimuli was presumed to be due to their overshadowing of the Pavlovian association between the target components and their following schedules. Experiment 2 also demonstrated a consistent relation between response rate and preference in a conditioned reinforcement procedure. In the absence of a strong Pavlovian association, anticipatory contrast, like other forms of contrast in free-operant procedures, reflects an increase in the value of the target component with an unchanged reinforcement schedule.     

The effect of rate of reinforcement and time in session on preference for variability

Pigeons pecked keys on concurrent-chains schedules that provided a variable interval 30-sec schedule in the initial link. One terminal link provided reinforcers in a fixed manner; the other provided reinforcers in a variable manner with the same arithmetic mean as the fixed alternative. In Experiment 1, the terminal links provided fixed and variable interval schedules. In Experiment 2, the terminal links provided reinforcers after a fixed or a variable delay following the response that produced them. In Experiment 3, the terminal links provided reinforcers that were fixed or variable in size. Rate of reinforcement was varied by changing the scheduled interreinforcer interval in the terminal link from 5 to 225 sec. The subjects usually preferred the variable option in Experiments 1 and 2 but differed in preference in Experiment 3. The preference for variability was usually stronger for lower (longer terminal links) than for higher (shorter terminal links) rates of reinforcement. Preference did not change systematically with time in the session. Some aspects of these results are inconsistent with explanations for the preference for variability in terms of scaling factors, scalar expectancy theory, risk-sensitive models of optimal foraging theory, and habituation to the reinforcer. Initial-link response rates also changed within sessions when the schedules provided high, but not low, rates of reinforcement. Within-session changes in responding were similar for the two initial links. These similarities imply that habituation to the reinforcer is represented differently in theories of choice than are other variables related to reinforcement.     

Effects of stimulus change in prepunishment alley segments on self-punitive behavior

Three experiments investigated the effects of stimulus change during extinction on self-punitive behavior. In Experiment 1, changing alley brightness cues in all three segments of the alley prior to extinction eliminated self-punitive behavior. That is, subjects given shock in the third alley segment during extinction did not differ from nonshocked subjects in alley speed or in the number of trials to extinction. In Experiment 2, with shock also administered in the third alley segment, self-punitive behavior was eliminated when the stimulus change was made in segment 1 or in segments 1 and 2 but was obtained when the change occurred in segment 2 or in the lower startbox. In Experiment 3, shock was administered in the second alley segment. Self-punitive behavior was not obtained when the lower startbox cues were changed but was obtained with stimulus change in the upper startbox or in segment 1. The results are consistent with an expanded version of the Mowrer-Brown conditioned-fear hypothesis.     

Partial reinforcement delayed extinction effect after regularly alternating reward training

When extinction is delayed very long, the superior resistance to extinction of the random schedule group relative to the alternating schedule group disappears (partial reinforcement delayed extinction effect, PRDE). Two experiments assessed the effects of reinforcement/nonreinforcement on Trial 1 on the PRDE. Following extended partial reinforcement acquisition training in a runway, rats received extinction training after a short (1-day) or long (23-day) retention interval. The schedules used in Experiment 1 were: a single-alternation (SA) schedule beginning each day with a rewarded (r) trial, for Group r-SA; an SA schedule beginning with a nonrewarded (n) trial, for Group n-SA; and a random (Rd) schedule, for Group Rd. The schedules and group names used in Experiment 2 were r-SA, Rd, and r-Rd. The results were that (1) rats given r-SA schedules yielded considerable resistance under delayed extinction, (2) those given Rd and r-Rd schedules showed a decline in resistance to extinction over a long retention interval, (3) those given the n-SA schedule showed relatively low resistance at both retention intervals, although retention deficit was not greater than in the case of the Rd schedule, and thus, (4) the PRDE was found in both experiments, although only weakly in Experiment 1. The results indicated that a regularly alternating reward pattern was a more important determinant than was type of reward on Trial 1 for the PRDE. The PRDE due to differential retention deficits among schedules is discussed on the basis of dual-process associative sequential mechanisms and cognitive rule-encoding mechanisms.     

Behavioral contrast and reinforcement value

Behavioral contrast was produced in two target components of a four-component multiple schedule by having two target stimuli followed either by a higher rate of reinforcement or by extinction. Response rate was higher in the target followed by extinction. Periodic probe trials were then presented in which the two target stimuli were presented together. Choice on these probe trials was in favor of the stimulus followed by the higher rate of reinforcement during regular training. Experiment 2 replicated this finding but with probe trials presented throughout training. Here, preference for the stimulus followed by the higher rate of reinforcement was evident early in training, substantially before the contrast effects developed. The results challenge interpretations of contrast based on the concept of relative value.