Ontogeny of persistence: Immediate extinction effects in preweanling and weanling rats

In Experiment I rats were trained for 21÷2 days under partial (PRF) or continuous reinforcement (CRF) conditions starting at 18, 22, 28, or 36 days of age and were then subjected to immediate extinction. At all ages there was a strong partial reinforcement extinction effect (PREE), and absolute size of PREE was greatest in the youngest rats. Rate of extinction increased as a function of age following both CRF and PRF. In Experiment II the youngest and oldest age groups of Experiment I were run under the two reward conditions of Experiment I and in a third condition, PRF with number of rewards rather than trials equated to CRF (PRF-R). The PRF-R and PRF groups were not different in extinction, and both were more persistent than CRF. The youngest rats were again more persistent than the oldest, particularly after PRF training. In Experiment III it was shown that the well-known paradoxical effect, greater reward in CRF acquisition leads to faster extinction, operates in our youngest and oldest animals, but is more pronounced in the oldest. The results are discussed in terms of whether they require different explanations than those often applied to extinction data from adult rats.     

Instrumental escape learning in the rat at 24-hour intertriai interval: Effects of magnitude and schedule of reinforcement

Three experiments investigated the effects of magnitude and schedule of reinforcement and level of training in instrumental escape learning at a 24-h intertriai interval. In Experiment I, two magnitudes of reinforcement were factorially combined with two schedules of reinforcement (CRF and PRF). Under PRF, large reward produced greater resistance to extinction than did small reward, while the reverse was true under CRF. In Experiment II, two levels of acquisition training were factorially combined with three schedules of reinforcement (CRF, single-alternation, and nonalternated PRF). Patterned running was observed late in acquisition in the single-alternation extended-training condition. Resistance to extinction was greater for the nonalternated PRF condition than for the single-alternation condition following extended acquisition, and the reverse was true following limited acquisition. Experiment III confirmed the extinction findings of Experiment II. The results of all three experiments supported an analysis of escape learning at spaced trials in terms of Capaldi’s (1967) sequential theory.     

A behavioral field approach to instrumental learning in the rat: I. Partial reinforcement effects and sex differences

The behavioral field approach employs naturalistic observation and simultaneous, multiple recordings of ecologically relevant aspects of behavior-environment interactions. Applying this approach to runway learning in the rat, the inferior acquisition speed of partial reinforcement (PRF) subjects as compared to continuous reinforcement (CRF) subjects was found to result from greater response variability, more sniffing and goal-avoidance behavior, and slower dropping out of collateral behaviors (e.g., drinking and sand-digging). Extinction first produced an increase in exploratory behavior, then displacement activities (e.g., grooming and biting) and goal-avoidance. CRF subjects showed greater response persistence as measured by number of extinction trials to disrupt an established, favored path. PRF subjects showed greater goal persistence as measured by trials to retrace from goalbox. In extinction, while CRF subjects were more inclined to engage in drinking and sand-digging in the startbox, PRF subjects exhibited more biting behavior in the goalbox. The only sex-related differences were superior speeds by female CRF subjects, inferior goal speeds by female PRF subjects during acquisition, and superior goalbox escape learning by females in extinction.     

Transfer of escape conditioning to extinction of a food-reinforced response

Four experiments test the hypothesis that escape learning in response to shock will transfer to a similar food-reinforced response and affect resistance to appetitive extinction. In the first two experiments, subjects were given escape training in a straight alley followed by continuous food reinforcement and then extinction. Prior escape training resulted in greater resistance to extinction of the food-reinforced response as compared to several control procedures. In the third experiment, the escape response was manipulated to be compatible or incompatible with the subsequent food-reinforced response. Greater resistance to extinction was shown when the two responses were compatible. The fourth experiment confirmed and extended this finding. The relationship of the present results to Amsel’s theory of persistence was discussed.     

Inescapable shock interferes with the acquisition of an appetitive operant

This article reports the reinforcer generality of the interference effect resulting from exposure to inescapable shock. In Experiment 1, rats that received inescapable shock showed weak interference with the acquisition of an appetitive operant compared to animals exposed either to escapable or no shock. In Experiment 2, the response-reinforcer contingency was degraded by introducing a 1-sec delay of reinforcement on the appetitive task. Inescapable shock produced much stronger interference with the acquisition of the operant response than in Experiment 1. The results demonstrate reinforcer generality of the debilitating effects produced by inescapable shock.     

A behavior field approach to instrumental learning in the rat: II. Training parameters and a stage model of extinction

Three runway experiments tested a stage model of extinction which postulated an orderly succession of three qualitatively different stages: habit, trial and error, and resolution. The model predicted that Stage 1 should be characterized by perseveration of habitual routes (i.e., response persistence) and the absence of competing responses; Stage 2, by an increase in investigatory behavior (response variation and hole exploration) and biting behavior; Stage 3, by a decrease in the competing responses of Stage 2 and continued increase in goal avoidance and substitution behavior (e.g., sand-digging). These predictions were largely confirmed. Further, Experiments 1 and 2 showed that, as expected by the model, continuous reinforcement (CRF) resulted in more practice of habitual routes in acquisition and greater response persistence, while partial reinforcement (PRF) resulted in more route variation and hole exploration in acquisition and greater goal persistence which was attributable to prior reinforcement of a trial-and-error coping strategy. Results of Experiment 3, which combined training trials and reward magnitudes orthogonally, supported the prediction that response persistence was positively related to training trials, and goal persistence negatively related to reward magnitudes. All three experiments demonstrated an inverted-U function in investigatory and biting behavior, as predicted by the stage model.     

Stepwise reductions in US density and intensity: Maintenance and extinction of conditioned suppression

Information was sought on how the strength of fear, acquired and extinguished in a classical conditioning paradigm, might be affected by certain of the circumstances that normally are associated with the course of avoidance learning. Following preconditioning exposures to the conditioned stimulus, one group of rats (Group PRF) was given continuous conditioned stimulus-unconditioned stimulus (CS-US) acquisition trials followed by stepwise reductions in US probability (given the CS) over three phases to about 11% for the final phase. Another group, Group US(lo), was given continuous CS-US pairings throughout, but, following acquisition, received stepwise reductions in US intensity (to permit evaluation of a progressively changing feature of the US over a wide range without having to employ excessive shock) over the same phases. A third group, Group US(hi), received unchanging (from acquisition) CS-US pairings over these phases, and an explicitly unpaired control group (Group RU) was included. Although outcomes differed somewhat depending upon whether suppression ratios or absolute measures of responding were considered, the major findings were that suppression to the CS was complete by the end of acquisition and persisted thereafter throughout the three phases for all but control subjects. In contrast to Group US(lo), which showed relatively little resistance to extinction of suppression to subsequent CS-only exposures, Groups US(hi) and PRF displayed marked resistance over two separate, extended sets of extinction sessions. Suppression to context cues was pronounced only for Groups US(hi) and RU, and varied as a function of US parameters and not whether the shocks were or were not signaled. Theoretical reconciliation of these findings was most difficult for Groun PRF.     

Intensification of punishment effects through exposure to prolonged, fixed-duration shocks: The role of shock cues as a stimulus for fear

Three experiments tested the hypotheses that (1) the onsets of prolonged, fixed-duration treatment shocks (shock treatment, or ST) serve as cues for fear conditioning to the ongoing painful effects of these stimuli, and (2) this acquired fear transfers to and influences shock-motivated test performances in predictable ways. Experiments 1 and 2 involved spaced post-ST presentations of very brief shocks (a presumed analogue of the onsets of treatment shocks) as a means of extinguishing the fear putatively associated with shock cues. This procedure reduced defecation by ST subjects over blocks of extinction sessions and nullified the punishment intensification effect that was otherwise shown to be a consequence of ST. As a further test of the shock-cue hypothesis, Experiment 3 involved relatively massed presentations of these brief shocks prior to ST in a latent-inhibition procedure (Lubow, 1973). This briefshock regimen also nullified the ST punishment intensification effect but did not impair the transituational transfer of contextual fear. In contrast, the group that was given the same brief shock regimen following ST showed enhancement of the punishment effect. With respect to current theoretical accounts of ST effects, these data were most consistent with notions that rely on an acquired-fear construct.     

The effect of intensity of intermittent punishment in acquisition on resistance to extinction of an approach response

In a transfer of persistence paradigm, intensity of shock employed as intermittent punishment during the acquisition of a food-motivated instrumental response was manipulated. While intensities that increased response time during acquisition did increase resistance to extinction, an intensity that had no punishing effect did not increase persistence. The results indicated that there is both a lower and an upper limit to the transfer of an approach response from s_p to S_f and that these limits should be determined by behavioral rather than intensity measures.     

Context effects on conditioning,extinction, and reinstatement in an appetitive conditioning preparation

Three experiments with rat subjects examined the effects of contextual stimuli on performance in appetitive conditioning. A 10-sec tone conditioned stimulus (CS) was paired with a food-pellet unconditioned stimulus (US); conditioning was indexed by the observation of headjerking, a response of the rat to auditory stimuli associated with food. In Experiment 1, a context switch following initial conditioning did not affect conditioned responding to the tone; however, when the response was extinguished in the different context, a return to the original conditioning context “renewed” extinguished responding. These results were replicated in Experiments 2 and 3 after equating exposure to the two contexts (Experiment 2) and massing the conditioning and extinction trials (Experiment 3). The results of Experiment 1 also demonstrated that separate exposure to the US following extinction reinstates extinguished responding to the tone; this effect was further shown to depend at least partly on presenting the US in the context in which testing is to occur (Experiments 2 and 3). Overall, the results are consistent with previous data from aversive conditioning procedures. In either appetitive or aversive conditioning, the context may be especially important in affecting performance after extinction.     

Signaling a change in cue-outcome relations in human associative learning

In three experiments, we assessed the role of signals for changes in the consequences of cues as a potential account of the renewal effect. Experiment 1 showed recovery of responding following extinction when acquisition, extinction, and test phases occurred in different contexts. In addition, extinction treatment in multiple contexts attenuated context-induced response recovery. In Experiment 2, we used presentations of an extraneous stimulus (ES), instead of context shifts, and found that responding recovered from extinction only when the ES was presented both between acquisition and extinction and between extinction and test. In Experiment 3, we used a reversal learning design in which, during training, two cues were first paired with different outcomes, then paired with the alternative outcomes, and finally paired again with the original outcomes. In this experiment, presentation, just prior to testing, of an ES that had previously been presented between the different phases produced an expectation of reversal in the meaning of the cues.     

Discrimination training in infant,preweanling, and weanling rats: Effects of prior learning experiences with the discriminanda

Rats were given continuous reinforcement (CRF), partial reinforcement (PRF), or successive discrimination (D) training in an alley from 11–14 (Age 1) or 15–18 (Age 2) days of age. Reinforcement was the opportunity to suckle the dry nipples of an anesthetized dam. Following a 10-day interval, all animals were given 4 successive days of discrimination training with food pellets as reinforcement. Control groups were given only the second phase of training. In the first phase, D subjects of both ages responded appropriately to the discriminative stimuli, and the PRF subjects of both ages ran significantly slower than CRF subjects. In the second phase, only the CRF subjects of Age 1 showed behavioral discrimination. All three Age 2 groups discriminated, but the discrimination developed earliest after Phase I CRF and latest after Phase I PRF. Both Age 1 control groups showed a late-developing discrimination, but neither Age 2 control group discriminated. The results suggest that infant and preweanling rats can learn both positive and negative expectancies of appetitive events, respond appropriately, and retain and transfer these expectancies to new learning. The reinforcement value of dry suckling and the effects of stimulus preexposure in infant rats are also discussed.     

Blocking and enhancement of fear conditioning by appetitive CSs

Prior research on Pavlovian-to-instrumental transfer has shown that when a CS previously associated with shock (AvCS+) is presented contingent upon a choice response to a discriminative stimulus for food reinforcement, it facilitates discrimination learning. Conversely, a response-contingent CS previously associated with the absence of shock (AvCS?) retards discrimination learning. To evaluate whether these findings reflect across-reinforcement blocking and enhancement effects, two experiments investigated the effects of appetitively conditioned stimuli on fear conditioning to a novel stimulus that was serially compounded with the appetitive CS during conditioned-emotional-response (CER) training. Although there were no differential effects of the appetitive CSs in CER acquisition, Experiment 1, using a relatively weak shock US, showed that a CS previously associated with food (ApCS+) retarded CER extinction to the novel stimulus, in evidence of enhanced fear conditioning to that stimulus. In addition, Experiment 2, using a stronger shock US, showed that a CS previously associated with the absence of food (ApCS?) facilitated CER extinction to the novel stimulus, in evidence of weaker fear conditioning to that stimulus. These results parallel traditional blocking effects and indicate not only that an ApCS+ and an ApCS? are functionally similar to AvCSs of opposite sign, but that their functional similarity is mediated by common central emotional states.     

Resistance to discrimination and subsequent resistance to extinction as a function of the sequence of partial S+ reward in differential conditioning

The effects of transitions from nonrewarded (N) to rewarded (R) trials (N-R transitions) on discriminative behavior in differential conditioning and subsequent resistance to extinction were investigated in two experiments. In Experiment 1, groups given N-R transitions within S+ were more resistant to discrimination (ran fast in S?) and extinction than were groups given a partial reinforcement (PRF) schedule in S+ devoid of N-R transitions. Experiment 2 indicated that N-R transitions that occur when an N trial in S? is followed by an R trial in S+ are as effective in increasing resistance to discrimination, but not resistance to extinction, as are N-R transitions that occur within S+. The sequential effects obtained here were highly similar to those in conventional PRF and support the view that differential conditioning and PRF are highly interrelated phenomena. The results are discussed in terms of the extension of sequential theory to differential conditioning and the importance of internal reward-produced cues in discrimination learning.     

Partial reinforcement effect: The expectancy of reward on nonreward trials

In order to determine the importance of the development of expectancy of reward prior to partial reward trials; rats were given 20 continuously reinforced trials prior to 20 partially reinforced trials (CRF-PRF) and compared to Ss given only 20 partially reinforced trials (PRF). Control groups received 20 or 40 continuously reinforced trials (CRF-20, CRF-40) to determine the effect of differing numbers of acquisition trials. Results showed that terminal acquisition differences were minimal in the run segment of the alley and that Group CRF-PRF was more resistant to extinction than Group PRF, and both were more resistant to extinction than the CRF-20 and CRF-40 groups, which did not differ from each other. These results were interpreted as supporting the notion that the expectancy of reward on nonreward trials during partial reinforcement acquisition is a determiner of the magnitude of the partial reinforcement extinction effect.     

A comparison of resistance to satiation and resistance to extinction

In two experiments, resistance to satiation was compared with resistance to extinction. In Experiment 1, rats given initial trials in a straight-alley runway while satiated failed to show increased resistance to satiation in a later test phase. This negative finding contrasts with the increased resistance to extinction usually found following initial nonrewarded trials in a straight alley. In Experiment 2, rats were extinguished or were run while satiated following deprived acquisition, and then were either shifted to the other condition or maintained under the same condition. A greater response decrement was produced by extinction than by satiation, both when current performance was examined and when the persistent effect of satiation or extinction on later performance was examined. These results show that there are important dissimilarities in the effects of satiation and extinction, dissimilarities that suggest that extinction is more nonrewarding or aversive than satiation. It seems likely that extinction involves processes (such as frustration, arousal of aversive motivation, and conditioned inhibition) not involved in satiation, which account for the greater response decrement in extinction as compared with satiation.     

Reacquisition following extinction in appetitive conditioning

In four experiments utilizing an appetitive conditioning preparation, reacquisition of conditioned responding was found to occur both rapidly and slowly following extinction. In Experiment 1, acquisition of responding to a tone that had been conditioned and extinguished occurred more rapidly than acquisition in either a group that received equivalent exposure to the food unconditioned stimulus or a “rest” control group that received only exposure to the apparatus in the first two phases. However, reacquisition was impaired relative to acquisition in a “learning-experienced” group that had previously received conditioning and extinction with a different stimulus. Experiments 2 and 3 produced similar results, but also found that high responding during reacquisition was confined to trials that followed reinforced, rather than nonreinforced, trials. Experiment 4, in which very few initial conditioning trials were used, produced reacquisition that was slow compared with both learning-experienced and rest controls. The results are consistent with a role for sequential learning: Reacquisition is rapid when animals have learned that reinforced trials signal other reinforced trials.     

The effects of qualitative and quantitative variation in the US on individual components of Pavlovian appetitive conditioned behavior in rats

The form of rats’ Pavlovian conditioned responses to visual and auditory conditioned stimuli (CSs) paired with a variety of unconditioned stimuli (USs) was examined in three experiments using direct behavioral observation techniques. In Experiment 1, the form of conditioned behavior occurring most frequently during later portions of the CS-US interval depended only on which of several appetitive USs was used, but the form of behavior occurring most frequently during early portions of the CS-US interval depended only on the nature of the CS. US-dependent behaviors resembled the response to the US, and CS-dependent behaviors resembled the original orienting response (OR) to the CS. In Experiment 2, the use of larger magnitude appetitive USs resulted in higher frequencies of US-dependent behaviors, but lower frequencies of CS-dependent behaviors in the presence of auditory and visual CSs. In Experiment 3, US-dependent conditioned behavior to auditory and visual CSs paired with shock was more frequent when high-intensity shocks were used, but CS-dependent behavior was more frequent when low-intensity shocks were used. These results suggested that Pavlovian conditioned responding may involve two independent types of behavior—one appropriate to the US and another based on the original OR to the CS.     

Transfer of persistence from fixed-ratio barpress training to runway extinction

Two experiments were performed to investigate transfer of persistence across different situations and response topographies. Experiment I demonstrated that FR 100 barpress training increased resistance to runway extinction as compared with a control. In Experiment II, resistance to extinction in the runway was systematically and positively related to terminal ratio requirements of prior barpress training.     

Failure to block control by a relevant stimulus

Pigeons were trained to depress a treadle in the presence of a discriminative stimulus, either a tone or illumination of red houselights, in order to obtain access to grain or avoid electric shock. In avoidance training, the auditory discriminative stimulus yielded faster acquisition than did the visual one. In appetitive training, the visual discriminative stimulus yielded faster acquisition than the auditory one. Experiments 2 and 3 used these stimuli in Kamin’s (1969) blocking design. In Experiment 2, when the pigeons were trained to depress a treadle in the presence of tone to obtain grain and then red light was added as the redundant stimulus, the light acquired stimulus control over treadlepressing; blocking was not observed. In Experiment 3, when the pigeons were trained to depress a treadle in the presence of red light to avoid electric shock and then tone was added as the redundant stimulus, the tone acquired stimulus control over treadle-pressing. Again, blocking was not observed. The implications of these results for several models of stimulus control are discussed.