Partial reinforcement delayed extinction effect after regularly alternating reward training

When extinction is delayed very long, the superior resistance to extinction of the random schedule group relative to the alternating schedule group disappears (partial reinforcement delayed extinction effect, PRDE). Two experiments assessed the effects of reinforcement/nonreinforcement on Trial 1 on the PRDE. Following extended partial reinforcement acquisition training in a runway, rats received extinction training after a short (1-day) or long (23-day) retention interval. The schedules used in Experiment 1 were: a single-alternation (SA) schedule beginning each day with a rewarded (r) trial, for Group r-SA; an SA schedule beginning with a nonrewarded (n) trial, for Group n-SA; and a random (Rd) schedule, for Group Rd. The schedules and group names used in Experiment 2 were r-SA, Rd, and r-Rd. The results were that (1) rats given r-SA schedules yielded considerable resistance under delayed extinction, (2) those given Rd and r-Rd schedules showed a decline in resistance to extinction over a long retention interval, (3) those given the n-SA schedule showed relatively low resistance at both retention intervals, although retention deficit was not greater than in the case of the Rd schedule, and thus, (4) the PRDE was found in both experiments, although only weakly in Experiment 1. The results indicated that a regularly alternating reward pattern was a more important determinant than was type of reward on Trial 1 for the PRDE. The PRDE due to differential retention deficits among schedules is discussed on the basis of dual-process associative sequential mechanisms and cognitive rule-encoding mechanisms.     

Training reinforcement rates,resistance to extinction,and the role of context in reinstatement

Behavior reduced as a consequence of extinction or intervention can relapse. According to behavioral momentum theory, the extent to which behavior persists and relapses once it has been eliminated depends on the relative training reinforcement rate among discriminative stimuli. In addition, studies of context renewal reveal that relapse depends on the similarity between the training stimulus context and the test stimulus context following disruption by extinction. In the present experiments with pigeons, we arranged different reinforcement rates in the presence of distinct discriminative stimuli across components of a multiple schedule. Following extinction, we attempted to reinstate responding in the presence of those target components with response-independent food presentations. Importantly, we arranged the reinstating food presentations either within the target components or in separate components, either paired with extinction (Experiment 1) or reinforcement (Experiment 2) during baseline. Reinstatement increased with greater training reinforcement rates when the reinstating food presentations were arranged in the target components and the separate components paired with reinforcement during training. Reinstatement was smaller and was not systematically related to training reinforcement rates in the target components when reinstating food presentation occurred in separate components paired with extinction. These findings suggest that relapse depends on the history of reinforcement associated with the discriminative stimuli in which the relapse-inducing event occurs.     

Partial reinforcement effects on learning and extinction of place preferences in the water maze

In two experiments, two groups of rats were trained in a navigation task according to either a continuous or a partial schedule of reinforcement. In Experiment 1, animals that were given continuous reinforcement extinguished the spatial response of approaching the goal location more readily than animals given partial reinforcement—a partial reinforcement extinction effect. In Experiment 2, after partially or continuously reinforced training, animals were trained in a new task that made use of the same reinforcer according to a continuous reinforcement schedule. Animals initially given partial reinforcement performed better in the novel task than did rats initially given continuous reinforcement. These results replicate, in the spatial domain, well-known partial reinforcement phenomena typically observed in the context of Pavlovian and instrumental conditioning, suggesting that similar principles govern spatial and associative learning. The results reported support the notion that salience modulation processes play a key role in determining partial reinforcement effects.     

A comparison of resistance to satiation and resistance to extinction

In two experiments, resistance to satiation was compared with resistance to extinction. In Experiment 1, rats given initial trials in a straight-alley runway while satiated failed to show increased resistance to satiation in a later test phase. This negative finding contrasts with the increased resistance to extinction usually found following initial nonrewarded trials in a straight alley. In Experiment 2, rats were extinguished or were run while satiated following deprived acquisition, and then were either shifted to the other condition or maintained under the same condition. A greater response decrement was produced by extinction than by satiation, both when current performance was examined and when the persistent effect of satiation or extinction on later performance was examined. These results show that there are important dissimilarities in the effects of satiation and extinction, dissimilarities that suggest that extinction is more nonrewarding or aversive than satiation. It seems likely that extinction involves processes (such as frustration, arousal of aversive motivation, and conditioned inhibition) not involved in satiation, which account for the greater response decrement in extinction as compared with satiation.     

Replacement of event-generated memories of nonreinforcement with signal-generated memories of reinforcement during partial reinforcement training: Effect on resistance to extinction

Event-generated memory refers to the memory of a reinforcement (R) or nonreinforcement (N) event from an immediately preceding trial;signal-generated memory refers to the memory of a temporally remote R or N, retrieval of which is generated by presentation of a signal with which the memory is associated (Haggbloom, 1988). In each of three experiments, Group Signal-R received runway discrimination training in Phase 1 to establish a stimulus as a signal for R, and partial reinforcement training in Phase 2. An extinction test measured learning about the memory of nonreward (S^N)—learning that occurs when S^N is retrieved on R trials that follow N trials. In Group Signal-H, those R trials were accompanied by the signal for R, a treatment we hypothesized would generate retrieval of the memory of reinforcement (S^R) so that signal-generated S^R would replace event-generated S^N as the operative memory, thereby eliminating the increased resistance to extinction normally produced by PRF training. In each experiment, Group Signal-R was less resistant to extinction than was a control group conditioned to respond to-event-generated S^N. Extinction was as rapid in Group Signal-R as it was in a consistent reinforcement control group (Experiment 1) and in a group given intertrial reinforcements to interfere with learning about S^N (Experiment 3). Experiment 2 tested two alternative interpretations of the failure to learn about S^N in Group Signal-R. Those alternatives were found to be less viable than the hypothesis that the signal for R actively recruited retrieval of a competing memory.     

Observational extinction: Observation of nonreinforced responding reduces resistance to extinction in rats

Rats trained to push a joystick to the left or right for food reward were given two successive tests in which neither response was reinforced. Prior to Test 1, subjects were either confined in the apparatus with a passive conspecific (Group None), or allowed to observe a conspecific demonstrator making 50 nonreinforced responses in the direction that had beeirrewarded during observer training (Group Same) or in the opposite direction (Group Different). In Test 1, Group Same made fewer previously reinforced responses than did Group Different, which made fewer than Group None, and Groups Same and Different each made fewer previously nonreinforced responses than did Group None. In Test 2, Group Same made fewer previously reinforced responses than did Group None. These results indicate that observation of nonreinforced responding can reduce resistance to extinction (Test 1) and spontaneous recovery (Test 2) in rats.     

Classical conditioning of the rabbit nictitating membrane response: Effects of reinforcement schedule on response maintenance and resistance to extinction

Two experiments were conducted to assess the effects of the introduction of schedules of partial reinforcement (PRF), subsequent to continuous reinforcement training, on the maintenance and resistance to extinction of the rabbit’s nictitating membrane CR. Substantial response levels were maintained by schedules of reinforcement as lean as 15%, and the performance decrements, when observed to be reliable, could not be localized to the immediate effects of one, two, or three consecutive nonreinforced trials by the examination of conditional response probabilities. Moreover, reliable PRF extinction effects were obtained. The relevance of these findings to the purported empirical divergence of PRF effects on classical and instrumental conditioning were discussed.     

Partial reinforcement in autoshaping with pigeons

The acquisition, maintenance, and extinction of autoshaped responding in pigeons were studied under partial and continuous reinforcement. Five values of probability of reinforcement, ranging from .1 to 1.0, were combined factorially with five values of intertrial interval ranging from 15 to 250 sec for different groups. The number of trials required before autoshaped responding emerged varied inversely with the duration of the intertriai interval and probability of reinforcment, but partial reinforcement did not increase the number of reinforcers before acquisition. During maintained training, partial reinforcement increased the overall rate of responding. A temporal gradient of accelerated responding over the trial duration emerged during maintenance training for partial reinforcement groups, and was evident for all groups in extinction. Partial reinforcement groups responded more than continuous reinforcement groups over an equivalent number of trials in extinction. However, this partial-reinforcment extinction effect disappeared when examined in terms of the omission of “expected” reinforcers.     

Differential reinforcement and resistance to change of divided-attention performance

Behavioral momentum theory provides a framework for understanding how conditions of reinforcement influence instrumental response strength under conditions of disruption (i.e., resistance to change). The present experiment examined resistance to change of divided-attention performance when different overall probabilities of reinforcement were arranged across two components of a multiple schedule. Pigeons responded in a delayed-matching-to-sample procedure with compound samples (color + line orientation) and element comparisons (two colors or two line orientations). Reinforcement ratios of 1:9, 1:1, and 9:1 for accurate matches on the two types of comparison trials were examined across conditions using reinforcement probabilities (color/lines) of .9/.1, .5/.5, and .1/.9 in the rich component and .18/.02, .1/.1, and .02/.18 in the lean component. Relative accuracy with color and line comparisons was an orderly function of relative reinforcement, but this relation did not depend on the overall rate of reinforcement between components. The resistance to change of divided-attention performance was greater for both trial types in the rich component with presession feeding and extinction, but not with decreases in sample duration. These findings suggest promise for the applicability of quantitative models of operant behavior to divided-attention performance, but they highlight the need to further explore conditions impacting the resistance to change of attending.     

Partial reinforcement effects on discrimination learning

Rats were trained on a series of reversals of a successive discrimination in which the percentage of S+ trials ending in food was varied. Changes in the discrimination index occurred more slowly with 50% reinforcement than with 100% reinforcement when the number of training trials was equated across conditions, but were approximately invariant when the conditions were equated with respect to the number of obtained reinforcements. Presentation of free reinforcement during the intertrial intervals reduced the overall rate of discrimination acquisition, but left this invariance unaffected. Invariance in reinforcements necessary to attain acquisition also occurred when different discriminations correlated with different percentages of reinforcement were intermixed within experimental sessions. The failure of the invariance effect to be disrupted by either manipulation suggests that previous accounts of the invariance effect in terms of “comparator” models of conditioning (e.g., Gibbon & Balsam, 1981) are inadequate.     

Durability of the partial reinforcement and partial delay of reinforcement extinction effects after minimal acquisition training

Four groups of 10 rats each were given six acquisition trials (Phase 1) under continuous reinforcement (CR), partial reinforcement (PR), constant delay (CD), or partial delay of reinforcement (PD) conditions. In Phase 2, all Ss were given 18 nonreinforced trials, followed by 12 continuously reinforced trials in Phase 3. In Phase 4, all Ss were given 12 more extinction trials. A constant 24-h ITI was observed throughout the experiment. A strong partial reinforcement extinction effect (PREE) was obtained in both Phases 2 and 4. Only a temporary partial delay of reinforcement effect (PDRE) was observed, which was restricted to the first nine trials of the first extinction phase. No constant delay of reinforcement effect (CDRE) was observed in either extinction phase. The results were discussed in terms of both frustration and sequential theories.     

Dissociation of patterned alternation learning and the partial reinforcement extinction effect in preweanling rats

Sprague-Dawley rat pups aged 14 or 18 days were trained on a patterned (single) alternation schedule with either an 8- or a 105-sec intertriai interval (ITI). At the 8-sec ITI, alternation learning was obtained at both ages, but the older age group learned more rapidly. There was no evidence of response alternation at the 105-sec ITI at either age. Continuously reinforced (CRF) and partially reinforced (PRF) groups trained and extinguished along with the patterned alternation (PA) group at the 105-sec ITI showed a robust partial reinforcement extinction effect (PREE) at both ages. Moreover, there was no difference in the rate of extinction of the PRF and PA groups at either age (i.e., no effect of N-length). A PREE can therefore be obtained in infant rats under conditions that apparently preclude the formation of sequential associations. The implications of this finding for the ontogeny of instrumental learning and extinction are discussed.     

Reductions in resistance to extinction and spontaneous recovery as a function of changes in transportational and contextual stimuli

Thirty rats received 10 sessions of baseline training in which leverpressing was reinforced according to a variable-interval (VI) 60-sec schedule. Twenty-four of the subjects were then assigned to one of four groups that received five sessions of extinction, with groups being differentiated in a 2 by 2 factorial design on the basis of: (1) changes in stimuli accompanying transportation of subjects from home cages to the laboratory and placement in the apparatus, and/or (2) changes in contextual stimuli within the apparatus. During the sixth session of extinction, the transportational and contextual stimuli previously associated with baseline training were reinstated. The remaining six rats experienced changes in both transportational and contextual stimuli but were maintained on the VI 60-sec schedule of reinforcement. Changes in either transportational or contextual stimuli reduced resistance to extinction and spontaneous recovery, and substantial increments in responding occurred upon reinstatement of the transportational and contextual stimuli associated with baseline training. Evidence for summation of the two sources of stimulus change was obtained. Changes in transportational and contextual stimuli produced only a brief disruption in responding when reinforcement of leverpressing was continued.     

A behavioral field approach to instrumental learning in the rat: I. Partial reinforcement effects and sex differences

The behavioral field approach employs naturalistic observation and simultaneous, multiple recordings of ecologically relevant aspects of behavior-environment interactions. Applying this approach to runway learning in the rat, the inferior acquisition speed of partial reinforcement (PRF) subjects as compared to continuous reinforcement (CRF) subjects was found to result from greater response variability, more sniffing and goal-avoidance behavior, and slower dropping out of collateral behaviors (e.g., drinking and sand-digging). Extinction first produced an increase in exploratory behavior, then displacement activities (e.g., grooming and biting) and goal-avoidance. CRF subjects showed greater response persistence as measured by number of extinction trials to disrupt an established, favored path. PRF subjects showed greater goal persistence as measured by trials to retrace from goalbox. In extinction, while CRF subjects were more inclined to engage in drinking and sand-digging in the startbox, PRF subjects exhibited more biting behavior in the goalbox. The only sex-related differences were superior speeds by female CRF subjects, inferior goal speeds by female PRF subjects during acquisition, and superior goalbox escape learning by females in extinction.     

Latent inhibition in honeybees

Aversive conditioning was studied in individual honeybees flying back and forth between the hive and the sill of an open laboratory window, where they took sucrose solution from a target so constructed that shock could be delivered while the proboscis was in contact with the solution. During feeding, a conditioned stimulus—substrate vibration or airstream—was paired with brief shock avoidable by interruption of feeding. In Experiment 1, unreinforced preexposure of the conditioned stimulus was found to retard acquisition (latent inhibition). In Experiment 2, which was designed to inquire into the stimulus specificity of the effect, differential conditioning was found to be impaired by unreinforced preexposure of the positive stimulus and facilitated by unreinforced preexposure of the negative stimulus. In Experiment 3, a summation experiment designed to test various alternative explanations of the effect, a preexposed stimulus was found to suppress response to an excitatory conditioned stimulus when the two stimuli were presented together.     

Intramodal blocking in honeybees

Foraging honeybees were trained in a concurrent blocking design with a compound stimulus (AX) reinforced and one of its components (A) either reinforced for a blocking group or nonreinforced for a control group. In Experiment 1, a compound of two colors was used; in Experiment 2, a compound of two odors was used; in Experiment 3, a color-position compound, with position defined in terms of proximity to a distinctive visual landmark, was used; and, in Experiment 4, an odor-position compound was used. In each of the first three experiments, the blocking group responded less than did the control group in a subsequent test with X; in the fourth experiment, the two groups did not differ. The results are in accord with expectations based on those of previous experiments with honeybees in which the independence assumption was found to hold for intermodal compounds but not for intramodal compounds.     

Effects of discrimination training on stimulus generalization in pigeons: Role of resistance to extinction and response-produced stimuli

In Experiment 1, two groups (n = 10) of pigeons received 17 sessions of TD (true discrimination) or ND (nondifferential) training with line angles. Seventeen sessions of SS (single stimulus) training with a wavelength preceded this training and two followed it. Subsequent wavelength generalization testing in extinction revealed a sharper TD than ND gradient. This slope difference was evident from the very first test stimulus presentation and remained stable throughout testing. As a consequence of substantial overtraining, there was no reduction of response strength and no sharpening of generalization during testing for either group. In Experiment 2, two groups (n = 16) of pigeons received 10 sessions of TD or PD (pseudodiscrimination) training with line angles, followed by four sessions of SS training with a single wavelength. During this training and in subsequent wavelength generalization testing in extinction, brief blackouts separated stimulus presentations. Again, the TD group yielded the sharper gradient. Although responding weakened and the gradients sharpened during the test, these effects were comparable in the two groups. Furthermore, gradients based on the percentage of trials with at least one response showed the same TD-PD slope difference. This finding indicates that differential control over responding by response-produced feedback is inadequate to account for the TD-PD difference in generalization slope. Both experiments indicate that a purported difference in resistance to extinction is also an inadequate explanation.     

Partial reinforcement effect: The expectancy of reward on nonreward trials

In order to determine the importance of the development of expectancy of reward prior to partial reward trials; rats were given 20 continuously reinforced trials prior to 20 partially reinforced trials (CRF-PRF) and compared to Ss given only 20 partially reinforced trials (PRF). Control groups received 20 or 40 continuously reinforced trials (CRF-20, CRF-40) to determine the effect of differing numbers of acquisition trials. Results showed that terminal acquisition differences were minimal in the run segment of the alley and that Group CRF-PRF was more resistant to extinction than Group PRF, and both were more resistant to extinction than the CRF-20 and CRF-40 groups, which did not differ from each other. These results were interpreted as supporting the notion that the expectancy of reward on nonreward trials during partial reinforcement acquisition is a determiner of the magnitude of the partial reinforcement extinction effect.     

Persistence early and late in extinction as a function of number of continuous reinforcements preceding partial reinforcement training

In two experiments rats were given straight-alley training in the following sequence: continuous reward (CR), partial reward (PR), extinction (EXT). Independent groups differed only in the amount of CR training. In both experiments,. early-EXT performance was directly related to amount of CR training and late-EXT performance was inversely related to amount of CR training. These data were related to a possible specific s_F intensity hypothesis, an extension of frustration theory.