Durability of the partial reinforcement and partial delay of reinforcement extinction effects after minimal acquisition training

Four groups of 10 rats each were given six acquisition trials (Phase 1) under continuous reinforcement (CR), partial reinforcement (PR), constant delay (CD), or partial delay of reinforcement (PD) conditions. In Phase 2, all Ss were given 18 nonreinforced trials, followed by 12 continuously reinforced trials in Phase 3. In Phase 4, all Ss were given 12 more extinction trials. A constant 24-h ITI was observed throughout the experiment. A strong partial reinforcement extinction effect (PREE) was obtained in both Phases 2 and 4. Only a temporary partial delay of reinforcement effect (PDRE) was observed, which was restricted to the first nine trials of the first extinction phase. No constant delay of reinforcement effect (CDRE) was observed in either extinction phase. The results were discussed in terms of both frustration and sequential theories.     

Ontogeny of persistence: Immediate extinction effects in preweanling and weanling rats

In Experiment I rats were trained for 21÷2 days under partial (PRF) or continuous reinforcement (CRF) conditions starting at 18, 22, 28, or 36 days of age and were then subjected to immediate extinction. At all ages there was a strong partial reinforcement extinction effect (PREE), and absolute size of PREE was greatest in the youngest rats. Rate of extinction increased as a function of age following both CRF and PRF. In Experiment II the youngest and oldest age groups of Experiment I were run under the two reward conditions of Experiment I and in a third condition, PRF with number of rewards rather than trials equated to CRF (PRF-R). The PRF-R and PRF groups were not different in extinction, and both were more persistent than CRF. The youngest rats were again more persistent than the oldest, particularly after PRF training. In Experiment III it was shown that the well-known paradoxical effect, greater reward in CRF acquisition leads to faster extinction, operates in our youngest and oldest animals, but is more pronounced in the oldest. The results are discussed in terms of whether they require different explanations than those often applied to extinction data from adult rats.     

Instrumental escape learning in the rat at 24-hour intertriai interval: Effects of magnitude and schedule of reinforcement

Three experiments investigated the effects of magnitude and schedule of reinforcement and level of training in instrumental escape learning at a 24-h intertriai interval. In Experiment I, two magnitudes of reinforcement were factorially combined with two schedules of reinforcement (CRF and PRF). Under PRF, large reward produced greater resistance to extinction than did small reward, while the reverse was true under CRF. In Experiment II, two levels of acquisition training were factorially combined with three schedules of reinforcement (CRF, single-alternation, and nonalternated PRF). Patterned running was observed late in acquisition in the single-alternation extended-training condition. Resistance to extinction was greater for the nonalternated PRF condition than for the single-alternation condition following extended acquisition, and the reverse was true following limited acquisition. Experiment III confirmed the extinction findings of Experiment II. The results of all three experiments supported an analysis of escape learning at spaced trials in terms of Capaldi’s (1967) sequential theory.     

Hurdle-jump responding in the rat as a function of conspecific odor of reward and nonreward

A hurdle-jump escape response was employed to assess the laboratory rat’s aversion or attraction to different types of conspecific odor. Odorant donor subjects received 112 runway acquisition trials on a continuous reward schedule followed by 32 extinction trials, 112 acquisition trials on a 50% schedule of reward and nonreward followed by 32 extinction trials, or 144 “neutral” trials with no reward in the alley. Different groups of test subjects escaped from odor excreted by odorant subjects on (a) nonrewarded acquisition and extinction trials, (b) rewarded trials during continuous reinforcement, (c) rewarded trials during partial reinforcement, or (d) neutral trials; others escaped from a clean box. The principal findings were: (1) significant aversion to “odor of nonreward” appeared after the donor odorants had received 12 exposures to reward; (2) production of odor of nonreward by odorant subjects changed as a function of training experience with reward; (3) after repeated exposure to odor of nonreward, the escape response habituated; (4) greater or different odor excretion in extinction resulted from subjects trained on a continuous reward schedule than on a partial reward schedule. Relationships of the data to frustration theory were discussed, assuming that inferred differences in production of odor reflect differences in frustration reaction.     

Effect of prior nonreward on subsequent incentive growth during brief acquisition

Four experiments compared runway extinction or hurdle-jumping from nonreward performance following brief (10 trials) continuous or partial reinforcement acquisition. Some of the partial groups received all nonrewarded trials prior to any rewards. The major findings were that (l) rats receiving all nonrewarded experiences prior to rewarded ones were more persistent during extinction than continuously rewarded subjects; (2) rats receiving nonrewarded placements prior to rewarded ones in one compartment of a two-compartment box, failed to learn a hurdle-jumping response to escape nonreward, whereas rats not receiving the initial nonrewards did learn the escape response; (3) increasing the number of rewarded placements following initial nonrewarded ones offset the effect noted in (2). The results, which are discussed in the context of a frustration analysis of the small-trials partial reinforcement effect, suggest that incentive growth over rewarded trials is retarded when the rewards have been preceded by nonrewards. The similarity of these results to those investigating the phenomenon of latent inhibition is apparent, and possible mechanisms responsible for the present results are suggested in current theoretical accounts of latent inhibition.     

Dissociation of patterned alternation learning and the partial reinforcement extinction effect in preweanling rats

Sprague-Dawley rat pups aged 14 or 18 days were trained on a patterned (single) alternation schedule with either an 8- or a 105-sec intertriai interval (ITI). At the 8-sec ITI, alternation learning was obtained at both ages, but the older age group learned more rapidly. There was no evidence of response alternation at the 105-sec ITI at either age. Continuously reinforced (CRF) and partially reinforced (PRF) groups trained and extinguished along with the patterned alternation (PA) group at the 105-sec ITI showed a robust partial reinforcement extinction effect (PREE) at both ages. Moreover, there was no difference in the rate of extinction of the PRF and PA groups at either age (i.e., no effect of N-length). A PREE can therefore be obtained in infant rats under conditions that apparently preclude the formation of sequential associations. The implications of this finding for the ontogeny of instrumental learning and extinction are discussed.     

A behavior field approach to instrumental learning in the rat: II. Training parameters and a stage model of extinction

Three runway experiments tested a stage model of extinction which postulated an orderly succession of three qualitatively different stages: habit, trial and error, and resolution. The model predicted that Stage 1 should be characterized by perseveration of habitual routes (i.e., response persistence) and the absence of competing responses; Stage 2, by an increase in investigatory behavior (response variation and hole exploration) and biting behavior; Stage 3, by a decrease in the competing responses of Stage 2 and continued increase in goal avoidance and substitution behavior (e.g., sand-digging). These predictions were largely confirmed. Further, Experiments 1 and 2 showed that, as expected by the model, continuous reinforcement (CRF) resulted in more practice of habitual routes in acquisition and greater response persistence, while partial reinforcement (PRF) resulted in more route variation and hole exploration in acquisition and greater goal persistence which was attributable to prior reinforcement of a trial-and-error coping strategy. Results of Experiment 3, which combined training trials and reward magnitudes orthogonally, supported the prediction that response persistence was positively related to training trials, and goal persistence negatively related to reward magnitudes. All three experiments demonstrated an inverted-U function in investigatory and biting behavior, as predicted by the stage model.     

Partial reinforcement effects on learning and extinction of place preferences in the water maze

In two experiments, two groups of rats were trained in a navigation task according to either a continuous or a partial schedule of reinforcement. In Experiment 1, animals that were given continuous reinforcement extinguished the spatial response of approaching the goal location more readily than animals given partial reinforcement—a partial reinforcement extinction effect. In Experiment 2, after partially or continuously reinforced training, animals were trained in a new task that made use of the same reinforcer according to a continuous reinforcement schedule. Animals initially given partial reinforcement performed better in the novel task than did rats initially given continuous reinforcement. These results replicate, in the spatial domain, well-known partial reinforcement phenomena typically observed in the context of Pavlovian and instrumental conditioning, suggesting that similar principles govern spatial and associative learning. The results reported support the notion that salience modulation processes play a key role in determining partial reinforcement effects.     

Partial reinforcement delayed extinction effect after regularly alternating reward training

When extinction is delayed very long, the superior resistance to extinction of the random schedule group relative to the alternating schedule group disappears (partial reinforcement delayed extinction effect, PRDE). Two experiments assessed the effects of reinforcement/nonreinforcement on Trial 1 on the PRDE. Following extended partial reinforcement acquisition training in a runway, rats received extinction training after a short (1-day) or long (23-day) retention interval. The schedules used in Experiment 1 were: a single-alternation (SA) schedule beginning each day with a rewarded (r) trial, for Group r-SA; an SA schedule beginning with a nonrewarded (n) trial, for Group n-SA; and a random (Rd) schedule, for Group Rd. The schedules and group names used in Experiment 2 were r-SA, Rd, and r-Rd. The results were that (1) rats given r-SA schedules yielded considerable resistance under delayed extinction, (2) those given Rd and r-Rd schedules showed a decline in resistance to extinction over a long retention interval, (3) those given the n-SA schedule showed relatively low resistance at both retention intervals, although retention deficit was not greater than in the case of the Rd schedule, and thus, (4) the PRDE was found in both experiments, although only weakly in Experiment 1. The results indicated that a regularly alternating reward pattern was a more important determinant than was type of reward on Trial 1 for the PRDE. The PRDE due to differential retention deficits among schedules is discussed on the basis of dual-process associative sequential mechanisms and cognitive rule-encoding mechanisms.     

Prolonged,unsignaled, inescapable shocks increase persistence in subsequent appetitive instrumental learning

In the first experiment, a prolonged period of intermittent, unsignaled shocks preceded appetitive runway acquisition, under either continuous (CRF) or partial reinforcement (PRF) and extinction. In the second experiment, the shock treatment came between CRF or PRF acquisition and extinction; and in the third experiment, the shocks intervened between appetitive CRF acquisition and shock-punishment extinction. The main finding was that compared with an unshocked control, shock facilitated acquisition in Experiment 1, and led to increased resistance to extinction and/or punishment in all experiments. In Experiment 1, the shock effect in appetitive extinction was seen mainly in the CRF group; in Experiment 2, the effect was to increase persistence in both the CRF and PRF groups; and in Experiment 3, shock treatment produced stronger resistance to punished extinction. The discussion is in terms of habituation and a general theory of persistence, and the concept of helplessness.     

A behavioral field approach to instrumental learning in the rat: I. Partial reinforcement effects and sex differences

The behavioral field approach employs naturalistic observation and simultaneous, multiple recordings of ecologically relevant aspects of behavior-environment interactions. Applying this approach to runway learning in the rat, the inferior acquisition speed of partial reinforcement (PRF) subjects as compared to continuous reinforcement (CRF) subjects was found to result from greater response variability, more sniffing and goal-avoidance behavior, and slower dropping out of collateral behaviors (e.g., drinking and sand-digging). Extinction first produced an increase in exploratory behavior, then displacement activities (e.g., grooming and biting) and goal-avoidance. CRF subjects showed greater response persistence as measured by number of extinction trials to disrupt an established, favored path. PRF subjects showed greater goal persistence as measured by trials to retrace from goalbox. In extinction, while CRF subjects were more inclined to engage in drinking and sand-digging in the startbox, PRF subjects exhibited more biting behavior in the goalbox. The only sex-related differences were superior speeds by female CRF subjects, inferior goal speeds by female PRF subjects during acquisition, and superior goalbox escape learning by females in extinction.     

Partial reinforcement in autoshaping with pigeons

The acquisition, maintenance, and extinction of autoshaped responding in pigeons were studied under partial and continuous reinforcement. Five values of probability of reinforcement, ranging from .1 to 1.0, were combined factorially with five values of intertrial interval ranging from 15 to 250 sec for different groups. The number of trials required before autoshaped responding emerged varied inversely with the duration of the intertriai interval and probability of reinforcment, but partial reinforcement did not increase the number of reinforcers before acquisition. During maintained training, partial reinforcement increased the overall rate of responding. A temporal gradient of accelerated responding over the trial duration emerged during maintenance training for partial reinforcement groups, and was evident for all groups in extinction. Partial reinforcement groups responded more than continuous reinforcement groups over an equivalent number of trials in extinction. However, this partial-reinforcment extinction effect disappeared when examined in terms of the omission of “expected” reinforcers.     

Primary frustration differences following brief partial-reinforcement acquisition under varying magnitudes of reward

In Experiment I, rats which had received six partially reinforced runway acquisition trials, with a reward magnitude of 60 sec access to wet mash on rewarded trials, showed less persistent responding over highly massed extinction trials than subjects which had received the same acquisition schedule but reward magnitudes of either 1 or 10 45-mg pellets. In Experiment II, rats which had received six partially reinforced placements into one compartment of a two-compartment box, with 60 sec access to mash on rewarded placements, jumped a hurdle faster to escape nonreward than subjects which had received the same reward schedule but 10 45-mg pellets on rewarded trials. The data supported a primary frustration analysis for reward-magnitude manipulations within brief partial-reinforcement schedules.     

Effects of varying sucrose reinforcers and amobarbital sodium on positive contrast in rats

The effect of quantity and quality of reinforcement on performance change following a shift to uniform high reward was studied in four groups of rats. Twenty or 200 licks of a 5% or 20% sucrose solution constituted the four incentive conditions. Two additional subject groups were run in the high (20%–200 licks) and low (5%-20 licks) reward conditions to determine how amobarbital sodium, an emotional depressant, influences incentive shift performance. All six groups received 60 preshift runway trials (6/day), followed by 30 high reward trials. Twenty-four extinction trials contrasted drugged and normal performance relating to high and low reward Postshift positive contrast appeared in all nondrugged groups. An emotional base for positive contrast is considered.     

Resistance to discrimination and subsequent resistance to extinction as a function of the sequence of partial S+ reward in differential conditioning

The effects of transitions from nonrewarded (N) to rewarded (R) trials (N-R transitions) on discriminative behavior in differential conditioning and subsequent resistance to extinction were investigated in two experiments. In Experiment 1, groups given N-R transitions within S+ were more resistant to discrimination (ran fast in S?) and extinction than were groups given a partial reinforcement (PRF) schedule in S+ devoid of N-R transitions. Experiment 2 indicated that N-R transitions that occur when an N trial in S? is followed by an R trial in S+ are as effective in increasing resistance to discrimination, but not resistance to extinction, as are N-R transitions that occur within S+. The sequential effects obtained here were highly similar to those in conventional PRF and support the view that differential conditioning and PRF are highly interrelated phenomena. The results are discussed in terms of the extension of sequential theory to differential conditioning and the importance of internal reward-produced cues in discrimination learning.     

Extinction responding following partial reinforcement: The effects of number of rewarded trials and magnitude of reward

Rats were trained on a daily partial reward schedule of small magnitude of reward (S), nonreward (N), and large magnitude of reward (L), which began with SN or SSNN for all animals. The remainder of the daily schedule was defined by the factorial combination of the number of rewards (1 vs. 3) and the magnitude of reward (S vs. L). Following 18 days of such training, 20 trials of extinction were administered. It was found that increasing the number of rewarded trials in a partial reinforcement schedule decreased resistance to extinction. Furthermore, increased number of large-magnitude rewards reduced resistance to extinction to a greater extent than increased number of small-magnitude rewards.     

Classical conditioning of the rabbit nictitating membrane response: Effects of reinforcement schedule on response maintenance and resistance to extinction

Two experiments were conducted to assess the effects of the introduction of schedules of partial reinforcement (PRF), subsequent to continuous reinforcement training, on the maintenance and resistance to extinction of the rabbit’s nictitating membrane CR. Substantial response levels were maintained by schedules of reinforcement as lean as 15%, and the performance decrements, when observed to be reliable, could not be localized to the immediate effects of one, two, or three consecutive nonreinforced trials by the examination of conditional response probabilities. Moreover, reliable PRF extinction effects were obtained. The relevance of these findings to the purported empirical divergence of PRF effects on classical and instrumental conditioning were discussed.     

Effect of N-R transitions during partial reinforcement pretraining on subsequent resistance to discrimination

Three groups of 12 rats received 25 pretraining trials to each future discriminandum employed in a subsequent differential brightness conditioning problem. Groups NR and RN received partial reinforcement (PRF) pretraining either with or without, respectively, transitions from nonrewarded to rewarded trials (N-R transitions). Group CRF received consistent reinforcement during pretraining. A fourth group (n=12), Group NP, received no pretraining. During discrimination learning, one-half of the rats in each group received all their daily S+ trials preceding their daily S? trials (+? sequence); the remainder of the rats received an intermixed sequence of trials to S+ and S? (+?+ sequence). Discrimination learning was faster under the +? sequence than under the +?+ condition, and discrimination learning was retarded in Group NR relative to the other three groups, which did not differ from one another, under both the +? and +?+ discrimination sequence conditions. The results are discussed with Reference to previous experiments demonstrating N-R transition effects on discrimination learning, a theoretical extension of sequential theory to discrimination learning, and the effects of nondifferential reinforcement prior to discrimination learning on learned irrelevance.     

Transfer between nonreward and delay of reward following minimal acquisition training

Four groups of rats were given six acquisition trials under continuous reward, continuous delay of reward, partial reward, or partial delay of reward, following which all Ss received continuous delay. It was found that the partial reward and the partial delay of reward groups were significantly more persistent during the shift phase than the continuous reward group. No differences were found over trials between either the two partial groups or between each one and the continuous delay group.     

Replacement of event-generated memories of nonreinforcement with signal-generated memories of reinforcement during partial reinforcement training: Effect on resistance to extinction

Event-generated memory refers to the memory of a reinforcement (R) or nonreinforcement (N) event from an immediately preceding trial;signal-generated memory refers to the memory of a temporally remote R or N, retrieval of which is generated by presentation of a signal with which the memory is associated (Haggbloom, 1988). In each of three experiments, Group Signal-R received runway discrimination training in Phase 1 to establish a stimulus as a signal for R, and partial reinforcement training in Phase 2. An extinction test measured learning about the memory of nonreward (S^N)—learning that occurs when S^N is retrieved on R trials that follow N trials. In Group Signal-H, those R trials were accompanied by the signal for R, a treatment we hypothesized would generate retrieval of the memory of reinforcement (S^R) so that signal-generated S^R would replace event-generated S^N as the operative memory, thereby eliminating the increased resistance to extinction normally produced by PRF training. In each experiment, Group Signal-R was less resistant to extinction than was a control group conditioned to respond to-event-generated S^N. Extinction was as rapid in Group Signal-R as it was in a consistent reinforcement control group (Experiment 1) and in a group given intertrial reinforcements to interfere with learning about S^N (Experiment 3). Experiment 2 tested two alternative interpretations of the failure to learn about S^N in Group Signal-R. Those alternatives were found to be less viable than the hypothesis that the signal for R actively recruited retrieval of a competing memory.