Instrumental escape learning in the rat at 24-hour intertriai interval: Effects of magnitude and schedule of reinforcement

Three experiments investigated the effects of magnitude and schedule of reinforcement and level of training in instrumental escape learning at a 24-h intertriai interval. In Experiment I, two magnitudes of reinforcement were factorially combined with two schedules of reinforcement (CRF and PRF). Under PRF, large reward produced greater resistance to extinction than did small reward, while the reverse was true under CRF. In Experiment II, two levels of acquisition training were factorially combined with three schedules of reinforcement (CRF, single-alternation, and nonalternated PRF). Patterned running was observed late in acquisition in the single-alternation extended-training condition. Resistance to extinction was greater for the nonalternated PRF condition than for the single-alternation condition following extended acquisition, and the reverse was true following limited acquisition. Experiment III confirmed the extinction findings of Experiment II. The results of all three experiments supported an analysis of escape learning at spaced trials in terms of Capaldi’s (1967) sequential theory.     

Ontogeny of persistence: Immediate extinction effects in preweanling and weanling rats

In Experiment I rats were trained for 21÷2 days under partial (PRF) or continuous reinforcement (CRF) conditions starting at 18, 22, 28, or 36 days of age and were then subjected to immediate extinction. At all ages there was a strong partial reinforcement extinction effect (PREE), and absolute size of PREE was greatest in the youngest rats. Rate of extinction increased as a function of age following both CRF and PRF. In Experiment II the youngest and oldest age groups of Experiment I were run under the two reward conditions of Experiment I and in a third condition, PRF with number of rewards rather than trials equated to CRF (PRF-R). The PRF-R and PRF groups were not different in extinction, and both were more persistent than CRF. The youngest rats were again more persistent than the oldest, particularly after PRF training. In Experiment III it was shown that the well-known paradoxical effect, greater reward in CRF acquisition leads to faster extinction, operates in our youngest and oldest animals, but is more pronounced in the oldest. The results are discussed in terms of whether they require different explanations than those often applied to extinction data from adult rats.     

Prolonged,unsignaled, inescapable shocks increase persistence in subsequent appetitive instrumental learning

In the first experiment, a prolonged period of intermittent, unsignaled shocks preceded appetitive runway acquisition, under either continuous (CRF) or partial reinforcement (PRF) and extinction. In the second experiment, the shock treatment came between CRF or PRF acquisition and extinction; and in the third experiment, the shocks intervened between appetitive CRF acquisition and shock-punishment extinction. The main finding was that compared with an unshocked control, shock facilitated acquisition in Experiment 1, and led to increased resistance to extinction and/or punishment in all experiments. In Experiment 1, the shock effect in appetitive extinction was seen mainly in the CRF group; in Experiment 2, the effect was to increase persistence in both the CRF and PRF groups; and in Experiment 3, shock treatment produced stronger resistance to punished extinction. The discussion is in terms of habituation and a general theory of persistence, and the concept of helplessness.     

A behavior field approach to instrumental learning in the rat: II. Training parameters and a stage model of extinction

Three runway experiments tested a stage model of extinction which postulated an orderly succession of three qualitatively different stages: habit, trial and error, and resolution. The model predicted that Stage 1 should be characterized by perseveration of habitual routes (i.e., response persistence) and the absence of competing responses; Stage 2, by an increase in investigatory behavior (response variation and hole exploration) and biting behavior; Stage 3, by a decrease in the competing responses of Stage 2 and continued increase in goal avoidance and substitution behavior (e.g., sand-digging). These predictions were largely confirmed. Further, Experiments 1 and 2 showed that, as expected by the model, continuous reinforcement (CRF) resulted in more practice of habitual routes in acquisition and greater response persistence, while partial reinforcement (PRF) resulted in more route variation and hole exploration in acquisition and greater goal persistence which was attributable to prior reinforcement of a trial-and-error coping strategy. Results of Experiment 3, which combined training trials and reward magnitudes orthogonally, supported the prediction that response persistence was positively related to training trials, and goal persistence negatively related to reward magnitudes. All three experiments demonstrated an inverted-U function in investigatory and biting behavior, as predicted by the stage model.     

Discrimination training in infant,preweanling, and weanling rats: Effects of prior learning experiences with the discriminanda

Rats were given continuous reinforcement (CRF), partial reinforcement (PRF), or successive discrimination (D) training in an alley from 11–14 (Age 1) or 15–18 (Age 2) days of age. Reinforcement was the opportunity to suckle the dry nipples of an anesthetized dam. Following a 10-day interval, all animals were given 4 successive days of discrimination training with food pellets as reinforcement. Control groups were given only the second phase of training. In the first phase, D subjects of both ages responded appropriately to the discriminative stimuli, and the PRF subjects of both ages ran significantly slower than CRF subjects. In the second phase, only the CRF subjects of Age 1 showed behavioral discrimination. All three Age 2 groups discriminated, but the discrimination developed earliest after Phase I CRF and latest after Phase I PRF. Both Age 1 control groups showed a late-developing discrimination, but neither Age 2 control group discriminated. The results suggest that infant and preweanling rats can learn both positive and negative expectancies of appetitive events, respond appropriately, and retain and transfer these expectancies to new learning. The reinforcement value of dry suckling and the effects of stimulus preexposure in infant rats are also discussed.     

Effects of signaled reward in instrumental conditioning: Enhanced learning on DRL and DRH schedules of reinforcement

The effects of signaled reward were examined using DRL and DRH schedules of reinforcement. In each case, one group of rats received a brief cue between the reinforced response and the reward, and a second group received brief cues at random times. With the DRL schedule (Experiment 1), signaled reward decreased response rate, increased response efficiency (number of responses per reinforcer), and increased resistence to satiation relative to the control group. With the DRH schedule (Experiment 2), signaled reward increased response rate, efficiency, and resistance to satiation. These results refute an overshadowing explanation of the effects of signaled reward and suggest that food-correlated cues enhance learning of the reinforcement contingencies.     

Dissociation of patterned alternation learning and the partial reinforcement extinction effect in preweanling rats

Sprague-Dawley rat pups aged 14 or 18 days were trained on a patterned (single) alternation schedule with either an 8- or a 105-sec intertriai interval (ITI). At the 8-sec ITI, alternation learning was obtained at both ages, but the older age group learned more rapidly. There was no evidence of response alternation at the 105-sec ITI at either age. Continuously reinforced (CRF) and partially reinforced (PRF) groups trained and extinguished along with the patterned alternation (PA) group at the 105-sec ITI showed a robust partial reinforcement extinction effect (PREE) at both ages. Moreover, there was no difference in the rate of extinction of the PRF and PA groups at either age (i.e., no effect of N-length). A PREE can therefore be obtained in infant rats under conditions that apparently preclude the formation of sequential associations. The implications of this finding for the ontogeny of instrumental learning and extinction are discussed.     

Reward sequence and reinforcement level as determinants of S− behavior in differential conditioning

In two differential conditioning experiments, groups of 10 rats each differed with respect to average reward and schedule of reward received in S+. Nonreward (N) occurred on all S? trials. In both experiments, extinction of responding to S? (resistance to discrimination) was extensively regulated by reward sequence and was largely independent of average reward. In Experiment 1, resistance to discrimination was a function of transitions from N to rewarded (R) trials (N-R transitions). In Experiment 2, resistance to discrimination was increased by large reward on the R trial of N-R transitions and decreased by large reward on the R trial of R-N transitions. These schedule effects on resistance to discrimination parallel the effects of comparable schedules on resistance to extinction following partial reinforcement. The results are discussed in terms of sequential theory, reinforcement level theory, and their implications for various schedule manipulations that have previously shown S? behavior to be inversely related to average reward in S+.     

Signaling functions of the second-order CS: Partial reinforcement during second-order conditioning of the pigeon’s keypeck

The signaling function of the second-order CS (S2) was manipulated in second-order autoshaping by arranging a partial reinforcement schedule. S2 was paired with a well-conditioned first-order CS (SI) on a continuous reinforcement or a 25% reinforcement schedule in different groups. Schedule of reinforcement did not influence the number of S2-S1 pairings required to establish keypecking to S2. However, in the postacquisition sessions, responding to S2 was initially weaker but persisted for many more sessions on the 25% schedule than on the 100% schedule. The data indicate that S2-S1 pairings are responsible both for the acquisition of second-order keypecking to S2 and for the subsequent conversion of S2 into an inhibitory stimulus.     

Extinction responding following partial reinforcement: The effects of number of rewarded trials and magnitude of reward

Rats were trained on a daily partial reward schedule of small magnitude of reward (S), nonreward (N), and large magnitude of reward (L), which began with SN or SSNN for all animals. The remainder of the daily schedule was defined by the factorial combination of the number of rewards (1 vs. 3) and the magnitude of reward (S vs. L). Following 18 days of such training, 20 trials of extinction were administered. It was found that increasing the number of rewarded trials in a partial reinforcement schedule decreased resistance to extinction. Furthermore, increased number of large-magnitude rewards reduced resistance to extinction to a greater extent than increased number of small-magnitude rewards.     

A behavior systems view of responding to probe stimuli during an interfood clock

Two experiments used a behavior systems approach to relate the form of responses during an interfood clock to the temporal distance of the individual clock stimuli to food. Stimuli proximate to food should better control a focal search mode and related responses, whereas stimuli temporally distant from food should better control a general search mode and related responses. Experiment 1 conditioned two groups of rats with a sequence of four equal-length 12-sec clock stimuli that terminated with food and then tested for the conditioning of a general search mode by presenting an unconditioned moving probe stimulus (either a rolling ball bearing or a rotating mechanical door) during each of the clock stimuli. Consistent with a behavior systems view, contact with the ball bearing was markedly greater during a clock stimulus distant from food. The absence of similar differential contact of the door across the clock stimuli showed that the effect was specific to the ball bearing rather than a general response to stimulus dimensions of movement and sound. Experiment 2 showed that the general search mode was controlled by the clock stimulus rather than the passage of time.     

A behavioral field approach to instrumental learning in the rat: I. Partial reinforcement effects and sex differences

The behavioral field approach employs naturalistic observation and simultaneous, multiple recordings of ecologically relevant aspects of behavior-environment interactions. Applying this approach to runway learning in the rat, the inferior acquisition speed of partial reinforcement (PRF) subjects as compared to continuous reinforcement (CRF) subjects was found to result from greater response variability, more sniffing and goal-avoidance behavior, and slower dropping out of collateral behaviors (e.g., drinking and sand-digging). Extinction first produced an increase in exploratory behavior, then displacement activities (e.g., grooming and biting) and goal-avoidance. CRF subjects showed greater response persistence as measured by number of extinction trials to disrupt an established, favored path. PRF subjects showed greater goal persistence as measured by trials to retrace from goalbox. In extinction, while CRF subjects were more inclined to engage in drinking and sand-digging in the startbox, PRF subjects exhibited more biting behavior in the goalbox. The only sex-related differences were superior speeds by female CRF subjects, inferior goal speeds by female PRF subjects during acquisition, and superior goalbox escape learning by females in extinction.     

The stimulus-response overshadowing phenomenon with VI versus FI schedules of reinforcement

Previous research has shown that response rates on a variable interval (VI) schedule of reinforcement decrease if a brief response-produced signal is given prior to reward. One explanation is that the signal overshadows the response because it is a better predictor of reinforcement. The S-R overshadowing effect does not occur with variable ratio (VR) schedules, however. Tarpy, Lea, and Midgley (1983) explained this fact by suggesting that the signal functions to enhance the salience of the temporal interval offset on the VI schedule (a characteristic not possessed by VR schedules), which then overshadows the response. In this experiment, the salience of the temporal interval was enhanced in another way: signaled or unsignaled reward was provided to rats responding on either a VI or fixed interval (FI) reward schedule. As predicted, rates were lowest for animals receiving signaled reinforcement on an FI schedule and highest for those receiving unsignaled reinforcement on a VI schedule.     

Resistance to discrimination and subsequent resistance to extinction as a function of the sequence of partial S+ reward in differential conditioning

The effects of transitions from nonrewarded (N) to rewarded (R) trials (N-R transitions) on discriminative behavior in differential conditioning and subsequent resistance to extinction were investigated in two experiments. In Experiment 1, groups given N-R transitions within S+ were more resistant to discrimination (ran fast in S?) and extinction than were groups given a partial reinforcement (PRF) schedule in S+ devoid of N-R transitions. Experiment 2 indicated that N-R transitions that occur when an N trial in S? is followed by an R trial in S+ are as effective in increasing resistance to discrimination, but not resistance to extinction, as are N-R transitions that occur within S+. The sequential effects obtained here were highly similar to those in conventional PRF and support the view that differential conditioning and PRF are highly interrelated phenomena. The results are discussed in terms of the extension of sequential theory to differential conditioning and the importance of internal reward-produced cues in discrimination learning.     

The effects of reinforcement frequency on performance in a multiple fixed-ratio schedule

A barpress analog to the double-alley runway was sought by varying percentage reward in the first of two consecutive FR 18s. Groups of six rats each were given 0% 50%. or 100% reinforcement upon completion of the first FR 18: after a 5-sec midtnal imterval, the second FR 18 was administered on a separate lever and all groups received CRF reward upon its completion. Group 50 Ss performed faster after nonreward than after reward. Group 50 Ss performed faster after nonreward than did 0% Ss. A measure of midtnal behavior revealed a difference between groups in orienting to the bars. When all groups were shifted to a 50% first component schedule (Phase II), there were no statistically reliable effects of prior reinforcement history on rewarded or nonrewarded responding. The Phase 1 results were taken to demonstrate a frustration effect similar to that of the double alley     

Effect of N-R transitions during partial reinforcement pretraining on subsequent resistance to discrimination

Three groups of 12 rats received 25 pretraining trials to each future discriminandum employed in a subsequent differential brightness conditioning problem. Groups NR and RN received partial reinforcement (PRF) pretraining either with or without, respectively, transitions from nonrewarded to rewarded trials (N-R transitions). Group CRF received consistent reinforcement during pretraining. A fourth group (n=12), Group NP, received no pretraining. During discrimination learning, one-half of the rats in each group received all their daily S+ trials preceding their daily S? trials (+? sequence); the remainder of the rats received an intermixed sequence of trials to S+ and S? (+?+ sequence). Discrimination learning was faster under the +? sequence than under the +?+ condition, and discrimination learning was retarded in Group NR relative to the other three groups, which did not differ from one another, under both the +? and +?+ discrimination sequence conditions. The results are discussed with Reference to previous experiments demonstrating N-R transition effects on discrimination learning, a theoretical extension of sequential theory to discrimination learning, and the effects of nondifferential reinforcement prior to discrimination learning on learned irrelevance.     

Partial reinforcement effect: The expectancy of reward on nonreward trials

In order to determine the importance of the development of expectancy of reward prior to partial reward trials; rats were given 20 continuously reinforced trials prior to 20 partially reinforced trials (CRF-PRF) and compared to Ss given only 20 partially reinforced trials (PRF). Control groups received 20 or 40 continuously reinforced trials (CRF-20, CRF-40) to determine the effect of differing numbers of acquisition trials. Results showed that terminal acquisition differences were minimal in the run segment of the alley and that Group CRF-PRF was more resistant to extinction than Group PRF, and both were more resistant to extinction than the CRF-20 and CRF-40 groups, which did not differ from each other. These results were interpreted as supporting the notion that the expectancy of reward on nonreward trials during partial reinforcement acquisition is a determiner of the magnitude of the partial reinforcement extinction effect.     

Partial reinforcement delayed extinction effect after regularly alternating reward training

When extinction is delayed very long, the superior resistance to extinction of the random schedule group relative to the alternating schedule group disappears (partial reinforcement delayed extinction effect, PRDE). Two experiments assessed the effects of reinforcement/nonreinforcement on Trial 1 on the PRDE. Following extended partial reinforcement acquisition training in a runway, rats received extinction training after a short (1-day) or long (23-day) retention interval. The schedules used in Experiment 1 were: a single-alternation (SA) schedule beginning each day with a rewarded (r) trial, for Group r-SA; an SA schedule beginning with a nonrewarded (n) trial, for Group n-SA; and a random (Rd) schedule, for Group Rd. The schedules and group names used in Experiment 2 were r-SA, Rd, and r-Rd. The results were that (1) rats given r-SA schedules yielded considerable resistance under delayed extinction, (2) those given Rd and r-Rd schedules showed a decline in resistance to extinction over a long retention interval, (3) those given the n-SA schedule showed relatively low resistance at both retention intervals, although retention deficit was not greater than in the case of the Rd schedule, and thus, (4) the PRDE was found in both experiments, although only weakly in Experiment 1. The results indicated that a regularly alternating reward pattern was a more important determinant than was type of reward on Trial 1 for the PRDE. The PRDE due to differential retention deficits among schedules is discussed on the basis of dual-process associative sequential mechanisms and cognitive rule-encoding mechanisms.     

Response sequence learning as a function of primary versus conditioned reinforcement

Two groups of pigeons were required to generate a fixed sequence of responses on three keys, for example, middle-left-right. One group received a small food reward (S^Food) following each correct response except the terminal one, which was followed by a large food reward. The second group received conditioned reinforcement from an overhead light (S^Light) for each correct response, with the terminal correct response followed by both S^Light and the large food reward. We manipulated length of sequence (3 or 7 responses) and duration of required interresponse interval (IRI; 1 to 9 sec). S^Light contingencies generated more accurate performances than did S^Food when sequence length was 3 responses but not when it was 7 responses. IRI duration influenced accuracy under the S^Light contingencies but not under S^Food. These results show that conditioned reinforcers sometimes generate more accurate sequence learning than do primary reinforcers, and that schedule contigencies influence which type of feedback will optimize performance. The results parallel those from the matching-to-sample and conditional discrimination literature.     

Hurdle-jump responding in the rat as a function of conspecific odor of reward and nonreward

A hurdle-jump escape response was employed to assess the laboratory rat’s aversion or attraction to different types of conspecific odor. Odorant donor subjects received 112 runway acquisition trials on a continuous reward schedule followed by 32 extinction trials, 112 acquisition trials on a 50% schedule of reward and nonreward followed by 32 extinction trials, or 144 “neutral” trials with no reward in the alley. Different groups of test subjects escaped from odor excreted by odorant subjects on (a) nonrewarded acquisition and extinction trials, (b) rewarded trials during continuous reinforcement, (c) rewarded trials during partial reinforcement, or (d) neutral trials; others escaped from a clean box. The principal findings were: (1) significant aversion to “odor of nonreward” appeared after the donor odorants had received 12 exposures to reward; (2) production of odor of nonreward by odorant subjects changed as a function of training experience with reward; (3) after repeated exposure to odor of nonreward, the escape response habituated; (4) greater or different odor excretion in extinction resulted from subjects trained on a continuous reward schedule than on a partial reward schedule. Relationships of the data to frustration theory were discussed, assuming that inferred differences in production of odor reflect differences in frustration reaction.