Primary frustration differences following brief partial-reinforcement acquisition under varying magnitudes of reward

In Experiment I, rats which had received six partially reinforced runway acquisition trials, with a reward magnitude of 60 sec access to wet mash on rewarded trials, showed less persistent responding over highly massed extinction trials than subjects which had received the same acquisition schedule but reward magnitudes of either 1 or 10 45-mg pellets. In Experiment II, rats which had received six partially reinforced placements into one compartment of a two-compartment box, with 60 sec access to mash on rewarded placements, jumped a hurdle faster to escape nonreward than subjects which had received the same reward schedule but 10 45-mg pellets on rewarded trials. The data supported a primary frustration analysis for reward-magnitude manipulations within brief partial-reinforcement schedules.     

Effect of prior nonreward on subsequent incentive growth during brief acquisition

Four experiments compared runway extinction or hurdle-jumping from nonreward performance following brief (10 trials) continuous or partial reinforcement acquisition. Some of the partial groups received all nonrewarded trials prior to any rewards. The major findings were that (l) rats receiving all nonrewarded experiences prior to rewarded ones were more persistent during extinction than continuously rewarded subjects; (2) rats receiving nonrewarded placements prior to rewarded ones in one compartment of a two-compartment box, failed to learn a hurdle-jumping response to escape nonreward, whereas rats not receiving the initial nonrewards did learn the escape response; (3) increasing the number of rewarded placements following initial nonrewarded ones offset the effect noted in (2). The results, which are discussed in the context of a frustration analysis of the small-trials partial reinforcement effect, suggest that incentive growth over rewarded trials is retarded when the rewards have been preceded by nonrewards. The similarity of these results to those investigating the phenomenon of latent inhibition is apparent, and possible mechanisms responsible for the present results are suggested in current theoretical accounts of latent inhibition.     

Extinction responding following partial reinforcement: The effects of number of rewarded trials and magnitude of reward

Rats were trained on a daily partial reward schedule of small magnitude of reward (S), nonreward (N), and large magnitude of reward (L), which began with SN or SSNN for all animals. The remainder of the daily schedule was defined by the factorial combination of the number of rewards (1 vs. 3) and the magnitude of reward (S vs. L). Following 18 days of such training, 20 trials of extinction were administered. It was found that increasing the number of rewarded trials in a partial reinforcement schedule decreased resistance to extinction. Furthermore, increased number of large-magnitude rewards reduced resistance to extinction to a greater extent than increased number of small-magnitude rewards.     

The effects of reinforcement frequency on performance in a multiple fixed-ratio schedule

A barpress analog to the double-alley runway was sought by varying percentage reward in the first of two consecutive FR 18s. Groups of six rats each were given 0% 50%. or 100% reinforcement upon completion of the first FR 18: after a 5-sec midtnal imterval, the second FR 18 was administered on a separate lever and all groups received CRF reward upon its completion. Group 50 Ss performed faster after nonreward than after reward. Group 50 Ss performed faster after nonreward than did 0% Ss. A measure of midtnal behavior revealed a difference between groups in orienting to the bars. When all groups were shifted to a 50% first component schedule (Phase II), there were no statistically reliable effects of prior reinforcement history on rewarded or nonrewarded responding. The Phase 1 results were taken to demonstrate a frustration effect similar to that of the double alley     

Partial reinforcement effect: The expectancy of reward on nonreward trials

In order to determine the importance of the development of expectancy of reward prior to partial reward trials; rats were given 20 continuously reinforced trials prior to 20 partially reinforced trials (CRF-PRF) and compared to Ss given only 20 partially reinforced trials (PRF). Control groups received 20 or 40 continuously reinforced trials (CRF-20, CRF-40) to determine the effect of differing numbers of acquisition trials. Results showed that terminal acquisition differences were minimal in the run segment of the alley and that Group CRF-PRF was more resistant to extinction than Group PRF, and both were more resistant to extinction than the CRF-20 and CRF-40 groups, which did not differ from each other. These results were interpreted as supporting the notion that the expectancy of reward on nonreward trials during partial reinforcement acquisition is a determiner of the magnitude of the partial reinforcement extinction effect.     

An analysis of “signaled double-alternation patterning” in the rat

Rats given training with double alternation of rewards and nonrewards in which the first reward or nonreward of each pair occurred in a black runway and the second in a white runway developed fast running on rewarded trials in both runways and slow running on nonrewarded trials in both runways—signaled double alternation patterning. A subsequent shift in the reinforcement schedule produced a period of reversed patterning—slow on rewarded trials and fast on nonrewarded trials. The results are consistent with a compound stimulus discrimination interpretation of signaled double-alternation patterning rather than with a selective memory-retrieval explanation.     

Instrumental escape learning in the rat at 24-hour intertriai interval: Effects of magnitude and schedule of reinforcement

Three experiments investigated the effects of magnitude and schedule of reinforcement and level of training in instrumental escape learning at a 24-h intertriai interval. In Experiment I, two magnitudes of reinforcement were factorially combined with two schedules of reinforcement (CRF and PRF). Under PRF, large reward produced greater resistance to extinction than did small reward, while the reverse was true under CRF. In Experiment II, two levels of acquisition training were factorially combined with three schedules of reinforcement (CRF, single-alternation, and nonalternated PRF). Patterned running was observed late in acquisition in the single-alternation extended-training condition. Resistance to extinction was greater for the nonalternated PRF condition than for the single-alternation condition following extended acquisition, and the reverse was true following limited acquisition. Experiment III confirmed the extinction findings of Experiment II. The results of all three experiments supported an analysis of escape learning at spaced trials in terms of Capaldi’s (1967) sequential theory.     

Approach and escape to conspecific odors of reward and nonreward in rats

A three-compartment box was used, and a reward odor, or nonreward (extinction) odor, produced by another rat, was present in the middle compartment. Two control odor procedures were also used. The results showed that rats will approach a location in which another rat has previously been given reward more rapidly than they will escape from that location, but showed the opposite effect when the odor was produced by a rat undergoing extinction. The mere presence of an odor associated with another rat had the effect of producing much slower locomotion as compared to a no-odor control condition.     

Reward sequence and reinforcement level as determinants of S− behavior in differential conditioning

In two differential conditioning experiments, groups of 10 rats each differed with respect to average reward and schedule of reward received in S+. Nonreward (N) occurred on all S? trials. In both experiments, extinction of responding to S? (resistance to discrimination) was extensively regulated by reward sequence and was largely independent of average reward. In Experiment 1, resistance to discrimination was a function of transitions from N to rewarded (R) trials (N-R transitions). In Experiment 2, resistance to discrimination was increased by large reward on the R trial of N-R transitions and decreased by large reward on the R trial of R-N transitions. These schedule effects on resistance to discrimination parallel the effects of comparable schedules on resistance to extinction following partial reinforcement. The results are discussed in terms of sequential theory, reinforcement level theory, and their implications for various schedule manipulations that have previously shown S? behavior to be inversely related to average reward in S+.     

Long-term reaction of the rat to conspecific (frustration) odor

Rats’ choices in a T-maze were observed to determine (a) their long-term tendencies, in the apparent absence of reinforcement, to approach or avoid a goalbox containing a “frustration odor” stimulus generated by prior placements into the box of other rats given frustrative nonreward, and (b) the extent to which preferences for one or the other goalbox persisted following cessation of odor placements. The initial response to frustration odor was avoidance, though it took a few trials to develop. Avoidance was short-lived for most subjects, diminishing quickly and turning to a stable approach reaction, although three subjects developed an equally stable avoidance. These changes in response direction appeared to result from altered perception or interpretation of the odor rather than changed responsivity or alterations in the odor product itself. Preferences for one or the other of the goalboxes per se were but little affected by experiencing odor there.     

Partial reinforcement delayed extinction effect after regularly alternating reward training

When extinction is delayed very long, the superior resistance to extinction of the random schedule group relative to the alternating schedule group disappears (partial reinforcement delayed extinction effect, PRDE). Two experiments assessed the effects of reinforcement/nonreinforcement on Trial 1 on the PRDE. Following extended partial reinforcement acquisition training in a runway, rats received extinction training after a short (1-day) or long (23-day) retention interval. The schedules used in Experiment 1 were: a single-alternation (SA) schedule beginning each day with a rewarded (r) trial, for Group r-SA; an SA schedule beginning with a nonrewarded (n) trial, for Group n-SA; and a random (Rd) schedule, for Group Rd. The schedules and group names used in Experiment 2 were r-SA, Rd, and r-Rd. The results were that (1) rats given r-SA schedules yielded considerable resistance under delayed extinction, (2) those given Rd and r-Rd schedules showed a decline in resistance to extinction over a long retention interval, (3) those given the n-SA schedule showed relatively low resistance at both retention intervals, although retention deficit was not greater than in the case of the Rd schedule, and thus, (4) the PRDE was found in both experiments, although only weakly in Experiment 1. The results indicated that a regularly alternating reward pattern was a more important determinant than was type of reward on Trial 1 for the PRDE. The PRDE due to differential retention deficits among schedules is discussed on the basis of dual-process associative sequential mechanisms and cognitive rule-encoding mechanisms.     

Durability of the partial reinforcement and partial delay of reinforcement extinction effects after minimal acquisition training

Four groups of 10 rats each were given six acquisition trials (Phase 1) under continuous reinforcement (CR), partial reinforcement (PR), constant delay (CD), or partial delay of reinforcement (PD) conditions. In Phase 2, all Ss were given 18 nonreinforced trials, followed by 12 continuously reinforced trials in Phase 3. In Phase 4, all Ss were given 12 more extinction trials. A constant 24-h ITI was observed throughout the experiment. A strong partial reinforcement extinction effect (PREE) was obtained in both Phases 2 and 4. Only a temporary partial delay of reinforcement effect (PDRE) was observed, which was restricted to the first nine trials of the first extinction phase. No constant delay of reinforcement effect (CDRE) was observed in either extinction phase. The results were discussed in terms of both frustration and sequential theories.     

Successive acquisition and extinction in a runway with the domestic Muscovy duck (Cairina moschata)

Twenty domestic Muscovy ducks were trained to traverse a runway for food reward. Subjects were then randomly assigned to treatments. In one treatment, subjects received six nonrewarded trials followed by six rewarded trials every day for 12 days; and, in the other treatment, subjects received six rewarded trials followed by six nonrewarded trials every day for 12 days. These successive acquisition and extinction (SAE) treatments were selected because different extinction rates on the nonrewarded trials are expected on the bases of previous research performed with rats. Analyses of variance revealed the former treatment yielded significantly greater resistance to extinction than did the latter treatment. It was concluded that ducklings performed similarly to rats in the above SAE situation.     

Retention of a running response following appetitive acquisition,extinction, and discrimination

In Experiment 1, hungry rats received 30 rewarded runway trials and then either extinction trials followed by retention tests or just retention tests. Different groups were tested after retention intervals of 1 min, 1, 3, or 24 h, or 30 days. Retention of extinction training was a nonmonotonic, cubic function of time for the early portion of the response chain, with good retention at 1 min and 3 h and little retention at 1 h, 24 h, or 30 days. In the latter portions of the response chain, retention of extinction decreased monotonically with time. Retention following reward-only training varied little in time, though slight losses occurred after 30 days. Experiments 2–3 differed from Experiment 1 in imposing nonchoice discrimination training (reward vs. nonreward) instead of extinction following 30 rewarded trials. After different time intervals (.017, .75, 1.25, 3, and 24 h in Experiment 1; and .017, 1, and 3 h in Experiment 2), retention tests revealed poorest discrimination at intermediate intervals in the initial portion of the response chain, i.e., a Kamin effect appeared. The deficit seemed the result of a loss of response suppression to the cue that signaled nonreward. In latter segments of the response chain, a Kamin effect tended not to appear. Implications for a number of observations and theoretical views are noted.     

Effects of sucrose concentrations on single-alternation performance in rats

In Experiment 1, rats received single-alternation training with 32% or 4% sucrose reward (Phase 1) followed by a shift in reward from 32% to 4%, and vice versa (Phase 2). In Phase 1, high reward facilitated alternation performance over low reward. In Phase 2, performance on rewarded trials increased as reward increased but was unchanged as reward decreased. Performance on nonrewarded trials showed negligible effects of shifts in reward. In Experiment 2, rats received goalbox placements with 32% or 4% sucrose alternated with nonreward in Phase 1; and in Phase 2, they received alternation runway training with the same or the opposite reward from that of placements. Performance on rewarded trials was faster, the higher the reward in runway training; performance on nonrewarded trials was slower, the higher the reward in placements. In Experiment 3, Phase 1 provided placements with 64%, 32%, 16%, or 4% sucrose or dry mash alternated with nonreward; Phase 2 provided alternation runway training with dry mash reward. Alternation prerformance developed more rapidly, the higher the sucrose concentration in placements. Only 64% sucrose produced performance superior to that for dry-mash placements.     

A behavioral field approach to instrumental learning in the rat: I. Partial reinforcement effects and sex differences

The behavioral field approach employs naturalistic observation and simultaneous, multiple recordings of ecologically relevant aspects of behavior-environment interactions. Applying this approach to runway learning in the rat, the inferior acquisition speed of partial reinforcement (PRF) subjects as compared to continuous reinforcement (CRF) subjects was found to result from greater response variability, more sniffing and goal-avoidance behavior, and slower dropping out of collateral behaviors (e.g., drinking and sand-digging). Extinction first produced an increase in exploratory behavior, then displacement activities (e.g., grooming and biting) and goal-avoidance. CRF subjects showed greater response persistence as measured by number of extinction trials to disrupt an established, favored path. PRF subjects showed greater goal persistence as measured by trials to retrace from goalbox. In extinction, while CRF subjects were more inclined to engage in drinking and sand-digging in the startbox, PRF subjects exhibited more biting behavior in the goalbox. The only sex-related differences were superior speeds by female CRF subjects, inferior goal speeds by female PRF subjects during acquisition, and superior goalbox escape learning by females in extinction.     

Memories of reward events and expectancies of reward events may work in tandem

A common assumption is that expectancies of reward events in instrumental tasks are established on the basis of Pavlovian conditioning. According to the tandem hypothesis, tested in the four runway investigations reported here employing rats, memories of reward events may serve as the conditioned stimuli eliciting expectancies. In Experiments 1–3, rats were trained under a schedule of partial reward (P), which did not produce increased resistance to extinction, and subsequently shifted to consistent reward (C). According to the tandem hypothesis, the shift to the C schedule should result in increased resistance to extinction if, as hypothesized, under the P schedule the memory of reward, S^R, came to elicit the expectancy of nonreward,_EN. This hypothesis was confirmed under a variety of conditions. It was shown that increased resistance to extinction could not be attributed to the P schedule alone, to the rats receiving two schedules, P and C, to stimuli other than S^R eliciting E_N, or to the rats forgetting reward-produced memories when expecting nonreward (Experiment 4). It was shown that the tandem hypothesis could explain the divergent findings obtained in prior studies employing a shift from P to C as well as in the present study.     

A role of stimulus compounds in eliciting responses: Relatively spaced extinction following massed acquisition

One of two schedules of rewarded (R) and nonrewarded (N) trials (RNR vs. RRN) was combined factorially with intertrial interval (ITI) in acquisition (1 vs. 45 min), with extinction occurring at a 45-min ITI. The RNR schedule produced greater resistance to extinction than the RRN schedule regardless of acquisition ITI. The shift in ITI from acquisition to extinction reduced resistance to extinction slightly in the RNR group but not in the RRN group. These findings suggest that there are cues common to 1-min and 45-min ITIs. It was suggested that these ITI-associated cues enter into compound with memories to control instrumental responding.     

Different odors in rats from large and small rewards

Although rats emit different odors in a goalbox upon receipt of unsignaled reward and nonreward, recent studies have failed to find odor differences from unsignaled large and small reward, apparently disconfirming the generalization that frustration-producing manipulations lead to a distinctive emission. Experiment 1 tested the hypothesis that providing discriminative runway signals, permitting subjects to anticipate the large and small rewards, would lead to different odor emissions. Results supported the hypothesis: donor rats emitted odors sufficiently different to provide cues for differential responding in test subjects. However, an attempt to replicate the recent observations on unsignaled rewards suggested that these, too, occasioned goalbox emissions. Therefore, Experiment 2 again gave donors unsignaled large and small reward, but increased the incentive for discrimination in test rats. Test subjects readily discriminated, confirming the conclusion that large and small rewards occasion different emissions even without discriminative signals and supporting a frustration-odor interpretation.     

The acquisition of a running response as a function of odor buildup,squad rotation,and introduction of naive subjects

A three-phase experiment was conducted in which rats received a double-alternation schedule of reward and nonreward. During Phase 1, the baseline period, double-alternation behavior was displayed earlier and more strongly by subjects run last in the daily sequence. This finding suggests that both reward and nonreward odor cues are cumulative over subjects. During Phase 2, a subject-rotation procedure was initiated; that is, each day the last subject in the previous day’s running sequence was moved to the first position in the sequence, etc. Rotation to the first position in the group led to an immediate disruption of responding. During Phase 3, two naive rats were inserted at the beginning of the running sequence and two at the end. The results, which showed that the naive animals placed at the end of the sequence acquired the patterning response much faster than those placed in the beginning positions, are interpreted as reflecting preparedness to respond to such intensified odors.