Hurdle-jump responding in the rat as a function of conspecific odor of reward and nonreward

A hurdle-jump escape response was employed to assess the laboratory rat’s aversion or attraction to different types of conspecific odor. Odorant donor subjects received 112 runway acquisition trials on a continuous reward schedule followed by 32 extinction trials, 112 acquisition trials on a 50% schedule of reward and nonreward followed by 32 extinction trials, or 144 “neutral” trials with no reward in the alley. Different groups of test subjects escaped from odor excreted by odorant subjects on (a) nonrewarded acquisition and extinction trials, (b) rewarded trials during continuous reinforcement, (c) rewarded trials during partial reinforcement, or (d) neutral trials; others escaped from a clean box. The principal findings were: (1) significant aversion to “odor of nonreward” appeared after the donor odorants had received 12 exposures to reward; (2) production of odor of nonreward by odorant subjects changed as a function of training experience with reward; (3) after repeated exposure to odor of nonreward, the escape response habituated; (4) greater or different odor excretion in extinction resulted from subjects trained on a continuous reward schedule than on a partial reward schedule. Relationships of the data to frustration theory were discussed, assuming that inferred differences in production of odor reflect differences in frustration reaction.     

Extinction responding following partial reinforcement: The effects of number of rewarded trials and magnitude of reward

Rats were trained on a daily partial reward schedule of small magnitude of reward (S), nonreward (N), and large magnitude of reward (L), which began with SN or SSNN for all animals. The remainder of the daily schedule was defined by the factorial combination of the number of rewards (1 vs. 3) and the magnitude of reward (S vs. L). Following 18 days of such training, 20 trials of extinction were administered. It was found that increasing the number of rewarded trials in a partial reinforcement schedule decreased resistance to extinction. Furthermore, increased number of large-magnitude rewards reduced resistance to extinction to a greater extent than increased number of small-magnitude rewards.     

Effects of partial vs consistent reward in noncontingent pairings of stimuli with reward: A noncontingently produced positive contrast effect

Rats received pairings of two stimuli with reward noncontingently in the Skinner box. During noncontingent pairings, the bar was immobilized. For Group CC 100% of the presentations of both stimuli were rewarded (S₁ ±, S₂ ±), for Group PP 50% of the presentations of each stimulus were rewarded (S₁, ±, S₂±), and for Group PC one stimulus was followed by reward on 50% of its presentations, while the second stimulus was followed by reward on 100% of its presentations (S₁ ±, S₂ ±). A fourth group received the stimuli and reward nonpaired. In a subsequent rewarded test phase, the response facilitating effects of the stimuli were evaluated. In the test phase all groups that received reward paired with S₁, and S₂ performed better in the presence of S₁ and S₂ than the group for which the stimuli were not paired with reward. For groups that received the stimuli paired with reward, a difference due to schedule of reward occurred when schedule of reward was varied within Ss (Group PC), but not when varied between Ss (Group PP vs Group CC). The specific form of this finding was that Group PC’s performance in the presence of S₂ ± was more vigorous than its performance in the presence of S₁ ± and was more vigorous than the performance of Groups PP and CC to S₂. Group PC’s performance to S₁ ± did not differ from that of Groups PP and CC to S₁.     

Discriminating between reward-produced memories: Effects of differences in reward magnitude

The greater the dissimilarity between exteroceptive stimuli, the easier it is to discriminate between them. To determine whether a similar relationship holds for memories produced by reward events, rats in three runway investigations received trials in pairs, the number of food pellets (0.045 g) occurring on Trial 1 indicating whether reward or nonreward would occur on Trial 2. In each investigation, discriminative responding on Trial 2 was better the larger the difference in reward magnitude on Trial 1. This finding was obtained under a wide variety of conditions: for example, when the larger of two reward magnitudes on Trial 1 signaled nonreward on Trial 2 (Experiment 1, 10 vs. 2 pellets); when the smaller of two reward magnitudes on Trial 1 signaled nonreward on Trial 2 (Experiment 2, 10 vs. 2 pellets); and when the same magnitude of reward on Trial 1 signaled nonreward on Trial 2 (Experiment 3, either 5 pellets or 0 pellets). The findings obtained here indicate that the greater the dissimilarity between reward magnitudes, the greater the dissimilarity between the memories they produced and, thus, the easier it is to discriminate between them. It is suggested that the present results may provide a basis for understanding findings obtained in other instrumental learning investigations in which reward magnitude is varied.     

Effects of varied and partial reward on discrete-trial patterning of rats

Rats were trained in the discrete-trial operant apparatus with single alternation of large reward and small reward or large reward and nonreward. followed by either transfer or extinction. The results showed that both groups acquired appropriate response patterning, that patterning is conditioned, and that the stimuli which control patterning are derived from reward conditions on immediately preceding trials. A modification of the sequential hypothesis of instrumental learning was proposed to account for the results.     

Primary frustration differences following brief partial-reinforcement acquisition under varying magnitudes of reward

In Experiment I, rats which had received six partially reinforced runway acquisition trials, with a reward magnitude of 60 sec access to wet mash on rewarded trials, showed less persistent responding over highly massed extinction trials than subjects which had received the same acquisition schedule but reward magnitudes of either 1 or 10 45-mg pellets. In Experiment II, rats which had received six partially reinforced placements into one compartment of a two-compartment box, with 60 sec access to mash on rewarded placements, jumped a hurdle faster to escape nonreward than subjects which had received the same reward schedule but 10 45-mg pellets on rewarded trials. The data supported a primary frustration analysis for reward-magnitude manipulations within brief partial-reinforcement schedules.     

Response events as well as goal events as sources of animal memory

Current approaches emphasize the control exercised over behavior by various internal stimuli arising from goal events such as reward and nonreward. In contrast, certain earlier views emphasized the behavioral control exercised by internal stimuli arising from responding. The purpose of the three experiments reported here was to determine if stimuli arising both from goal events and from response events control instrumental behavior, and, if so, how much behavioral control is exercised by each. In the three experiments reported, rats received partial reinforcement in a runway either at a 24-h ITI or at both a 24-h ITI and a shorter ITI (2 min, 15 min, 30 min, or 1 h), with extinction always occurring at the shorter ITI. The results suggest that both goal-produced and response-produced internal stimuli control behavior simultaneously. Too, it was found that at ITIs as long as 30 min, response-produced cues regulate responding about as strongly as goal-produced cues.

     

Forming chunks in instrumental learning: The role of overshadowing

Lower order units combined into higher order functional units (e.g., letters into words) are calledchunks. The hypothesis tested here suggests that chunks are formed when memories of goal events signal other goal events, a signal capacity that can be reduced when memories are seriously overshadowed by other more valid cues calledgrouping cues. In support of this hypothesis, it was found, in each of three experiments employing rats in runways, that the capacity of the memory of nonreward to signal reward depended on its validity relative to that of two other situational cues, runway brightness cues and a change in brightness cues, from one trial to the next. Specifically, the capacity of the memory of nonreward to signal reward was much more seriously reduced when it was in competition with more valid situational cues (.33 vs. 1.00, Experiment 1; .50 vs. 1.00, Experiment 2) than when it was in competition with situational cues having either the same validity (.33, Experiment 1) or a slightly higher validity (.33 vs. .50, Experiment 3). It was also found that grouping each of two series was less effective in producing chunking than was grouping only one of the series.     

Partial reinforcement delayed extinction effect after regularly alternating reward training

When extinction is delayed very long, the superior resistance to extinction of the random schedule group relative to the alternating schedule group disappears (partial reinforcement delayed extinction effect, PRDE). Two experiments assessed the effects of reinforcement/nonreinforcement on Trial 1 on the PRDE. Following extended partial reinforcement acquisition training in a runway, rats received extinction training after a short (1-day) or long (23-day) retention interval. The schedules used in Experiment 1 were: a single-alternation (SA) schedule beginning each day with a rewarded (r) trial, for Group r-SA; an SA schedule beginning with a nonrewarded (n) trial, for Group n-SA; and a random (Rd) schedule, for Group Rd. The schedules and group names used in Experiment 2 were r-SA, Rd, and r-Rd. The results were that (1) rats given r-SA schedules yielded considerable resistance under delayed extinction, (2) those given Rd and r-Rd schedules showed a decline in resistance to extinction over a long retention interval, (3) those given the n-SA schedule showed relatively low resistance at both retention intervals, although retention deficit was not greater than in the case of the Rd schedule, and thus, (4) the PRDE was found in both experiments, although only weakly in Experiment 1. The results indicated that a regularly alternating reward pattern was a more important determinant than was type of reward on Trial 1 for the PRDE. The PRDE due to differential retention deficits among schedules is discussed on the basis of dual-process associative sequential mechanisms and cognitive rule-encoding mechanisms.     

Replacement of event-generated memories of nonreinforcement with signal-generated memories of reinforcement during partial reinforcement training: Effect on resistance to extinction

Event-generated memory refers to the memory of a reinforcement (R) or nonreinforcement (N) event from an immediately preceding trial;signal-generated memory refers to the memory of a temporally remote R or N, retrieval of which is generated by presentation of a signal with which the memory is associated (Haggbloom, 1988). In each of three experiments, Group Signal-R received runway discrimination training in Phase 1 to establish a stimulus as a signal for R, and partial reinforcement training in Phase 2. An extinction test measured learning about the memory of nonreward (S^N)—learning that occurs when S^N is retrieved on R trials that follow N trials. In Group Signal-H, those R trials were accompanied by the signal for R, a treatment we hypothesized would generate retrieval of the memory of reinforcement (S^R) so that signal-generated S^R would replace event-generated S^N as the operative memory, thereby eliminating the increased resistance to extinction normally produced by PRF training. In each experiment, Group Signal-R was less resistant to extinction than was a control group conditioned to respond to-event-generated S^N. Extinction was as rapid in Group Signal-R as it was in a consistent reinforcement control group (Experiment 1) and in a group given intertrial reinforcements to interfere with learning about S^N (Experiment 3). Experiment 2 tested two alternative interpretations of the failure to learn about S^N in Group Signal-R. Those alternatives were found to be less viable than the hypothesis that the signal for R actively recruited retrieval of a competing memory.     

Partial reinforcement effect: The expectancy of reward on nonreward trials

In order to determine the importance of the development of expectancy of reward prior to partial reward trials; rats were given 20 continuously reinforced trials prior to 20 partially reinforced trials (CRF-PRF) and compared to Ss given only 20 partially reinforced trials (PRF). Control groups received 20 or 40 continuously reinforced trials (CRF-20, CRF-40) to determine the effect of differing numbers of acquisition trials. Results showed that terminal acquisition differences were minimal in the run segment of the alley and that Group CRF-PRF was more resistant to extinction than Group PRF, and both were more resistant to extinction than the CRF-20 and CRF-40 groups, which did not differ from each other. These results were interpreted as supporting the notion that the expectancy of reward on nonreward trials during partial reinforcement acquisition is a determiner of the magnitude of the partial reinforcement extinction effect.     

Approach and escape to conspecific odors of reward and nonreward in rats

A three-compartment box was used, and a reward odor, or nonreward (extinction) odor, produced by another rat, was present in the middle compartment. Two control odor procedures were also used. The results showed that rats will approach a location in which another rat has previously been given reward more rapidly than they will escape from that location, but showed the opposite effect when the odor was produced by a rat undergoing extinction. The mere presence of an odor associated with another rat had the effect of producing much slower locomotion as compared to a no-odor control condition.     

Effects of intertrial reward and/or nonreward placements during a 20-second intertrial interval on single-alternation training in rats

Seventy male hooded rats received single-alternation runway training in which goalbox placements were interpolated during the 20-sec intertrial interval. Placements provided alternating reward/ nonreward, random reward/nonreward, continuous reward, or continuous nonreward. Relative to nonplaced controls, alternation performance was reliably facilitated only by intertrial alternation placements which re-presented the goal event of each immediately preceding instrumental trial. All other intertrial procedures reliably impaired alternation performance. Degree of impairment was graded from least to most as follows: intertrial alternation placements with the goal event opposite to that of each immediately preceding instrumental trial, intertrial placements with continuous non-reward, intertrial placements with continuous reward, and intertrial placements with random reward/nonreward.     

Instrumental performance and time between reinforcements: Intimate relation to learning or memory retrieval?

According to scalar expectancy theory (SET), instrumental performance is determined by the ratio of the time between reinforcements in the trial (T _T) to the overall time between reinforcements (T _O). Groups for which theT _O|T _T ratio is the same should perform similarly. According to the sequential-memory view, the memory of nonreward becomes a signal for reward, and thereby promotes strong responding, when that memory is retrieved on a reward trial. In each of three runway investigations employing rats in a runway, two groups were compared that had the sameT _O |T _T ratio but that differed in the tendency to retrieve the memory of nonreward on a rewarded trial. In each investigation faster running on critical nonrewarded trials was associated with the group having the stronger tendency to retrieve the memory of nonreward on a rewarded trial. These findings are consistent with the predictions of the sequential-memory view, as well as with certain earlier findings, but are inconsistent with SET. It was indicated that the groups compared here were matched along a considerable number of dimensions—an unprecedented number for a varied reward investigation.     

Reward sequence and reinforcement level as determinants of S− behavior in differential conditioning

In two differential conditioning experiments, groups of 10 rats each differed with respect to average reward and schedule of reward received in S+. Nonreward (N) occurred on all S? trials. In both experiments, extinction of responding to S? (resistance to discrimination) was extensively regulated by reward sequence and was largely independent of average reward. In Experiment 1, resistance to discrimination was a function of transitions from N to rewarded (R) trials (N-R transitions). In Experiment 2, resistance to discrimination was increased by large reward on the R trial of N-R transitions and decreased by large reward on the R trial of R-N transitions. These schedule effects on resistance to discrimination parallel the effects of comparable schedules on resistance to extinction following partial reinforcement. The results are discussed in terms of sequential theory, reinforcement level theory, and their implications for various schedule manipulations that have previously shown S? behavior to be inversely related to average reward in S+.     

Resistance to extinction as a function of percentage of reward: A reinforcement-level interpretation

Three experiments investigated the effects of percentage of reinforcement on the resistance to extinction of an instrumental running response. In Experiment 1, with N-length held constant, 47% reinforcement during acquisition generated greater resistance to extinction (R_n) than did 77%. In Experiment 2, this result was replicated with both functional N-length and number of N-R transitions held constant. In Experiment 3, R_n was shown to be a function of both N-length and percentage of reinforcement. The results of all three experiments were discussed in terms of Capaldi’s reinforcement level theory and possible alternative explanations.     

Associatively activated representations of food events resemble food outcome expectancies more closely than they resemble food-based memories

When Pavlovian stimuli activate representations of food, do these representations resemble memories of food consumed in the recent past or expectancies of food that is imminent? In Experiments 1A and 1B, this question was addressed by training pigeons on a symbolic matching-to-sample task involving different grains as memory cues or as expectancy cues for correct choices. Autoshaping trials involving these same grains were interspersed among matching-to-sample trials, as were test trials involving the substitution of autoshaping stimuli for cues in the matching-to-sample task. Control over choices transferred to autoshaping stimuli in both experiments, suggesting that associatively activated representations of food resemble both memories and expectancies. In Experiment 2, pigeons were trained on a symbolic matching-to-sample task in which food and no-food memory cues (i.e., the samples) were juxtaposed with no-food and food expectancy cues. Subsequently, autoshaping stimuli, which activated representations of food and no food, were substituted for the samples. Choices by the pigeons indicated that associatively activated representations of food-related events resemble expectancies more closely than they do memories.     

The acquisition of a running response as a function of odor buildup,squad rotation,and introduction of naive subjects

A three-phase experiment was conducted in which rats received a double-alternation schedule of reward and nonreward. During Phase 1, the baseline period, double-alternation behavior was displayed earlier and more strongly by subjects run last in the daily sequence. This finding suggests that both reward and nonreward odor cues are cumulative over subjects. During Phase 2, a subject-rotation procedure was initiated; that is, each day the last subject in the previous day’s running sequence was moved to the first position in the sequence, etc. Rotation to the first position in the group led to an immediate disruption of responding. During Phase 3, two naive rats were inserted at the beginning of the running sequence and two at the end. The results, which showed that the naive animals placed at the end of the sequence acquired the patterning response much faster than those placed in the beginning positions, are interpreted as reflecting preparedness to respond to such intensified odors.     

The effects of reinforcement frequency on performance in a multiple fixed-ratio schedule

A barpress analog to the double-alley runway was sought by varying percentage reward in the first of two consecutive FR 18s. Groups of six rats each were given 0% 50%. or 100% reinforcement upon completion of the first FR 18: after a 5-sec midtnal imterval, the second FR 18 was administered on a separate lever and all groups received CRF reward upon its completion. Group 50 Ss performed faster after nonreward than after reward. Group 50 Ss performed faster after nonreward than did 0% Ss. A measure of midtnal behavior revealed a difference between groups in orienting to the bars. When all groups were shifted to a 50% first component schedule (Phase II), there were no statistically reliable effects of prior reinforcement history on rewarded or nonrewarded responding. The Phase 1 results were taken to demonstrate a frustration effect similar to that of the double alley     

Effects of signaled reward in instrumental conditioning: Enhanced learning on DRL and DRH schedules of reinforcement

The effects of signaled reward were examined using DRL and DRH schedules of reinforcement. In each case, one group of rats received a brief cue between the reinforced response and the reward, and a second group received brief cues at random times. With the DRL schedule (Experiment 1), signaled reward decreased response rate, increased response efficiency (number of responses per reinforcer), and increased resistence to satiation relative to the control group. With the DRH schedule (Experiment 2), signaled reward increased response rate, efficiency, and resistance to satiation. These results refute an overshadowing explanation of the effects of signaled reward and suggest that food-correlated cues enhance learning of the reinforcement contingencies.