Discriminating between reward-produced memories: Effects of differences in reward magnitude

The greater the dissimilarity between exteroceptive stimuli, the easier it is to discriminate between them. To determine whether a similar relationship holds for memories produced by reward events, rats in three runway investigations received trials in pairs, the number of food pellets (0.045 g) occurring on Trial 1 indicating whether reward or nonreward would occur on Trial 2. In each investigation, discriminative responding on Trial 2 was better the larger the difference in reward magnitude on Trial 1. This finding was obtained under a wide variety of conditions: for example, when the larger of two reward magnitudes on Trial 1 signaled nonreward on Trial 2 (Experiment 1, 10 vs. 2 pellets); when the smaller of two reward magnitudes on Trial 1 signaled nonreward on Trial 2 (Experiment 2, 10 vs. 2 pellets); and when the same magnitude of reward on Trial 1 signaled nonreward on Trial 2 (Experiment 3, either 5 pellets or 0 pellets). The findings obtained here indicate that the greater the dissimilarity between reward magnitudes, the greater the dissimilarity between the memories they produced and, thus, the easier it is to discriminate between them. It is suggested that the present results may provide a basis for understanding findings obtained in other instrumental learning investigations in which reward magnitude is varied.     

Ordinal position in the serial learning of rats

Rats were runway trained on each of two, three-trial series consisting of different varieties of reward (X, Y, and Z) and nonreward (N) serving as trial outcomes. The two series are represented as XNY and ZNN. Distinguishing the two series were different brightness and texture cues on the runway floor. Transfer tests, conducted after the rats had developed faster running for rewarded trials than for nonrewarded trials and slower running on Trial 2 of ZNN than on Trial 2 of XNY, provided evidence that trial position, rather than item memories, was controlling the discriminations. In Experiment 1, reversing the floor cues completely reversed the discriminations. In Experiment 2, transfer to NNN did not change the routine patterns of approach that had been established.     

Reward magnitude and shock variables (continuity and intensity) in shuttlebox-avoidance learning

In Experiment I, eight groups of rats (n = 20) were given shuttlebox-avoidance training. Two levels of shock (.3 and 1.6 mA) were combined factorially with two levels of reward (large and small) under both continuous and discontinuous (.75 sec on and 2.00 sec off) shock. Visual situational cues were absent after a shuttle response for the large-reward condition and present for the small-reward condition. Superior performance was obtained with weak rather than strong shock under both reward conditions and with large rather than small reward only under the weak-shock condition. Continuity of shock had no differential effect on performance. Experiment II allowed the conclusion that the reward effect was attributable to a reinforcement mechanism. The data were taken as support for the effective reinforcement theory, which emphasizes the importance in avoidance learning of fear conditioned to situational cues.     

Memories of reward events and expectancies of reward events may work in tandem

A common assumption is that expectancies of reward events in instrumental tasks are established on the basis of Pavlovian conditioning. According to the tandem hypothesis, tested in the four runway investigations reported here employing rats, memories of reward events may serve as the conditioned stimuli eliciting expectancies. In Experiments 1–3, rats were trained under a schedule of partial reward (P), which did not produce increased resistance to extinction, and subsequently shifted to consistent reward (C). According to the tandem hypothesis, the shift to the C schedule should result in increased resistance to extinction if, as hypothesized, under the P schedule the memory of reward, S^R, came to elicit the expectancy of nonreward,_EN. This hypothesis was confirmed under a variety of conditions. It was shown that increased resistance to extinction could not be attributed to the P schedule alone, to the rats receiving two schedules, P and C, to stimuli other than S^R eliciting E_N, or to the rats forgetting reward-produced memories when expecting nonreward (Experiment 4). It was shown that the tandem hypothesis could explain the divergent findings obtained in prior studies employing a shift from P to C as well as in the present study.     

Retention of a running response following appetitive acquisition,extinction, and discrimination

In Experiment 1, hungry rats received 30 rewarded runway trials and then either extinction trials followed by retention tests or just retention tests. Different groups were tested after retention intervals of 1 min, 1, 3, or 24 h, or 30 days. Retention of extinction training was a nonmonotonic, cubic function of time for the early portion of the response chain, with good retention at 1 min and 3 h and little retention at 1 h, 24 h, or 30 days. In the latter portions of the response chain, retention of extinction decreased monotonically with time. Retention following reward-only training varied little in time, though slight losses occurred after 30 days. Experiments 2–3 differed from Experiment 1 in imposing nonchoice discrimination training (reward vs. nonreward) instead of extinction following 30 rewarded trials. After different time intervals (.017, .75, 1.25, 3, and 24 h in Experiment 1; and .017, 1, and 3 h in Experiment 2), retention tests revealed poorest discrimination at intermediate intervals in the initial portion of the response chain, i.e., a Kamin effect appeared. The deficit seemed the result of a loss of response suppression to the cue that signaled nonreward. In latter segments of the response chain, a Kamin effect tended not to appear. Implications for a number of observations and theoretical views are noted.     

Effects of sucrose concentrations on single-alternation performance in rats

In Experiment 1, rats received single-alternation training with 32% or 4% sucrose reward (Phase 1) followed by a shift in reward from 32% to 4%, and vice versa (Phase 2). In Phase 1, high reward facilitated alternation performance over low reward. In Phase 2, performance on rewarded trials increased as reward increased but was unchanged as reward decreased. Performance on nonrewarded trials showed negligible effects of shifts in reward. In Experiment 2, rats received goalbox placements with 32% or 4% sucrose alternated with nonreward in Phase 1; and in Phase 2, they received alternation runway training with the same or the opposite reward from that of placements. Performance on rewarded trials was faster, the higher the reward in runway training; performance on nonrewarded trials was slower, the higher the reward in placements. In Experiment 3, Phase 1 provided placements with 64%, 32%, 16%, or 4% sucrose or dry mash alternated with nonreward; Phase 2 provided alternation runway training with dry mash reward. Alternation prerformance developed more rapidly, the higher the sucrose concentration in placements. Only 64% sucrose produced performance superior to that for dry-mash placements.     

Preexposure to situational cues produces a direct relationship between two-way avoidance learning and shock intensity

In Experiment 1, four groups of subjects (n = 16 each) were exposed to the situational stimuli of a shuttlebox apparatus for 4 h. Subsequently, 200 two-way avoidance trials were administered (100/day) with either .3- or 1.6-mA shock and with either small or large reward (presence or absence of visual stimuli following the response). Avoidance performance was directly related to shock intensity on both days and to magnitude of reward on the 2nd day. In Experiment 2, four groups of subjects (n = 24 each) were given 4 h of exposure either to the situational stimuli of the shuttlebox or to a neutral box. Then, 10 two-way avoidance trials were given with 1.6-mA shock. Subsequently, subjects were allowed to escape from one of the shuttlebox compartments to an adjacent safe box. Following preexposure to situational stimuli, avoidance performance was superior whereas escape-from-fear performance was inferior. This latter finding demonstrated that less fear of situational cues was present during avoidance training in the preexposed condition. All of these results support the effective reinforcement theory, an extension of two-factor theory, which emphasizes the importance for avoidance learning of the amount of fear of situational cues present following a response.     

Different odors in rats from large and small rewards

Although rats emit different odors in a goalbox upon receipt of unsignaled reward and nonreward, recent studies have failed to find odor differences from unsignaled large and small reward, apparently disconfirming the generalization that frustration-producing manipulations lead to a distinctive emission. Experiment 1 tested the hypothesis that providing discriminative runway signals, permitting subjects to anticipate the large and small rewards, would lead to different odor emissions. Results supported the hypothesis: donor rats emitted odors sufficiently different to provide cues for differential responding in test subjects. However, an attempt to replicate the recent observations on unsignaled rewards suggested that these, too, occasioned goalbox emissions. Therefore, Experiment 2 again gave donors unsignaled large and small reward, but increased the incentive for discrimination in test rats. Test subjects readily discriminated, confirming the conclusion that large and small rewards occasion different emissions even without discriminative signals and supporting a frustration-odor interpretation.     

Conditions that potentiate the effects of electroconvulsive shock administered 24 hours after avoidance training

Three experiments assessed the conditions that potentiate effects of an electroconvulsive shock (ECS) administered 24 h after avoidance training. Stimuli present immediately prior to the ECS were systematically varied. In Experiment 1, which employed a passive avoidance task, the primary determinant of whether the ECS disrupted retention was whether the situational cues present at the time of ECS delivery were those associated with the initial training experience: ECS disrupted performance only when it was administered in the original training apparatus, regardless of whether or not a footshock was presented immediately prior to ECS. In Experiment 2, which employed an active, shuttlebox avoidance task, both the situational cues from the training apparatus and a footshock were necessary to potentiate the disruptive effects of the ECS. Experiment 3 revealed that ECS effects on performance of the active avoidance task can also be potentiated by a combination of apparatus cues and the warning signal used in initial training. These results are interpreted as indicating that informational functions of stimuli present when an ECS is administered are important determinants of the effects of the ECS.     

The acquisition of a running response as a function of odor buildup,squad rotation,and introduction of naive subjects

A three-phase experiment was conducted in which rats received a double-alternation schedule of reward and nonreward. During Phase 1, the baseline period, double-alternation behavior was displayed earlier and more strongly by subjects run last in the daily sequence. This finding suggests that both reward and nonreward odor cues are cumulative over subjects. During Phase 2, a subject-rotation procedure was initiated; that is, each day the last subject in the previous day’s running sequence was moved to the first position in the sequence, etc. Rotation to the first position in the group led to an immediate disruption of responding. During Phase 3, two naive rats were inserted at the beginning of the running sequence and two at the end. The results, which showed that the naive animals placed at the end of the sequence acquired the patterning response much faster than those placed in the beginning positions, are interpreted as reflecting preparedness to respond to such intensified odors.     

Temporal cuing of runs in series of reward events reduces interevent anticipation

In Experiment 1, a group of rats were runway trained on each of two reward series for 32 days. The two series consisted of three runs, the first two of which were, respectively, rewarded and nonrewarded; the third run was rewarded in one series but nonrewarded in the other. A 40-min interval separated the two series; the first and second runs within the series were separated by a 10-min interval, whereas the second and third runs were separated by a 30-sec interval. The reward (and nonreward) events and temporal cues of the two series are designated R-NR/R-NN. A second group was similarly trained, with the exception that the 10-min interval separated the second and third runs (RN-R/RN-N). Both groups developed appropriate differential running on the third run of the two series, and the RN-R/RN-N animals ran appropriately (slowly) on the second run of both series. Appropriate Run 2 performance appeared in one half of the R-NR/R-NN animals (depending upon order of series presentation); the remaining half ran faster on Run 2 of the R-NR series than on the same run of the R-NN series, an effect currently termed interevent anticipation. A cue shift phase in which all within-series intervals were 30 sec showed that the temporal intervals were controlling performance before the shift. Experiment 2 showed that interevent anticipation appears when all within-series intervals are either 10 min or 30 sec from the beginning of training, suggesting that the elimination of interevent anticipation in Experiment 1 was due to the differential cuing of runs by the temporal intervals rather than the particular interval duration. The overall findings suggest that the similarity of Run 2 and Run 3 performance termed interevent anticipation may be due to a failure to discriminate the ordinal position of runs within a series.     

Preference for food and water in rats as a function of delay of reward

In Experiment I, rats which were both hungry and thirsty were given a choice between a food reward and a water reward. The animals preferred food to water when the reward was delivered immediately, but preferred water to food when a 30-sec delay was imposed in the goalbox before the reward was received. Experiment II replicated the results of the first experiment and showed, in addition, that when the delay was imposed in a separate delay chamber devoid of differential goalbox cues, subjects preferred food to water, similar to the immediate group. The results were discussed in terms of an incentive value process and a competing response hypothesis.     

Instrumental performance and time between reinforcements: Intimate relation to learning or memory retrieval?

According to scalar expectancy theory (SET), instrumental performance is determined by the ratio of the time between reinforcements in the trial (T _T) to the overall time between reinforcements (T _O). Groups for which theT _O|T _T ratio is the same should perform similarly. According to the sequential-memory view, the memory of nonreward becomes a signal for reward, and thereby promotes strong responding, when that memory is retrieved on a reward trial. In each of three runway investigations employing rats in a runway, two groups were compared that had the sameT _O |T _T ratio but that differed in the tendency to retrieve the memory of nonreward on a rewarded trial. In each investigation faster running on critical nonrewarded trials was associated with the group having the stronger tendency to retrieve the memory of nonreward on a rewarded trial. These findings are consistent with the predictions of the sequential-memory view, as well as with certain earlier findings, but are inconsistent with SET. It was indicated that the groups compared here were matched along a considerable number of dimensions—an unprecedented number for a varied reward investigation.     

Serial learning,interitem associations,phrasing cues,interference, overshadowing,chunking, memory,and extinction

Two experiments indicated that two approaches to serial learning are too extreme—the classical view that it consists only of interitem associations and various recent views that it involves no interitem associations. The novel assumption introduced here was that phrasing cues, normally conceptualized as merely segregating long series into smaller units or chunks, may also enter into associations with items, thereby reducing interitem interference and facilitating serial learning. It was found that one item could become a signal for another item, an interitem association, or be overshadowed by a phrasing cue, such as a brightness and temporal cue, also signaling that item. The items were .045-g pellets. Rats traversed a runway for items arranged in ordered series, 14-7-3-1-0 pellets (Experiment 1) or 10-2-0-10 (Experiment 2). Complete tracking of, for example, the 10-2-0-10 series would consist of fastest running to 10 pellets and slowest running to 0 pellets. In both investigations, the interitem association overshadowed was that between 0 pellets and the subsequent rewarded item, 0 → 14 (Experiment 1) or 0 → 10 (Experiment 2). Either repetitions of the 14-7-3-1-0 subpattern (Experiment 1) or merely the terminal 10-pellet item (Experiment 2) were phrased, both methods producing identical results. Overshadowing the 0-pellet item produced superior serial learning, more rapid extinction, and, in Experiment 1, considerable elevation of responding when the brightness phrasing cue was introduced in extinction, an effect said to be conceptually identical to spontaneous recovery and one demonstrating directly that phrasing cues are in reality overshadowing cues. It was suggested that many effects attributed to forgetting may be due to unrecognized overshadowing of memory cues by phrasing cues, giving rise to exaggerated estimates of forgetting.     

Hurdle-jump responding in the rat as a function of conspecific odor of reward and nonreward

A hurdle-jump escape response was employed to assess the laboratory rat’s aversion or attraction to different types of conspecific odor. Odorant donor subjects received 112 runway acquisition trials on a continuous reward schedule followed by 32 extinction trials, 112 acquisition trials on a 50% schedule of reward and nonreward followed by 32 extinction trials, or 144 “neutral” trials with no reward in the alley. Different groups of test subjects escaped from odor excreted by odorant subjects on (a) nonrewarded acquisition and extinction trials, (b) rewarded trials during continuous reinforcement, (c) rewarded trials during partial reinforcement, or (d) neutral trials; others escaped from a clean box. The principal findings were: (1) significant aversion to “odor of nonreward” appeared after the donor odorants had received 12 exposures to reward; (2) production of odor of nonreward by odorant subjects changed as a function of training experience with reward; (3) after repeated exposure to odor of nonreward, the escape response habituated; (4) greater or different odor excretion in extinction resulted from subjects trained on a continuous reward schedule than on a partial reward schedule. Relationships of the data to frustration theory were discussed, assuming that inferred differences in production of odor reflect differences in frustration reaction.     

Analysis of alternation patterning in goldfish

In Experiment 1, goldfish trained with alternation of reward (R) and nonreward (N) for responding to a single color gave clear evidence of patterning (more rapid responding on R than on N trials). In Experiment 2, patterning was found for each of two colors alternately rewarded and nonrewarded in the sequence blue R, yellow R, blue N, yellow N, …. Changes in performance with subsequent changes in the sequence of the two colors suggested that the patterning was based on carryover rather than on associative memory of R and N.     

Extinction responding following partial reinforcement: The effects of number of rewarded trials and magnitude of reward

Rats were trained on a daily partial reward schedule of small magnitude of reward (S), nonreward (N), and large magnitude of reward (L), which began with SN or SSNN for all animals. The remainder of the daily schedule was defined by the factorial combination of the number of rewards (1 vs. 3) and the magnitude of reward (S vs. L). Following 18 days of such training, 20 trials of extinction were administered. It was found that increasing the number of rewarded trials in a partial reinforcement schedule decreased resistance to extinction. Furthermore, increased number of large-magnitude rewards reduced resistance to extinction to a greater extent than increased number of small-magnitude rewards.     

Differential Effects of Salient Visual Events on Memory-Guided Attention in Adults and Children

Both salient visual events and scene-based memories can influence attention, but it is unclear how they interact in children and adults. In Experiment 1, children (N = 27; ages 7–12) were faster to discriminate targets when they appeared at the same versus different location as they had previously learned or as a salient visual event. In contrast, adults (N = 30; ages 18–31) responded faster only when cued by visual events. While Experiment 2 confirmed that adults (N = 27) can use memories to orient attention, Experiment 3 showed that, even in the absence of visual events, the effects of memories on attention were larger in children (N = 27) versus adults (N = 28). These findings suggest that memories may be a robust source of influence on children's attention.     

Effects of stimulus change in prepunishment alley segments on self-punitive behavior

Three experiments investigated the effects of stimulus change during extinction on self-punitive behavior. In Experiment 1, changing alley brightness cues in all three segments of the alley prior to extinction eliminated self-punitive behavior. That is, subjects given shock in the third alley segment during extinction did not differ from nonshocked subjects in alley speed or in the number of trials to extinction. In Experiment 2, with shock also administered in the third alley segment, self-punitive behavior was eliminated when the stimulus change was made in segment 1 or in segments 1 and 2 but was obtained when the change occurred in segment 2 or in the lower startbox. In Experiment 3, shock was administered in the second alley segment. Self-punitive behavior was not obtained when the lower startbox cues were changed but was obtained with stimulus change in the upper startbox or in segment 1. The results are consistent with an expanded version of the Mowrer-Brown conditioned-fear hypothesis.     

Interaction between conflicting memories in the rat: Contextual pretest cuing reverses control of behavior by testing context

Two experiments were run in order to investigate the influence of external contextual cues on the interaction between two conflicting memories. Rats were trained, in the same apparatus, on a passive-avoidance task and, 10 min later, on an active-avoidance task, then submitted to a 24-h delayed test (without reinforcement). When contextual cues remained unchanged throughout these three phases, the animals exhibited proactive interference, as shown by longer response latencies than those of control animals that had learned only the active-avoidance task (Experiment 1). When training contexts were made slightly different from one task to the other (by the presence of a continuous tone during either the first or the second task), the animals behaved at subsequent testing in accordance with the response contingencies present in the context exactly similar to the test context; this control of behavior by testing context was demonstrated both when the tone was absent (Experiments 1 and 2) and present (Experiment 2) at testing. A cuing procedure—a 90-sec exposure to the tone in the experimental room 5 min prior to testing—led the animals to behave in accordance with the response previously acquired in the presence of the tone (Experiments 1 and 2), exactly in the same way as animals tested in the presence of the tone (Experiment 2). The same cuing treatment was ineffective when administered 1 h before testing (Experiment 2). These results are interpreted in terms of a dual function of contextual cues at the time of retrieval: the general contextual information present during testing or during pretest cuing is assumed to induce concurrent reactivation of both memories. Consequently, the experimentally manipulated contextual cue (the tone) would have a discriminative function, leading the animals to choose between the two equally available representations.