Effects of partial vs consistent reward in noncontingent pairings of stimuli with reward: A noncontingently produced positive contrast effect

Rats received pairings of two stimuli with reward noncontingently in the Skinner box. During noncontingent pairings, the bar was immobilized. For Group CC 100% of the presentations of both stimuli were rewarded (S₁ ±, S₂ ±), for Group PP 50% of the presentations of each stimulus were rewarded (S₁, ±, S₂±), and for Group PC one stimulus was followed by reward on 50% of its presentations, while the second stimulus was followed by reward on 100% of its presentations (S₁ ±, S₂ ±). A fourth group received the stimuli and reward nonpaired. In a subsequent rewarded test phase, the response facilitating effects of the stimuli were evaluated. In the test phase all groups that received reward paired with S₁, and S₂ performed better in the presence of S₁ and S₂ than the group for which the stimuli were not paired with reward. For groups that received the stimuli paired with reward, a difference due to schedule of reward occurred when schedule of reward was varied within Ss (Group PC), but not when varied between Ss (Group PP vs Group CC). The specific form of this finding was that Group PC’s performance in the presence of S₂ ± was more vigorous than its performance in the presence of S₁ ± and was more vigorous than the performance of Groups PP and CC to S₂. Group PC’s performance to S₁ ± did not differ from that of Groups PP and CC to S₁.     

Learning in honeybees as a function of amount and frequency of reward

In a series of four experiments with free-flying honeybees, individual foragers were trained with targets of two different colors that contained 5 or 20 μl of 50% sucrose solution. The two targets were singly presented in quasi-random sequences on each visit, with the amount of reward to be found on each target perfectly predictable from its color. The number of training visits (4–32) was varied both within and between experiments, and so also was the relative frequency of trials with the 5- and 20-μl targets (1:1, 2:1, 3:1, and 9:1). At the conclusion of training under each condition, unrewarded responses to the targets were measured in a 10-min extinction test, with the targets presented either separately to two different groups of animals (Experiment 1) or as a pair (Experiments 2–4). When the number of training trials with each target was the same (Experiments 1 and 2), the animals responded more in extinction to the 20-μl target than to the 5-μl target, although there was a decline in the overall level of responding to both targets (an overlearning-extinction effect) as the number of training trials increased. After nine times as many, or only three times as many, training trials with the5- μl target as with the 20-μl target, the animals responded more in extinction to the 5-μl target (Experiment 3); after twice as many training trials with the 5-μl target as with the 20-μl target, there was equal responding to both (Experiment 4). The preferences shown in the choice tests of Experiments 2–4 could be simulated rather accurately on the assumptions of a model previously developed to deal with the discrete-trials choice behavior of honeybees and the further assumption that associative strength grows at a rate increasing with amount of reward to an asymptote independent of amount of reward.     

Effects of sucrose concentrations on single-alternation performance in rats

In Experiment 1, rats received single-alternation training with 32% or 4% sucrose reward (Phase 1) followed by a shift in reward from 32% to 4%, and vice versa (Phase 2). In Phase 1, high reward facilitated alternation performance over low reward. In Phase 2, performance on rewarded trials increased as reward increased but was unchanged as reward decreased. Performance on nonrewarded trials showed negligible effects of shifts in reward. In Experiment 2, rats received goalbox placements with 32% or 4% sucrose alternated with nonreward in Phase 1; and in Phase 2, they received alternation runway training with the same or the opposite reward from that of placements. Performance on rewarded trials was faster, the higher the reward in runway training; performance on nonrewarded trials was slower, the higher the reward in placements. In Experiment 3, Phase 1 provided placements with 64%, 32%, 16%, or 4% sucrose or dry mash alternated with nonreward; Phase 2 provided alternation runway training with dry mash reward. Alternation prerformance developed more rapidly, the higher the sucrose concentration in placements. Only 64% sucrose produced performance superior to that for dry-mash placements.     

On the effects of training inductive reasoning: How far does it transfer and how long do the effects persist?

Using the same program, two training experiments have been conducted in a Dutch and in a German elementary school. The common expectation was that training in inductive reasoning would transfer both on intelligence tests measuring inductive reasoning and on math performance. Furthermore, it was expected that the training effects would persist for at least some months after training had ended. In experiment 1 (N=34), a rather short training period turned out to be effective with respect to the intelligence test performance but not with respect to math performance. In experiment 2 (N=23), the amount of training in inductive reasoning was systematically varied. It could be shown that transfer on intelligence test as well as on math performance was linearly dependent on the amount of prior training. The training effects were found to persist between four and nine months after training.     

Preexposure to situational cues produces a direct relationship between two-way avoidance learning and shock intensity

In Experiment 1, four groups of subjects (n = 16 each) were exposed to the situational stimuli of a shuttlebox apparatus for 4 h. Subsequently, 200 two-way avoidance trials were administered (100/day) with either .3- or 1.6-mA shock and with either small or large reward (presence or absence of visual stimuli following the response). Avoidance performance was directly related to shock intensity on both days and to magnitude of reward on the 2nd day. In Experiment 2, four groups of subjects (n = 24 each) were given 4 h of exposure either to the situational stimuli of the shuttlebox or to a neutral box. Then, 10 two-way avoidance trials were given with 1.6-mA shock. Subsequently, subjects were allowed to escape from one of the shuttlebox compartments to an adjacent safe box. Following preexposure to situational stimuli, avoidance performance was superior whereas escape-from-fear performance was inferior. This latter finding demonstrated that less fear of situational cues was present during avoidance training in the preexposed condition. All of these results support the effective reinforcement theory, an extension of two-factor theory, which emphasizes the importance for avoidance learning of the amount of fear of situational cues present following a response.     

Learning in honeybees as a function of amount of reward: Tests of the equal-asymptote assumption

In Experiments 1 and 2, honeybee foragers visiting the laboratory were fed on targets of two different colors, one containing 5 μl and the other containing 20 μl of 50% sucrose solution. The targets were presented singly in quasi-random sequences on the training visits, after which preference was measured in an unrewarded choice test. In Experiment 1, 16 differentially rewarded training trials with each color were followed by the same number of trials with the color-amount relation reversed; no preference for either color was found in the subsequent choice test. In Experiment 2, 20 differentially rewarded training trials with each color—enough to produce a clear preference for the 20-μl color when given directly after pretraining—were given after 10 feedings to repletion on each color that were calculated to generate near-asymptotic associative strength; no preference for either color was found in the subsequent choice test. In Experiment 3, there were 12 feedings to repletion on one color and, on the other, 12 feedings to repletion followed by 15 trials with a small (5 μl) reward; no preference was found in a subsequent choice test. The results of all three experiments support a nonrepresentational interpretation of the role of amount of reward in the learning of honeybees.     

General transfer across sensory modalities survives reductions in the original conditioned reflex in the rabbit

Two experiments demonstrated that transfer of training between CSs from different sensory modalities survived substantial reductions in responding to the first CS. In both experiments, animals received three stages of training. Stage 1 entailed CS-US training with a CS from one modality (e.g., light), and Stage 3 entailed CS-US training with a CS from another modality (e.g., tone). The experiments differed in treatment during Stage 2. In Experiment 1, animals either remained in their home cages or received unreinforced exposures to the first CS, which extinguished the original CR. In Experiment 2, the animals received either continued CS-US training or exposure to the CS and US but at a long interval (2,800 msec), which eliminated the original CR. As the baseline for detection of transfer effects, each experimental group had a control group that received Stage 1 training with a 2,800-msec CS-US interval, which produced minimal CR acquisition. The results of both experiments revealed substantial positive transfer across CS modalities regardless of the treatment during Stage 2. The transfer did not appear immediately on test presentations of the second CS in Stage 3. Rather, the transfer appeared as an enhancement in the rate of CR acquisition after reinforced training with the second CS had commenced. The results are discussed with respect to stimulus generalization, neutralization of background stimuli, and learning processes superordinate to specific associations.     

The effects of differing type and magnitude of reward on the contrafreeloading phenomenon in rats

This investigation was made to determine the effects of the magnitude of reward on contrafreeloading, using food or water as reward. Two quantities were selected for each level of reward quality—a 20-mg-pellet food reward, a 45-mg-pellet food reward, a .01-cc water reward, and a .1-cc water reward. Seven days of training were followed by three test sessions. There was a significantly higher percent of contrafreeloding demonstrated with food as reward than with water and higher number of barpresses with small reward than with large. It was argued that a more appropriate measure should include reference to performance during training. In this approach, contrafreeloading with food and water were virtually the same.     

Facilitating stimulus effects of reward and punishment in discrimination learning

To demonstrate a facilitating stimulus effect, as opposed to an incentive effect, of food reward, rats were trained on an easy, light-dark discrimination with different amounts of reward for correct and incorrect responses (1-0, 2-0, 3-1, and 5-1 pellets, respectively), and with shock or no shock administered in the correct goalbox. Both errors and trials to criterion were fewer with a large reward differential (LRD: 2-0 and 5-1), as compared with a small reward differential (SRD: 1-0 and 3-1), but were not affected by the “base” reinforcement condition of either 1 or 0 pellets for the incorrect response. In addition, choice and arm speeds during early training were positively related to the combined, or average, number of pellets contingent upon both correct and incorrect responses, indicating a generalization of reward expectancies. Although shock uniformly suppressed arm speeds under all reward conditions, it facilitated discrimination learning in the SRD conditions. That such facilitation occurred only when the conditions of reward for correct and incorrect responses were relatively similar indicates that not only shock, but also food can function as a distinctive cue: As a stimulus selectively applied to one response, it can decrease the similarity of the alternatives, and, in this manner, it can faciltate performance.     

The role of within-trial location of the retention interval in rats’ delayed conditional discrimination performance

Rats were trained and tested on delayed conditional discriminations (DCDs) consisting of four possible light and tone stimulus sequences: light-light, tone-tone, light-tone, and tone-light. A lever was presented after the offset of the second or test stimulus, S₂. Two retention intervals (RIs) were present within the DCD task, one (RI-1) between the sample or first stimulus (S₁) and S₂, and the other (RI-2) between S₂ and presentation of the lever. Liquid reinforcement was contingent upon pressing only when S₂ matched S₁ in Experiment 1 or only when S₂ differed from S₁ in Experiment 2. RI-1 and RI-2 were separately increased to 5,10, and 20 sec from 1-sec training conditions. Increasing RI-1 produced greater declines in performance to light S₁ than to tone S₁ in both experiments. No such stimulus modality effect occurred for increases in RI-2 in these experiments. These results indicate that retrospection of S₁ occurred during RI-1 and prospection of a response decision and reward expectancy primarily occurred during RI-2.     

Sample duration and type of stimuli in delayed matching-to-sample in rhesus monkeys

Two experiments were performed to determine the effect of sample duration (0.1, 2, and 4 sec), delay interval (.03, 4, 8, 16, and 32 sec), and type of stimulus (color and shape) on the matching performance of rhesus monkeys. In Experiment 1, the 15 possible delay-duration combinations were randomly presented in blocks of 15 trials. In Experiment 2, each duration was held constant and the five delays randomly presented. Then each delay interval was held constant with the three durations randomly varied. Matching performance increased as sample duration increased (ps < .01 and .005), while length of delay did not significantly affect performance. The type of stimuli paired in the matching test significantly affected performance (ps < .05 and .10) with the shape/shape choices leading to the poorest performance. Stimulus discriminability and amount of training with brief sample durations were implicated as significant determinants of matching performance.     

A behavior field approach to instrumental learning in the rat: II. Training parameters and a stage model of extinction

Three runway experiments tested a stage model of extinction which postulated an orderly succession of three qualitatively different stages: habit, trial and error, and resolution. The model predicted that Stage 1 should be characterized by perseveration of habitual routes (i.e., response persistence) and the absence of competing responses; Stage 2, by an increase in investigatory behavior (response variation and hole exploration) and biting behavior; Stage 3, by a decrease in the competing responses of Stage 2 and continued increase in goal avoidance and substitution behavior (e.g., sand-digging). These predictions were largely confirmed. Further, Experiments 1 and 2 showed that, as expected by the model, continuous reinforcement (CRF) resulted in more practice of habitual routes in acquisition and greater response persistence, while partial reinforcement (PRF) resulted in more route variation and hole exploration in acquisition and greater goal persistence which was attributable to prior reinforcement of a trial-and-error coping strategy. Results of Experiment 3, which combined training trials and reward magnitudes orthogonally, supported the prediction that response persistence was positively related to training trials, and goal persistence negatively related to reward magnitudes. All three experiments demonstrated an inverted-U function in investigatory and biting behavior, as predicted by the stage model.     

Instrumental escape learning in the rat at 24-hour intertriai interval: Effects of magnitude and schedule of reinforcement

Three experiments investigated the effects of magnitude and schedule of reinforcement and level of training in instrumental escape learning at a 24-h intertriai interval. In Experiment I, two magnitudes of reinforcement were factorially combined with two schedules of reinforcement (CRF and PRF). Under PRF, large reward produced greater resistance to extinction than did small reward, while the reverse was true under CRF. In Experiment II, two levels of acquisition training were factorially combined with three schedules of reinforcement (CRF, single-alternation, and nonalternated PRF). Patterned running was observed late in acquisition in the single-alternation extended-training condition. Resistance to extinction was greater for the nonalternated PRF condition than for the single-alternation condition following extended acquisition, and the reverse was true following limited acquisition. Experiment III confirmed the extinction findings of Experiment II. The results of all three experiments supported an analysis of escape learning at spaced trials in terms of Capaldi’s (1967) sequential theory.     

Effects of an aversive CS+ and CS− under deprivation upon successive classical appetitive and aversive conditioning

Rabbits under high or moderate water deprivation received in Stage 1 either paired (CS+), unpaired (CS?), or no-tone/shock presentations, with the pairings being appropriate for nictitating membrane conditioning. In Stage 2, all groups were given paired tone and water deliveries for jaw-movement conditioning, while, in Stage 3, all group received the tone and shock paired together for membrane conditioning. In Stage 2, the previously established aversive CS+ suppressed jaw-movement conditioning under high deprivation, and membrane CR decrements were directly related to deprivation. Also in Stage 2, the aversive CS? raised jaw-movement conditioning under moderate deprivation. In Stage 3, membrane CR performance immediately returned in the aversive CS+ group. For the other groups, conditioning was faster under high, relative to moderate, deprivation; however, the initial membrane CR occurrence required more trials if unpaired presentations were used in Stage 1. These results suggest that CSs can acquire both opponent-process and associative effects expressed according to the prevailing training conditions.     

Instrumental performance and time between reinforcements: Intimate relation to learning or memory retrieval?

According to scalar expectancy theory (SET), instrumental performance is determined by the ratio of the time between reinforcements in the trial (T _T) to the overall time between reinforcements (T _O). Groups for which theT _O|T _T ratio is the same should perform similarly. According to the sequential-memory view, the memory of nonreward becomes a signal for reward, and thereby promotes strong responding, when that memory is retrieved on a reward trial. In each of three runway investigations employing rats in a runway, two groups were compared that had the sameT _O |T _T ratio but that differed in the tendency to retrieve the memory of nonreward on a rewarded trial. In each investigation faster running on critical nonrewarded trials was associated with the group having the stronger tendency to retrieve the memory of nonreward on a rewarded trial. These findings are consistent with the predictions of the sequential-memory view, as well as with certain earlier findings, but are inconsistent with SET. It was indicated that the groups compared here were matched along a considerable number of dimensions—an unprecedented number for a varied reward investigation.     

Reward magnitude and shock variables (continuity and intensity) in shuttlebox-avoidance learning

In Experiment I, eight groups of rats (n = 20) were given shuttlebox-avoidance training. Two levels of shock (.3 and 1.6 mA) were combined factorially with two levels of reward (large and small) under both continuous and discontinuous (.75 sec on and 2.00 sec off) shock. Visual situational cues were absent after a shuttle response for the large-reward condition and present for the small-reward condition. Superior performance was obtained with weak rather than strong shock under both reward conditions and with large rather than small reward only under the weak-shock condition. Continuity of shock had no differential effect on performance. Experiment II allowed the conclusion that the reward effect was attributable to a reinforcement mechanism. The data were taken as support for the effective reinforcement theory, which emphasizes the importance in avoidance learning of fear conditioned to situational cues.     

Contributions of Reward Sensitivity to Ventral Striatum Activity Across Adolescence and Early Adulthood

It was examined how ventral striatum responses to rewards develop across adolescence and early adulthood and how individual differences in state‐ and trait‐level reward sensitivity are related to these changes. Participants (aged 8–29 years) were tested across three waves separated by 2 years (693 functional MRI scans) in an accelerated longitudinal design. The results confirmed an adolescent peak in reward‐related ventral striatum, specifically nucleus accumbens, activity. In early to mid‐adolescence, increases in reward activation were related to trait‐level reward drive. In mid‐adolescence to early adulthood decreases in reward activation were related to decreases in state‐level hedonic reward pleasure. This study demonstrates that state‐ and trait‐level reward sensitivity account for reward‐related ventral striatum activity in different phases of adolescence and early adulthood.     

Reward contrast in delay and probability discounting

In this study, we examined whether reward contrast influences choice between delayed and probabilistic outcomes. Specifically, we predicted that the subjective value of an intermediate reward would seem relatively larger or smaller, respectively, if it followed choices involving a smaller or larger reward and would produce corresponding changes in rates of delay and probability discounting. In Experiment 1, subjects made choices about hypothetical 5,000 or5,000 or 50 outcomes and then made choices about 500 outcomes. Delay-discounting rates for the500 outcomes. Delay-discounting rates for the 500 outcome were larger for Group 5,000 than for Group5,000 than for Group 50, whereas the opposite result was obtained for probability-discounting rates. In Experiment 2, we used a design that allowed for contrast effects to be assessed within subjects. Two groups made choices about delayed or probabilistic rewards. After completing question blocks in which the amount was 5,000 or5,000 or 50, subjects responded to questions with an intermediate amount (475/475/525). For Group Delay, the present value of the intermediate reward was greater after the 50 block than after the50 block than after the 5,000 block, whereas the opposite was obtained for Group Probability. The results from both experiments confirmed the predictions of reward contrast and suggested that the subjective value of a monetary reward varies inversely with the prior reward amount.     

Effects of the amount of acquisition and contextual generalization on the renewal of instrumental behavior after extinction

Four experiments with rat subjects examined the role of context during the extinction of instrumental (free-operant) behavior. In all experiments, leverpressing was first reinforced on a variable-interval 30-s schedule and then extinguished before being tested in the extinction and renewal contexts. The results identified three important variables affecting the renewal effect after instrumental extinction. First, ABA and ABC forms of renewal were strengthened by increasing the amount of acquisition training. This suggests that the strength of the association learned during acquisition, or the final level of performance, influences the degree of renewal after extinction. The effect of the amount of training was modulated by the second factor, the degrees of generalization from the acquisition and extinction contexts to the test context. The third variable was acquisition training in multiple contexts, which was shown to strengthen ABC renewal. Methodological, theoretical, and practical implications are discussed.     

Spatial localization of a goal: Beacon homing and landmark piloting by rats on a radial maze

We performed six experiments in order to examine the ability of rats to use moving beacons and landmarks as cues to the location of reward on an eight-arm radial maze. In Experiments 1–4, the cues and goals were moved before each trial, and groups in which a single beacon was placed on the rewarded arm, a single landmark indicated that reward was on the arm immediately to the left of a landmark, or two landmarks were placed on each side of the reward arm were compared. The rats rapidly learned to track the reward in the beacon condition, failed to find the reward sooner than chance expectation with a single landmark, and did only slightly better than chance with two landmarks. In Experiments 5 and 6, the rats were trained in five trials per day, with the landmark and goal locations constant over daily rewarded trials, and in two extinction trials that were inserted among the rewarded trials. The rats found the goal arm at substantially better than chance expectancy with both one and two landmarks. Our results, in agreement with data from recent swimming pool experiments (A. D. L. Roberts & Pearce, 1998), show that rats will use the relationship between moving landmarks and a goal in order to find reward.