Learning in honeybees as a function of amount and frequency of reward

In a series of four experiments with free-flying honeybees, individual foragers were trained with targets of two different colors that contained 5 or 20 μl of 50% sucrose solution. The two targets were singly presented in quasi-random sequences on each visit, with the amount of reward to be found on each target perfectly predictable from its color. The number of training visits (4–32) was varied both within and between experiments, and so also was the relative frequency of trials with the 5- and 20-μl targets (1:1, 2:1, 3:1, and 9:1). At the conclusion of training under each condition, unrewarded responses to the targets were measured in a 10-min extinction test, with the targets presented either separately to two different groups of animals (Experiment 1) or as a pair (Experiments 2–4). When the number of training trials with each target was the same (Experiments 1 and 2), the animals responded more in extinction to the 20-μl target than to the 5-μl target, although there was a decline in the overall level of responding to both targets (an overlearning-extinction effect) as the number of training trials increased. After nine times as many, or only three times as many, training trials with the5- μl target as with the 20-μl target, the animals responded more in extinction to the 5-μl target (Experiment 3); after twice as many training trials with the 5-μl target as with the 20-μl target, there was equal responding to both (Experiment 4). The preferences shown in the choice tests of Experiments 2–4 could be simulated rather accurately on the assumptions of a model previously developed to deal with the discrete-trials choice behavior of honeybees and the further assumption that associative strength grows at a rate increasing with amount of reward to an asymptote independent of amount of reward.     

Control of performance by short-term memory in honeybees

Individual honeybees foraging at a laboratory window were trained with a correction method to choose between blue and yellow targets, one of which contained sucrose solution. There were two trials on each visit, with the locus of the sucrose predictable only on the second. Animals differentially rewarded on Trial 2 for choosing the rewarded color of Trial 1, for choosing the alternative color, or for choosing the target in the rewarded position of Trial 1 independently of its color, all showed a small but persistent preference for the rewarded color, with no significant preference for the rewarded position. When the positions of the colored targets were the same on Trial 2 as on Trial 1 (color and position confounded), there was a more substantial but equally persistent preference on Trial 2 for the rewarded color-position of Trial 1, whether the animals were differentially rewarded for perseveration or for alternation. The results provide further evidence of unlearned control of performance by short-term memory in honeybees but no indication of learned control.     

Learning in honeybees as a function of amount of reward: Rejection of the equal-asymptote assumption

Foraging honeybees were trained individually with successively presented targets differing in odor, one containing 5 µl and the other 20 µl of a 50% sucrose solution, after which preferences were measured in choice tests. In Experiment 1, there were either 8 training trials with each target, 16 trials with each, or 8 trials with the 20-µ1 target and 16 trials with the 5-µl target. In Experiments 2 and 3, the odor-amount relation was reversed after either 24 or 16 trials with each target. In Experiment 4, differential reward was introduced only after two, four, or six feedings-to-repletion on each target. All of the results could be simulated quantitatively and with considerable accuracy on the assumption that the attractiveness of an odor is given by the strength of its association with sucrose; that asymptotic associative strength is an increasing function of amount of reward; and that choice between two odors is determined by their relative associative strength.     

An analysis of “signaled double-alternation patterning” in the rat

Rats given training with double alternation of rewards and nonrewards in which the first reward or nonreward of each pair occurred in a black runway and the second in a white runway developed fast running on rewarded trials in both runways and slow running on nonrewarded trials in both runways—signaled double alternation patterning. A subsequent shift in the reinforcement schedule produced a period of reversed patterning—slow on rewarded trials and fast on nonrewarded trials. The results are consistent with a compound stimulus discrimination interpretation of signaled double-alternation patterning rather than with a selective memory-retrieval explanation.     

Short-term spatial memory in honeybees

Foraging honeybees were trained individually in two-choice spatial problems. Differentially rewarded for spatial alternation in Experiment 1 (“win-shift” training), they showed instead a clear tendency to perseverate—that is, to prefer on each trial the location of reward on the immediately preceding trial. On the basis of the results of Experiments 2 and 3, in which one location was rewarded over shorter or longer series of consecutive trials, an associative interpretation of the perseveration found in the first experiment was rejected in favor of an interpretation in terms of short-term spatial memory. Experiment 4, in which the animals were rewarded on each trial for choosing either location, also showed perseveration. Honeybees, like rats, seem to remember a rewarded location recently visited, but tend to return to it rather than, like rats, to avoid it.     

Effects of sequences of sucrose reward magnitudes with short ITIs in rats

Two experiments assessed the role of aftereffect learning in rats rewarded with sucrose solutions. In Experiment 1, rats were trained in a single straight runway for two trials on each of 18 days, each trial terminating with either large (20% scurose) or small (3% sucrose) reward. The ITI was 3–5 min. The sequence of daily rewards for each of four groups was small-small (SS), small-large, (SL), large-small (LS), or large-large (LL). Response patterning and a simultaneous negative contrast effect were observed in LS and SL relative to the consistently rewarded controls. During 10 massed extinction trials, resistance to extinction was greatest for Group SL, followed in order by Groups SS, LL, and LS. Experiment 2 examined single alternation of large and small rewards administered for 10 trials on each of 31 days with an ITI of 60 sec. Reward for one group was 20% or 3% sucrose while another received 1 or 10 45-mg Noyes pellets. Appropriate patterning developed only in the food-pellet rewarded animals. The overall results suggest that sucrose rewards may produce high-amplitude and long-duration aftereffects which interfere with learning in designs employing several massed daily trials, but which may facilitate learning—relative to food-pellet rewards—with longer intertrial intervals and fewer daily trials.     

Analysis of alternation patterning in goldfish

In Experiment 1, goldfish trained with alternation of reward (R) and nonreward (N) for responding to a single color gave clear evidence of patterning (more rapid responding on R than on N trials). In Experiment 2, patterning was found for each of two colors alternately rewarded and nonrewarded in the sequence blue R, yellow R, blue N, yellow N, …. Changes in performance with subsequent changes in the sequence of the two colors suggested that the patterning was based on carryover rather than on associative memory of R and N.     

Effects of sucrose concentrations on single-alternation performance in rats

In Experiment 1, rats received single-alternation training with 32% or 4% sucrose reward (Phase 1) followed by a shift in reward from 32% to 4%, and vice versa (Phase 2). In Phase 1, high reward facilitated alternation performance over low reward. In Phase 2, performance on rewarded trials increased as reward increased but was unchanged as reward decreased. Performance on nonrewarded trials showed negligible effects of shifts in reward. In Experiment 2, rats received goalbox placements with 32% or 4% sucrose alternated with nonreward in Phase 1; and in Phase 2, they received alternation runway training with the same or the opposite reward from that of placements. Performance on rewarded trials was faster, the higher the reward in runway training; performance on nonrewarded trials was slower, the higher the reward in placements. In Experiment 3, Phase 1 provided placements with 64%, 32%, 16%, or 4% sucrose or dry mash alternated with nonreward; Phase 2 provided alternation runway training with dry mash reward. Alternation prerformance developed more rapidly, the higher the sucrose concentration in placements. Only 64% sucrose produced performance superior to that for dry-mash placements.     

Reward and learning in honeybees: analysis of an overshadowing effect

Previous experiments have shown that honeybees trained with colored targets baited with 5- versus 20-µl drops of sucrose solution fail to develop a preference for the 20-µl color when the location of the drop on each target is marked by a white dot (dot-color overshadowing) but that discrimination is not impaired by dots when the targets differ in odor rather than in color. In Experiments 1–3, dot-color overshadowing failed to appear with differences in concentration rather than amount of sucrose (50% vs. 20% or 0%), but it did appear in Experiments 4 and 5 with a difference in probability of reward (consistent vs. partial). Experiment 6 showed no dot-odor overshadowing with a difference in probability of reward. The results are not generally predictable from the Rescorla-Wagner principle of shared associative strength, but point instead (in conjunction with those of earlier experiments) to competition for visual attention.     

Stimulus duration and conditioned reinforcing value measured by a learning-tests procedure

The relationship between the duration of stimuli and their conditioned reinforcing effect was investigated using a learning-tests procedure. In Experiment 1, stimuli were the same duration on training (stimulus → reward) and test (choice response → stimulus). Ten- and 30-sec stimuli provided effective differential conditioned reinforcement but 3-sec stimuli did not. In Experiment 2, different pigeons had each combination of the 3- and 30-sec stimuli on training and test trials. Evidence of conditioned reinforcement was obtained only for the birds with 30-sec stimuli on both training and test. The results were interpreted as indicating that stimuli become effective conditioned reinforcers on test trials only when their duration exceeds the duration of differential short-term memory cues resulting from a difference in the events that precede them on training and test trials.     

The effect of nonnutritive satiation on the learning of a flavor preference by rats

On each day of training in Experiment 1, hungry rats were given one flavored saccharin solution followed by a differently flavored saccharin solution. The rats drank more of the first flavor during training, but preferred the second flavor in a subsequent choice test. In Experiment 2, the two flavored saccharin solutions were provided on alternate days, with one flavor being preceded by nothing and the other flavor by plain saccharin. The rats drank more of the flavor preceded by nothing during training, but preferred the other flavor in a subsequent choice test. These results suggest that a state of nonnutritive satiation can reinforce a flavor preference.     

Reciprocal overshadowing in the discrimination of color-odor compounds by honeybees: Further tests of a continuity model

Honeybees were trained to discriminate between simultaneously presented color-odor compounds, one group with color and odor confounded and a control group with color relevant and odor irrelevant; in subsequent differentially reinforced training with the colors in the absence of the odors, the performance of the two groups was the same (Experiment 1). When, however, response to the colors was measured in a 10-min extinction test, discrimination was found to be poorer after confounded training (Experiment 2), and like results were obtained in an extinction test with the odors after control animals had been trained with odor relevant and color irrelevant, the confounded animals showing poorer discrimination of the odors than the controls (Experiment 3). The results of the first two experiments, in which overshadowing of color by odor was found only with an extinction test, require us to take seriously the possibility that our previous modeling experiments (with probability of correct choice in differentially reinforced training as the measure of performance) may have been insufficiently sensitive to noncontinuity effects. Our first efforts to model extinction suggest, however, that all the results of the present experiments can be understood without sacrifice of the parsimonious independence principle.     

Learning in honeybees as a function of amount of reward: Control of delay

Six experiments on learning in honeybees were prompted by the possibility that results previously attributed to a difference in amount of reward (20- versus 5-μl drops of sucrose solution presented on colored targets) might be due at least in part to a difference in delay of reward attendant on greater difficulty in locating the 5-μ1 drops. Substantial reduction in the diameter of the targets, which was designed to facilitate location of the drops, impaired discrimination of the colors, perhaps because their salience was reduced in the process (Experiment 3). White dots used to mark the location of the drops on larger targets also impaired discrimination of the colors, which presumably were overshadowed by the dots (Experiments 1, 2, and 4). That the dots did not serve merely to equate delay but were themselves discriminated was demonstrated in Experiment 5, which produced as well the first indication of an effect of amount of reward uncontaminated by the possibility of differential delay: Animals trained with a 5-μl drop on a dotted target of one color and a drop of the same size on an undotted target of a second color preferred the dotted target, but animals trained with a 5-μl drop on a dotted target of one color and a 20-μl drop on an undotted target of a second color preferred the undotted target. In Experiment 6, with odors substituted for the colors on the assumption that they were less likely to be overshadowed by the dots, what could be interpreted as a pure amount effect was found again. Aside from their relevance to questions about the role of amount of reward, the results have some interesting implications for the theory of discriminative learning in honeybees.     

Foraging on the radial-arm maze: Effects of altering the reward at a target location

Thirsty rats were trained to collect small water rewards from the end of each arm of an eight-arm radial maze. During these training trials and subsequent testing trials, the subjects were allowed to choose a maximum of eight arms. “Preference” for a target maze location was studied by noting when, in the sequence of eight choices, the target was selected. During testing, when one maze location was consistently devoid of water, rats decreased their preference for this arm over trials (Experiment 1). Similarly, rats that learned a saccharin-lithium association demonstrated lower preferences for a maze location that consistently held the conditioned saccharin solution. This was true for animals that received saccharin-lithium conditioning on the maze (Experiment 3A) and for animals conditioned to saccharin in a separate context (Experiment 3B). An increase in preference for a target maze location consistently containing a sweet chocolate milk solution was observed in animals that were water- and food-deprived (Experiment 2). These studies demonstrate that animals will modify their responses toward (preferences for) maze locations that predictably contain an altered reward.     

Learning in honeybees as a function of amount of reward: Further experiments with color

Foraging honeybees were trained individually with successively presented targets differing in color, one containing 5 µl and the other 20 µl of 50% sucrose solution, after which preferences were measured in unrewarded choice tests. The targets were conical, designed to control for the possibility of differential delay of reward stemming from the greater detectability of the larger as compared with the smaller drops of sucrose when the drops were presented on the conventional flat targets. The new results for color, like recent results for odor, can be understood on the assumption that the attractiveness of a stimulus increases as a function of the strength of its association with reward and that the effect of amount of reward is on asymptotic strength.     

Primary frustration differences following brief partial-reinforcement acquisition under varying magnitudes of reward

In Experiment I, rats which had received six partially reinforced runway acquisition trials, with a reward magnitude of 60 sec access to wet mash on rewarded trials, showed less persistent responding over highly massed extinction trials than subjects which had received the same acquisition schedule but reward magnitudes of either 1 or 10 45-mg pellets. In Experiment II, rats which had received six partially reinforced placements into one compartment of a two-compartment box, with 60 sec access to mash on rewarded placements, jumped a hurdle faster to escape nonreward than subjects which had received the same reward schedule but 10 45-mg pellets on rewarded trials. The data supported a primary frustration analysis for reward-magnitude manipulations within brief partial-reinforcement schedules.     

The effects of differing type and magnitude of reward on the contrafreeloading phenomenon in rats

This investigation was made to determine the effects of the magnitude of reward on contrafreeloading, using food or water as reward. Two quantities were selected for each level of reward quality—a 20-mg-pellet food reward, a 45-mg-pellet food reward, a .01-cc water reward, and a .1-cc water reward. Seven days of training were followed by three test sessions. There was a significantly higher percent of contrafreeloding demonstrated with food as reward than with water and higher number of barpresses with small reward than with large. It was argued that a more appropriate measure should include reference to performance during training. In this approach, contrafreeloading with food and water were virtually the same.     

Hurdle-jump responding in the rat as a function of conspecific odor of reward and nonreward

A hurdle-jump escape response was employed to assess the laboratory rat’s aversion or attraction to different types of conspecific odor. Odorant donor subjects received 112 runway acquisition trials on a continuous reward schedule followed by 32 extinction trials, 112 acquisition trials on a 50% schedule of reward and nonreward followed by 32 extinction trials, or 144 “neutral” trials with no reward in the alley. Different groups of test subjects escaped from odor excreted by odorant subjects on (a) nonrewarded acquisition and extinction trials, (b) rewarded trials during continuous reinforcement, (c) rewarded trials during partial reinforcement, or (d) neutral trials; others escaped from a clean box. The principal findings were: (1) significant aversion to “odor of nonreward” appeared after the donor odorants had received 12 exposures to reward; (2) production of odor of nonreward by odorant subjects changed as a function of training experience with reward; (3) after repeated exposure to odor of nonreward, the escape response habituated; (4) greater or different odor excretion in extinction resulted from subjects trained on a continuous reward schedule than on a partial reward schedule. Relationships of the data to frustration theory were discussed, assuming that inferred differences in production of odor reflect differences in frustration reaction.     

Effects of forced-choice runway variations on rats’ T-maze serial pattern learning

Rats learned an ordered RNR/RNN serial pattern task in a T-maze where they were shifted to a different runway on Trial 3 only in the RNR series (shift-win/stay-lose group) or only in the RNN series (stay-win/shift-lose group). The shift-win/stay-lose group developed faster speeds on Trial 3 of the RNR than on Trial 3 of the RNN series more easily than the stay-win/shift-lose group. This difference occurred whether all rats were forced onto the same runway on the first two trials (Experiment 1) or onto a different runway on Trial 2 from that on Trial 1 in each series (Experiment 2). Posttraining probe tests revealed that the shift-win/stay-lose group in each experiment relied on the runway shift event in Trial 3 or on the series position to anticipate the second reward within a series. Such reward expectancies were greater when the runway shift occurred in the same series position as during training. These probe tests revealed that the stay-win/shift-lose group relied only on the series position in Experiment 2. Our findings do not support predictions based on an associative predictive validity model. Rather, they reflect rats’ predisposition to spontaneously alternate choices in the T-maze, a tendency corresponding to their inherent win-shift foraging strategy. Rats in each group also reduced their speeds less on the nonrewarded Trial 2 when it preceded a rewarded rather than a nonrewarded Trial 3. This effect suggests that rats were able to determine which series contained a second rewarded trial. We discuss the theoretical implications of this Trial 2 speed effect in terms of rats’ uncertainty about where this second rewarded trial might occur in the RNR series.     

Reward sequence and reinforcement level as determinants of S− behavior in differential conditioning

In two differential conditioning experiments, groups of 10 rats each differed with respect to average reward and schedule of reward received in S+. Nonreward (N) occurred on all S? trials. In both experiments, extinction of responding to S? (resistance to discrimination) was extensively regulated by reward sequence and was largely independent of average reward. In Experiment 1, resistance to discrimination was a function of transitions from N to rewarded (R) trials (N-R transitions). In Experiment 2, resistance to discrimination was increased by large reward on the R trial of N-R transitions and decreased by large reward on the R trial of R-N transitions. These schedule effects on resistance to discrimination parallel the effects of comparable schedules on resistance to extinction following partial reinforcement. The results are discussed in terms of sequential theory, reinforcement level theory, and their implications for various schedule manipulations that have previously shown S? behavior to be inversely related to average reward in S+.