Learning in honeybees as a function of amount of reward: Control of delay

Six experiments on learning in honeybees were prompted by the possibility that results previously attributed to a difference in amount of reward (20- versus 5-μl drops of sucrose solution presented on colored targets) might be due at least in part to a difference in delay of reward attendant on greater difficulty in locating the 5-μ1 drops. Substantial reduction in the diameter of the targets, which was designed to facilitate location of the drops, impaired discrimination of the colors, perhaps because their salience was reduced in the process (Experiment 3). White dots used to mark the location of the drops on larger targets also impaired discrimination of the colors, which presumably were overshadowed by the dots (Experiments 1, 2, and 4). That the dots did not serve merely to equate delay but were themselves discriminated was demonstrated in Experiment 5, which produced as well the first indication of an effect of amount of reward uncontaminated by the possibility of differential delay: Animals trained with a 5-μl drop on a dotted target of one color and a drop of the same size on an undotted target of a second color preferred the dotted target, but animals trained with a 5-μl drop on a dotted target of one color and a 20-μl drop on an undotted target of a second color preferred the undotted target. In Experiment 6, with odors substituted for the colors on the assumption that they were less likely to be overshadowed by the dots, what could be interpreted as a pure amount effect was found again. Aside from their relevance to questions about the role of amount of reward, the results have some interesting implications for the theory of discriminative learning in honeybees.     

Learning in honeybees as a function of amount of reward: Tests of the equal-asymptote assumption

In Experiments 1 and 2, honeybee foragers visiting the laboratory were fed on targets of two different colors, one containing 5 μl and the other containing 20 μl of 50% sucrose solution. The targets were presented singly in quasi-random sequences on the training visits, after which preference was measured in an unrewarded choice test. In Experiment 1, 16 differentially rewarded training trials with each color were followed by the same number of trials with the color-amount relation reversed; no preference for either color was found in the subsequent choice test. In Experiment 2, 20 differentially rewarded training trials with each color—enough to produce a clear preference for the 20-μl color when given directly after pretraining—were given after 10 feedings to repletion on each color that were calculated to generate near-asymptotic associative strength; no preference for either color was found in the subsequent choice test. In Experiment 3, there were 12 feedings to repletion on one color and, on the other, 12 feedings to repletion followed by 15 trials with a small (5 μl) reward; no preference was found in a subsequent choice test. The results of all three experiments support a nonrepresentational interpretation of the role of amount of reward in the learning of honeybees.     

Learning in honeybees as a function of amount of reward: Rejection of the equal-asymptote assumption

Foraging honeybees were trained individually with successively presented targets differing in odor, one containing 5 µl and the other 20 µl of a 50% sucrose solution, after which preferences were measured in choice tests. In Experiment 1, there were either 8 training trials with each target, 16 trials with each, or 8 trials with the 20-µ1 target and 16 trials with the 5-µl target. In Experiments 2 and 3, the odor-amount relation was reversed after either 24 or 16 trials with each target. In Experiment 4, differential reward was introduced only after two, four, or six feedings-to-repletion on each target. All of the results could be simulated quantitatively and with considerable accuracy on the assumption that the attractiveness of an odor is given by the strength of its association with sucrose; that asymptotic associative strength is an increasing function of amount of reward; and that choice between two odors is determined by their relative associative strength.     

Reward and learning in honeybees: analysis of an overshadowing effect

Previous experiments have shown that honeybees trained with colored targets baited with 5- versus 20-µl drops of sucrose solution fail to develop a preference for the 20-µl color when the location of the drop on each target is marked by a white dot (dot-color overshadowing) but that discrimination is not impaired by dots when the targets differ in odor rather than in color. In Experiments 1–3, dot-color overshadowing failed to appear with differences in concentration rather than amount of sucrose (50% vs. 20% or 0%), but it did appear in Experiments 4 and 5 with a difference in probability of reward (consistent vs. partial). Experiment 6 showed no dot-odor overshadowing with a difference in probability of reward. The results are not generally predictable from the Rescorla-Wagner principle of shared associative strength, but point instead (in conjunction with those of earlier experiments) to competition for visual attention.     

Short-term spatial memory in honeybees

Foraging honeybees were trained individually in two-choice spatial problems. Differentially rewarded for spatial alternation in Experiment 1 (“win-shift” training), they showed instead a clear tendency to perseverate—that is, to prefer on each trial the location of reward on the immediately preceding trial. On the basis of the results of Experiments 2 and 3, in which one location was rewarded over shorter or longer series of consecutive trials, an associative interpretation of the perseveration found in the first experiment was rejected in favor of an interpretation in terms of short-term spatial memory. Experiment 4, in which the animals were rewarded on each trial for choosing either location, also showed perseveration. Honeybees, like rats, seem to remember a rewarded location recently visited, but tend to return to it rather than, like rats, to avoid it.     

Performance of honeybees in analogues of the rodent radial maze

The performance of individual honeybees pretrained to forage at a laboratory window was studied in three rudimentary analogues of the radial maze designed for the study of short-term spatial memory in rats. A linear arrangement of three targets was used in Experiment 1, a triangular arrangement of three targets in Experiment 2, and a rectangular arrangement of four targets in Experiment 3, with reward only for the first response to each of the targets presented on any given trial. Several systematic patterns of responding were observed, with no indication that the choices made by the animals were influenced by memory of targets recently visited.     

Savings in classical conditioning in the rabbit as a function of extended extinction

In the present experiments, savings phenomena following a limited amount of initial acquisition and extended extinction were examined. Experiments 1 and 2 compared rates of reacquisition following brief acquisition and various amounts of extinction in conditioning of the rabbit’s nictitating membrane and heart rate response, respectively. Experiment 3 compared rates of acquisition to a novel stimulus (e.g., light) following brief acquisition and various amounts of extinction to another stimulus (e.g., tone). In addition, in Experiment 3 recovery of responding to the extinguished stimulus during acquisition to the novel, cross-modal stimulus was examined. Experiments 1, 2, and 3 demonstrated that with a limited number of acquisition trials (1) there was a graded reduction in the rate of reacquisition as a function of the number of extinction trials in both conditioning preparations, (2) there was a graded reduction in the rate of cross-modal acquisition as a function of the number of extinction trials, but (3), in Experiment 3, recovery of responding to the extinguished stimulus during cross-modal training of the novel stimulus appeared uniformly robust even in the face of extended extinction.     

Effects of sequences of sucrose reward magnitudes with short ITIs in rats

Two experiments assessed the role of aftereffect learning in rats rewarded with sucrose solutions. In Experiment 1, rats were trained in a single straight runway for two trials on each of 18 days, each trial terminating with either large (20% scurose) or small (3% sucrose) reward. The ITI was 3–5 min. The sequence of daily rewards for each of four groups was small-small (SS), small-large, (SL), large-small (LS), or large-large (LL). Response patterning and a simultaneous negative contrast effect were observed in LS and SL relative to the consistently rewarded controls. During 10 massed extinction trials, resistance to extinction was greatest for Group SL, followed in order by Groups SS, LL, and LS. Experiment 2 examined single alternation of large and small rewards administered for 10 trials on each of 31 days with an ITI of 60 sec. Reward for one group was 20% or 3% sucrose while another received 1 or 10 45-mg Noyes pellets. Appropriate patterning developed only in the food-pellet rewarded animals. The overall results suggest that sucrose rewards may produce high-amplitude and long-duration aftereffects which interfere with learning in designs employing several massed daily trials, but which may facilitate learning—relative to food-pellet rewards—with longer intertrial intervals and fewer daily trials.     

Learning in honeybees as a function of amount of reward: Further experiments with color

Foraging honeybees were trained individually with successively presented targets differing in color, one containing 5 µl and the other 20 µl of 50% sucrose solution, after which preferences were measured in unrewarded choice tests. The targets were conical, designed to control for the possibility of differential delay of reward stemming from the greater detectability of the larger as compared with the smaller drops of sucrose when the drops were presented on the conventional flat targets. The new results for color, like recent results for odor, can be understood on the assumption that the attractiveness of a stimulus increases as a function of the strength of its association with reward and that the effect of amount of reward is on asymptotic strength.     

Retention of a running response following appetitive acquisition,extinction, and discrimination

In Experiment 1, hungry rats received 30 rewarded runway trials and then either extinction trials followed by retention tests or just retention tests. Different groups were tested after retention intervals of 1 min, 1, 3, or 24 h, or 30 days. Retention of extinction training was a nonmonotonic, cubic function of time for the early portion of the response chain, with good retention at 1 min and 3 h and little retention at 1 h, 24 h, or 30 days. In the latter portions of the response chain, retention of extinction decreased monotonically with time. Retention following reward-only training varied little in time, though slight losses occurred after 30 days. Experiments 2–3 differed from Experiment 1 in imposing nonchoice discrimination training (reward vs. nonreward) instead of extinction following 30 rewarded trials. After different time intervals (.017, .75, 1.25, 3, and 24 h in Experiment 1; and .017, 1, and 3 h in Experiment 2), retention tests revealed poorest discrimination at intermediate intervals in the initial portion of the response chain, i.e., a Kamin effect appeared. The deficit seemed the result of a loss of response suppression to the cue that signaled nonreward. In latter segments of the response chain, a Kamin effect tended not to appear. Implications for a number of observations and theoretical views are noted.     

Extinction responding following partial reinforcement: The effects of number of rewarded trials and magnitude of reward

Rats were trained on a daily partial reward schedule of small magnitude of reward (S), nonreward (N), and large magnitude of reward (L), which began with SN or SSNN for all animals. The remainder of the daily schedule was defined by the factorial combination of the number of rewards (1 vs. 3) and the magnitude of reward (S vs. L). Following 18 days of such training, 20 trials of extinction were administered. It was found that increasing the number of rewarded trials in a partial reinforcement schedule decreased resistance to extinction. Furthermore, increased number of large-magnitude rewards reduced resistance to extinction to a greater extent than increased number of small-magnitude rewards.     

Ontogeny of persistence: Immediate extinction effects in preweanling and weanling rats

In Experiment I rats were trained for 21÷2 days under partial (PRF) or continuous reinforcement (CRF) conditions starting at 18, 22, 28, or 36 days of age and were then subjected to immediate extinction. At all ages there was a strong partial reinforcement extinction effect (PREE), and absolute size of PREE was greatest in the youngest rats. Rate of extinction increased as a function of age following both CRF and PRF. In Experiment II the youngest and oldest age groups of Experiment I were run under the two reward conditions of Experiment I and in a third condition, PRF with number of rewards rather than trials equated to CRF (PRF-R). The PRF-R and PRF groups were not different in extinction, and both were more persistent than CRF. The youngest rats were again more persistent than the oldest, particularly after PRF training. In Experiment III it was shown that the well-known paradoxical effect, greater reward in CRF acquisition leads to faster extinction, operates in our youngest and oldest animals, but is more pronounced in the oldest. The results are discussed in terms of whether they require different explanations than those often applied to extinction data from adult rats.     

Foraging on the radial-arm maze: Effects of altering the reward at a target location

Thirsty rats were trained to collect small water rewards from the end of each arm of an eight-arm radial maze. During these training trials and subsequent testing trials, the subjects were allowed to choose a maximum of eight arms. “Preference” for a target maze location was studied by noting when, in the sequence of eight choices, the target was selected. During testing, when one maze location was consistently devoid of water, rats decreased their preference for this arm over trials (Experiment 1). Similarly, rats that learned a saccharin-lithium association demonstrated lower preferences for a maze location that consistently held the conditioned saccharin solution. This was true for animals that received saccharin-lithium conditioning on the maze (Experiment 3A) and for animals conditioned to saccharin in a separate context (Experiment 3B). An increase in preference for a target maze location consistently containing a sweet chocolate milk solution was observed in animals that were water- and food-deprived (Experiment 2). These studies demonstrate that animals will modify their responses toward (preferences for) maze locations that predictably contain an altered reward.     

Landmark use by squirrel monkeys (Saimiri sciureus)

Two squirrel monkeys searched for a reward buried in 1 of 144 holes that formed a 12×12 grid (48×50 cm). An array of vertical, colored landmarks was placed on the grid, and their locations on the grid were changed from trial to trial. During training trials, the mealworm reward was placed either in the center of a square array of landmarks (Experiments 1 and 3) or midway between two landmarks (Experiment 2). On nonrewarded test trials, the monkeys searched among landmarks placed in the same arrays as those used in training and among landmarks placed in an expanded array (Experiments 1 and 2) or in an array intermediate between the two arrays used in training (Experiment 3). Distributions of searches on test trials indicated that the monkeys searched mostly within the configuration of the landmarks but that they had not coded the location of the reward as being either in the middle of the landmarks or at a fixed distance and direction from an individual landmark.     

Acquisition and extinction of facilitation in the C57BL/6J mouse

Acquisition, extinction, and transfer of facilitation were explored in a series of experiments with C57BL/6J mice. With a procedure in which an auditory target was followed by food only in the presence of a visual facilitator, Experiments 1—4 showed that the facilitator promoted magazine entries to the auditory target. This enhancement effect was eliminated by training the facilitator as a conditioned inhibitor (Experiments 1 and 3B). Enhancement was also reduced by nonreinforced presentations of the facilitator in a discrimination procedure (Experiment 1) and by simple nonreinforcement of the facilitator (Experiments 2, 3A, and 4). In contrast to the results obtained with a facilitator, simple nonreinforcement of an inhibitor, a visual cue that had signaled when an auditory target would not be reinforced, did not reduce its ability to modulate responding to that target (Experiment 4). However, both the facilitator and the inhibitor were found to transfer their modulatory effects to other targets (Experiment 4). Finally, mice demonstrated no evidence of differential responding on a biconditional discrimination procedure in which one auditory target (A1) was reinforced in the presence of one visual stimulus (L1) but not in the presence of another (L2), and a different auditory target (A2) was reinforced in L2 but not in L1 (Experiment 5). The implications of these results for analysis of the function of a facilitator are discussed.     

Effects of retraining following response prevention on extinction of escape behavior

Sixty female hooded rats received a sequence of 40 shock-escape training trials, 20 response prevention (or pseudoprevention) trials, either 0, 5, or 20 shock-escape retraining trials and then nonshock extinction procedures. Results of extinction using escape speed and trials-to-criterion indices showed that, in each retraining condition, response prevention reliably facilitated extinction relative to pseudoprevention controls, the degree of facilitation decreasing as amount of retraining increased. In 5-and 20-trial retraining conditions, prevention procedures also reliably impaired shock-escape performance on early retraining trials, this effect dissipating before the start of extinction.     

Reward sequence and reinforcement level as determinants of S− behavior in differential conditioning

In two differential conditioning experiments, groups of 10 rats each differed with respect to average reward and schedule of reward received in S+. Nonreward (N) occurred on all S? trials. In both experiments, extinction of responding to S? (resistance to discrimination) was extensively regulated by reward sequence and was largely independent of average reward. In Experiment 1, resistance to discrimination was a function of transitions from N to rewarded (R) trials (N-R transitions). In Experiment 2, resistance to discrimination was increased by large reward on the R trial of N-R transitions and decreased by large reward on the R trial of R-N transitions. These schedule effects on resistance to discrimination parallel the effects of comparable schedules on resistance to extinction following partial reinforcement. The results are discussed in terms of sequential theory, reinforcement level theory, and their implications for various schedule manipulations that have previously shown S? behavior to be inversely related to average reward in S+.     

The use of local features and global spatial context for object recognition in a visuomotor task in the Mongolian gerbil

Mongolian gerbils were trained to jump across gaps of randomly varying width in order to obtain a food reward. During training, gerbils learned to jump to each of two landing platforms differing in width. These landing platforms were associated with unique spatial contexts (Experiment 1), local features (Experiment 2), or both (Experiment 3). In test sessions, a landing platform was substituted that was intermediate in width between the two training platforms of novel width, in order to determine whether gerbils could use retinal image size to calibrate distance. Experiments 1 and 2 suggested that gerbils used spatial context but not local feature information to identify the target platforms. In Experiment 3, when the probe platform was presented with the same local feature information and in the same context as seen in training sessions, gerbils over- or underjumped in a manner predicted if they were using the retinal image size of the target to calibrate the distance across the gap. When the probe trials contained a mismatch between context and local feature information, no systematic over- or under-jumps were seen, and jumping accuracy was decreased. Taken together, these results suggest that, in this task, objects are identified primarily on the basis of the spatial context in which they are seen and not on the basis of their local features.     

Instrumental escape learning in the rat at 24-hour intertriai interval: Effects of magnitude and schedule of reinforcement

Three experiments investigated the effects of magnitude and schedule of reinforcement and level of training in instrumental escape learning at a 24-h intertriai interval. In Experiment I, two magnitudes of reinforcement were factorially combined with two schedules of reinforcement (CRF and PRF). Under PRF, large reward produced greater resistance to extinction than did small reward, while the reverse was true under CRF. In Experiment II, two levels of acquisition training were factorially combined with three schedules of reinforcement (CRF, single-alternation, and nonalternated PRF). Patterned running was observed late in acquisition in the single-alternation extended-training condition. Resistance to extinction was greater for the nonalternated PRF condition than for the single-alternation condition following extended acquisition, and the reverse was true following limited acquisition. Experiment III confirmed the extinction findings of Experiment II. The results of all three experiments supported an analysis of escape learning at spaced trials in terms of Capaldi’s (1967) sequential theory.     

Frustration and sexual behavior in male rats

Frustration is an emotional state produced by the surprising omission in quantity and/or quality of an appetitive reinforcer. The aversive properties of stressors, such as electric shocks, produce responses similar to those elicited by a state of frustration. In this set of three experiments, we assessed the effects of water immersion (WIM, in Experiment 1)—that is, a physical stressor—and first (in Experiments 1 and 2) and second trials of a consummatory extinction (cE; i.e., a surprising reward omission; in Experiment 3) on the sexual behavior of male rats, as compared with nonstressed animals. The results showed a sexual deficit in the animals subjected to either WIM or cE, relative to control subjects, although these experimental conditions differed in the component of the male sexual response that was affected. The present results accord with the fear = frustration hypothesis, and with Amsel’s frustration theory.