Rats’ choices between one and two delayed reinforcers

Rats chose between alternatives that differed in the number of reinforcers and in the delay to each reinforcer. A left leverpress led to two reinforcers, each delivered after a fixed delay. A right leverpress led to one reinforcer after an adjusting delay. The adjusting delay was increased or decreased many times in a session, depending on the rat’s choices, in order to estimate an indifference point& #x2014;a delay at which the two alternatives were chosen about equally often. Both the number of reinforcers and their individual delays affected the indifference points. The overall pattern of results was well described by the hyperbolic-decay model, which states that each additional reinforcer delivered by an alternative increases preference for that alternative but that a reinforcer’s effect is inversely related to its delay. Two other possible delay-discounting equations, an exponential equation and a reciprocal equation, did not produce satisfactory predictions for these data. Adding an additional free parameter to the hyperbolic equation as an exponent for delay did not appreciably improve the predictions, suggesting that raising delay to some power other than 1.0 was unnecessary. The results were qualitatively similar to those from a previous experiment with pigeons (Mazur, 1986), but quantitative differences suggested that the rates of delay discounting were several times slower for rats than for pigeons.     

Choice, delay, probability, and conditioned reinforcement

The hyperbolic-decay model is a mathematical expression of the relation between delay and reinforcer value. The model has been used to predict choices in discrete-trial experiments on delay-amount tradeoffs, on preference for variable over fixed delays, and on probabilistic reinforcement. Experiments manipulating the presence or absence of conditioned reinforcers on trials that end without primary reinforcement have provided evidence that the hyperbolic-decay model actually predicts the strength of conditioned reinforcers rather than the strength of delayed primary reinforcers. The model states that the strength of a conditioned reinforcer is inversely related to the time spent in its presence before a primary reinforcer is delivered. A possible way to integrate the model with Grace’s (1994) contextual-choice model for concurrent-chain schedules is presented. Also discussed are unresolved difficulties in determining exactly when a stimulus will or will not serve as a conditioned reinforcer.     

Reinforcement contexts effects on fixed-interval responding

Three rats responding on fixed-interval schedules received either 1 or 4 pellets at the end of 2-min intervals. Five experimental conditions manipulated the relative probabilities of these two reinforcers. Response rates following the 1-pellet reinforcer were always higher than the rates following the 4-pellet reinforcer. The rates after the 1-pellet reinforcer were also highest in those experimental conditions where it was delivered with low probability. Contrast effects were observed when two sequential fixed intervals differed in reinforcer magnitudes. It was concluded that the context of reinforcement as well as the specific reinforcer magnitude affects responding under fixed-interval schedules.     

A matching law analysis of the reinforcing efficacy of wheel running in rats

Previous research has demonstrated that running in a rotating wheel functions as a reinforcer for leverpressing in rats. In these studies, the pattern of responding was similar to the pattern of responding maintained by consummatory reinforcers, such as food and water. The present study investigated quantitative features of responding maintained by running. In previous experiments in which responses were reinforced according to variable-interval (VI) schedules and food and water served as the reinforcer, the equation for a rectangular hyperbola described the relationship between response rate and reinforcement rate. This experiment tested whether this quantitative regularity also applies to leverpressing maintained by the opportunity to run in a wheel. Fourteen male Wistar rats responded on levers for the opportunity to run. In each session, subjects were exposed to a series of VI schedules. An opportunity to run for 60 sec was the reinforcing consequence. Results showed that response rate was a negatively accelerated function of reinforcement rate, and the relationship between these two variables was described well by the equation for a rectangular hyperbola. To further test the similarity between running and consummatory reinforcers, the response requirement and access were manipulated. In previous experiments with food and water, these types of manipulations differentially changed the two parameters of the hyperbola. A similar pattern of results was obtained with wheel running. Thus, the equation appears to apply to running about as well as it does to consummatory reinforcers.     

Reward contrast in delay and probability discounting

In this study, we examined whether reward contrast influences choice between delayed and probabilistic outcomes. Specifically, we predicted that the subjective value of an intermediate reward would seem relatively larger or smaller, respectively, if it followed choices involving a smaller or larger reward and would produce corresponding changes in rates of delay and probability discounting. In Experiment 1, subjects made choices about hypothetical 5,000 or5,000 or 50 outcomes and then made choices about 500 outcomes. Delay-discounting rates for the500 outcomes. Delay-discounting rates for the 500 outcome were larger for Group 5,000 than for Group5,000 than for Group 50, whereas the opposite result was obtained for probability-discounting rates. In Experiment 2, we used a design that allowed for contrast effects to be assessed within subjects. Two groups made choices about delayed or probabilistic rewards. After completing question blocks in which the amount was 5,000 or5,000 or 50, subjects responded to questions with an intermediate amount (475/475/525). For Group Delay, the present value of the intermediate reward was greater after the 50 block than after the50 block than after the 5,000 block, whereas the opposite was obtained for Group Probability. The results from both experiments confirmed the predictions of reward contrast and suggested that the subjective value of a monetary reward varies inversely with the prior reward amount.     

Control of pecking response form in the pigeon: Topography of ingestive behaviors and conditioned keypecks with food and water reinforcers

In Experiment 1, the form of keypecks produced in an autoshaping procedure with food or water reinforcers was compared with that of eating and drinking responses. Because the responses involve a number of different effector systems, several elements of response form were measured, including peck force and duration, gape, and eye closure. Gape was the only measure to reliably distinguish between both ingestive responses and between conditioned keypecks reinforced with food or water. With either reinforcer, keypecks had greater force than did ingestive behaviors. In Experiment 2, a transition between two forms of keypeck was produced by manipulating deprivation and reinforcer conditions. Some measures appeared to vary in a dichotomous manner between two discrete response forms; gape showed a gradual and continuous change involving the production of intermediate forms of the response. It was concluded that the control of conditioned response form involves theconstruction of the response from movements produced by several effector systems, each with potentially different sources of control.     

Concurrent schedules of wheel-running reinforcement: Choice between different durations of opportunity to run in rats

How do animals choose between opportunities to run of different durations? Are longer durations preferred over shorter durations because they permit a greater number of revolutions? Are shorter durations preferred because they engender higher rates of running? Will longer durations be chosen because running is less constrained? The present study reports on three experiments that attempted to address these questions. In the first experiment, five male Wistar rats chose between 10-sec and 50-sec opportunities to run on modified concurrent variable-interval (VI) schedules. Across conditions, the durations associated with the alternatives were reversed. Response, time, and reinforcer proportions did not vary from indifference. In a second experiment, eight female Long-Evans rats chose between opportunities to run of equal (30 sec) and unequal durations (10 sec and 50 sec) on concurrent variable-ratio (VR) schedules. As in Experiment 1, between presentations of equal duration conditions, 10-sec and 50-sec durations were reversed. Results showed that response, time, and reinforcer proportions on an alternative did not vary with reinforcer duration. In a third experiment, using concurrent VR schedules, durations were systematically varied to decrease the shorter duration toward 0 sec. As the shorter duration decreased, response, time, and reinforcer proportions shifted toward the longer duration. In summary, differences in durations of opportunities to run did not affect choice behavior in a manner consistent with the assumption that a longer reinforcer is a larger reinforcer.     

Blocking despite changes in reinforcer identity

Rats were trained on a discriminated operant barpressing task according to a standard blocking design. In some conditions, the reinforcer was changed between the pretraining and compound conditioning phases; for other conditions, the reinforcer remained the same across phases. In three separate experiments using both between- and within-subject designs, strong blocking effects occurred regardless of the change in the reinforcer. In a fourth experiment, a multiple schedule of reinforcement was used in which response-independent reinforcers were superimposed on the schedule of response-contingent reinforcers. The degree of response suppression caused by the free reinforcer was greater when the free reinforcers were the same as the response-contingent reinforcers than when they were different. The role played by the reinforcer identity in contingency experiments thus appears to be different from the role it plays in blocking experiments.     

Self-control in the monkeymacaca fascicularis

Two experiments tested two cynomolgus monkeys’ self-control—choice of a longer, more delayed reinforcer over a shorter, less delayed reinforcer. In Experiment 1, subjects exhibited significant selfcontrol in a procedure in which reinforcer amounts and delays were held constant throughout a condition. In Experiment 2, subjects exhibited significantly greater sensitivity to variation in reinforcer amount than to variation in reinforcer delay in a procedure in which the reinforcer delay associated with the self-control alternative was adjusted until each macaque was indifferent between the two alternatives. Both experiments indicated that, in laboratory paradigms in which humans show self-control and pigeons and rats show impulsiveness, macaques show self-control. These results are inconsistent with the hypothesis that species differences in self-control are a function of language ability or of specific types of prior training. The results are consistent with the hypothesis that species differences in self-control are related to the ratio of brain size to body weight (a possible indicator of general cognitive ability) or to shared phylogeny.     

The role of response-reinforcer associations increases throughout extended instrumental training

Two experiments examined the effects of extended training on the development of response-reinforcer associations. Rats were trained by using various food reinforcers to make multiple instrumental responses. Subsequently, those reinforcers were devalued by being paired with a toxin. The presence of response-reinforcer associations was inferred from the decrease in the likelihood of a response following devaluation of its reinforcer. Such response-reinforcer associations are known to contribute to performance after moderate amounts of training. These experiments addressed the question of whether the contribution of those associations remains constant, increases, or decreases with more extended training. Experiment 1 found that even after a response had been extensively trained with one reinforcer, the substitution of a new reinforcer produced new associations between the response and that new reinforcer. After extended training, a response continued to acquire new associations with a reinforcer, as indexed by the impact of a devaluation procedure. Experiment 2 directly compared the contribution of reinforcers used extensively and moderately with the same response. It found that devaluation of the extensively used reinforcer more effectively reduced performance of the response, suggesting that the associations formed with additional training contribute to performance of the response. These experiments indicate that the contribution of response-reinforcer associations does not decrease but, instead, continues to grow throughout the course of extended instrumental training.     

Impulsiveness in pigeons living in the experimental chamber

Pigeons’ choices between larger, more delayed and smaller, less delayed reinforcers were examined while the pigeons lived in the experimental chamber for 23-h sessions. In Condition 1, 4 pigeons were food deprived prior to each session and exposed to one session every 4th day. Condition 2 was identical except that the pigeons began each session at their ad-lib weights. Condition 3 was identical to Condition 2 except that sessions were conducted on consecutive days. Condition 4 was identical to Condition 3 except that the subjects (2 pigeons from Conditions 1–3 plus a naive pigeon) could obtain reinforcers much less frequently. In all of the conditions, the pigeons consistently chose the smaller, less delayed reinforcers; the pigeons were impulsive. The restriction of food access caused a disruption in the diurnal pattern of feeding, but did not decrease impulsiveness even in this 23-h live-in procedure.     

Delay discounting in rhesus monkeys: equivalent discounting of more and less preferred sucrose concentrations

Humans discount larger amounts of a delayed reinforcer less steeply than smaller amounts, but studies with pigeons and rats have yet to reveal such a magnitude effect, suggesting that the effect may be unique to humans. The present study examined whether the magnitude effect is observed in a species phylogenetically closer to humans, by comparing the rates at which rhesus monkeys discounted 10% and 20% concentrations of sucrose. There were no systematic differences in the rates at which the monkeys discounted the two sucrose concentrations, despite the fact that they strongly preferred the 20% concentration. Interestingly, the monkeys discounted delayed sucrose at a rate higher than was observed with delayed cocaine, and lower than was observed with delayed saccharin in previous studies (Freeman et al. Behavioural Processes, 82, 214-218, 2009; Woolverton et al. Experimental and Clinical Psychopharmacology, 15, 238-244, 2007). Taken together, these findings suggest that although both quantitative and qualitative differences can affect monkeys’ preferences between immediate reinforcers, qualitative differences between types of reinforcers (e.g., sucrose vs. cocaine) can affect monkeys’ discounting rates in a way that quantitative differences within a reinforcer (e.g., 10% vs. 20% sucrose) do not.     

Effects of response effort on self-control in rats

Five rats served as subjects in an experiment that examined the effect of increasing response effort on self-control (choice of a larger, more delayed reinforcer over a smaller, less delayed reinforcer). The rats made significantly more self-control choices as the force required to respond on each lever increased from 0.1 to 0.8 N. As the force required to press the levers increased to 1 N and then began to decrease, some of the rats stopped responding. For those rats that continued to respond, self-control tended to decrease. The results suggest that increasing the required response force can increase selfcontrol choices, as long as the required response force is low enough that some responding occurs.     

Autoshaping of initial feeding and drinking reactions in newly hatched chicks

Newly hatched chicks were force-fed food and water throughout rearing, and food, water, or sand reinforcers during exposure to an omission-training procedure. The chicks were thus prevented from performing approach and contact responses to the reinforcer at any time in their lives. Nevertheless, the subjects displayed approach and species-specific feeding or drinking reactions directed toward an illuminated key paired with food or water, but not with sand. Illumination of a key either uncorrelated or negatively correlated with food or water did not engender appreciable responding. Feeding and drinking reactions were topographically distinct, determined by the type of reinforcer, but were not elicited by the reinforcer. These findings support a “learned release” view of autoshaping, according to which phylogenetically preorganized behavior patterns are triggered by distal stimuli paired with biologically significant proximal stimulation, and suggest a close relationship between autoshaping and primitive instances of visual object recognition.     

COMPARING METHODS OF IDENTIFYING REINFORCING STIMULI IN SCHOOL CONSULTATION

Reinforcement-based interventions, the most frequently used treatments for school-age children, rely on accurately identifying stimuli that will serve to reinforce appropriate classroom behavior. Research has consistently demonstrated that the results from a forced-choice pairing procedure are the best predictors of reinforcing stimuli. Interestingly, systematic evaluation of potential reinforcers is rarely implemented in the school consultation setting. Considering the importance of the reinforcer on reinforcement-based interventions, and the literature focusing on the significance of the selection procedure on accurately identifying a reinforcer, this is concerning. The purpose of these two studies was to examine the effectiveness of identifying reinforcing stimuli for students in the consultation setting using two different methods: stimulus forced-choice and asking the teacher to identify potential reinforcers. The effectiveness of the selected stimuli as reinforcers was studied on two student outcomes: academic production and on-task behavior. The results of the two studies suggested that the reinforcers selected using a forced-choice procedure were more effective than the reinforcers selected from a teacher-identification procedure. Further, results indicated that although stimuli derived from both reinforcer assessment methods were useful at increasing rates of desired behavior, stimuli derived from the forced-choice reinforcer assessment were more consistently effective.     

Choice and percentage reinforcement in pigeons

Pigeons responded on a two-key concurrent chains choice procedure with the same level of percentage reinforcement on each key. During the initial links, a choice response on either key occasionally produced a conditioned reinforcer—which on one key was associated with a 15-sec, and on the other key with a 30-sec, interreinforcement interval—or an extinction stimulus. In Part 1, the initial links were equal. With successive decreases in the probability of a reinforcer, choice shifted from preference for the 15-sec terminal link toward indifference. In Part 2, the initial links were unequal and were arranged so that the shorter initial link preceded the 30-sec terminal link. At a high probability of a reinforcer, the pigeons again preferred the 15-sec terminal link. However, at a low probability, the pigeons reversed and preferred the alternate key. It was concluded that the conditioned reinforcers tended to become functionally equivalent at a low probability of a reinforcer, despite the nominally different interreinforcement intervals, with the result that choice was then modulated by the relative size of the initial links. The data are inconsistent with the view that choice and the strength of conditioned reinforcers are isomorphic with the reduction in delay to reward correlated with terminal link stimuli.     

Experience with the reinforcer and the preference for earned rather than free reinforcers in rats

Manipulating experience with the reinforcer, through either home cage presentations of Noyes pellets or availability of the free reinforcer immediately prior to testing, attenuated the preference for earned as opposed to free reinforcers. Similarly, changing the reinforcer to one of a different flavor at testing increased the preference for the noncontingent reinforcer. These results are consistent with an interpretation of earned reinforcer preference which emphasizes the role of the reinforcer as a discriminative signal for further instrumental responding. It is suggested that the tendency to perform instrumental responses for reinforcers when free reinforcers are available can be explained in terms of traditional learning processes.     

Social working memory: Memory for another rat's spatial choices can increase or decrease choice tendencies

In two experiments using a radial-arm maze, pairs of rats made choices among eight maze locations, each containing a large quantity of one of two food types. The choices made by 1 rat affected the choices made by the other rat. Under most conditions, visits by 1 rat increased the tendency of the other rat to subsequently choose that maze location. However, the effect depended on the quality of the food available in a particular location. When it was possible for the rats to observe each other on the maze arms and a rat had experienced that a location contained the less preferred food type, a previous visit to that location by the foraging partner decreased the tendency to visit that location. These effects are attributed to working memory for the spatial choices of another rat, and they indicate that memory produced by a rat’s own visit to a maze location is integrated with memory for the behavior of another rat to determine spatial choice     

Sign-tracking (autoshaping) in rats: A comparison of cocaine and food as unconditioned stimuli

A series of experiments was performed to determine whether sign-tracking would occur in rats with intravenous (i.v.) cocaine as the unconditioned stimulus. In Experiment 1, a retractable lever paired with food produced strong sign-tracking, but a lever paired with one of three doses of i.v. cocaine did not elicit any approach or contact behavior. Experiment 2 demonstrated that doses of cocaine that did not elicit sign-tracking would function as a positive reinforcer for a lever contact operant. In Experiment 3, an artificialconsummatory response was added to make the cocaine reinforcement episode more behaviorally comparable to that occasioned by food. Although the rats readily performed this response when it was required to receive cocaine infusions, they still did not contact a lever that signaled the availability of these infusions. It appears that cocaine is different from other positive reinforcers (e.g., food, water, warmth, or intracranial stimulation) in that it will not produce sign-tracking in rats.