Excitatory and inhibitory effects of free reinforcers

Rats’ responding was maintained on a random-interval 1-min food schedule. In addition, non-contingent pellets were delivered, independently of the animals’ behavior, at either fixed intervals (Experiment 1) or at random intervals (Experiment 2). As the rate of delivery of the periodic and aperiodic free reinforcers increased, the rate of responding decreased. But these free reinforcers, in addition to having this inhibitory effect, had also a local excitatory effect upon responding: lever-pressing increased to a level above its mean rate following the delivery of a free food pellet. The time course of this behavioral aftereffect of free reinforcers, for both fixed and random intervals, was dependent upon the proportion of the interval between successive free food deliveries. The relation of these results to those obtained with response-contingent reinforcement, free punishment, and in schedule-induced phenomena is discussed.     

Experience with the reinforcer and the preference for earned rather than free reinforcers in rats

Manipulating experience with the reinforcer, through either home cage presentations of Noyes pellets or availability of the free reinforcer immediately prior to testing, attenuated the preference for earned as opposed to free reinforcers. Similarly, changing the reinforcer to one of a different flavor at testing increased the preference for the noncontingent reinforcer. These results are consistent with an interpretation of earned reinforcer preference which emphasizes the role of the reinforcer as a discriminative signal for further instrumental responding. It is suggested that the tendency to perform instrumental responses for reinforcers when free reinforcers are available can be explained in terms of traditional learning processes.     

Resistance to extinction: contingency termination and generalization decrement

We present an algebraic model of resistance to extinction that is consistent with research on resistance to change. The model assumes that response strength is a power function of reinforcer rate and that extinction involves two additive, decremental processes: (1) the termination of the reinforcement contingency and (2) generalization decrement resulting from reinforcer omission. The model was supported by three experiments. In Experiment 1, 4 pigeons were trained on two-component multiple variable-interval (VI) 60-sec, VI 240-sec schedules. In two conditions, resistance to change was tested by terminating the response-reinforcer contingency and presenting response-independent reinforcers at the same rate as in training. In two further conditions, resistance to change was tested by prefeeding and by extinction. In Experiment 2, 6 pigeons were trained on two-component multiple VI 150-sec schedules with 8-sec or 2-sec reinforcers, and resistance to change was tested by terminating the response-reinforcer contingency in three conditions. In two of those conditions, brief delays were interposed between responses and response-independent reinforcers. In both Experiments 1 and 2, response rate was more resistant to change in the richer component, except for extinction in Experiment 1. In Experiment 3, 8 pigeons were trained on multiple VI 30-sec, VI 120-sec schedules. During extinction, half of the presentations of each component were accompanied by a novel stimulus to produce generalization decrement. The extinction data of Experiments 1 and 3 were well described by our model. The value of the exponent relating response strength and reinforcement was similar in all three experiments.     

Remembering: The role of extraneous reinforcement

In two experiments, pigeons' responding on an extraneous task was explicitly reinforced during delayed matching-to-sample trials. In Experiment 1, red or green sample stimuli were followed by retention intervals of 0.2, 1, 4, or 12 sec, during which pecks to a white center key were reinforced with 2.5-sec access to wheat according to extinction, variable-interval 30-sec, and variable-interval 15-sec schedules in different conditions. A proportion of .2, .5, .7, or .9 of subsequent red or green choice responses that matched the sample were reinforced with 3-sec access to wheat. The result was that increasing center key reinforcement, or reducing reinforcer probability, lowered overall accuracy. Initial discriminability fell, but with no change in the rate of forgetting. In Experiment 2, initial discriminability was affected by extraneous reinforcers that were contingent on center key pecking, but not by noncontingent reinforcers. A plausible conclusion is that initial discriminability decreases when reinforcers strengthen competing behaviors.     

Rats show no preference between free and earned water in an advance-response procedure

Although an arbitrarily specified instrumental response may persist when free reinforcers are concurrently available, the interpretation that earned reinforcers are preferred is tenuous. The present advance-response procedure used both time allocation and advance response rates as indices of preference between free and earned water in rats. When multiple schedule components were two response-dependent schedules with different overall reinforcement rates, higher rates of reinforcement were preferred. However, when the multiple schedule consisted of response-dependent and response-independent components equated for overall rates of reinforcement, no consistent preference for free or earned reinforcers was evident. That a preference for free reinforcers was not obtained is difficult to reconcile with concepts of least effort.     

The role of response-reinforcer associations increases throughout extended instrumental training

Two experiments examined the effects of extended training on the development of response-reinforcer associations. Rats were trained by using various food reinforcers to make multiple instrumental responses. Subsequently, those reinforcers were devalued by being paired with a toxin. The presence of response-reinforcer associations was inferred from the decrease in the likelihood of a response following devaluation of its reinforcer. Such response-reinforcer associations are known to contribute to performance after moderate amounts of training. These experiments addressed the question of whether the contribution of those associations remains constant, increases, or decreases with more extended training. Experiment 1 found that even after a response had been extensively trained with one reinforcer, the substitution of a new reinforcer produced new associations between the response and that new reinforcer. After extended training, a response continued to acquire new associations with a reinforcer, as indexed by the impact of a devaluation procedure. Experiment 2 directly compared the contribution of reinforcers used extensively and moderately with the same response. It found that devaluation of the extensively used reinforcer more effectively reduced performance of the response, suggesting that the associations formed with additional training contribute to performance of the response. These experiments indicate that the contribution of response-reinforcer associations does not decrease but, instead, continues to grow throughout the course of extended instrumental training.     

Multiple determinants of “blocking” effects on operant behavior

Blocking was investigated in a free-operant procedure by presenting a response-contingent signal prior to reinforcer delivery. At issue was the way in which blocking effects previously reported with this procedure are related to conditioned reinforcement effects, also previously found with similar procedures. Signal presentation decreased response rate when delay of reinforcement was 0 or .5 sec, but the signal increased response rate when the delay of reinforcement was increased to 3 sec. Thus, which effect (blocking or conditioned reinforcement) occurred depended critically on the response-reinforcer interval.     

The effect of rate of reinforcement and time in session on preference for variability

Pigeons pecked keys on concurrent-chains schedules that provided a variable interval 30-sec schedule in the initial link. One terminal link provided reinforcers in a fixed manner; the other provided reinforcers in a variable manner with the same arithmetic mean as the fixed alternative. In Experiment 1, the terminal links provided fixed and variable interval schedules. In Experiment 2, the terminal links provided reinforcers after a fixed or a variable delay following the response that produced them. In Experiment 3, the terminal links provided reinforcers that were fixed or variable in size. Rate of reinforcement was varied by changing the scheduled interreinforcer interval in the terminal link from 5 to 225 sec. The subjects usually preferred the variable option in Experiments 1 and 2 but differed in preference in Experiment 3. The preference for variability was usually stronger for lower (longer terminal links) than for higher (shorter terminal links) rates of reinforcement. Preference did not change systematically with time in the session. Some aspects of these results are inconsistent with explanations for the preference for variability in terms of scaling factors, scalar expectancy theory, risk-sensitive models of optimal foraging theory, and habituation to the reinforcer. Initial-link response rates also changed within sessions when the schedules provided high, but not low, rates of reinforcement. Within-session changes in responding were similar for the two initial links. These similarities imply that habituation to the reinforcer is represented differently in theories of choice than are other variables related to reinforcement.     

Choice, delay, probability, and conditioned reinforcement

The hyperbolic-decay model is a mathematical expression of the relation between delay and reinforcer value. The model has been used to predict choices in discrete-trial experiments on delay-amount tradeoffs, on preference for variable over fixed delays, and on probabilistic reinforcement. Experiments manipulating the presence or absence of conditioned reinforcers on trials that end without primary reinforcement have provided evidence that the hyperbolic-decay model actually predicts the strength of conditioned reinforcers rather than the strength of delayed primary reinforcers. The model states that the strength of a conditioned reinforcer is inversely related to the time spent in its presence before a primary reinforcer is delivered. A possible way to integrate the model with Grace’s (1994) contextual-choice model for concurrent-chain schedules is presented. Also discussed are unresolved difficulties in determining exactly when a stimulus will or will not serve as a conditioned reinforcer.     

Reinforcement contexts effects on fixed-interval responding

Three rats responding on fixed-interval schedules received either 1 or 4 pellets at the end of 2-min intervals. Five experimental conditions manipulated the relative probabilities of these two reinforcers. Response rates following the 1-pellet reinforcer were always higher than the rates following the 4-pellet reinforcer. The rates after the 1-pellet reinforcer were also highest in those experimental conditions where it was delivered with low probability. Contrast effects were observed when two sequential fixed intervals differed in reinforcer magnitudes. It was concluded that the context of reinforcement as well as the specific reinforcer magnitude affects responding under fixed-interval schedules.     

Differences in responding for sucrose and wheel-running reinforcement: Excitatory stimulus effects or inhibitory after-effects?

Previous research showed that sucrose and wheel-running reinforcement of leverpressing generate different response rate asymptotes. To investigate the basis of this difference, the present study assessed the role of inhibitory after-effects and excitatory stimulus effects on measures of responding in rats exposed to fixed-interval schedules that randomly produced either sucrose or wheel-running reinforcers. Different discriminative stimuli were associated with each reinforcer type. Inhibitory aftereffects and excitatory stimulus effects were assessed by the pattern of postreinforcement pauses and local response rates across the four different combinations of previous and upcoming reinforcer types: wheel-wheel, wheel-sucrose, sucrose-wheel, and sucrose-sucrose. Results showed that, regardless of the prior type of reinforcer, response rates were higher and pauses were shorter in the presence of a stimulus signaling sucrose reinforcement. This suggests that differences in response rate asymptotes generated by these qualitatively different reinforcers may have more to do with differences in excitatory stimulus effects than with inhibitory after-effects.     

COMPARING METHODS OF IDENTIFYING REINFORCING STIMULI IN SCHOOL CONSULTATION

Reinforcement-based interventions, the most frequently used treatments for school-age children, rely on accurately identifying stimuli that will serve to reinforce appropriate classroom behavior. Research has consistently demonstrated that the results from a forced-choice pairing procedure are the best predictors of reinforcing stimuli. Interestingly, systematic evaluation of potential reinforcers is rarely implemented in the school consultation setting. Considering the importance of the reinforcer on reinforcement-based interventions, and the literature focusing on the significance of the selection procedure on accurately identifying a reinforcer, this is concerning. The purpose of these two studies was to examine the effectiveness of identifying reinforcing stimuli for students in the consultation setting using two different methods: stimulus forced-choice and asking the teacher to identify potential reinforcers. The effectiveness of the selected stimuli as reinforcers was studied on two student outcomes: academic production and on-task behavior. The results of the two studies suggested that the reinforcers selected using a forced-choice procedure were more effective than the reinforcers selected from a teacher-identification procedure. Further, results indicated that although stimuli derived from both reinforcer assessment methods were useful at increasing rates of desired behavior, stimuli derived from the forced-choice reinforcer assessment were more consistently effective.     

Choice between small certain and large uncertain reinforcers

When choosing between two alternatives that deliver the same amount of food per trial in the long run, organisms are calledrisk-averse if they choose a small certain reinforcer over a larger probabilistic reinforcer. They are calledrisk-prone if they choose the larger probabilistic reinforcer. This experiment attempted to predict whether rats would be risk-prone or risk-averse on the basis of their separate choices between reinforcers differing in probability and reinforcers differing in amount. Choice was measured with an adjusting-delay procedure, which provided estimates of indifference points, or pairs of alternatives that a subject chose about equally often. The subjects were usually more responsive to differences in amount than to differences in probability, leading to predictions of risk-proneness for choices between two probability-amount combinations. The predictions were confirmed in almost every case. As the number of food pellets delivered by the two alternatives was increased while maintaining a 2:1 difference between them, the tendency toward risk-proneness declined. These results suggest an explanation of the inconsistent findings obtained in previous experiments on risk-taking by rats.     

Choice and percentage reinforcement in pigeons

Pigeons responded on a two-key concurrent chains choice procedure with the same level of percentage reinforcement on each key. During the initial links, a choice response on either key occasionally produced a conditioned reinforcer—which on one key was associated with a 15-sec, and on the other key with a 30-sec, interreinforcement interval—or an extinction stimulus. In Part 1, the initial links were equal. With successive decreases in the probability of a reinforcer, choice shifted from preference for the 15-sec terminal link toward indifference. In Part 2, the initial links were unequal and were arranged so that the shorter initial link preceded the 30-sec terminal link. At a high probability of a reinforcer, the pigeons again preferred the 15-sec terminal link. However, at a low probability, the pigeons reversed and preferred the alternate key. It was concluded that the conditioned reinforcers tended to become functionally equivalent at a low probability of a reinforcer, despite the nominally different interreinforcement intervals, with the result that choice was then modulated by the relative size of the initial links. The data are inconsistent with the view that choice and the strength of conditioned reinforcers are isomorphic with the reduction in delay to reward correlated with terminal link stimuli.     

Instrumental responding by rats on free-operant schedules with components that schedule response-dependent reinforcer omission: Implications for optimization theories

In two experiments, we assessed whether rats optimize the number of reinforcers per response. In Experiment 1, one group of rats was trained to leverpress for food reinforcement on a simple variable-interval (VI) 60-sec schedule. For another group, a negative fixed-ratio component was imposed on the VI schedule to produce a conjoint contingency in which reinforcement became available on the VI schedule but was omitted when the ratio criterion was satisfied. In Experiment 2, one group of rats responded on a VI schedule and also received response-independent food. For another group, responding above a certain rate canceled the response-independent food. Despite the negative contingency experienced by the groups in Experiments 1 and 2, responding was maintained at a level at which the number of obtained reinforcers was reduced substantially below the maximum number possible. In addition, in both experiments, the groups that experienced the negative contingency responded at a lower rate than did a yoked control group that experienced the same frequency of reinforcement but lacked the negative component. These results demonstrate that although rats do not optimize their behavior with respect to reinforcement, they are nevertheless sensitive to some aspect of the instrumental contingency in operation.     

A matching law analysis of the reinforcing efficacy of wheel running in rats

Previous research has demonstrated that running in a rotating wheel functions as a reinforcer for leverpressing in rats. In these studies, the pattern of responding was similar to the pattern of responding maintained by consummatory reinforcers, such as food and water. The present study investigated quantitative features of responding maintained by running. In previous experiments in which responses were reinforced according to variable-interval (VI) schedules and food and water served as the reinforcer, the equation for a rectangular hyperbola described the relationship between response rate and reinforcement rate. This experiment tested whether this quantitative regularity also applies to leverpressing maintained by the opportunity to run in a wheel. Fourteen male Wistar rats responded on levers for the opportunity to run. In each session, subjects were exposed to a series of VI schedules. An opportunity to run for 60 sec was the reinforcing consequence. Results showed that response rate was a negatively accelerated function of reinforcement rate, and the relationship between these two variables was described well by the equation for a rectangular hyperbola. To further test the similarity between running and consummatory reinforcers, the response requirement and access were manipulated. In previous experiments with food and water, these types of manipulations differentially changed the two parameters of the hyperbola. A similar pattern of results was obtained with wheel running. Thus, the equation appears to apply to running about as well as it does to consummatory reinforcers.     

Rats’ choices between one and two delayed reinforcers

Rats chose between alternatives that differed in the number of reinforcers and in the delay to each reinforcer. A left leverpress led to two reinforcers, each delivered after a fixed delay. A right leverpress led to one reinforcer after an adjusting delay. The adjusting delay was increased or decreased many times in a session, depending on the rat’s choices, in order to estimate an indifference point& #x2014;a delay at which the two alternatives were chosen about equally often. Both the number of reinforcers and their individual delays affected the indifference points. The overall pattern of results was well described by the hyperbolic-decay model, which states that each additional reinforcer delivered by an alternative increases preference for that alternative but that a reinforcer’s effect is inversely related to its delay. Two other possible delay-discounting equations, an exponential equation and a reciprocal equation, did not produce satisfactory predictions for these data. Adding an additional free parameter to the hyperbolic equation as an exponent for delay did not appreciably improve the predictions, suggesting that raising delay to some power other than 1.0 was unnecessary. The results were qualitatively similar to those from a previous experiment with pigeons (Mazur, 1986), but quantitative differences suggested that the rates of delay discounting were several times slower for rats than for pigeons.     

Concurrent VR VI schedules: primacy of molar control of preference and molecular control of response rates

In the first condition in Experiment 1, 6 rats were exposed to concurrent variable ratio (VR) 30, variable interval (VI) 30-sec schedules. In the next two conditions, the subjects were exposed to concurrent VI VI schedules and concurrent tandem VI-differential-reinforcement-of-high-rate VI schedules. For the latter conditions, the overall and relative reinforcer rates equaled those in the first condition. Only minor differences appeared in time allocation (a molar measure) across conditions. However, local response rate differences (a molecular measure) appeared between schedule types consistently with the interresponse times these schedules reinforced. In Experiment 2, these findings reappeared when the prior experiment was replicated with 5 subjects, except that the VR schedule was replaced by a VI plus linear feedback schedule. These results suggest that within the context tested, the molar factor of relative reinforcement rate controls preference, whereas the molecular factor of the relation between interresponse times and reinforcer probability controls the local response rate.     

Renewal after the extinction of free operant behavior

Four experiments were performed to explore the role of context in operant extinction. In all experiments, leverpressing in rats was first reinforced with food pellets on a variable interval 30-s schedule, then extinguished, and finally tested in the same and a different physical context. The experiments demonstrated a clear ABA renewal effect, a recovery of extinguished responding when conditioning, extinction, and testing occurred in contexts A, B, and A, respectively. They also demonstrated ABC renewal (where conditioning extinction and testing occurred in contexts A, B, and C) and, for the first time in operant conditioning, AAB renewal (where conditioning, extinction, and testing occurred in contexts A, A, and B). The latter two phenomena indicate that tests outside the extinction context are sufficient to cause a recovery of extinguished operant behavior and, thus, that operant extinction, like Pavlovian extinction, is relatively specific to the context in which it is learned. AAB renewal was not weakened by tripling the amount of extinction training. ABA renewal was stronger than AAB, but not merely because of context A’s direct association with the reinforcer.