Qualitatively different reinforcers and parameters of Herrnstein’s (1970) response-strength equation

Previous research that compared the estimated parameters (i.e.,k andR _e) from Herrnstein’s (1970) hyperbolic matching law equation within the same individuals responding for qualitatively different consummatory reinforcers (i.e., water and sucrose solution) found similar asymptotic response rates (k). The present study compared these parameters within subjects responding on levers for consummatory and nonconsummatory reinforcers. Male Wistar rats responded on a lever in a running wheel on a series of tandem FR 1 VI schedules for either 0.1 ml of a 15% sucrose solution or the opportunity to run for 15 sec. Herrnstein’s hyperbola was fit to response and reinforcement rates from each session. Results showed thatk values were significantly higher for sucrose than for wheel-running reinforcement. On average,R _e was lower for sucrose than for wheel-running reinforcement, though not significantly lower. The results of the present study appear to violate the assumption of the constancy ofk in Herrnstein’s matching law analysis.     

Differences in responding for sucrose and wheel-running reinforcement: Excitatory stimulus effects or inhibitory after-effects?

Previous research showed that sucrose and wheel-running reinforcement of leverpressing generate different response rate asymptotes. To investigate the basis of this difference, the present study assessed the role of inhibitory after-effects and excitatory stimulus effects on measures of responding in rats exposed to fixed-interval schedules that randomly produced either sucrose or wheel-running reinforcers. Different discriminative stimuli were associated with each reinforcer type. Inhibitory aftereffects and excitatory stimulus effects were assessed by the pattern of postreinforcement pauses and local response rates across the four different combinations of previous and upcoming reinforcer types: wheel-wheel, wheel-sucrose, sucrose-wheel, and sucrose-sucrose. Results showed that, regardless of the prior type of reinforcer, response rates were higher and pauses were shorter in the presence of a stimulus signaling sucrose reinforcement. This suggests that differences in response rate asymptotes generated by these qualitatively different reinforcers may have more to do with differences in excitatory stimulus effects than with inhibitory after-effects.     

Prediction of concurrent keypeck treadle-press responding from simple schedule performance

Four pigeons pecked keys and pressed treadles for food reinforcers delivered by several variable-interval schedules of reinforcement. Then the subjects responded on several concurrent schedules. Keypecking produced reinforcers in one component, and treadle-pressing produced reinforcers in the other. The changeover delay, which prevented reinforcement after all switches from one response to the other, was 0, 5, or 20 sec long. An equation proposed by Kerrnstein (1970) described the rates of treadle-pressing and keypecking emitted during the variable-interval schedules. The k parameter of this equation was larger for keypecking than for treadle-pressing. The R₀ parameters were not systematically different for the two responses. The rates of keypecking and treadle-pressing emitted during the components of the concurrent schedules correlated with, but were not equal to, the rates of responding predicted by Herrnstein’s equation and the subject’s simple schedule responding. The ratios of the rates of responding emitted during, and the ratios of the time spent responding on, the components of the concurrent schedules conformed to an equation proposed by Baum (1974), but not to Herrnstein’s equation.     

Resistance to extinction: contingency termination and generalization decrement

We present an algebraic model of resistance to extinction that is consistent with research on resistance to change. The model assumes that response strength is a power function of reinforcer rate and that extinction involves two additive, decremental processes: (1) the termination of the reinforcement contingency and (2) generalization decrement resulting from reinforcer omission. The model was supported by three experiments. In Experiment 1, 4 pigeons were trained on two-component multiple variable-interval (VI) 60-sec, VI 240-sec schedules. In two conditions, resistance to change was tested by terminating the response-reinforcer contingency and presenting response-independent reinforcers at the same rate as in training. In two further conditions, resistance to change was tested by prefeeding and by extinction. In Experiment 2, 6 pigeons were trained on two-component multiple VI 150-sec schedules with 8-sec or 2-sec reinforcers, and resistance to change was tested by terminating the response-reinforcer contingency in three conditions. In two of those conditions, brief delays were interposed between responses and response-independent reinforcers. In both Experiments 1 and 2, response rate was more resistant to change in the richer component, except for extinction in Experiment 1. In Experiment 3, 8 pigeons were trained on multiple VI 30-sec, VI 120-sec schedules. During extinction, half of the presentations of each component were accompanied by a novel stimulus to produce generalization decrement. The extinction data of Experiments 1 and 3 were well described by our model. The value of the exponent relating response strength and reinforcement was similar in all three experiments.     

Concurrent schedules of wheel-running reinforcement: Choice between different durations of opportunity to run in rats

How do animals choose between opportunities to run of different durations? Are longer durations preferred over shorter durations because they permit a greater number of revolutions? Are shorter durations preferred because they engender higher rates of running? Will longer durations be chosen because running is less constrained? The present study reports on three experiments that attempted to address these questions. In the first experiment, five male Wistar rats chose between 10-sec and 50-sec opportunities to run on modified concurrent variable-interval (VI) schedules. Across conditions, the durations associated with the alternatives were reversed. Response, time, and reinforcer proportions did not vary from indifference. In a second experiment, eight female Long-Evans rats chose between opportunities to run of equal (30 sec) and unequal durations (10 sec and 50 sec) on concurrent variable-ratio (VR) schedules. As in Experiment 1, between presentations of equal duration conditions, 10-sec and 50-sec durations were reversed. Results showed that response, time, and reinforcer proportions on an alternative did not vary with reinforcer duration. In a third experiment, using concurrent VR schedules, durations were systematically varied to decrease the shorter duration toward 0 sec. As the shorter duration decreased, response, time, and reinforcer proportions shifted toward the longer duration. In summary, differences in durations of opportunities to run did not affect choice behavior in a manner consistent with the assumption that a longer reinforcer is a larger reinforcer.     

Habituation may contribute to within-session decreases in responding under high-rate schedules of reinforcement

Two experiments tested the hypothesis that habituation contributes to within-session decreases in responding. In Experiment 1, rats’ leverpressing was reinforced under a fixed ratio (FR) 4 schedule throughout the baseline sessions. During the dishabituation sessions, the first 21 min and the last 21 min were FR 4; dishabituating events occurred during the middle 3 min. The dishabituating events altered the manner of reinforcer delivery in four different ways. Response rates increased after all dishabituating events. In Experiment 2, rats responded on several FR and variable ratio (VR) schedules. The ratio requirement varied from 3 to 15. Within-session decreases in responding were steeper during FR schedules than during VR schedules. In addition, response rates were well described as linear functions of cumulative number of food pellets eaten within sessions. These results support the habituation hypothesis but do not rule out the possibility that other satiety variables might contribute simultaneously.     

Reinforcement contexts effects on fixed-interval responding

Three rats responding on fixed-interval schedules received either 1 or 4 pellets at the end of 2-min intervals. Five experimental conditions manipulated the relative probabilities of these two reinforcers. Response rates following the 1-pellet reinforcer were always higher than the rates following the 4-pellet reinforcer. The rates after the 1-pellet reinforcer were also highest in those experimental conditions where it was delivered with low probability. Contrast effects were observed when two sequential fixed intervals differed in reinforcer magnitudes. It was concluded that the context of reinforcement as well as the specific reinforcer magnitude affects responding under fixed-interval schedules.     

Sum of responding as a function of sum of reinforcement on two-key concurrent schedules

Four pigeons pecked for food reinforcers delivered by several two-key concurrent schedules. The sum of the rates of reinforcement provided by the component schedules varied from 25 to 300 reinforcers/h. The ratio of the rates of reinforcement remained constant at 1:4. The sum of the rates of responding generated by the component schedules increased with increases in the sum of the rates of reinforcement which the components provided. The increase in response rate was predicted by equations proposed by Catania (1963) and by Herrnstein (1970). But the data conformed more closely to Herrnstein’s equation.     

Rates of responding in the pigeon generated by simple and complex schedules which provide the same rates of reinforcement

Four pigeons pecked for food reinforcement on variable interval 1-min schedules and on the variable-interval 1-min components of multiple, concurrent, and pseudoconcurrent schedules. The pseudoconcurrent schedule provided only one schedule of reinforcement; but, any reinforcer could be collected by responding on either of two keys. The rate of responding generated by the variable interval schedule was not greater than the rates of responding generated by the components of the complex schedules. But, the rate of reinforcement obtained from the variable interval schedule was greater than the rates of reinforcement obtained from the components of the multiple schedule. These results may contradict the equation proposed by Herrnstein (1970). The equation predicts that the rate of responding generated by a schedule of reinforcement will be greater when the schedule appears alone, than when it appears as one component of a complex schedule.     

Instrumental responding by rats on free-operant schedules with components that schedule response-dependent reinforcer omission: Implications for optimization theories

In two experiments, we assessed whether rats optimize the number of reinforcers per response. In Experiment 1, one group of rats was trained to leverpress for food reinforcement on a simple variable-interval (VI) 60-sec schedule. For another group, a negative fixed-ratio component was imposed on the VI schedule to produce a conjoint contingency in which reinforcement became available on the VI schedule but was omitted when the ratio criterion was satisfied. In Experiment 2, one group of rats responded on a VI schedule and also received response-independent food. For another group, responding above a certain rate canceled the response-independent food. Despite the negative contingency experienced by the groups in Experiments 1 and 2, responding was maintained at a level at which the number of obtained reinforcers was reduced substantially below the maximum number possible. In addition, in both experiments, the groups that experienced the negative contingency responded at a lower rate than did a yoked control group that experienced the same frequency of reinforcement but lacked the negative component. These results demonstrate that although rats do not optimize their behavior with respect to reinforcement, they are nevertheless sensitive to some aspect of the instrumental contingency in operation.     

Tracking of the expected time to reinforcement in temporal conditioning procedures

In one experiment, the rate and pattern of responding (head entry into the food cup) under different distributions of intervals between food deliveries were examined. Separate groups of rats received fixed-time (45, 90, 180, or 360 sec), random-time (45, 90, 180, or 360 sec), or tandem fixed-time (45 or 90 sec) random-time (45 or 90 sec) schedules of reinforcement. Schedule type affected the pattern of responding as a function of time, whereas mean interval duration affected the mean rate of responding. Responses occurred in bouts with characteristics that were invariant across conditions. Packet theory, which assumes that the momentary probability of bout occurrence is negatively related to the conditional expected time remaining until the next reinforcer, accurately predicted global and local measures of responding. The success of the model advances the prediction of multiple measures of responding across different types of time-based schedules.     

The relation of multiple-schedule behavioral contrast to deprivation,time in session,and within-session changes in responding

Pigeons' keypecking was reinforced by food on baseline schedules of multiple variable interval (VI) x VI x and on contrast schedules of multiple VI x VI y. Deprivation of food was varied by maintaining subjects at 75%, 85%, and 95% (+/- 2%) of their free-feeding weights. Positive and negative behavioral contrast were observed. The size of the contrast was not systematically altered by changes in deprivation. Positive and negative contrast were both larger later in the session than they were earlier. Within-session decreases in responding were steeper for the baseline than for the contrast schedules for positive contrast. Within-session decreases were steeper for the contrast than for the baseline schedules for negative contrast. These results were predicted by the idea that different amounts of habituation to the reinforcer during the baseline and contrast schedules contribute to behavioral contrast. The results show that contrast occurs under conditions that reduce the effect of the following component. The results support the assumption that positive and negative contrast are produced by symmetrical theoretical variables.     

Intermittent reinforcement and salt arousal of thirst in rats

Sated rats, previously trained to leverpress for H₂O reinforcement on continuous or variable-interval (10-sec or 60-sec) schedules, were given NaCl injections and tested for leverpressing. Under all schedules, responding was an inverted U function of NaCl concentration (0.15M to 3.0M). However, NaCl thirst produced relatively little change in behavior under the VI 60-sec schedule.     

The role of response-reinforcer correlation in signaled reinforcement effects

In three experiments, we examined the effects of signaling reinforcement during operant responding in order to illuminate the factors underlying instrumental overshadowing and potentiation effects. Specifically, we examined whether signaling reinforcement produces an enhancement and attentuation of responding when the response-reinforcer correlation is weak and strong, respectively. In Experiment 1, rats responded on variable-ratio (VR) or variable-interval (VI) schedules that were equated for the number of responses emitted per reinforcer. A signal correlated with reinforcement enhanced response rates on the VR schedule, but attenuated response rates were produced by the signal on the VI schedule. In Experiment 2, two groups of rats responded on a VI schedule while the two other groups received a conjoint VI, negative fixed-ratio schedule in which the subjects lost the availability of reinforcements if they emitted high response rates. A reinforcement signal attenuated responding for the simple VI groups but not for the animals given the negative fixed-ratio component, although the signal improved response efficiency in both groups. In Experiment 3, a poor correlation between responding and reinforcement was produced by a VI schedule onto which the delivery of response-independent food was superimposed. A signal for reinforcement initially elevated responding on this schedule, relative to an unsignaled condition; however, this pattern was reversed with further training. In sum, the present experiments provide little support for the view that signaling reinforcement enhances responding when the response-reinforcer correlation is weak and attenuates responding when this correlation is strong.     

Behavioral contrast as a function of component duration for leverpressing using a within-session procedure

Ten rats pressed levers for food reinforcers delivered by multiple schedules. Behavioral contrast was measured using a within-session procedure that presented the baseline and contrast schedules within single sessions. The absolute sizes of both positive and negative contrast increased and then decreased as components lengthened. Negative induction occurred when components were very short. These results question theories that predict that the size of contrast will vary inversely with component duration. They support theories that attribute positive and negative contrast to similar theoretical mechanisms. A comparison of the present results with those of past studies indicates that keypecking by pigeons and leverpressing by rats change as different functions of component duration. Treadlepressing by pigeons and leverpressing by rats change as similar functions. These findings challenge general process theories that argue that all responses obey the same behavioral laws.     

The effect of rate of reinforcement and time in session on preference for variability

Pigeons pecked keys on concurrent-chains schedules that provided a variable interval 30-sec schedule in the initial link. One terminal link provided reinforcers in a fixed manner; the other provided reinforcers in a variable manner with the same arithmetic mean as the fixed alternative. In Experiment 1, the terminal links provided fixed and variable interval schedules. In Experiment 2, the terminal links provided reinforcers after a fixed or a variable delay following the response that produced them. In Experiment 3, the terminal links provided reinforcers that were fixed or variable in size. Rate of reinforcement was varied by changing the scheduled interreinforcer interval in the terminal link from 5 to 225 sec. The subjects usually preferred the variable option in Experiments 1 and 2 but differed in preference in Experiment 3. The preference for variability was usually stronger for lower (longer terminal links) than for higher (shorter terminal links) rates of reinforcement. Preference did not change systematically with time in the session. Some aspects of these results are inconsistent with explanations for the preference for variability in terms of scaling factors, scalar expectancy theory, risk-sensitive models of optimal foraging theory, and habituation to the reinforcer. Initial-link response rates also changed within sessions when the schedules provided high, but not low, rates of reinforcement. Within-session changes in responding were similar for the two initial links. These similarities imply that habituation to the reinforcer is represented differently in theories of choice than are other variables related to reinforcement.     

Excitatory and inhibitory effects of free reinforcers

Rats’ responding was maintained on a random-interval 1-min food schedule. In addition, non-contingent pellets were delivered, independently of the animals’ behavior, at either fixed intervals (Experiment 1) or at random intervals (Experiment 2). As the rate of delivery of the periodic and aperiodic free reinforcers increased, the rate of responding decreased. But these free reinforcers, in addition to having this inhibitory effect, had also a local excitatory effect upon responding: lever-pressing increased to a level above its mean rate following the delivery of a free food pellet. The time course of this behavioral aftereffect of free reinforcers, for both fixed and random intervals, was dependent upon the proportion of the interval between successive free food deliveries. The relation of these results to those obtained with response-contingent reinforcement, free punishment, and in schedule-induced phenomena is discussed.     

Mechanisms of inhibitory discriminative control

Rats were trained to discriminate between trials signaled by a tone, during which leverpressing was reinforced with food, and trials signaled by the tone in compound with a light stimulus, during which no reinforcers were delivered. A subsequent transfer test suggested that the light had acquired the ability to suppress operant responding; there was no evidence that this suppression could be attributed to Pavlovian inhibitory conditioning. It is argued that these data cannot be accommodated by current accounts of discriminative control.     

Positive behavioral contrast as a function of time-out duration when pigeons peck keys on a within-session procedure

Three pigeons pecked keys for food reinforcers delivered by multiple variable interval variable interval schedules in the first part of each session (baseline) and by multiple variable interval extinction schedules in the second part of each session (contrast). The variable interval schedules delivered reinforcers after an average of 4 min or 30 sec in different conditions. The duration of a time-out between the components varied in five steps from 5 to 120 sec. Positive contrast occurred for all time-out durations in both experiments. That is, the rate of responding emitted during the constant, variable interval component was greater during the contrast than during the baseline schedules. The size of contrast did not change systematically with changes in timeout duration. These results violate most theories of contrast. They are compatible with the idea that animals integrate reinforcers over intervals longer than 2 min.