Multiple determinants of transfer of evaluative function after conditioning with free-operant schedules of reinforcement

The aim of the four present experiments was to explore how different schedules of reinforcement influence schedule-induced behavior, their impact on evaluative ratings given to conditioned stimuli associated with each schedule through evaluative conditioning, and the transfer of these evaluations through derived stimulus networks. Experiment 1 compared two contrasting response reinforcement rules (variable ratio [VR], variable interval [VI]). Experiment 2 varied the response to reinforcement rule between two schedules but equated the outcome to response rate (differential reinforcement of high rate [DRH] vs. VR). Experiment 3 compared molar and molecular aspects of contingencies of reinforcement (tandem VIVR vs. tandem VRVI). Finally, Experiment 4 employed schedules that induced low rates of responding to determine whether, under these circumstances, responses were more sensitive to the molecular aspects of a schedule (differential reinforcement of low rate [DRL] vs. VI). The findings suggest that the transfer of evaluative functions is determined mainly by differences in response rate between the schedules and the molar aspects of the schedules. However, when neither schedule was based on a strong response reinforcement rule, the transfer of evaluative judgments came under the control of the molecular aspects of the schedule.     

Differential conditionability: Reinforcing grooming in golden hamsters

Differential conditionability is the empirical finding that not all responses are equally amenable to the same conditioning paradigm. One phenomenon associated with the conditioning of grooming behavior (a difficult-to-condition response) is a decrease in its average duration when followed by food reinforcement. The first experiment investigated this phenomenon by reinforcing golden hamsters (Mesocricetus auratus) with food for grooming or open rearing (a readily conditionable response) under three duration-dependent reinforcement schedules. The obtained data showed that different densities of food delivery had no differential effects on the average durations of grooming responses, indicating that the decreases were not the result of reinforcement-produced interruption. In the second experiment, golden hamsters were reinforced with food for grooming or for open rearing or received free food, under three interval reinforcement schedules. This experiment demonstrated that decreases in the average duration of grooming are independent of grooming behavior’s resistance to conditioning. Furthermore, although duration-dependent reinforcement schedules are largely ineffective in conditioning grooming behavior, interval schedules are shown to be quite effective in increasing rates of grooming.     

Interval timing under a behavioral microscope: Dissociating motivational and timing processes in fixed-interval performance

The distribution of latencies and interresponse times (IRTs) of rats was compared between two fixed-interval (FI) schedules of food reinforcement (FI 30 s and FI 90 s), and between two levels of food deprivation. Computational modeling revealed that latencies and IRTs were well described by mixture probability distributions embodying two-state Markov chains. Analysis of these models revealed that only a subset of latencies is sensitive to the periodicity of reinforcement, and prefeeding only reduces the size of this subset. The distribution of IRTs suggests that behavior in FI schedules is organized in bouts that lengthen and ramp up in frequency with proximity to reinforcement. Prefeeding slowed down the lengthening of bouts and increased the time between bouts. When concatenated, latency and IRT models adequately reproduced sigmoidal FI response functions. These findings suggest that behavior in FI schedules fluctuates in and out of schedule control; an account of such fluctuation suggests that timing and motivation are dissociable components of FI performance. These mixture-distribution models also provide novel insights on the motivational, associative, and timing processes expressed in FI performance. These processes may be obscured, however, when performance in timing tasks is analyzed in terms of mean response rates.     

Operant matching is not a logical consequence of maximizing reinforcement rate

The distribution of behavior between concurrently available schedules of reinforcement approximates the distribution of reinforcements between the schedules. This equality, called matching, has been explained as an instance of the principle that organisms maximize reinforcement rate. However, a precise account of the relationship between the distribution of behavior and reinforcement rate on the standard concurrent schedule shows that matching and maximizing are different.     

Factors affecting schedule-induced wood-chewing in rats: Percentage and rate of reinforcement,and operant requirement

In Experiment 1, rats were allowed to acquire either schedule-induced drinking or schedule-induced wood-chewing behavior under a fixed-interval (FI) 60-sec schedule of food reinforcement, following which food was omitted from 20% and then 50% of interreinforcement intervals. Omission of food severely disrupted induced drinking but had relatively little effect on induced wood-chewing. Experiment 2 investigated wood-chewing as a function of reinforcement rate, using a range of FI schedules from 5 to 180 sec in duration. Both the amount of chewing per session and the relative time spent chewing were bitonically related to reinforcement rate. In Experiment 3, schedule-induced chewing that had been acquired under a response-dependent schedule was found to persist under a response-independent schedule. Induced wood-chewing resembles other induced behaviors in important respects, but quantitative differences are also apparent.     

Adjunctive behavior in multiple schedules of reinforcement

A recent theory of timing (Killeen & Fetterman, 1988) suggests that adjunctive behaviors may act as discriminative cues for the passage of time and that the rate of transition between those behaviors is affected by the rate of reinforcement within the experimental context. Is the rate of transition between behaviors correlated with the rate of reinforcement? What is the context in which rate of reinforcement is calibrated? If rate of transition is correlated with reinforcement frequency, does this correlation change with extended training? Four pigeons were trained on multiple fixed-time schedules of reinforcement, with one component always FT 15 sec, the other either FT 15 sec, FT 45 sec, or FT 5 sec. Behavior was coded into one of 12 categories. Response distributions in the constant component shifted when rate of reinforcement was varied in the other component and eventually shifted back toward their original location.     

The maintenance of matching behavior when contrast occurs in multiple concurrent concurrent schedules of reinforcement

Pigeons were trained on a multiple schedule of reinforcement in which each component was a concurrent schedule. The concurrent schedules were programmed by the changeover-key procedure. The primary purpose was to determine if the relative behavior allocated to two response alternatives is affected when absolute changes in these behaviors occur; i.e., to determine if matching is affected when positive behavioral contrast occurs. Results showed that (1) relative behavior in the unaltered component of the multiple schedule is not disrupted when positive contrast occurs in that component, (2) positive contrast occurred when the overall frequency of reinforcement in the reinforcement-correlated component(s) was high, but not when it was low, (3) changeover behavior was susceptible to positive contrast effects, and (4) changeover contrast and food-key contrast are independent phenomena.     

Changeover contingencies and choice on concurrent schedules

Pigeons were trained on a multiple concurrent schedule with two components per session. In one component, changing schedules required the completion of a small fixed ratio on the switching key (a fixed-ratio changeover, or FRCO), and in the other component, changing schedules required only one switching response but engaged a changeover delay (COD) during which keypecks were not reinforced. Response ratios overmatched reinforcer ratios under the FRCO but undermatched under the COD. There was no difference in time allocation. In addition to these molar regularities in behavior, there were characteristic differences in performance at the molecular level. These local patterns of behavior, which can be explained within the context of contingencies created by the different changeover requirements, appear to underlie differences in performance at the molar level. Obtained molar differences in performance are not compatible with the assumption that there is a “general outcome” on concurrent schedules; and explaining these molar differences in performance in terms of the local contingencies of reinforcement is contrary to the assumption that behavior is allocated as a function of molar distributions of reinforcers.     

Differences in food consumption under intermittent and continuous reinforcement schedules of water delivery: Some implications for schedule-induced behavior

Consumption of food pellets was examined in four water-deprived rats during 1-h sessions in which water was presented once every 30, 60, or 120 sec independently of the rats’ behavior according to three fixed-time (FT) schedules. Correlated with each FT condition was a continuous reinforcement (CRF) control condition in which the rats received, at the start of the session, the number of dipper presentations that were programmed to occur during the corresponding FT condition. During both the FT and CRF conditions, pellets per dipper presentation decreased and food intake rate increased with rate of water presentation, and there was a direct linear relation between log food intake and log water intake. For each of these three measures there was less eating under the FT condition than under the CRF condition, but the difference between the FT and CRF functions decreased at shorter FT values. These data are discussed in terms of the effects of amount of water on food consumption and the principle of temporal summation.     

Behavioral contrast and type of reward: Role of elicited response topography

Groups of pigeons were exposed to multiple variable-interval variable-interval and multiple variable-interval extinction schedules of either food or water reinforcement for keypecking. Discriminative stimuli associated with component schedules were located either on the operant key or on a second “signal” key. When the stimuli were projected on the operant key, positive contrast appeared during discrimination conditions with either food or water as the reinforcer. When the stimuli were projected on the signal key, overall responding to the operant and signal keys showed contrast with food, but negative induction with water as the reinforcer. In the latter condition, the signal for the variable-interval shcedule of water reinforcement elicited a variety of water-related behavior, only some of which was directed at the signal. Thus, the type of reward and location of discriminative stimuli interacted to determine the presence or absence of behavioral contrast effects. In large part, these results support and extend the autoshaping view of contrast.     

Behavioral momentum and relapse of extinguished operant responding

Previous experiments on behavioral momentum have shown that relative resistance to extinction of operant behavior in the presence of a stimulus depends on the rate of reinforcement associated with that stimulus, even if some of those reinforcers occur independently of the behavior. We present three experiments examining whether the rate of reinforcement in the presence of a stimulus similarly modulates the relative relapse of operant behavior produced by reinstatement, resurgence, and renewal paradigms. During baseline conditions, pigeons responded for food reinforcement on variable-interval 120-sec schedules in alternating periods of exposure to two stimuli arranged by a multiple schedule. Additional response-independent food presentations were also delivered in the presence of one of the multiple-schedule stimuli. Consistent with previous research, baseline response rates were lower in the presence of the stimulus with the added response-independent reinforcement, and relative resistance to extinction was greater in the presence of that stimulus. In addition, following extinction, the relative relapse of responding produced by reinstatement, resurgence, and renewal paradigms was greater in the presence of the stimulus associated with the higher rate of reinforcement. We suggest that a model of extinction from behavioral momentum theory may be useful for understanding these results.     

Extinction of operant behavior: An analysis based on foraging considerations

In two experiments, the frequency of food reinforcement provided by variable interval (VI) schedules prior to extinction was varied. In the first experiment, two-component multiple schedules resulted in a greater number of responses in extinction in the presence of the stimulus previously associated with the richer of the two component schedules than that previously associated with the leaner schedule. In the second experiment, different groups of animals were trained on different VI schedules. Responding in extinction was analyzed into bouts of responding showing that the number of response bouts increased and the number of responses per bout decreased with decreasing frequency of reinforcement during training. These data are compatible with an analysis of operant behavior based on an analogy to processes that presumably-occur-in naturalistic foraging situations. According to this analogy, behavior associated with search for a food source (i.e., number of response bouts) and that of procurement of food from a source (i.e., responses per bout) represent aspects of behavior that are differentially strengthened by different VI schedules. Extinction serves to reveal this differential strengthening.     

Optimization of reward rate on concurrent variable-interval variable-interval schedules

Six rats were placed on concurrent variable-interval variable-interval schedules with a 15-sec changeover delay (COD). The variable-interval schedules were varied such that the COD comprised between 25% and 100% of the average interreinforcement interval of the more favorable alternative. The obtained reinforcement rate and the rate of changing from one schedule to the other were compared to predictions of Houston and McNamara’s (1981) optimality model of concurrent choice. The pattern of behavioral allocation was consistent with the predictions of the model, although none of the animals was able to achieve optimal performance on any of the presented schedules. Observed behavior reliably tracked optimal behavior in that the ratio of obtained reinforcers to the optimum predicted by Houston and McNamara did not vary as the underlying schedule parameters was changed.     

Choice responding following multiple schedule training

Pigeons’ choice responding on 10-sec interpolated probes was studied after baseline training on multiple variable-interval variable-interval schedules of food reinforcement. Unreinforced choice following training with three different relative reinforcement rates (Experiment 1), with a 3-ply multiple schedule (Experiment 2), and with three different relative reinforcement durations (Experiment 3) was examined. Least squares lines were fit to choice relative response rate and schedule relative response rate as functions of training relative reinforcement rate; choice slope was significantly greater than schedule slope in all three experiments. This result is counter to the prediction of Herrnstein’s (1970) theory that these slopes should not differ. Luce’s (1959) theory also failed to account for the data. It was concluded that choice responding was controlled by both approach to the stimulus associated with the smaller mean interreinforcer interval or the longer duration, and avoidance of the other stimulus.     

Rates of responding in the pigeon generated by simple and complex schedules which provide the same rates of reinforcement

Four pigeons pecked for food reinforcement on variable interval 1-min schedules and on the variable-interval 1-min components of multiple, concurrent, and pseudoconcurrent schedules. The pseudoconcurrent schedule provided only one schedule of reinforcement; but, any reinforcer could be collected by responding on either of two keys. The rate of responding generated by the variable interval schedule was not greater than the rates of responding generated by the components of the complex schedules. But, the rate of reinforcement obtained from the variable interval schedule was greater than the rates of reinforcement obtained from the components of the multiple schedule. These results may contradict the equation proposed by Herrnstein (1970). The equation predicts that the rate of responding generated by a schedule of reinforcement will be greater when the schedule appears alone, than when it appears as one component of a complex schedule.     

Decreasing “obscene” language of behaviorally disordered children through the use of a DRL schedule

The purpose of the present study was to determine the value of DRL schedules in reducing the behavior of a group of behaviorally disordered children. During the DRL procedure, reinforcement was made available for inappropriate language that was less than a prescribed limit for each day. When the DRL limit was successively decreased across weeks, the behavior of the students systematically decreased. Data indicated that reinforcing low rates of behavior may be an effective alternative to the use of aversive contingencies to reduce the misbehavior of such children.     

Stimulus control of behavior during the postreinforcement pause of FI schedules

Pigeons were given the opportunity to terminate certain segments of fixed intervals by pecking a control key. When 30-sec segments of negative and positive stimuli alternated across the interreinforcement interval (Experiment 1), most birds terminated a large proportion of negative segments. However, few control-key responses were made during the negative segment immediately following food presentation. Under schedules during which only one negative segment was programmed, during the first 30 sec of 1-min intervals (Experiment 2), control-key responses, when they occurred at all, were made after several seconds of the interval had elapsed. Similar findings were obtained when a peck on the control key merely changed the color on the food key (Experiment 3). These findings suggest that the post-reinforcement extinction state (Schneider, 1969) during fixed-interval schedules consists of two phases: an immediate postreinforcement inhibitory phase, followed by a second phase during which a control-key response may occur. These two phases and their associated behavior may be related to Staddon’s (1977) distinction between interim and facultative activities.     

Procedure for preventing response strain on random interval schedules with a linear feedback loop

An experiment examined the impact of a procedure designed to prevent response or extinction strain occurring on random interval schedules with a linear feedback loop (i.e., an RI+ schedule). Rats lever-pressed for food reinforcement on either a RI+ or a random interval (RI) schedule that was matched to the RI+ schedule in terms of reinforcement rate. Two groups of rats responded on an RI+ and two on an RI schedule matched for rate of reinforcement. One group on each schedule also received response-independent food if there had been no response for 60 s, and response-independent food continued to be delivered on an RT-60 schedule until a response was made. Rats on the RI and RI+ obtained similar rates of reinforcement and had similar reinforced inter-response times to one another. On the schedules without response-independent food, rats had similar rates of response to one another. However, while the delivery of response-independent food reduced rates of response on an RI schedule, they enhanced response rates on an RI+ schedule. These results suggest that rats can display sensitivity to the molar aspects of the free-operant contingency, when procedures are implemented to reduce the impact of factors such as extinction-strain.     

The effects of variations in the interpellet interval on wheel running in the rat

Three rats were given access to a running wheel during spaced food deliveries. As the interpellet interval increased, the overall amount of running increased, the rate of running neither systematically increased nor decreased, and the locus of maximal running occurred later in the interpellet interval. Both the overall amount and the temporal distribution of running exhibited considerable within-schedule changes during successive exposures to the different schedules. These data are discussed in terms of Staddon’s (1977) assignment of behavior into different functional classes, that is, schedule-induced and facultative.