Effect on subsequent fixed-interval schedule performance of prior exposure to ratio and interval schedules of reinforcement

In three experiments, we examined the effect on the patterns of responding noted on fixed interval (FI) schedules of prior exposure to a range of interval and ratio schedules. Rats leverpressed for food reinforcement on random ratio (RR), random interval (RI), or variable interval (VI) schedules prior to transfer to FI schedules. In Experiment 1, prior exposure to an RR schedule retarded the development of typical FI patterns of responding. Exposure to a yoked RI schedule produced even greater retardation of typical FI performance. This effect was replicated in Experiment 2, using a within-subjects design. Rats responded on a multiple RR-RI schedule prior to a multiple FI-FI schedule. Typical FI performance emerged more slowly in the component previously associated with the RI than with that associated with the RR. In Experiment 3, exposure to an RR schedule retarded the development of FI performance to a greater extent than did exposure to a VR schedule. The latter schedule was programmed to allow the possibility that inhibitory control would develop after reinforcement. These results confirm that ratio schedules independently result in the disruption of FI responding. This effect was not long lasting and cannot be used plausibly to explain species differences in responding to FI schedules. However, it does suggest that temporal control—as manifested by the transfer of inhibitory control from one schedule to another—could facilitate movement between interval schedules.     

Procedure for preventing response strain on random interval schedules with a linear feedback loop

An experiment examined the impact of a procedure designed to prevent response or extinction strain occurring on random interval schedules with a linear feedback loop (i.e., an RI+ schedule). Rats lever-pressed for food reinforcement on either a RI+ or a random interval (RI) schedule that was matched to the RI+ schedule in terms of reinforcement rate. Two groups of rats responded on an RI+ and two on an RI schedule matched for rate of reinforcement. One group on each schedule also received response-independent food if there had been no response for 60 s, and response-independent food continued to be delivered on an RT-60 schedule until a response was made. Rats on the RI and RI+ obtained similar rates of reinforcement and had similar reinforced inter-response times to one another. On the schedules without response-independent food, rats had similar rates of response to one another. However, while the delivery of response-independent food reduced rates of response on an RI schedule, they enhanced response rates on an RI+ schedule. These results suggest that rats can display sensitivity to the molar aspects of the free-operant contingency, when procedures are implemented to reduce the impact of factors such as extinction-strain.     

Resurgence of behavior during extinction depends on previous rate of response

In two experiments, food-deprived rat subjects leverpressed for food in three successive training phases. In the first phase of both experiments, rats were exposed to a multiple schedule, one component of which produced a high rate of response, and the other of which produced a lower rate of response (multiple random ratio [RR], random interval [RI] in Experiment 1, and multiple differential reinforcement of high rate, differential reinforcement of low rate in Experiment 2). Rats were then transferred to a multiple fixed interval (FI; 60-sec, 60-sec) schedule, until the effects of the first phase on response rate were no longer apparent and their response rates did not differ from those of rats responding on a multiple FI 60-sec, FI 60-sec schedule without previously experiencing a multiple RR, RI schedule. During the third stage oftraining, all rats were placed into extinction. During extinction, rates of responding were higher in the component previously associated with the high rate of responding in Phase 1, and they were lower in the component previously associated with low rates of responding in Phase 1. These results suggest that resurgence effects, like other history effects, are controlled by previous rates of responding.     

Conjunctive schedules of reinforcement IV: Effects on the pattern of responding of changes in requirement at reinforcement

The effects of different schedule requirements at reinforcement on patterns of responding by pigeons were assessed under conjunctive schedules with comparable response-number requirements, Under one conjunctive schedule (conjunctive fixed-interval fixed-ratio schedule), a response was reinforced after a 6-min interval had elapsedand a specific minimum number of responses had been emitted, Under a second conjunctive schedule, a response was reinforced after the 6-min fixed interval and upon completion of a tandem schedule requirement (conjunctive fixed-interval tandem schedule), This schedule retained the same required minimum number of responses as the first conjunctive schedule, but responses were never reinforced according to a fixed-ratio schedule; the tandem schedule was comprised of a fixed-ratio and a small (.1 to 10.0 sec) fixed-interval schedule, Under the conjunctive fixed-interval fixed-ratio schedule, responding was characterized by an initial pause, an abrupt transition to a high response rate, and a second transition to a lower rate that prevailed or slightly increased up to reinforcement, Under the conjunctive fixed-interval tandem schedule, pauses were extended, response rates were lower, and the initial high rate of responding was generally absent, The above effects depended upon the size of the fixed interval of the tandem schedule, The distinct pattern of responding generated by conjunctive fixed-interval fixed-ratio schedules depends upon occasional reinforcement of fixed-ratio responding and not merely on the addition of a minimum number of required responses.     

Preference for the interval schedule following multiple variable-ratio yoked-variable-interval schedule training

Pigeons were studied on multiple variable-ratio yoked-variable-interval schedules in which components had equal rates of food reinforcement and appeared equally often on each of two keys. Interpolated between component changes on the final multiple schedule were 10-sec probes in which both schedule stimuli were present, one on each key. During multiple schedule training, variable-ratio response rates were greater than yoked-variable-interval rates; however, response rate differences in the components were not a function of the mean ratio value for the 40-to-320-ratio range studied. During the choice probes, subjects responded more to the stimulus associated with the interval schedule than to the one associated with the ratio schedule. It was concluded that pigeons prefer interval schedules over equal reinforcement rate ratio schedules, because the former generate fewer responses per reinforcement.     

Multiple determinants of transfer of evaluative function after conditioning with free-operant schedules of reinforcement

The aim of the four present experiments was to explore how different schedules of reinforcement influence schedule-induced behavior, their impact on evaluative ratings given to conditioned stimuli associated with each schedule through evaluative conditioning, and the transfer of these evaluations through derived stimulus networks. Experiment 1 compared two contrasting response reinforcement rules (variable ratio [VR], variable interval [VI]). Experiment 2 varied the response to reinforcement rule between two schedules but equated the outcome to response rate (differential reinforcement of high rate [DRH] vs. VR). Experiment 3 compared molar and molecular aspects of contingencies of reinforcement (tandem VIVR vs. tandem VRVI). Finally, Experiment 4 employed schedules that induced low rates of responding to determine whether, under these circumstances, responses were more sensitive to the molecular aspects of a schedule (differential reinforcement of low rate [DRL] vs. VI). The findings suggest that the transfer of evaluative functions is determined mainly by differences in response rate between the schedules and the molar aspects of the schedules. However, when neither schedule was based on a strong response reinforcement rule, the transfer of evaluative judgments came under the control of the molecular aspects of the schedule.     

Behavioral contrast in pigeons and rats: A comparative analysis

The effects of reinforcement rate on behavioral contrast were examined in pigeons and rats. Each species was exposed to a series of 12 multiple variable-interval schedules, divided into four 3-schedule series. Each series consisted of a standard contrast manipulation, and baseline schedules provided a different rate of reinforcement in each of the series. The functions relating reinforcement rate to the magnitude of contrast were different across species. Rats showed a U-shaped function, with reliable contrast occurring only at high reinforcement rates. Pigeons showed an inverted U-shaped function, with contrast occurring on all schedules except the schedule providing the lowest rate of reinforcement. Pigeons discriminated between schedule components better than rats did, although differences in discrimination were probably not responsible for the differences in contrast. The results suggest that behavioral contrast in rats may be a different phenomenon from behavioral contrast in pigeons. The results cannot be explained by current theories, which view contrast as the product of a single general process.     

Antecedent reinforcement schedule training and operant response reinstatement in rats

Rats were trained on reinforcement schedules which generated high or low response rates. After extinguishing responding by eliminating food-reinforcement delivery, response-independent food presentations reinstated responding. Higher response rates occurred if the schedule preceding extinction controlled high response rates, suggesting that discriminative stimulus properties of the reinforcer were a function of antecedent training schedules.     

Rates of responding in the pigeon generated by simple and complex schedules which provide the same rates of reinforcement

Four pigeons pecked for food reinforcement on variable interval 1-min schedules and on the variable-interval 1-min components of multiple, concurrent, and pseudoconcurrent schedules. The pseudoconcurrent schedule provided only one schedule of reinforcement; but, any reinforcer could be collected by responding on either of two keys. The rate of responding generated by the variable interval schedule was not greater than the rates of responding generated by the components of the complex schedules. But, the rate of reinforcement obtained from the variable interval schedule was greater than the rates of reinforcement obtained from the components of the multiple schedule. These results may contradict the equation proposed by Herrnstein (1970). The equation predicts that the rate of responding generated by a schedule of reinforcement will be greater when the schedule appears alone, than when it appears as one component of a complex schedule.     

Rats show no preference between free and earned water in an advance-response procedure

Although an arbitrarily specified instrumental response may persist when free reinforcers are concurrently available, the interpretation that earned reinforcers are preferred is tenuous. The present advance-response procedure used both time allocation and advance response rates as indices of preference between free and earned water in rats. When multiple schedule components were two response-dependent schedules with different overall reinforcement rates, higher rates of reinforcement were preferred. However, when the multiple schedule consisted of response-dependent and response-independent components equated for overall rates of reinforcement, no consistent preference for free or earned reinforcers was evident. That a preference for free reinforcers was not obtained is difficult to reconcile with concepts of least effort.     

Preference between fixed-interval and variable-interval schedules of reinforcement: Separate roles of temporal scaling and predictability

Pigeons’ preference between fixed-interval and variable-interval schedules was examined using a concurrent-chains procedure. Responses to two concurrently available keys in the initial links of the concurrent chains occasionally produced terminal links where further responses were reinforced under either a fixed- or variable-interval schedule. In previous studies, preferences for the variable schedule with such a procedure have been interpreted as reflecting atemporal scaling process that heavily weights the shorter intervals in the variable schedule. The present experiment examined whetherpredictability, i.e., the presence of external stimuli correlated with the reinforcement interval, might also influence preference in such situations. When the two intervals in a variable schedule were made predictable by being associated with different key colors, preference for that schedule increased. This increase was reliable but small in magnitude and transient when initial-link responses only occasionally produced terminal links; it was large in magnitude when only one response in the initial link was required to produce the appropriate terminal-link schedule. The results suggest that preference between fixed and variable schedules may be influenced both by temporal scaling and to a lesser extent by predictability of the reinforcement intervals.     

The role of response-reinforcer correlation in signaled reinforcement effects

In three experiments, we examined the effects of signaling reinforcement during operant responding in order to illuminate the factors underlying instrumental overshadowing and potentiation effects. Specifically, we examined whether signaling reinforcement produces an enhancement and attentuation of responding when the response-reinforcer correlation is weak and strong, respectively. In Experiment 1, rats responded on variable-ratio (VR) or variable-interval (VI) schedules that were equated for the number of responses emitted per reinforcer. A signal correlated with reinforcement enhanced response rates on the VR schedule, but attenuated response rates were produced by the signal on the VI schedule. In Experiment 2, two groups of rats responded on a VI schedule while the two other groups received a conjoint VI, negative fixed-ratio schedule in which the subjects lost the availability of reinforcements if they emitted high response rates. A reinforcement signal attenuated responding for the simple VI groups but not for the animals given the negative fixed-ratio component, although the signal improved response efficiency in both groups. In Experiment 3, a poor correlation between responding and reinforcement was produced by a VI schedule onto which the delivery of response-independent food was superimposed. A signal for reinforcement initially elevated responding on this schedule, relative to an unsignaled condition; however, this pattern was reversed with further training. In sum, the present experiments provide little support for the view that signaling reinforcement enhances responding when the response-reinforcer correlation is weak and attenuates responding when this correlation is strong.     

Human causality judgments and response rates on DRL and DRH schedules of reinforcement

The effect of various relationships between a response (an investment made in the context of a game) and an outcome (a return on the investment) on judgments of the causal effectiveness of the response was examined. In Experiment 1, response rates and causal judgments were higher for a differential-reinforcement-of-high-rate (DRH) schedule relative to a variable-ratio (VR) schedule with the same probability of outcome following a response. Response rates were also higher for a DRH than for a variable-interval schedule matched for reinforcement rate. In Experiment 2, response rates and causal judgments were lower for a differential-reinforcement-of-low-rate schedule relative to a VR schedule with the same probability of outcome following a response. These results corroborate the view that schedules are a determinant of both response rates and causal judgments, and that few current theories of causal judgment explicitly predict this pattern of results.     

Reversibility of reinforcement between eating and running by schedule changes: A comparison of hypotheses and models

Rats increased eating that produced access to a running-wheel or increased running that produced access to food, depending on which response was potentially deprived, relative to baseline, by the scheduled ratio of responding. Under both schedules, instrumental responding significantly exceeded appropriate baselines of the noncontingent effects of the schedule. The results contradicted the hypothesis that reinforcement is produced by an overall or momentary probability differential between two responses; instead, they supported the condition of response deprivation as a key determinant of reinforcement. Of several recent quantitative models that predict reversibility of reinforcement by schedule changes, only the predictions of the relative response-deprivation model did not differ significantly from the data of either schedule.     

Generalization of causal efficacy judgments after evaluative learning

In three experiments, we examined the effect of response-outcome relations on human ratings of causal efficacy and demonstrated that such efficacy ratings transfer to novel situations through derived stimulus relations. Causal efficacy ratings were higher, and probability of an outcome given a response was lower, for a differential reinforcement of high rate schedule than for either a differential reinforcement of low rate schedule (Experiment 1) or a variable interval schedule (Experiment 2). In Experiment 3, we employed schedules that were equated for outcome probability and noted that ratings of causal efficacy and the rate of response were higher on a variable ratio than on a variable interval schedule. For participants in all three experiments, causal efficacy ratings transferred to the stimulus present during each schedule and generalized to novel stimuli through derived relations. The results corroborate the view that schedules are a determinant of both response rates and causal efficacy ratings. In addition, the novel demonstration of a mechanism of generalization of these ratings via derived relations has clinical implications.     

Competing sources of stimulus value in anticipatory contrast

In previous studies of anticipatory contrast, identical target components (A and B) preceded either a lower (extinction) or a richer schedule. Higher response rates occurred during the target preceding the lower rate of reinforcement, whereas preference was in favor of the target preceding the richer schedule. In Experiment 1, the response and preference measures were positively related when additional stimuli, with no reinforcement of their own, preceded the target components. The effect of these additional stimuli was presumed to be due to their overshadowing of the Pavlovian association between the target components and their following schedules. Experiment 2 also demonstrated a consistent relation between response rate and preference in a conditioned reinforcement procedure. In the absence of a strong Pavlovian association, anticipatory contrast, like other forms of contrast in free-operant procedures, reflects an increase in the value of the target component with an unchanged reinforcement schedule.     

Stimulus preference and the transitivity of preference

Four pigeons were exposed to multiple schedules with concurrent variable interval (VI) components and then tested for preference transfer. Half of the pigeons were trained on a multiple concurrent VI 20-sec, VI 40-sec/cuncurrent VI 4G-sec5 VI 80-sec schedule. The remaining pigeons were trained on a multiple concurrent VI 80-sec, VI 40-sec/concurrent VI 40-sec, VI 20-sec schedule-After stability criteria for time and response proportions were simultaneously met, four preference transfer tests were conducted with the stimuli associated with the VI 40-sec schedules. During the transfer tests, each pigeon allocated a greater proportion of responses (M=0,79) and time (M=0.82) to the stimulus associated with the VI 40-sec schedule that was paired with the VI 80-sec schedule than lo the VI 40-sec schedule stimulus paired with the VI 20-sec schedule. Absolute reinforcement rates on the two VI 40 sec schedules were approximately equal and unlikely to account for the observed preference. Nor was the preference consistent with the differences in local reinforcement rates associated with the two stimuli. Instead, the results were interpreted in terms of the differential value that stimuli acquire as a function of previous pairings with alternative schedules of reinforcement.     

Time and response matching with topographically different responses

Four pigeons were exposed to several nonindependent concurrent variable-interval schedules of reinforcement. One schedule component required a keypecking response; the other component required a treadlepressing response. The birds matched the ratio of their behavior (as measured by responses and time) between the two topographically different responses to the ratio of reinforcement in those two components. When additional foods not contingent on a keypeck or treadle-press were then added, the birds matched time spent in the components to total rates of food delivered in those components; response matching was somewhat disrupted. The matching law, developed under concurrent variable-interval schedules requiring similar responses, can thus account for choice behavior involving topographically different responses.     

The sensitivity of the pigeon’s keypeck to the differential reinforcement of long interresponse times

Keypecking of pigeons was studied under differential-reinforcement-of-low-rate (DRL) and variable-interval (VI) schedules in which the interreinforcement times on the two schedules were equated by a yoking procedure. Each schedule was available for half of every session and a change of schedule was signaled by a change of key color. The value of the DRL schedule was varied from .5 to 300 sec. Response rates were always higher in the VI schedule, but within sessions there was a sharp change in response rate coincident with the change in schedule only under lower schedule values. A group without prior training was tested with a 180-sec schedule value, and it, too, developed a higher response rate during the VI schedule, showing that the effect was not dependent on prior experience under low schedule values. In all conditions except the .5- and 1-sec values of the schedule, the mean proportion of responses emitted during the VI schedule was approximately .85 of the responses emitted during both schedules. The conclusion was that the requirement of a minimum interresponse time for reinforcement may work its effect by determining which responses may occur just prior to the reinforced response and thus receive delayed reinforcement.     

Concurrent VR VI schedules: primacy of molar control of preference and molecular control of response rates

In the first condition in Experiment 1, 6 rats were exposed to concurrent variable ratio (VR) 30, variable interval (VI) 30-sec schedules. In the next two conditions, the subjects were exposed to concurrent VI VI schedules and concurrent tandem VI-differential-reinforcement-of-high-rate VI schedules. For the latter conditions, the overall and relative reinforcer rates equaled those in the first condition. Only minor differences appeared in time allocation (a molar measure) across conditions. However, local response rate differences (a molecular measure) appeared between schedule types consistently with the interresponse times these schedules reinforced. In Experiment 2, these findings reappeared when the prior experiment was replicated with 5 subjects, except that the VR schedule was replaced by a VI plus linear feedback schedule. These results suggest that within the context tested, the molar factor of relative reinforcement rate controls preference, whereas the molecular factor of the relation between interresponse times and reinforcer probability controls the local response rate.