Pavlovian contingencies and anticipatory contrast

Pigeons were trained on a four-component multiple schedule in which two target components with identical reinforcement schedules were followed by other components with either higher or lower reinforcement rates. The Pavlovian signal properties of the target-component stimuli were varied by changes in their duration relative to the following components, and by whether the two following components were cued by the same or different stimuli, When different stimuli occurred in the following components, response rates were higher in the target component preceding the following component with the lower reinforcement rate, and these contrast effects were larger with shorter relative durations. But with nondifferential stimuli in the following components, contrast consistently occurred only with the longer durations of the target components. Moreover, several subjects with the shorter duration target stimuli had higher response rates in the target followed by the richer schedule—that is, Pavlovian conditioning occurred to the target stimuli. This interaction suggests that the processes underlying anticipatory contrast and Pavlovian conditioning are in opposition, and that the Pavlovian effect can dominate ifthe signaling properties of the target components are sufficiently enhanced.     

Training reinforcement rates,resistance to extinction,and the role of context in reinstatement

Behavior reduced as a consequence of extinction or intervention can relapse. According to behavioral momentum theory, the extent to which behavior persists and relapses once it has been eliminated depends on the relative training reinforcement rate among discriminative stimuli. In addition, studies of context renewal reveal that relapse depends on the similarity between the training stimulus context and the test stimulus context following disruption by extinction. In the present experiments with pigeons, we arranged different reinforcement rates in the presence of distinct discriminative stimuli across components of a multiple schedule. Following extinction, we attempted to reinstate responding in the presence of those target components with response-independent food presentations. Importantly, we arranged the reinstating food presentations either within the target components or in separate components, either paired with extinction (Experiment 1) or reinforcement (Experiment 2) during baseline. Reinstatement increased with greater training reinforcement rates when the reinstating food presentations were arranged in the target components and the separate components paired with reinforcement during training. Reinstatement was smaller and was not systematically related to training reinforcement rates in the target components when reinstating food presentation occurred in separate components paired with extinction. These findings suggest that relapse depends on the history of reinforcement associated with the discriminative stimuli in which the relapse-inducing event occurs.     

Choice responding following multiple schedule training

Pigeons’ choice responding on 10-sec interpolated probes was studied after baseline training on multiple variable-interval variable-interval schedules of food reinforcement. Unreinforced choice following training with three different relative reinforcement rates (Experiment 1), with a 3-ply multiple schedule (Experiment 2), and with three different relative reinforcement durations (Experiment 3) was examined. Least squares lines were fit to choice relative response rate and schedule relative response rate as functions of training relative reinforcement rate; choice slope was significantly greater than schedule slope in all three experiments. This result is counter to the prediction of Herrnstein’s (1970) theory that these slopes should not differ. Luce’s (1959) theory also failed to account for the data. It was concluded that choice responding was controlled by both approach to the stimulus associated with the smaller mean interreinforcer interval or the longer duration, and avoidance of the other stimulus.     

Resurgence of behavior during extinction depends on previous rate of response

In two experiments, food-deprived rat subjects leverpressed for food in three successive training phases. In the first phase of both experiments, rats were exposed to a multiple schedule, one component of which produced a high rate of response, and the other of which produced a lower rate of response (multiple random ratio [RR], random interval [RI] in Experiment 1, and multiple differential reinforcement of high rate, differential reinforcement of low rate in Experiment 2). Rats were then transferred to a multiple fixed interval (FI; 60-sec, 60-sec) schedule, until the effects of the first phase on response rate were no longer apparent and their response rates did not differ from those of rats responding on a multiple FI 60-sec, FI 60-sec schedule without previously experiencing a multiple RR, RI schedule. During the third stage oftraining, all rats were placed into extinction. During extinction, rates of responding were higher in the component previously associated with the high rate of responding in Phase 1, and they were lower in the component previously associated with low rates of responding in Phase 1. These results suggest that resurgence effects, like other history effects, are controlled by previous rates of responding.     

The influence of brief stimuli uncorrelated with reinforcement on choice between variable-ratio schedules

Experiment 1 investigated the behavior of rats trained to leverpress on a concurrent variable ratio (VR) 30 VR-30 schedule with a brief, 500-msec, light occurring at the midpoint of the ratio on one of the levers. Higher response rates were recorded on the lever associated with this stimulus, a finding that paralleled the effect produced by inserting primary reinforcement at the midpoint (i.e., by training on a concurrent VR-30 VR-15 schedule). Similar results were found in Experiment 2 using a concurrent VR-20 VR-20 schedule with a 2-sec visual stimulus presented midway through one of the components. In addition, a brief stimulus inserted midway through the VR-20 component of a concurrent VR-20 VR-10 schedule retarded the development of a difference in response rates between the components relative to a VR-20 VR-10 group lacking the signal. In Experiment 3, multiple VR VR schedules were used. Again, the response rate was higher in the component that had the added stimulus or, for a second group of subjects, on the component with the smaller response requirement. Probe-choice trials revealed a preference for the component that generated the higher rate in both groups. Presenting a stimulus partway through a ratio appears to reduce the effect on response rate and choice of a large ratio value.     

Resistance to extinction: contingency termination and generalization decrement

We present an algebraic model of resistance to extinction that is consistent with research on resistance to change. The model assumes that response strength is a power function of reinforcer rate and that extinction involves two additive, decremental processes: (1) the termination of the reinforcement contingency and (2) generalization decrement resulting from reinforcer omission. The model was supported by three experiments. In Experiment 1, 4 pigeons were trained on two-component multiple variable-interval (VI) 60-sec, VI 240-sec schedules. In two conditions, resistance to change was tested by terminating the response-reinforcer contingency and presenting response-independent reinforcers at the same rate as in training. In two further conditions, resistance to change was tested by prefeeding and by extinction. In Experiment 2, 6 pigeons were trained on two-component multiple VI 150-sec schedules with 8-sec or 2-sec reinforcers, and resistance to change was tested by terminating the response-reinforcer contingency in three conditions. In two of those conditions, brief delays were interposed between responses and response-independent reinforcers. In both Experiments 1 and 2, response rate was more resistant to change in the richer component, except for extinction in Experiment 1. In Experiment 3, 8 pigeons were trained on multiple VI 30-sec, VI 120-sec schedules. During extinction, half of the presentations of each component were accompanied by a novel stimulus to produce generalization decrement. The extinction data of Experiments 1 and 3 were well described by our model. The value of the exponent relating response strength and reinforcement was similar in all three experiments.     

The effects of changeover delays of fixed or variable duration on concurrent variable-interval performance in pigeons

The effects of changeover delays of fixed or variable duration on concurrent variable-interval performance in pigeons were investigated in a series of three experiments. Experiment 1 compared the effects of a fixed, variable, or variable signaled changeover delay on interchangeover times and responding during and after the changeover delay. The duration of the changeover delays was systematically varied in Experiment 2, and the relative reinforcement frequencies were manipulated in Experiment 3. Interchangeover times were found to be shorter when changeover delays of variable duration were compared with those of fixed duration. Changeover delays of fixed duration produced higher response rates during the changeover delay than after the changeover delay had elapsed; changeover delays of variable duration produced such differences to a lesser extent. It was concluded that the changeover delay in concurrent variable-interval schedules of reinforcement functionally acts as a delay period to the next opportunity for reinforcement, possibly serving as a conditioned reinforcer for the behavior preceding it (the interchangeover time) and as a discriminative stimulus for the behavior in its presence (response rates during the delay).     

The effects of stimulus similarity on different types of behavioral contrast

Pigeons were trained on multiple schedules with component stimuli of different degrees of similarity. In Experiment 1, a two-component schedule was used in which the two stimuli were either two line orientations or a line orientation versus a diffuse color. Reinforcement rate was varied in one component to determine the effects of stimulus similarity on different aspects of behavioral contrast. Contrast in terms of average response rates (molar contrast) was larger with less similar stimuli. Local contrast effects at the beginning of the component were larger for more similar stimuli, but these effects were more variable and did not attain statistical significance. Independent of the level of molar contrast, the local pattern of schedule interaction differed for the two levels of similarity: with more similar stimuli, the maximum degree of interaction occurred at the beginning of the components and then decreased; with less similar stimuli, the degree of interaction increased throughout the components and was at its maximum near their end. In Experiment 2, the same three stimuli were used while reinforcement rate in the middle component of a three-component sequence was varied; this isolated the effects of the preceding schedule from those of the following schedule. Contrast effects were generally greater in the target component preceding the variable schedule, and these were enhanced by less similar stimuli. Contrast in the target component following the variable schedule was manifested primarily in terms of the behavior at the beginning of the component, and these effects were inconsistently related to stimulus similarity. The functional separation of the effects of stimulus similarity on the different locations of contrast suggest that “anticipatory contrast” and “local contrast” depend upon different mechanisms, thus excluding any account of contrast solely in terms of relative rate of reinforcement.     

Factors affecting schedule-induced wood-chewing in rats: Percentage and rate of reinforcement,and operant requirement

In Experiment 1, rats were allowed to acquire either schedule-induced drinking or schedule-induced wood-chewing behavior under a fixed-interval (FI) 60-sec schedule of food reinforcement, following which food was omitted from 20% and then 50% of interreinforcement intervals. Omission of food severely disrupted induced drinking but had relatively little effect on induced wood-chewing. Experiment 2 investigated wood-chewing as a function of reinforcement rate, using a range of FI schedules from 5 to 180 sec in duration. Both the amount of chewing per session and the relative time spent chewing were bitonically related to reinforcement rate. In Experiment 3, schedule-induced chewing that had been acquired under a response-dependent schedule was found to persist under a response-independent schedule. Induced wood-chewing resembles other induced behaviors in important respects, but quantitative differences are also apparent.     

Behavioral contrast and reinforcement value

Behavioral contrast was produced in two target components of a four-component multiple schedule by having two target stimuli followed either by a higher rate of reinforcement or by extinction. Response rate was higher in the target followed by extinction. Periodic probe trials were then presented in which the two target stimuli were presented together. Choice on these probe trials was in favor of the stimulus followed by the higher rate of reinforcement during regular training. Experiment 2 replicated this finding but with probe trials presented throughout training. Here, preference for the stimulus followed by the higher rate of reinforcement was evident early in training, substantially before the contrast effects developed. The results challenge interpretations of contrast based on the concept of relative value.     

Effects of component training and subsequent sequencing of stimuli on shuttlebox avoidance responding of rats

The serial presentation of two different CSs, with each stimulus having an 8-sec duration (S1₈/S2₈), consistently has resulted in most of the shuttlebox avoidance responses being recorded to the S2 component. Experiment 1 attempted to attenuate this serial CS, delayed-response effect by conditioning the separate components of a serial CS prior to ordering them sequentially. Ten component-training trials were administered, with subjects receiving CS-US pairing to S1 only, S2 only, or to both S1 and S2 presented on separate trials. Two CS durations (8 or 16 sec) during this phase also were compared. Subjects were then given 100 avoidance test trials using the standard serial procedure. The 10 best avoidance responders in each group were selected for analysis. Shorter avoidance latencies were obtained only for subjects receiving component conditioning to S1. CS duration was not a factor in establishing the shorter latencies. Component conditioning to S2 resulted in increasing the total avoidances. Experiment 2 increased the number of component-training trials and the generality of the findings by using a different strain of rats and by extending the testing phase of the study so that all subjects could be included in the analysis. Comparable results were obtained. The theoretical implications of these data were discussed.     

Competing sources of stimulus value in anticipatory contrast

In previous studies of anticipatory contrast, identical target components (A and B) preceded either a lower (extinction) or a richer schedule. Higher response rates occurred during the target preceding the lower rate of reinforcement, whereas preference was in favor of the target preceding the richer schedule. In Experiment 1, the response and preference measures were positively related when additional stimuli, with no reinforcement of their own, preceded the target components. The effect of these additional stimuli was presumed to be due to their overshadowing of the Pavlovian association between the target components and their following schedules. Experiment 2 also demonstrated a consistent relation between response rate and preference in a conditioned reinforcement procedure. In the absence of a strong Pavlovian association, anticipatory contrast, like other forms of contrast in free-operant procedures, reflects an increase in the value of the target component with an unchanged reinforcement schedule.     

Stimulus duration and conditioned reinforcing value measured by a learning-tests procedure

The relationship between the duration of stimuli and their conditioned reinforcing effect was investigated using a learning-tests procedure. In Experiment 1, stimuli were the same duration on training (stimulus → reward) and test (choice response → stimulus). Ten- and 30-sec stimuli provided effective differential conditioned reinforcement but 3-sec stimuli did not. In Experiment 2, different pigeons had each combination of the 3- and 30-sec stimuli on training and test trials. Evidence of conditioned reinforcement was obtained only for the birds with 30-sec stimuli on both training and test. The results were interpreted as indicating that stimuli become effective conditioned reinforcers on test trials only when their duration exceeds the duration of differential short-term memory cues resulting from a difference in the events that precede them on training and test trials.     

Duration of components and response rates on multiple fixed-ratio schedules

Pigeons were exposed to a two-component multiple fixed-ratio X fixed-ratio Y schedule of reinforcement in which X was always less than Y. Components were equal in duration and alternated at rates that varied between 2 sec and 23.5 h. Relative response rate in the FR X component: (1) increased as the duration of components increased between 2 sec and 15 min, (2) was at a maximum between 15 min and 6 h, and (3) decreased as the duration of components increased from 6 h to 23.5 h. The changes in relative response rate were attributable primarily to changes in absolute response rates during the FR Y schedule as absolute response rates during the FR X schedule were relatively invariant. These results pose complexities for several theoretical formulations.     

Concurrent schedules of wheel-running reinforcement: Choice between different durations of opportunity to run in rats

How do animals choose between opportunities to run of different durations? Are longer durations preferred over shorter durations because they permit a greater number of revolutions? Are shorter durations preferred because they engender higher rates of running? Will longer durations be chosen because running is less constrained? The present study reports on three experiments that attempted to address these questions. In the first experiment, five male Wistar rats chose between 10-sec and 50-sec opportunities to run on modified concurrent variable-interval (VI) schedules. Across conditions, the durations associated with the alternatives were reversed. Response, time, and reinforcer proportions did not vary from indifference. In a second experiment, eight female Long-Evans rats chose between opportunities to run of equal (30 sec) and unequal durations (10 sec and 50 sec) on concurrent variable-ratio (VR) schedules. As in Experiment 1, between presentations of equal duration conditions, 10-sec and 50-sec durations were reversed. Results showed that response, time, and reinforcer proportions on an alternative did not vary with reinforcer duration. In a third experiment, using concurrent VR schedules, durations were systematically varied to decrease the shorter duration toward 0 sec. As the shorter duration decreased, response, time, and reinforcer proportions shifted toward the longer duration. In summary, differences in durations of opportunities to run did not affect choice behavior in a manner consistent with the assumption that a longer reinforcer is a larger reinforcer.     

Preference and resistance to change in concurrent variable-interval schedules

Pigeons were trained on a multiple schedule in which separate concurrent schedules were presented in the two components of the schedule. During one component, concurrent variable-interval 40-sec variableinterval 80-sec schedules operated. In the second component, concurrent variable-interval 40-sec variableinterval 20-sec schedules operated. After stable baseline performance was obtained in both components, extinction probe choice tests were presented to assess preference between the variable-interval 40-sec schedules from the two components. The variable-interval 40-sec schedule paired with the variableinterval 80-sec schedule was preferred over the variable-interval 40-sec schedule paired with the variableinterval 20-sec schedule. The subjects were also exposed to several resistance-to-change manipulations: (1) prefeeding prior to the experimental session, (2) a free-food schedule added to timeout periods separating components, and (3) extinction. The results indicated that preference and resistance to change do not necessarily covary.     

How reinforcement context affects temporal production and categorization

The behavioral theory of timing assumes that timing is governed by a pacemaker whose pulses move organisms from one state to the next, and that the speed of the pacemaker covaries with the rate of reinforcement in the experimental context. The goal of the present experiments was to clarify just what constitutes that context. In Experiment 1, pigeons responded on signaled fixed-interval 20-sec and 40-sec schedules of food reinforcement that were presented randomly within sessions (alternating condition) or between sessions (isolated condition). In Experiment 2, pigeons categorized the duration of a short or a long set of intervals in the alternating or the isolated condition. Performance in both experiments was under strong control by the signals, with scalar timing between long and short sets, but no significant differences between the alternating and isolated conditions. The context of reinforcement that determines pacemaker period can thus be specific to a particular timing task and signal.     

Signal duration and choosing between signaled and unsignaled reward

The effects of signal duration when choosing between signaled and unsignaled response-independent reinforcers were examined in two experiments. In Experiment 1, albino rats were given a choice between signaled and unsignaled food delivered on variable-time 60-sec schedules using a 20-sec signal. All subjects preferred the signaled schedule at a level comparable to that reported in an earlier study using a 5-sec signal. Experiment 2 presented six rats with a direct choice between a 5-sec and a 20-sec signal condition, and three rats with a choice between a 1.5-sec and a 5-sec signal duration. Subjects preferred the 20-sec signal over the 5-sec signal, but no pReference was found with the 1.5-sec vs. a 5-sec signal. Current theoretical views, such as delay reduction and behavioral competition, are considered.     

Specification of the stimulus-reinforcer relation in multiple schedules: Delay and probability of reinforcement

Control of pigeons’ keypecking by a stimulus-reinforcer contingency was investigated in the context of a four-component multiple schedule. In each of three experiments, pigeons were exposed to a schedule consisting of two two-component sequences. Discriminative stimuli identifying each sequence were present only in Component 1, which was 4, 6, or 8 sec in duration, while reinforcers could be earned only in Component 2 (30 sec in duration). Control by a stimulus-reinforcer contingency was sought during Component 1 by arranging a differential relation between Component 1 cues and schedule of reinforcement in Component 2. In Experiment 1, rate of keypecking during Component 1 varied with the presence and absence of a stimulus-reinforcer contingency. When a contingency was introduced, rate of keypecking increased during the Component 1 cue associated with the availability of reinforcement in Component 2. In Experiment 2, the stimulus-reinforcer contingency was manipulated parametrically by varying the correlation between Component 1 cues and Component 2 schedules of reinforcement. Responding in Component 1 varied as a function of strength of the stimulus-reinforcer contingency. The relatively high rates of Component 1 responding observed in Experiments 1 and 2 pose difficulties for conceptions of stimulus-reinforcer control based on probability of reinforcement. In these two experiments, the stimulus-associated probabilities of reinforcement in Component 1 were invariant at zero. An alternate dimension of stimulus-reinforcer control was explored in Experiment 3, in which Component 1 cues were differentially associated with delay to reinforcement in Component 2, while probability of reinforcement was held constant across components. When the stimulus-reinforcer contingency was in force, rate of responding in Component 1 varied inversely with delay to reinforcement in Component 2. In a quantitative analysis of data from Experiments 2 and 3, relative rate of responding during Component 1 was strongly correlated with two measures of relative delay to reinforcement.     

Effects of intertrial interval and exteroceptive feedback duration on discriminative avoidance acquisition in the gerbil

A series of studies of shuttlebox-avoidance learning in the gerbil evaluated the efficacy of an exteroceptive feedback stimulus (FS). Experiment 1 assessed the relative effectiveness of a FS at 30- and 90-sec intertriai intervals (ITIs), and found that the FS and warning signal termination contingencies were additive sources of avoidance reinforcement; i.e., they produced “supernormal acquisition” at the short ITI, but not at the 90-sec ITI. The effectiveness of a FS at the 30-sec ITI was further explored in Experiments 2 and 3, in which FS duration was varied in delayed and trace avoidance conditioning, respectively. In both studies, a FS facilitated acquisition but FS duration was not a critical determinant of performance. These results were interpreted in terms of an expectancy account of the informational value of a FS, and the problem of experimentally distinguishing between cognitive and inhibition-of-fear accounts of avoidance learning was discussed.