Conjunctive schedules of reinforcement IV: Effects on the pattern of responding of changes in requirement at reinforcement

The effects of different schedule requirements at reinforcement on patterns of responding by pigeons were assessed under conjunctive schedules with comparable response-number requirements, Under one conjunctive schedule (conjunctive fixed-interval fixed-ratio schedule), a response was reinforced after a 6-min interval had elapsedand a specific minimum number of responses had been emitted, Under a second conjunctive schedule, a response was reinforced after the 6-min fixed interval and upon completion of a tandem schedule requirement (conjunctive fixed-interval tandem schedule), This schedule retained the same required minimum number of responses as the first conjunctive schedule, but responses were never reinforced according to a fixed-ratio schedule; the tandem schedule was comprised of a fixed-ratio and a small (.1 to 10.0 sec) fixed-interval schedule, Under the conjunctive fixed-interval fixed-ratio schedule, responding was characterized by an initial pause, an abrupt transition to a high response rate, and a second transition to a lower rate that prevailed or slightly increased up to reinforcement, Under the conjunctive fixed-interval tandem schedule, pauses were extended, response rates were lower, and the initial high rate of responding was generally absent, The above effects depended upon the size of the fixed interval of the tandem schedule, The distinct pattern of responding generated by conjunctive fixed-interval fixed-ratio schedules depends upon occasional reinforcement of fixed-ratio responding and not merely on the addition of a minimum number of required responses.     

Habituation may contribute to within-session decreases in responding under high-rate schedules of reinforcement

Two experiments tested the hypothesis that habituation contributes to within-session decreases in responding. In Experiment 1, rats’ leverpressing was reinforced under a fixed ratio (FR) 4 schedule throughout the baseline sessions. During the dishabituation sessions, the first 21 min and the last 21 min were FR 4; dishabituating events occurred during the middle 3 min. The dishabituating events altered the manner of reinforcer delivery in four different ways. Response rates increased after all dishabituating events. In Experiment 2, rats responded on several FR and variable ratio (VR) schedules. The ratio requirement varied from 3 to 15. Within-session decreases in responding were steeper during FR schedules than during VR schedules. In addition, response rates were well described as linear functions of cumulative number of food pellets eaten within sessions. These results support the habituation hypothesis but do not rule out the possibility that other satiety variables might contribute simultaneously.     

Responding on a multiple variable-time variable-time schedule as a function of component duration

Five pigeons pecked lighted keys for food reinforcers delivered by a multiple variable-time 30-sec variable-time 2-min schedule. The duration of the components varied from 5 sec to 16 min. The rate of responding generated by the more favorable component schedule decreased as component duration increased to an intermediate value and then increased with additional increases in duration. The decrease confirmed a prediction of additive theories of behavioral contrast. The rate of responding generated by the less favorable component did not increase as component duration increased. This decrease may represent a floor effect or it may violate a prediction of one additive theory of contrast.     

The role of trial tracking in rats’ working memory

The experiments reported in the present study tested whether decreasing intertrial intervals (ITIs) intensifies the disruptive effects of increasing retention intervals (RIs) in a delayed conditional discrimination by decreasing the animal’s trial tracking accuracy (Cohen & Armstrong, 1996; Cohen & Roberts, 1996). Rats responded on a fixed ratio (FR) 1 or fixed interval (FI) 10-sec reinforcement schedule at a second light or tone stimulus, S2, when the first light or tone stimulus, S1, had signaled an FI 10-sec or FR 1 schedule, respectively. RIs between S1 and S2 were increased from 3 to 24 sec and never exceeded ITIs that were reduced from 24 to 6 sec. For some rats, the trials were separated from each other by extending the lever at S1 and retracting it at the end of S2 (ITI lever-retracted group). For other, control rats, the lever remained extended throughout the session (lever-extended group, Experiment 1) or was extended and retracted with the onset and offset of each stimulus (RI/ITI lever-retracted group, Experiment 2). The rats under all trial conditions learned to delay leverpressing on the FI 10-sec schedule. Latency to begin leverpressing on the FI 10-sec schedule declined as RIs were increased, but this effect was attenuated in the ITI lever-retracted groups in both experiments, as would be predicted by thetrial tracking hypothesis. Decreasing ITIs from 24 to 6 sec intensified the disruptive effects of increasing RIs from 3 to 6 sec in the RI/ITI lever-retracted group (Experiment 2), as would be predicted by the trial tracking hypothesis.     

Relative preferences for various bivalued ratio schedules

Widely cited experiments on optimal foraging have used bivalued distributions as representing environmental stochasticity, characterizing these in terms of their arithmetic means. In contrast, research on free-operant choice has established that organisms prefer variable patterns of food delivery, relative to fixed patterns with the same mean values. To explore such departures from linear averaging, specifically with respect to bivalued alternatives, pigeons were given choices between a fixed-ratio (FR) schedule of food delivery that required 15, 30, or 60 responses and bivalued variable-ratio schedules with an arithmetic mean of 60,5 or 60, A bivalued schedule of 1 and 120 was preferred almost exclusively over each of the FR values. With a bivalued schedule of 15 and 105, there was a shift of preference, most notably in the FR-15 condition, but in no case was linear averaging a good predictor of the birds’ choices. Geometric averaging fared better, but even this failed to represent the apparent salience of the minimum value of the bivalued schedule in some conditions.     

Response-cost punishment with pigeons: Further evidence of response suppression via token loss

Four pigeons responded on a two-component multiple token-reinforcement schedule, in which tokens were produced according to a random-interval 30-sec schedule and exchanged according to a variable-ratio 4 schedule in both components. To assess the effects of contingent token loss, tokens were removed after every second response (i.e., fixed-ratio 2 loss) in one of the components. Response rates were selectively lower in the loss components relative to baseline (no-loss) conditions, as well as to the within-condition no-loss components. Response rates were decreased to a greater degree in the presence of tokens than in their absence. To control for the effects of changes in the density of token and food reinforcement, two parts consisted of additional conditions where food density and token loss were yoked to those in a previous loss condition. In the yoked-food condition, tokens were produced as usual in both components, but the overall density of food reinforcement in one of the components was yoked to that obtained during a previous token-loss condition. In the yoked token-loss condition, tokens were removed during one component of the multiple schedule at a rate that approximately matched the obtained rate of loss from a previous token-loss condition. Response rates in these yoked components were less affected than those in comparable loss components, despite similar densities of token, exchange, and food reinforcement. On the whole, the results support the conclusion that contingent token loss serves as an effective punisher with pigeons.     

Maintenance of observing responses with the less highly valued stimulus in pigeons

Previous research has shown that the more valued stimulus will maintain observing responses but the less valued stimulus will not. In the first part of this experiment, it was shown that the removal of the stimulus associated with the fixed-ratio component of a multiple schedule (mult FI 2 min, FR 35) abolished observing responses. In the next part, the ratio stimulus was phased out by making it available during some ratios but not during others. Observing responses were maintained following the phasing procedure even when all ratio stimuli were unavailable. In the third part, all stimuli, both ratio and interval, were unavailable. Observing responses declined under this condition and increased again when the stimulus associated with the interval was again made available. It is hypothesized that observing responses will be maintained as long as they permit the maintenance of a discrimination.     

Specification of the stimulus-reinforcer relation in multiple schedules: Delay and probability of reinforcement

Control of pigeons’ keypecking by a stimulus-reinforcer contingency was investigated in the context of a four-component multiple schedule. In each of three experiments, pigeons were exposed to a schedule consisting of two two-component sequences. Discriminative stimuli identifying each sequence were present only in Component 1, which was 4, 6, or 8 sec in duration, while reinforcers could be earned only in Component 2 (30 sec in duration). Control by a stimulus-reinforcer contingency was sought during Component 1 by arranging a differential relation between Component 1 cues and schedule of reinforcement in Component 2. In Experiment 1, rate of keypecking during Component 1 varied with the presence and absence of a stimulus-reinforcer contingency. When a contingency was introduced, rate of keypecking increased during the Component 1 cue associated with the availability of reinforcement in Component 2. In Experiment 2, the stimulus-reinforcer contingency was manipulated parametrically by varying the correlation between Component 1 cues and Component 2 schedules of reinforcement. Responding in Component 1 varied as a function of strength of the stimulus-reinforcer contingency. The relatively high rates of Component 1 responding observed in Experiments 1 and 2 pose difficulties for conceptions of stimulus-reinforcer control based on probability of reinforcement. In these two experiments, the stimulus-associated probabilities of reinforcement in Component 1 were invariant at zero. An alternate dimension of stimulus-reinforcer control was explored in Experiment 3, in which Component 1 cues were differentially associated with delay to reinforcement in Component 2, while probability of reinforcement was held constant across components. When the stimulus-reinforcer contingency was in force, rate of responding in Component 1 varied inversely with delay to reinforcement in Component 2. In a quantitative analysis of data from Experiments 2 and 3, relative rate of responding during Component 1 was strongly correlated with two measures of relative delay to reinforcement.     

The role of response-reinforcer correlation in signaled reinforcement effects

In three experiments, we examined the effects of signaling reinforcement during operant responding in order to illuminate the factors underlying instrumental overshadowing and potentiation effects. Specifically, we examined whether signaling reinforcement produces an enhancement and attentuation of responding when the response-reinforcer correlation is weak and strong, respectively. In Experiment 1, rats responded on variable-ratio (VR) or variable-interval (VI) schedules that were equated for the number of responses emitted per reinforcer. A signal correlated with reinforcement enhanced response rates on the VR schedule, but attenuated response rates were produced by the signal on the VI schedule. In Experiment 2, two groups of rats responded on a VI schedule while the two other groups received a conjoint VI, negative fixed-ratio schedule in which the subjects lost the availability of reinforcements if they emitted high response rates. A reinforcement signal attenuated responding for the simple VI groups but not for the animals given the negative fixed-ratio component, although the signal improved response efficiency in both groups. In Experiment 3, a poor correlation between responding and reinforcement was produced by a VI schedule onto which the delivery of response-independent food was superimposed. A signal for reinforcement initially elevated responding on this schedule, relative to an unsignaled condition; however, this pattern was reversed with further training. In sum, the present experiments provide little support for the view that signaling reinforcement enhances responding when the response-reinforcer correlation is weak and attenuates responding when this correlation is strong.     

Selected schedules of reinforcement in the black-tailed prairie dog (Cynomys ludovicianus)

Wild black-tailed prairie dogs were run on FR, FI, VR, and VI schedules for Noyes pellet reinforcement. Cumulative barpress responses, postreinforcement pause lengths, and responses per second were recorded. The highest response rates occurred in the VR schedules, with the lowest response rates coming in the FI schedules. Fixed-ratio schedules had the longest postreinforcement pauses, VI schedules had the shortest. At the upper levels of the fixed-ratio schedules (FR 90–100), the animals ceased to respond consistently. Generally, data from prairie dogs were consistent with data reported in studies from other mammalian species.     

Pavlovian contingencies and anticipatory contrast

Pigeons were trained on a four-component multiple schedule in which two target components with identical reinforcement schedules were followed by other components with either higher or lower reinforcement rates. The Pavlovian signal properties of the target-component stimuli were varied by changes in their duration relative to the following components, and by whether the two following components were cued by the same or different stimuli, When different stimuli occurred in the following components, response rates were higher in the target component preceding the following component with the lower reinforcement rate, and these contrast effects were larger with shorter relative durations. But with nondifferential stimuli in the following components, contrast consistently occurred only with the longer durations of the target components. Moreover, several subjects with the shorter duration target stimuli had higher response rates in the target followed by the richer schedule—that is, Pavlovian conditioning occurred to the target stimuli. This interaction suggests that the processes underlying anticipatory contrast and Pavlovian conditioning are in opposition, and that the Pavlovian effect can dominate ifthe signaling properties of the target components are sufficiently enhanced.     

Schedule-induced polydipsia contrast in the rat

Schedule-induced polydipsia was studied using a behavioral contrast paradigm. Food pellets were delivered to food-deprived rats on a response-independent FT 1-min schedule. Licking on a tube produced water on a MULT FR 10 FR 10, MULT FR 10 EXT, or MIXED FR 10 EXT for three rats (Experiment 1) and on a MULT VI VI, MULT VI EXT, or MIXED VI EXT schedule for three other rats (Experiment 2). On the FR schedules, rats could drink more water by increasing lick rates, but on the VI schedules the amount of drinking was fixed by the schedule parameters and was relatively unaffected by lick rates. Relative to MULT FR FR, positive polydipsia contrast was clearly demonstrated on MULT and MIXED FR EXT; but relative to MULT VI VI, contrast was not demonstrated on MULT and MIXED VI EXT. These data suggest that polydipsia contrast occurs only if increased licking permits increased drinking.     

Accurate DRL performance in the pigeon: Comparison between perching and treadle pressing

Homing pigeons were reinforced for emitting a perching response according to differential-reinforcement-of-low-rate (DRL) schedules. The spacing requirement between successive perchings was progressively increased by 1-sec steps up to 70 sec and then abruptly decreased to 60, 40, and 20 sec. IRT/OP (interresponse time/opportunity) functions were maximal near the time of reinforcement. The coefficients of variation of the IRT distributions (ratio between the interquartile range and median IRT) fluctuated around .32, testifying for equivalent levels of adjustment throughout the critical IRT range. The ratio between reinforced and total IRTs ranged between .90 and .20. These data contrast with the performance of another group of pigeons reinforced for a treadle-pressing response according to DRL schedules (flatter IRT/OP functions, high coefficients of variation, and low efficiencies). Despite these differences in temporal regulation between perching and treadle-pressing DRL, response rates and reinforcement rates followed the same trend in both cases: they decreased as schedule value increased. The DRL perching results are similar to previous results obtained in the same species when perching duration was reinforced.     

Signal properties of reinforcement and reinforcement omission on a multiple fixed-ratio schedule

A series of experiments used food-deprived pigeons to examine several parameters of reinforcement omission in an attempt to control changes of keypeck response measures on a subsequent schedule. In Experiments 1 and 2, the pigeons were tested with a multiple fixed-ratio schedule on which reinforcement was occasionally omitted at the completion of the first component. The duration of the delay occurring in lieu of reinforcement was systematically varied. In Experiment 3, the stimulus that signaled the second component of the schedule was altered to appear either more or less similar to the stimulus that signaled the first component. Two principal results are reported: (1) Response latency decreased and, to a much lesser extent, terminal response rate increased as the delay occurring in lieu of reinforcement decreased; and (2) both latency decrease and response-rate increase were enhanced by a second component stimulus which was similar to the first. The results are evaluated in terms of Amsel’s frustration theory and an analysis by Staddon which suggests that reinforcement inhibits responding. The data appear to support Staddon’s argument that rate increases and latency decreases following reinforcement omission are largely a function of an attenuation of the inhibitory influence of reinforcement, an effect that is enhanced by stimulus generalization. Accordingly, it is proposed that an animal’s response to reinforcement omission is determined by a stimulus complex that minimally includes the omission event and component cues.     

Operant response topographies of rats receiving food or water reinforcers on FR or FI reinforcement schedules

Separate groups of food- and water-deprived rats pressed a lever for food or water, respectively, on continuous reinforcement and various fixed-ratio and fixed-interval reinforcement schedules. Food-reinforced rats on continuous, FR 2-, or FI 10-sec schedules showed consistently longer mean lever contact durations per leverpress than did water-reinforced rats on the same schedules. Mean lever-contact-duration differences between food- and water-reinforced rats were greatly attenuated or disappeared under FR 4-, FR 8-, FI 20-sec, and FI 30-sec schedules of reinforcement. These results are interpreted as supporting earlier hypotheses that there are respondent components of operantly conditioned and autoshaped leverpresses, but that these respondent components weaken with partial reinforcement and the leverpress topography comes under the control of operant contingencies.     

Preference for the interval schedule following multiple variable-ratio yoked-variable-interval schedule training

Pigeons were studied on multiple variable-ratio yoked-variable-interval schedules in which components had equal rates of food reinforcement and appeared equally often on each of two keys. Interpolated between component changes on the final multiple schedule were 10-sec probes in which both schedule stimuli were present, one on each key. During multiple schedule training, variable-ratio response rates were greater than yoked-variable-interval rates; however, response rate differences in the components were not a function of the mean ratio value for the 40-to-320-ratio range studied. During the choice probes, subjects responded more to the stimulus associated with the interval schedule than to the one associated with the ratio schedule. It was concluded that pigeons prefer interval schedules over equal reinforcement rate ratio schedules, because the former generate fewer responses per reinforcement.     

Within-session changes in responding during delayed matching-to-sample and discrimination procedures

Two experiments examined within-session changes in responding during discrimination procedures. In Experiment 1, rate of responding changed significantly within sessions during symbolic delayed matching-to-sample tasks when the delay between the stimulus and the choice period was short (1–5 sec), but not when it was long (8–12 sec). The percentage of responses that were correct did not change within sessions. In Experiment 2, response rates increased and then decreased within sessions during both S1 and S2 when successive discrimination procedures provided high, but not low, rates of reinforcement. Discrimination ratios sometimes increased within sessions. These results question two potential definitions of attention as explanations for within-session changes in response rates. They are more compatible with explanations based on concepts such as arousal, satiation, habituation, and interfering responses.     

Second-order schedules: Manipulation of brief-stimulus duration at component completion

In a second-order schedule, fixed-interval components were reinforced according to a variable-interval schedule. A brief stimulus accompanied the completion of each fixed interval. Brief-stimulus duration was varied across conditions from 0.5 to 8 sec. Patterning was greater the longer the duration of the stimulus. Additionally, exposure to relatively long brief-stimulus durations enhanced patterning upon reexposure to shorter brief-stimulus durations.     

The influence of brief stimuli uncorrelated with reinforcement on choice between variable-ratio schedules

Experiment 1 investigated the behavior of rats trained to leverpress on a concurrent variable ratio (VR) 30 VR-30 schedule with a brief, 500-msec, light occurring at the midpoint of the ratio on one of the levers. Higher response rates were recorded on the lever associated with this stimulus, a finding that paralleled the effect produced by inserting primary reinforcement at the midpoint (i.e., by training on a concurrent VR-30 VR-15 schedule). Similar results were found in Experiment 2 using a concurrent VR-20 VR-20 schedule with a 2-sec visual stimulus presented midway through one of the components. In addition, a brief stimulus inserted midway through the VR-20 component of a concurrent VR-20 VR-10 schedule retarded the development of a difference in response rates between the components relative to a VR-20 VR-10 group lacking the signal. In Experiment 3, multiple VR VR schedules were used. Again, the response rate was higher in the component that had the added stimulus or, for a second group of subjects, on the component with the smaller response requirement. Probe-choice trials revealed a preference for the component that generated the higher rate in both groups. Presenting a stimulus partway through a ratio appears to reduce the effect on response rate and choice of a large ratio value.     

Positive behavioral contrast as a function of time-out duration when pigeons peck keys on a within-session procedure

Three pigeons pecked keys for food reinforcers delivered by multiple variable interval variable interval schedules in the first part of each session (baseline) and by multiple variable interval extinction schedules in the second part of each session (contrast). The variable interval schedules delivered reinforcers after an average of 4 min or 30 sec in different conditions. The duration of a time-out between the components varied in five steps from 5 to 120 sec. Positive contrast occurred for all time-out durations in both experiments. That is, the rate of responding emitted during the constant, variable interval component was greater during the contrast than during the baseline schedules. The size of contrast did not change systematically with changes in timeout duration. These results violate most theories of contrast. They are compatible with the idea that animals integrate reinforcers over intervals longer than 2 min.