Specification of the stimulus-reinforcer relation in multiple schedules: Delay and probability of reinforcement

Control of pigeons’ keypecking by a stimulus-reinforcer contingency was investigated in the context of a four-component multiple schedule. In each of three experiments, pigeons were exposed to a schedule consisting of two two-component sequences. Discriminative stimuli identifying each sequence were present only in Component 1, which was 4, 6, or 8 sec in duration, while reinforcers could be earned only in Component 2 (30 sec in duration). Control by a stimulus-reinforcer contingency was sought during Component 1 by arranging a differential relation between Component 1 cues and schedule of reinforcement in Component 2. In Experiment 1, rate of keypecking during Component 1 varied with the presence and absence of a stimulus-reinforcer contingency. When a contingency was introduced, rate of keypecking increased during the Component 1 cue associated with the availability of reinforcement in Component 2. In Experiment 2, the stimulus-reinforcer contingency was manipulated parametrically by varying the correlation between Component 1 cues and Component 2 schedules of reinforcement. Responding in Component 1 varied as a function of strength of the stimulus-reinforcer contingency. The relatively high rates of Component 1 responding observed in Experiments 1 and 2 pose difficulties for conceptions of stimulus-reinforcer control based on probability of reinforcement. In these two experiments, the stimulus-associated probabilities of reinforcement in Component 1 were invariant at zero. An alternate dimension of stimulus-reinforcer control was explored in Experiment 3, in which Component 1 cues were differentially associated with delay to reinforcement in Component 2, while probability of reinforcement was held constant across components. When the stimulus-reinforcer contingency was in force, rate of responding in Component 1 varied inversely with delay to reinforcement in Component 2. In a quantitative analysis of data from Experiments 2 and 3, relative rate of responding during Component 1 was strongly correlated with two measures of relative delay to reinforcement.     

Microanalysis of variable-interval performance during stimulus compounding

Additive summation is observed when more responses are emitted to the simultaneous presentation (tone-plus-light) of independently conditioned stimuli (tone and light) than to either stimulus presented alone. The current experiment sought to determine if this increased rate during tone-plus-light was a function of a new modal interresponse time (IRT) or a differential mixing of pauses with a modal IRT characteristic of the responding in tone and light alone. Three rats were trained on a three-component multiple schedule where tone and light were each associated with a variable-interval 30-sec schedule while a variable-interval 100-sec operated in the simultaneous absence of these stimuli, tone-off and light-out. Baseline response rates were 2–4 times as high in tone or light as in their absence. In testing, more responses were emitted to tone-plus-light than to tone or light by all animals, but the modal IRT was in the 0.2–0.4-sec IRT bin for all test conditions. Tone-plus-light controlled fewer long IRT values and more responses in the short modal category than tone or light alone. These results support the response mixing hypothesis of stimulus control; i.e., no “new” behavior was observed during the novel combination of stimulus elements, only a mixture of previously reinforced behavior patterns in different proportions.     

Attempts to demonstrate magnetic discrimination by homing pigeons in flight

Eight homing pigeons, trained to fly between two elevated feeders within a flight tunnel, were tested for their ability to discriminate between two magnetic field stimuli and two acoustic stimuli, using a unitary discrete-trials procedure with successive presentation of stimuli. Magnetic stimuli consisted of the ambient magnetic field and a reduced magnetic field in which the vertical component of the field was reduced to 50% of its ambient value. Acoustic stimuli consisted of an ambient white noise and the white noise plus a tone. Stimuli were paired with food reward and either a time penalty (Experiment 1) or electric shock (Experiment 2). Although subjects could discriminate sounds with our procedures, none of the subjects demonstrated discrimination of magnetic fields. The failure of pigeons to discriminate magnetic stimuli is discussed as a consequence of either the failure to provide conditions sufficient for such discrimination or the absence of a magnetic sense in these animals.     

Summation and configuration in patterning schedules with the rat and rabbit

Discrimination between a tone + light compound and its components in positive and negative patterning schedules was examined. In the positive schedule, reinforced compound presentations (C+) were intermixed with unreinforced component presentations (T?, L?). In the negative schedule, the compound was unreinforced (C?) and the components were reinforced (T+, L+). In Experiment 1, appetitive conditioning of rats’ anticipatory magazine responses was used, and in Experiment 2, aversive conditioning of the rabbit’s nictitating membrane response was used. Both experiments revealed that the positive patterning schedule consistently produced rapid acquisition of appropriate discriminative responding. The results of the negative patterning schedule were more complex. Specifically, the results of Experiment 1 demonstrated that naive rats initially showed rapid acquisition of the negative patterning discrimination. However, schedule reversals revealed that experience with the positive patterning schedule virtually abolished subsequent acquisition of discriminative responding under the negative patterning schedule. The results of Experiment 2 revealed that naive rabbits showed very slow acquisition of discriminative responding under the negative patterning schedule. The results are discussed in relation to the unique-stimulus hypothesis, a contextual encoding hypothesis, and a configural hypothesis.     

Pigeons and rats observe signals of when but not where food will occur

Pigeons and rats were exposed to a mixed variable-time extinction schedule of reinforcement. During the variable-time component of the schedule, response-independent food was delivered at either a left or a right feeder. The animals were allowed to perform observing responses to produce either stimuli paired with the component of the mixed schedule that was in effect (temporal information) or stimuli paired with the feeder that might deliver food (spatial information). Only stimuli conveying temporal information reinforced observing. This result contradicts a prediction of the “information hypothesis” of observing, but is consistent with various conditioned-reinforcement interpretations of observing.     

Preference for the interval schedule following multiple variable-ratio yoked-variable-interval schedule training

Pigeons were studied on multiple variable-ratio yoked-variable-interval schedules in which components had equal rates of food reinforcement and appeared equally often on each of two keys. Interpolated between component changes on the final multiple schedule were 10-sec probes in which both schedule stimuli were present, one on each key. During multiple schedule training, variable-ratio response rates were greater than yoked-variable-interval rates; however, response rate differences in the components were not a function of the mean ratio value for the 40-to-320-ratio range studied. During the choice probes, subjects responded more to the stimulus associated with the interval schedule than to the one associated with the ratio schedule. It was concluded that pigeons prefer interval schedules over equal reinforcement rate ratio schedules, because the former generate fewer responses per reinforcement.     

Preference between fixed-interval and variable-interval schedules of reinforcement: Separate roles of temporal scaling and predictability

Pigeons’ preference between fixed-interval and variable-interval schedules was examined using a concurrent-chains procedure. Responses to two concurrently available keys in the initial links of the concurrent chains occasionally produced terminal links where further responses were reinforced under either a fixed- or variable-interval schedule. In previous studies, preferences for the variable schedule with such a procedure have been interpreted as reflecting atemporal scaling process that heavily weights the shorter intervals in the variable schedule. The present experiment examined whetherpredictability, i.e., the presence of external stimuli correlated with the reinforcement interval, might also influence preference in such situations. When the two intervals in a variable schedule were made predictable by being associated with different key colors, preference for that schedule increased. This increase was reliable but small in magnitude and transient when initial-link responses only occasionally produced terminal links; it was large in magnitude when only one response in the initial link was required to produce the appropriate terminal-link schedule. The results suggest that preference between fixed and variable schedules may be influenced both by temporal scaling and to a lesser extent by predictability of the reinforcement intervals.     

The effects of stimulus similarity on different types of behavioral contrast

Pigeons were trained on multiple schedules with component stimuli of different degrees of similarity. In Experiment 1, a two-component schedule was used in which the two stimuli were either two line orientations or a line orientation versus a diffuse color. Reinforcement rate was varied in one component to determine the effects of stimulus similarity on different aspects of behavioral contrast. Contrast in terms of average response rates (molar contrast) was larger with less similar stimuli. Local contrast effects at the beginning of the component were larger for more similar stimuli, but these effects were more variable and did not attain statistical significance. Independent of the level of molar contrast, the local pattern of schedule interaction differed for the two levels of similarity: with more similar stimuli, the maximum degree of interaction occurred at the beginning of the components and then decreased; with less similar stimuli, the degree of interaction increased throughout the components and was at its maximum near their end. In Experiment 2, the same three stimuli were used while reinforcement rate in the middle component of a three-component sequence was varied; this isolated the effects of the preceding schedule from those of the following schedule. Contrast effects were generally greater in the target component preceding the variable schedule, and these were enhanced by less similar stimuli. Contrast in the target component following the variable schedule was manifested primarily in terms of the behavior at the beginning of the component, and these effects were inconsistently related to stimulus similarity. The functional separation of the effects of stimulus similarity on the different locations of contrast suggest that “anticipatory contrast” and “local contrast” depend upon different mechanisms, thus excluding any account of contrast solely in terms of relative rate of reinforcement.     

The sensitivity of the pigeon’s keypeck to the differential reinforcement of long interresponse times

Keypecking of pigeons was studied under differential-reinforcement-of-low-rate (DRL) and variable-interval (VI) schedules in which the interreinforcement times on the two schedules were equated by a yoking procedure. Each schedule was available for half of every session and a change of schedule was signaled by a change of key color. The value of the DRL schedule was varied from .5 to 300 sec. Response rates were always higher in the VI schedule, but within sessions there was a sharp change in response rate coincident with the change in schedule only under lower schedule values. A group without prior training was tested with a 180-sec schedule value, and it, too, developed a higher response rate during the VI schedule, showing that the effect was not dependent on prior experience under low schedule values. In all conditions except the .5- and 1-sec values of the schedule, the mean proportion of responses emitted during the VI schedule was approximately .85 of the responses emitted during both schedules. The conclusion was that the requirement of a minimum interresponse time for reinforcement may work its effect by determining which responses may occur just prior to the reinforced response and thus receive delayed reinforcement.     

Stimulus control of topographically tagged responding

Pigeons’ responses on an operant key were reinforced according to either multiple variable-interval variable-interval or multiple variable-interval extinction schedules. The multiple-schedule components were signaled by line-tilt stimuli on a second key (signal key). Signal-key responses never produced reinforcement, and operant-key responses were not reinforced if they followed within 1 sec of a signal-key response. Behavioral contrast was not observed on the operant key, although there was a small, but reliable, increase in signal-key responding in the variable-interval component of the multiple variable-interval extinction condition. Generalization tests were interspersed between sessions of multiple variable-interval extinction training. Generalization gradients along the line-tilt dimension exhibited peak shift for both operant-key and signal-key responding following intradimensional (line tilt) discrimination training. Line-tilt generalization gradients following interdimensional discrimination did not exhibit peak shift. Gradients following intradimensional discrimination were sharper than gradients following interdimensional discrimination for both operant-key and signal-key responding. It was concluded that dimensional stimulus control of topographically tagged responding maintained by the stimulusreinforcer relation parallels that maintained by the response-reinforcer relation.     

Chromatic adaptation in the pigeon

This study presents evidence that adaptation to colored light alters the apparent hue of subsequently presented stimuli in pigeons. During training, right and left keypecks were reinforced following responses to colored and nominally achromatic slides, respectively. During test sessions, subjects continued to observe and report on the two classes of slides while 6-min components alternated, such that the experimental chamber was illuminated with either a green flood lamp or a nominally white bulb. The proportion of right keypecks following achromatic slides was much higher during green components than during white components, indicating that the achromatic slides appeared more like the chromatic slides.     

Remembering: The role of extraneous reinforcement

In two experiments, pigeons' responding on an extraneous task was explicitly reinforced during delayed matching-to-sample trials. In Experiment 1, red or green sample stimuli were followed by retention intervals of 0.2, 1, 4, or 12 sec, during which pecks to a white center key were reinforced with 2.5-sec access to wheat according to extinction, variable-interval 30-sec, and variable-interval 15-sec schedules in different conditions. A proportion of .2, .5, .7, or .9 of subsequent red or green choice responses that matched the sample were reinforced with 3-sec access to wheat. The result was that increasing center key reinforcement, or reducing reinforcer probability, lowered overall accuracy. Initial discriminability fell, but with no change in the rate of forgetting. In Experiment 2, initial discriminability was affected by extraneous reinforcers that were contingent on center key pecking, but not by noncontingent reinforcers. A plausible conclusion is that initial discriminability decreases when reinforcers strengthen competing behaviors.     

Sustained attention: Stimulus control determined by schedule of cue production interacting with cue duration

In a study of sustained attention (“vigilance”), pigeons performed a conditional discrimination in a 3-key operant chamber. Pecking a white center key initiated a 0.2- or 2.0-sec cue (a red or green disk). The side keys then displayed white disks, and a peck on the right or left key was reinforced depending on whether the preceding cue was red or green. Pecks on the white center key initiated the cue according to one of two schedules of cue production (FR 1 or VI 7.5 sec). Control of side key choices by 0.2-sec cues was disturbed by transition from FR 1 to VI 7.5, and recovered after the schedule of cue production changed from VI 7.5 back to FR 1. Control of choices by 2.0-sec cues was not affected by changing schedules of cue production. Rates of pecking the cue were higher than rates of cue-producing responses.     

The isolation of stimulus-reinforcer associations established with multiple schedules

Multiple schedules established stimulus-reinforcer (S-S^R) associations on baselines in which equal response rates and patterning were maintained in all components. Subsequently, stimuli associated with an increase in reinforcement but no change in ongoing response rate were compounded. For one experimental group, free-operant avoidance (FOA) was programmed in tone and in light while variable-interval (VI) food reinforcement was effective in their simultaneous absence (T + L). The opposite stimulus-schedule combinations were programmed for the other. Both groups remained in their VI components 85% of the session on schedule preference tests, and on a stimulus compounding test emitted approximately 1.5 times as many responses to tone-plus-light (T + L) as to tone or light alone. This is the first report of additive summation to combined discriminative stimuli associated with only an increase in reinforcement. Nondifferentially trained controls who had the same contingency effective in tone, light, and T + L-VI or FOA—showed neither preference among schedule components or summation during stimulus compounding, indicating that nonassociative stimulus factors made no contribution to either resultant in the experimental animals. Evidence supporting an algebraic combination of response and reinforcement associations is presented, and functional similarities between transfer-of-control studies and the stimulus compounding tests of the experimental groups in the present experiment are discussed.     

Conjunctive schedules of reinforcement IV: Effects on the pattern of responding of changes in requirement at reinforcement

The effects of different schedule requirements at reinforcement on patterns of responding by pigeons were assessed under conjunctive schedules with comparable response-number requirements, Under one conjunctive schedule (conjunctive fixed-interval fixed-ratio schedule), a response was reinforced after a 6-min interval had elapsedand a specific minimum number of responses had been emitted, Under a second conjunctive schedule, a response was reinforced after the 6-min fixed interval and upon completion of a tandem schedule requirement (conjunctive fixed-interval tandem schedule), This schedule retained the same required minimum number of responses as the first conjunctive schedule, but responses were never reinforced according to a fixed-ratio schedule; the tandem schedule was comprised of a fixed-ratio and a small (.1 to 10.0 sec) fixed-interval schedule, Under the conjunctive fixed-interval fixed-ratio schedule, responding was characterized by an initial pause, an abrupt transition to a high response rate, and a second transition to a lower rate that prevailed or slightly increased up to reinforcement, Under the conjunctive fixed-interval tandem schedule, pauses were extended, response rates were lower, and the initial high rate of responding was generally absent, The above effects depended upon the size of the fixed interval of the tandem schedule, The distinct pattern of responding generated by conjunctive fixed-interval fixed-ratio schedules depends upon occasional reinforcement of fixed-ratio responding and not merely on the addition of a minimum number of required responses.     

Pavlovian contingencies and anticipatory contrast

Pigeons were trained on a four-component multiple schedule in which two target components with identical reinforcement schedules were followed by other components with either higher or lower reinforcement rates. The Pavlovian signal properties of the target-component stimuli were varied by changes in their duration relative to the following components, and by whether the two following components were cued by the same or different stimuli, When different stimuli occurred in the following components, response rates were higher in the target component preceding the following component with the lower reinforcement rate, and these contrast effects were larger with shorter relative durations. But with nondifferential stimuli in the following components, contrast consistently occurred only with the longer durations of the target components. Moreover, several subjects with the shorter duration target stimuli had higher response rates in the target followed by the richer schedule—that is, Pavlovian conditioning occurred to the target stimuli. This interaction suggests that the processes underlying anticipatory contrast and Pavlovian conditioning are in opposition, and that the Pavlovian effect can dominate ifthe signaling properties of the target components are sufficiently enhanced.     

Overshadowing by food of stimulus control by a keylight in a two-element serial compound

Pigeons were given successive discrimination training in which pecking during a choice period when the key was white was either reinforced or not, depending upon the prior presence or absence of a discriminative stimulus, which was a two-element serial compound. The compound consisted of a keylight and food, with food presented second or first in a forward or backward pairing for different groups of pigeons. In Experiment 1, the sequence was an S+ indicating reinforced trials, while in Experiment 2, the sequence was an S? indicating nonreinforced trials. Following acquisition of discriminated operant behavior, a sequence generalization test was administered during which all possible orders of the two stimuli were presented on test trials prior to the onset of the choice period. The results showed that food overshadowed stimulus control by the color of the light on the key on the sequence-generalization test, independently of whether food was presented first or second during training and independently of whether food was associated with reinforcement or nonreinforcement. The similarity of results for the two experiments suggests that overshadowing occurs independently of whether the compound is a discriminative stimulus for reinforcement or nonreinforcement. Simultaneous presentation of elements of a compound stimulus is not necessary for overshadowing because the phenomenon was captured with sequentially presented stimuli.     

Competing sources of stimulus value in anticipatory contrast

In previous studies of anticipatory contrast, identical target components (A and B) preceded either a lower (extinction) or a richer schedule. Higher response rates occurred during the target preceding the lower rate of reinforcement, whereas preference was in favor of the target preceding the richer schedule. In Experiment 1, the response and preference measures were positively related when additional stimuli, with no reinforcement of their own, preceded the target components. The effect of these additional stimuli was presumed to be due to their overshadowing of the Pavlovian association between the target components and their following schedules. Experiment 2 also demonstrated a consistent relation between response rate and preference in a conditioned reinforcement procedure. In the absence of a strong Pavlovian association, anticipatory contrast, like other forms of contrast in free-operant procedures, reflects an increase in the value of the target component with an unchanged reinforcement schedule.     

Stimulus preference and the transitivity of preference

Four pigeons were exposed to multiple schedules with concurrent variable interval (VI) components and then tested for preference transfer. Half of the pigeons were trained on a multiple concurrent VI 20-sec, VI 40-sec/cuncurrent VI 4G-sec5 VI 80-sec schedule. The remaining pigeons were trained on a multiple concurrent VI 80-sec, VI 40-sec/concurrent VI 40-sec, VI 20-sec schedule-After stability criteria for time and response proportions were simultaneously met, four preference transfer tests were conducted with the stimuli associated with the VI 40-sec schedules. During the transfer tests, each pigeon allocated a greater proportion of responses (M=0,79) and time (M=0.82) to the stimulus associated with the VI 40-sec schedule that was paired with the VI 80-sec schedule than lo the VI 40-sec schedule stimulus paired with the VI 20-sec schedule. Absolute reinforcement rates on the two VI 40 sec schedules were approximately equal and unlikely to account for the observed preference. Nor was the preference consistent with the differences in local reinforcement rates associated with the two stimuli. Instead, the results were interpreted in terms of the differential value that stimuli acquire as a function of previous pairings with alternative schedules of reinforcement.