Stimulus control of behavior during the postreinforcement pause of FI schedules

Pigeons were given the opportunity to terminate certain segments of fixed intervals by pecking a control key. When 30-sec segments of negative and positive stimuli alternated across the interreinforcement interval (Experiment 1), most birds terminated a large proportion of negative segments. However, few control-key responses were made during the negative segment immediately following food presentation. Under schedules during which only one negative segment was programmed, during the first 30 sec of 1-min intervals (Experiment 2), control-key responses, when they occurred at all, were made after several seconds of the interval had elapsed. Similar findings were obtained when a peck on the control key merely changed the color on the food key (Experiment 3). These findings suggest that the post-reinforcement extinction state (Schneider, 1969) during fixed-interval schedules consists of two phases: an immediate postreinforcement inhibitory phase, followed by a second phase during which a control-key response may occur. These two phases and their associated behavior may be related to Staddon’s (1977) distinction between interim and facultative activities.     

Overshadowing by food of stimulus control by a keylight in a two-element serial compound

Pigeons were given successive discrimination training in which pecking during a choice period when the key was white was either reinforced or not, depending upon the prior presence or absence of a discriminative stimulus, which was a two-element serial compound. The compound consisted of a keylight and food, with food presented second or first in a forward or backward pairing for different groups of pigeons. In Experiment 1, the sequence was an S+ indicating reinforced trials, while in Experiment 2, the sequence was an S? indicating nonreinforced trials. Following acquisition of discriminated operant behavior, a sequence generalization test was administered during which all possible orders of the two stimuli were presented on test trials prior to the onset of the choice period. The results showed that food overshadowed stimulus control by the color of the light on the key on the sequence-generalization test, independently of whether food was presented first or second during training and independently of whether food was associated with reinforcement or nonreinforcement. The similarity of results for the two experiments suggests that overshadowing occurs independently of whether the compound is a discriminative stimulus for reinforcement or nonreinforcement. Simultaneous presentation of elements of a compound stimulus is not necessary for overshadowing because the phenomenon was captured with sequentially presented stimuli.     

Effects of stimulus spacing on steady state gradients of inhibitory stimulus control

When the response of pigeons is maintained to a number of stimulus wavelengths, but extinguished to one (S?), the birds peck more rapidly at stimuli near the S? than at more distant stimuli. The present study explores this “dimensional contrast” effect as a function of the number and spacing of test wavelengths. A fixed portion of the wavelength continuum was spanned by 5, 9, 13, or 49 stimuli, which appeared in random sequence behind a standard pecking key. At the end of each 20-sec trial, pecks to test stimuli produced a conditioned reinforcer (sometimes followed by food), while pecks to the S? stimulus produced only darkness. Dimensional contrast “shoulders” developed to test stimuli on either side of the S?; these shoulders were of approximately the same height and wavelength position for all but the 5-stimulus (widely spaced) condition, and were comparable to the original contrast results with 25 stimuli. The results strongly suggest that the extent and locus of contrast shoulders are largely independent of the number and spacing of test stimuli.     

Gradient form and sequential effects during generalization testing in extinction

Four naive pigeons were given six generalization tests in extinction after periods of pretraining in which S+ appeared with food reinforcement and S? appeared in extinction. An analysis of sequential effects among presentations of test stimuli showed that the overall gradient was influenced differently by stimuli at the extremes of the continuum of test stimuli and by S+ and adjacent stimuli. Gradients consisting of responding in each stimulus when it was preceded by an extreme stimulus tended to peak at S+, while gradients produced when each stimulus was preceded by S+ or an adjacent stimulus tended to show a peak shift. This was true whether the overall gradient showed a peak shift or not. Two naive subjects were added and four additional tests were given after pretraining in which unequal frequencies of reinforcement accompanied both S+ and S?. Results of all 10 tests show that sequential effects occur during generalization testing in extinction and that these “local dimensional effects” are unlike local contrast. These stimulus-specific sequential effects may greatly influence overall gradient form.     

Remembering: The role of extraneous reinforcement

In two experiments, pigeons' responding on an extraneous task was explicitly reinforced during delayed matching-to-sample trials. In Experiment 1, red or green sample stimuli were followed by retention intervals of 0.2, 1, 4, or 12 sec, during which pecks to a white center key were reinforced with 2.5-sec access to wheat according to extinction, variable-interval 30-sec, and variable-interval 15-sec schedules in different conditions. A proportion of .2, .5, .7, or .9 of subsequent red or green choice responses that matched the sample were reinforced with 3-sec access to wheat. The result was that increasing center key reinforcement, or reducing reinforcer probability, lowered overall accuracy. Initial discriminability fell, but with no change in the rate of forgetting. In Experiment 2, initial discriminability was affected by extraneous reinforcers that were contingent on center key pecking, but not by noncontingent reinforcers. A plausible conclusion is that initial discriminability decreases when reinforcers strengthen competing behaviors.     

Flexible coding of temporal information by pigeons: Event durations as remember and forget cues for temporal samples

The ability of pigeons to use event durations as remember (R) and forget (F) cues for temporal samples was examined. Pigeons were required to indicate whether a houselight sample stimulus was short (2 sec) or long (6 sec) by pecking a red or a green comparison stimulus. After training with a constant 10-sec delay interval, temporal cues (illumination of the center key) were presented 2 sec after the offset of the temporal samples. For one group, a short (2-sec) temporal cue served as the R cue and a long (6-3ec) temporal cue served as the F cue. This was reversed for a second group of birds. During training, comparison stimuli were always presented following the temporal R cue, but never following the temporal F cue. Tests for the effectiveness of the temporal R and F cues showed that F cues were equally effective in reducing matching accuracy in both groups of birds. It was concluded that pigeons used the duration of the cue to determine whether or not to rehearse the memory code for the temporal sample.     

Preference and resistance to change in concurrent variable-interval schedules

Pigeons were trained on a multiple schedule in which separate concurrent schedules were presented in the two components of the schedule. During one component, concurrent variable-interval 40-sec variableinterval 80-sec schedules operated. In the second component, concurrent variable-interval 40-sec variableinterval 20-sec schedules operated. After stable baseline performance was obtained in both components, extinction probe choice tests were presented to assess preference between the variable-interval 40-sec schedules from the two components. The variable-interval 40-sec schedule paired with the variableinterval 80-sec schedule was preferred over the variable-interval 40-sec schedule paired with the variableinterval 20-sec schedule. The subjects were also exposed to several resistance-to-change manipulations: (1) prefeeding prior to the experimental session, (2) a free-food schedule added to timeout periods separating components, and (3) extinction. The results indicated that preference and resistance to change do not necessarily covary.     

Motivational mechanisms and schedule-induced behavioral stereotypy

In two experiments, behavioral stereotypies elicited by scheduled presentations of food and water were compared. In Experiment 1, pigeons were exposed to a fixed-time 30-sec (FT 30-sec) schedule of food or water deliveries with a brightening keylight stimulus signaling time to the unconditioned stimulus (UCS) on each trial. Food and water presentations both produced terminal autoshaped keypecking that was similarly distributed in the trial but differed in response topography and persistence. Locomotor interim behavior was different in the two motivational conditions: With food presentations, it consisted of a “retreat” to the rear of the chamber after UCS termination, followed by “pacing” in the midportion of trials. The water schedule produced very little locomotor activity with no regular distribution in the trial. Experiment 2, using a random-time 30-sec (RT 30-sec) schedule, showed that the differences in interim locomotor behavior persisted in the absence of temporal predictability of the UCS and the keypecking terminal response. The results are taken to support Timberlake’s (1983a) behavior-system theory.     

Required pecking and refraining from pecking alter judgments of time by pigeons

There is evidence that humans' perception of time is affected by the activity in which they are engaged while they are timing. The more demanding the task, the faster time appears to pass. A similar effect has been found in pigeons. Pigeons trained to discriminate between a short-duration (2-sec) and a long-duration (10-sec) stimulus were required to peck when the stimulus was one color and to refrain from pecking when it was a different color. On probe trials of intermediate durations, the bisection point (50% choice of the stimulus associated with both long and short stimuli) for trials in which the pigeons were required to peck was almost 1 sec longer than on trials in which the pigeons were required to refrain from pecking (Zentall, Friedrich, & Clement, 2006). In the present research, we replicated this effect and determined the relation between this effect and the typical bisection point that occurs when pecking is permitted but not required. Results indicated that the typical procedure results in a bisection point that is between required pecking and refraining from pecking. Furthermore, the rate of pecking when pecking is allowed but not required also falls between the rate of pecking for the required-pecking and refrain-from-pecking conditions. This result suggests that, similar to humans, pigeons underestimate the passage of time when they are active or when attention to time-related cues has to be shared with attention to satisfying the response requirement.     

A fading procedure facilitates the maintenance of observing responses when the more valued stimulus is not produced

Pigeons served in two experiments in which responding on an observing key converted a two-component mixed schedule to the corresponding multiple schedule of reinforcement. Presentation of the stimulus correlated with the more valued component was faded out (probabilistically) over sessions, so that ultimately an observing response produced only the stimulus correlated with the less valued component. Observing was well maintained after a fading procedure when a stimulus was produced by a single response, regardless of whether the less valued stimulus was associated with food or with extinction (Experiment 1). However, observing was not well maintained after a fading procedure when a stimulus was produced according to an intermittent schedule (Experiment 2). Taken together, the results of the two experiments suggest that the absence of an exteroceptive stimulus change after a single response may become discriminative in its own right for the more valued component, and that the fading procedure is an effective means of promoting this discrimination. However, if observing responses produce a stimulus change according to an intermittent schedule, then the absence of a stimulus change after a response is correlated with both components. Under these conditions, the absence of stimulus change is not discriminative for either component, even with fading, and observing is not maintained.     

Modification of chicks’ pecking preferences: Food imprinting or instrumental conditioning?

Using Hess’ technique, 35 small groups of chicks of age 2–3 days or beyond were reinforced in training with food for pecking at one visual stimulus (S+) and nonreinforced for pecking at another (S?) and subsequently tested in extinction. In training, pecking was strongly conditioned to S+, but in testing, both number of pecks to and preference for S+ declined as in instrumental conditioning, contrary to Hess’ original report. Results are discussed relative to studies exposing chicks to actual food objects, where an imprinting-like phenomenon has been found.     

Behavior patterns in pigeons during autoshaping with an incremental conditioned stimulus

Pigeons were trained on a variant of the autoshaping procedure in which a keylight stimulus of increasing brightness was used to signal the passing of a 30-sec interfood interval (IFI). Key-pecking developed in all subjects within the first session (65 trials). Within trials, pecking began midway through the IFI, increased throughout the remainder, and decreased just before food delivery. Other behavioral stereotypies were also recorded: Low light levels were associated with a retreat to the rear of the test chamber, and medium light levels (during the midportion of the IFI) were associated with high rates of pacing toward and away from the food source. Probe trials revealed that pecking, pacing, and retreat were all under strong stimulus control; that is, when the light was held constant at its lowest or highest brightness, or when the brightness ramp was presented in reverse order, the behavior pattern almost invariably remained tied to stimulus brightness. Results are discussed in terms of associative and nonassociative sources of the form and sequential characteristics of the behavior.     

Effects of trial duration on overall and momentary rates of maintained autoshaped keypecking: Choice and single-stimulus trials

Pigeons were exposed to differentially cued autoshaping trials in which conditioned stimuli were followed by food after 6 or 14 sec. Average and momentary rates of keypecking were examined on two types of unreinforced test trials: single-stimulus probe trials and simultaneous choice trials, each 40 sec in duration. Rates averaged over the 40-sec test trials did not favor the cue associated with the shorter delay to food (the short-delay cue) on either type of test trial; however, average rates prior to the scheduled time of food delivery were reliably higher for the short-delay cue on choice trials. Momentary rates of keypecking during choice trials varied as a function of both cue and elapsed time from trial onset. At short elapsed trial times, rate of pecking was higher for the short-delay cue, with this difference reversing at longer times. A reversal of the programmed relation between key color and delay to food presentation for 5 birds confirmed the generality of these findings. Implications of these data for models of Pavlovian conditioning and for methods of assessing conditioned response strength are discussed.     

Pigeons and rats observe signals of when but not where food will occur

Pigeons and rats were exposed to a mixed variable-time extinction schedule of reinforcement. During the variable-time component of the schedule, response-independent food was delivered at either a left or a right feeder. The animals were allowed to perform observing responses to produce either stimuli paired with the component of the mixed schedule that was in effect (temporal information) or stimuli paired with the feeder that might deliver food (spatial information). Only stimuli conveying temporal information reinforced observing. This result contradicts a prediction of the “information hypothesis” of observing, but is consistent with various conditioned-reinforcement interpretations of observing.     

Preference for the interval schedule following multiple variable-ratio yoked-variable-interval schedule training

Pigeons were studied on multiple variable-ratio yoked-variable-interval schedules in which components had equal rates of food reinforcement and appeared equally often on each of two keys. Interpolated between component changes on the final multiple schedule were 10-sec probes in which both schedule stimuli were present, one on each key. During multiple schedule training, variable-ratio response rates were greater than yoked-variable-interval rates; however, response rate differences in the components were not a function of the mean ratio value for the 40-to-320-ratio range studied. During the choice probes, subjects responded more to the stimulus associated with the interval schedule than to the one associated with the ratio schedule. It was concluded that pigeons prefer interval schedules over equal reinforcement rate ratio schedules, because the former generate fewer responses per reinforcement.     

The effect of the controllability of auditory discriminative stimuli in the performance of go/no-go discriminations by pigeons

Compared with their performance with localized (on-key) visual stimuli, pigeons are notoriously poor at performing go/no-go discriminations when keypecking for food in the presence of auditory discriminative stimuli. The difference might reflect the fact that an aversive visual onkey stimulus signaling nonreward can be escaped by looking away and not pecking, which contributes to the measure of good discriminative performance, while an auditory stimulus cannot be escaped. In Experiment 1, discriminative performance was significantly improved by providing pigeons with a response incompatible with keypecking by which they could escape a tone S+ and a tone S?. However, the pattern, frequency, and duration of escape responses were found to be insufficient to explain the improvement. In Experiment 2, it was found that the capacity to escape only S+ or only S? enhanced discriminative performance as much as the capacity to escape both. It is theorized that the Pavlovian relationship between the absence of the discriminative stimuli and the nonoccurrence of food might transfer to the instrumental relationships learned in a go/no-go discrimination. The possibility that intermittent stimuli command more attention than continuous stimuli is also considered.     

Multiple-schedule interactions and discrimination

Pigeons received variable-interval (VI) reinforcement for keypecking during randomized presentations of seven line-orientation stimuli forming a continuum ranging from horizontal (0 deg) to vertical (90 deg). Each line presentation lasted for 30 sec and was preceded and followed by 30-sec time-outs. After responding stabilized, only responding in the two extreme stimuli (0 and 90 deg) was reinforced. As discrimination training proceeded, strong behavioral contrast and dimensional contrast effects appeared. However, only marginal local effects (local contrast and local dimensional effects), exerted by one line-orientation component upon a second, appeared, indicating that behavioral and dimensional contrast may be independent of parallel local effects. An attempt was made to apply Blough’s (1975) quantitative model of operant generalization and discrimination to the present discrimination procedure. However, this model did not predict the generalization gradient shape that was experimentally obtained. This experiment also yielded two serendipitous findings: (1) Positive behavioral contrast appeared in an extinction-related stimulus (time-out) when other stimuli were switched from reinforcement to extinction (hitherto, positive behavioral contrast had been observed only in responding to a reinforcementrelated stimulus when other stimuli were switched from reinforcement to extinction), and (2) final responding was higher in the presence of an extinction stimulus that had always been an extinction stimulus than it was in the presence of other extinction stimuli that had previously been paired with VI reinforcement.     

Behavioral momentum and relapse of extinguished operant responding

Previous experiments on behavioral momentum have shown that relative resistance to extinction of operant behavior in the presence of a stimulus depends on the rate of reinforcement associated with that stimulus, even if some of those reinforcers occur independently of the behavior. We present three experiments examining whether the rate of reinforcement in the presence of a stimulus similarly modulates the relative relapse of operant behavior produced by reinstatement, resurgence, and renewal paradigms. During baseline conditions, pigeons responded for food reinforcement on variable-interval 120-sec schedules in alternating periods of exposure to two stimuli arranged by a multiple schedule. Additional response-independent food presentations were also delivered in the presence of one of the multiple-schedule stimuli. Consistent with previous research, baseline response rates were lower in the presence of the stimulus with the added response-independent reinforcement, and relative resistance to extinction was greater in the presence of that stimulus. In addition, following extinction, the relative relapse of responding produced by reinstatement, resurgence, and renewal paradigms was greater in the presence of the stimulus associated with the higher rate of reinforcement. We suggest that a model of extinction from behavioral momentum theory may be useful for understanding these results.     

Choice and percentage reinforcement in pigeons

Pigeons responded on a two-key concurrent chains choice procedure with the same level of percentage reinforcement on each key. During the initial links, a choice response on either key occasionally produced a conditioned reinforcer—which on one key was associated with a 15-sec, and on the other key with a 30-sec, interreinforcement interval—or an extinction stimulus. In Part 1, the initial links were equal. With successive decreases in the probability of a reinforcer, choice shifted from preference for the 15-sec terminal link toward indifference. In Part 2, the initial links were unequal and were arranged so that the shorter initial link preceded the 30-sec terminal link. At a high probability of a reinforcer, the pigeons again preferred the 15-sec terminal link. However, at a low probability, the pigeons reversed and preferred the alternate key. It was concluded that the conditioned reinforcers tended to become functionally equivalent at a low probability of a reinforcer, despite the nominally different interreinforcement intervals, with the result that choice was then modulated by the relative size of the initial links. The data are inconsistent with the view that choice and the strength of conditioned reinforcers are isomorphic with the reduction in delay to reward correlated with terminal link stimuli.