Remembering: The role of extraneous reinforcement

In two experiments, pigeons' responding on an extraneous task was explicitly reinforced during delayed matching-to-sample trials. In Experiment 1, red or green sample stimuli were followed by retention intervals of 0.2, 1, 4, or 12 sec, during which pecks to a white center key were reinforced with 2.5-sec access to wheat according to extinction, variable-interval 30-sec, and variable-interval 15-sec schedules in different conditions. A proportion of .2, .5, .7, or .9 of subsequent red or green choice responses that matched the sample were reinforced with 3-sec access to wheat. The result was that increasing center key reinforcement, or reducing reinforcer probability, lowered overall accuracy. Initial discriminability fell, but with no change in the rate of forgetting. In Experiment 2, initial discriminability was affected by extraneous reinforcers that were contingent on center key pecking, but not by noncontingent reinforcers. A plausible conclusion is that initial discriminability decreases when reinforcers strengthen competing behaviors.     

Pigeons’ memory for empty time intervals marked by visual or auditory stimuli

In Experiment 1, pigeons were trained to discriminate the duration (2 or 8 sec) of an empty interval separated by two 1325-Hz tone markers by responding to red and green comparison stimuli. During delay testing, a choose-short bias occurred at 1 sec, but a robust choose-long bias occurred at 9 sec. Responding in the absence of tone markers indicated that the pigeons were attending to the markers and not simply timing the total trial duration. The birds were then trained to match short (2-sec) or long (8-sec) empty intervals marked by light to blue/yellow comparisons. For both visual and auditory markers, delay testing produced a choose-short bias at 1 sec and a choose-long bias at 9 sec. In Experiment 2, the pigeons were shifted from a fixed to variable intertrial intervals (ITI) within sessions. On trials with tone markers, the duration of both the empty interval and the preceding ITI affected choice responding. On trials with light markers, only the duration of the empty interval influenced choice responding. Subsequent delay testing in the context of variable ITIs replicated the memory biases previously obtained. In Experiment 3, performance was assessed at various delay intervals on trials in which either the first or the second marker was omitted. The data from these omission tests indicated that the first marker initiated timing but that the second marker sometimes initiated the timing of a new interval. Explanations of these effects in terms of the internal clock model of timing are discussed, and a simple quantitative model of the delay interval data is tested.     

Serial conditioning as a function of parametric variations of an interfood clock

Two experiments examined the effects of differences in the parameters of a clocked interfood interval on the obtained distribution of responding to the stimuli in that interval. In the first experiment, a 3×3 factorial design assessed the effects of the number of components and the durations of those components. Food was presented irrespective of behavior following 5, 10, or 20 discrete stimuli across durations of 3, 6, or 12 sec each. Responding began at the approximate midpoint of the interval. More responding occurred in earlier portions of the last half of the interval as the number and the durations of individual stimuli decreased or as the overall interfood interval decreased. The second experiment, also a 3×3 factorial design, manipulated the probability and duration of food presentation following a 60-sec trial containing 10 discrete stimuli. A 2.0-, 3.5-, or 5.0-sec food presentation followed 100%, 25%, or 10% of the trials. Responding again began at the approximate midpoint of the interval. More responding occurred in earlier portions of the last half of the interval only when both food duration and the proportion of reinforced trials increased. Both experiments therefore showed that the onset of responding occurs at the approximate midpoint of clocked interfood intervals in spite of a wide variety of CS and US manipulations.     

Conjunctive schedules of reinforcement IV: Effects on the pattern of responding of changes in requirement at reinforcement

The effects of different schedule requirements at reinforcement on patterns of responding by pigeons were assessed under conjunctive schedules with comparable response-number requirements, Under one conjunctive schedule (conjunctive fixed-interval fixed-ratio schedule), a response was reinforced after a 6-min interval had elapsedand a specific minimum number of responses had been emitted, Under a second conjunctive schedule, a response was reinforced after the 6-min fixed interval and upon completion of a tandem schedule requirement (conjunctive fixed-interval tandem schedule), This schedule retained the same required minimum number of responses as the first conjunctive schedule, but responses were never reinforced according to a fixed-ratio schedule; the tandem schedule was comprised of a fixed-ratio and a small (.1 to 10.0 sec) fixed-interval schedule, Under the conjunctive fixed-interval fixed-ratio schedule, responding was characterized by an initial pause, an abrupt transition to a high response rate, and a second transition to a lower rate that prevailed or slightly increased up to reinforcement, Under the conjunctive fixed-interval tandem schedule, pauses were extended, response rates were lower, and the initial high rate of responding was generally absent, The above effects depended upon the size of the fixed interval of the tandem schedule, The distinct pattern of responding generated by conjunctive fixed-interval fixed-ratio schedules depends upon occasional reinforcement of fixed-ratio responding and not merely on the addition of a minimum number of required responses.     

Quantitative analysis of local-level resurgence

Resurgence is the recurrence of a previously reinforced and then extinguished behavior induced by the extinction of another more recently reinforced behavior. Resurgence provides insight into behavioral processes relevant to treatment relapse of a range of problem behaviors. Resurgence is typically studied across three phases: (1) reinforcement of a target response, (2) extinction of the target and concurrent reinforcement of an alternative response, and (3) extinction of the alternative response, resulting in the recurrence of target responding. Because each phase typically occurs successively and spans multiple sessions, extended time frames separate the training and resurgence of target responding. This study assessed resurgence more dynamically and throughout ongoing training in 6 pigeons. Baseline entailed 50-s trials of a free-operant psychophysical procedure, resembling Phases 1 and 2 of typical resurgence procedures. During the first 25 s, we reinforced target (left-key) responding but not alternative (right-key) responding; contingencies reversed during the second 25 s. Target and alternative responding followed the baseline reinforcement contingencies, with alternative responding replacing target responding across the 50 s. We observed resurgence of target responding during signaled and unsignaled probes that extended trial durations an additional 100 s in extinction. Furthermore, resurgence was greater and/or sooner when probes were signaled, suggesting an important role of discriminating transitions to extinction in resurgence. The data were well described by an extension of a stimulus-control model of discrimination that assumes resurgence is the result of generalization of obtained reinforcers across space and time. Therefore, the present findings introduce novel methods and quantitative analyses for assessing behavioral processes underlying resurgence.     

Positive behavioral contrast as a function of time-out duration when pigeons peck keys on a within-session procedure

Three pigeons pecked keys for food reinforcers delivered by multiple variable interval variable interval schedules in the first part of each session (baseline) and by multiple variable interval extinction schedules in the second part of each session (contrast). The variable interval schedules delivered reinforcers after an average of 4 min or 30 sec in different conditions. The duration of a time-out between the components varied in five steps from 5 to 120 sec. Positive contrast occurred for all time-out durations in both experiments. That is, the rate of responding emitted during the constant, variable interval component was greater during the contrast than during the baseline schedules. The size of contrast did not change systematically with changes in timeout duration. These results violate most theories of contrast. They are compatible with the idea that animals integrate reinforcers over intervals longer than 2 min.     

Temporal control during maintenance and extinction of conditioned keypecking in ring doves

Timing of conditioned keypecking was investigated using a delay autoshaping procedure. In two experiments, the CS-US interval (ISI) was varied and the temporal pattern of responding during unreinforced probe trials extending beyond the reinforced ISI was recorded. Both experiments showed temporal control of keypecking at ISIs of 4, 8, and 16 sec, regardless of whether the probe duration was held constant (Experiment 1) or covaried with the ISI (Experiment 2). Analyses showed that the timing of keypecking was scalar. Increased responding near the end of the probe was due possibly to independent timing of the probe duration. In a third experiment, a group of subjects trained at an ISI of 8 sec were extinguished by presentation of only probe trials. Although the maximal response rate declined progressively, timing of keypecking did not change.     

Short-term retention of “surprising” events following different training conditions

Terry and Wagner (1975) have suggested that short-term retention of information about an event is enhanced if the occurrence of the event is surprising. To investigate this idea, we trained two groups of pigeons in a preparatory-releaser procedure in which half the trials started with the presentation of food (the preparatory event). The preparatory food presensation was signaled by an 8-sec white keylight in the signaled, but not in the unsignaled, group. After a retention interval, varying between 2 and 32 sec, the releaser stimulus (CS_R), a red keylight, was presented for 8 sec in the absence of any reinforcement. The remaining trials were initiated by the presentation of CS_R, and the first peck occurring 8 sec after the onset of CS_R was reinforced by food. The preparatory event controlled responding to CS_R at the short retention interval, with the level of control declining systematically with increasing retention intervals. On probe test trials, the presentation of the preparatory food event was preceded by a stimulus that had previously been paired (CS+) or unpaired with food (CS?). Discriminative responding to CSr was better following CS? than following CS+ in the unsignaled, but not the signaled, group. These results suggest that the enhanced retention following surprising preparatory events reflects a generalization decrement induced by changing the signaling conditions between training and testing.     

How reinforcement context affects temporal production and categorization

The behavioral theory of timing assumes that timing is governed by a pacemaker whose pulses move organisms from one state to the next, and that the speed of the pacemaker covaries with the rate of reinforcement in the experimental context. The goal of the present experiments was to clarify just what constitutes that context. In Experiment 1, pigeons responded on signaled fixed-interval 20-sec and 40-sec schedules of food reinforcement that were presented randomly within sessions (alternating condition) or between sessions (isolated condition). In Experiment 2, pigeons categorized the duration of a short or a long set of intervals in the alternating or the isolated condition. Performance in both experiments was under strong control by the signals, with scalar timing between long and short sets, but no significant differences between the alternating and isolated conditions. The context of reinforcement that determines pacemaker period can thus be specific to a particular timing task and signal.     

Prospective representation: The effects of varied mapping of sample stimuli to comparison stimuli and differential trial outcomes on pigeons’ working memory

The mapping of sample stimuli onto comparison stimuli and the nature of trial outcomes were factorially manipulated in a delayed matching-to-sample procedure. In the many-to-one condition (MTO), responding to a particular comparison was correct following several sample stimuli, whereas in the one-to-many condition (OTM), responding to several comparison stimuli was correct following a particular sample. Probabilities of reinforcement for correct responding to comparison stimuli were either differential (DO) or nondifferential (NDO). Four groups of pigeons were trained under four combinations of mapping and outcome conditions, MTO-DO, MTO-NDO, OTM-DO, and OTM-NDO. Testing at delay intervals of 1, 2, 4, and 8 sec revealed significant effects due to both the mapping variable and the differential outcomes variable. It was argued that the poorer performance obtained in the OTM condition was due to the differential prospective coding requirements placed on reference and working memory by this mapping. In the OTM conditions, a greater number of response codes had to be retrieved from reference memory and multiple response codes may have overburdened working memory, which has a limited capacity.     

Conservation,matching, and the variable-interval schedule

The conservation model was generalized to the variable-interval schedule by incorporating the concept of unscheduled instrumental responses, those which occur in the time before the next setup is due. Thirsty rats responded in constant-duration sessions on two 7-sec schedules that required one leverpress for 25 and 50 licks at a water tube and on a 14-sec 25-lick schedule. In accordance with the model, total licks decreased linearly as total presses increased, and the schedules facilitated leverpressing and suppressed licking relative to paired baseline levels of responding. While the matching model also gave a satisfactory fit to instrumental responding under the schedules, its two constants, representing asymptotic rate of responding and extraneous reinforcement, had anomalous values which led the model to predict that response rate would decrease as the rate of reinforcement increased, directly opposing its prediction for the constant-consumption experiments of its previous tests.     

Partial reinforcement in autoshaping with pigeons

The acquisition, maintenance, and extinction of autoshaped responding in pigeons were studied under partial and continuous reinforcement. Five values of probability of reinforcement, ranging from .1 to 1.0, were combined factorially with five values of intertrial interval ranging from 15 to 250 sec for different groups. The number of trials required before autoshaped responding emerged varied inversely with the duration of the intertriai interval and probability of reinforcment, but partial reinforcement did not increase the number of reinforcers before acquisition. During maintained training, partial reinforcement increased the overall rate of responding. A temporal gradient of accelerated responding over the trial duration emerged during maintenance training for partial reinforcement groups, and was evident for all groups in extinction. Partial reinforcement groups responded more than continuous reinforcement groups over an equivalent number of trials in extinction. However, this partial-reinforcment extinction effect disappeared when examined in terms of the omission of “expected” reinforcers.     

Memory for empty time intervals in pigeons

Pigeons were trained to discriminate short (2 sec) and long (8 sec) empty intervals that began each trial. In group consistent, onset of an empty interval was marked by a brief presentation of red keylight, and termination of the interval was marked by a brief presentation of green keylight. In group inconsistent, red and green served equally often as the first and second markers across trials. Testing revealed that, in group consistent, (1) birds were sensitive to the relation between marker color and marker type and (2) presentation of the second marker did not initiate timing a new interval. Testing also revealed a robust choose-long effect at delays longer than the training delay and indifference between the comparisons on no-sample trials. Both of the latter findings differ from those typically obtained when filled intervals are employed. It was concluded that pigeons process filled and empty intervals differently.     

The “disinhibition” of extinguished operant behavior in pigeons: Trial-tempo shifts and novel stimulus effects

After conditioning and extinction of keypecking with 25-see intertrial intervals between key illuminations, an immediate change to 5-sec intertrial intervals reinstated pecking during trials. Brief illuminations of the chamber during intertrial intervals also temporarily restored extinguished keypecking. The same manipulations of trial tempo and chamber illumination usually weakened well established, still reinforced keypecking. No recovery of extinguished behavior occurred when the intertrial interval was shifted upward from 5 sec to 25 sec. These and other behavioral findings were examined in relation to (a) Pavlov’s and Skinner’s research and views on “disinhibition” and “external inhibition” and to (b) analogous phenomena in physiological studies of habituation and dishabituation. The reliable evidence of disinhibition obtained in the present experiments suggests the involvement of an inhibitory process in extinction.     

Multiple-schedule interactions and discrimination

Pigeons received variable-interval (VI) reinforcement for keypecking during randomized presentations of seven line-orientation stimuli forming a continuum ranging from horizontal (0 deg) to vertical (90 deg). Each line presentation lasted for 30 sec and was preceded and followed by 30-sec time-outs. After responding stabilized, only responding in the two extreme stimuli (0 and 90 deg) was reinforced. As discrimination training proceeded, strong behavioral contrast and dimensional contrast effects appeared. However, only marginal local effects (local contrast and local dimensional effects), exerted by one line-orientation component upon a second, appeared, indicating that behavioral and dimensional contrast may be independent of parallel local effects. An attempt was made to apply Blough’s (1975) quantitative model of operant generalization and discrimination to the present discrimination procedure. However, this model did not predict the generalization gradient shape that was experimentally obtained. This experiment also yielded two serendipitous findings: (1) Positive behavioral contrast appeared in an extinction-related stimulus (time-out) when other stimuli were switched from reinforcement to extinction (hitherto, positive behavioral contrast had been observed only in responding to a reinforcementrelated stimulus when other stimuli were switched from reinforcement to extinction), and (2) final responding was higher in the presence of an extinction stimulus that had always been an extinction stimulus than it was in the presence of other extinction stimuli that had previously been paired with VI reinforcement.     

Sources of errors by capuchin monkeys on delayed response

Six capuchin monkeys were trained on a series of indirect, delayed response problems with delay intervals ranging from 0 to 60 sec. Data were analyzed by sequential state theory in order to quantify random and nonrandom components of the subjects’ response sequences. A tendency of the subjects to repeat responses to the position chosen on the previous trial was a significant source of proactive interference even when effective orienting responses were elicited by the predelay cue. Changes in random and nonrandom response components with increasing delays suggested a two-phase process: the first for delays up to 7 sec characterized by increases in random and nonrandom error-producing responses, and the second for delays greater than 7 sec characterized by increases in random responding only.     

Tracking of the expected time to reinforcement in temporal conditioning procedures

In one experiment, the rate and pattern of responding (head entry into the food cup) under different distributions of intervals between food deliveries were examined. Separate groups of rats received fixed-time (45, 90, 180, or 360 sec), random-time (45, 90, 180, or 360 sec), or tandem fixed-time (45 or 90 sec) random-time (45 or 90 sec) schedules of reinforcement. Schedule type affected the pattern of responding as a function of time, whereas mean interval duration affected the mean rate of responding. Responses occurred in bouts with characteristics that were invariant across conditions. Packet theory, which assumes that the momentary probability of bout occurrence is negatively related to the conditional expected time remaining until the next reinforcer, accurately predicted global and local measures of responding. The success of the model advances the prediction of multiple measures of responding across different types of time-based schedules.     

Latent discrimination learning under a regular schedule of partial reinforcement

In this investigation, which employed rats in a runway, discriminative responding consisted of faster running on the reinforced than on the nonreinforced trials of either the 4NR or R4N schedule, both schedules containing fixed, repeated sequences of nonreinforced and reinforced trials. Under the 4NR schedule, four nonreinforced trials preceded a reinforced trial each day, and under the R4N schedule, a reinforced trial was followed by four nonreinforced trials each day. The major finding obtained was that under the 4NR schedule, discriminative responding was improved very substantially by a shift to extinction. Rats maintained on the 4NR schedule did not show improved discriminative responding, nor did discriminative responding improve in extinction following training under either the R4N schedule or a schedule of consistent reinforcement. Latent discrimination learning was defined as discriminative responding which fails to reflect adequately the amount of discrimination learning accomplished. The present findings demonstrate latent discrimination learning for regular schedules of partial reinforcement, something already demonstrated for brightness differential conditioning and possibly DRL schedules, as well.     

Specification of the stimulus-reinforcer relation in multiple schedules: Delay and probability of reinforcement

Control of pigeons’ keypecking by a stimulus-reinforcer contingency was investigated in the context of a four-component multiple schedule. In each of three experiments, pigeons were exposed to a schedule consisting of two two-component sequences. Discriminative stimuli identifying each sequence were present only in Component 1, which was 4, 6, or 8 sec in duration, while reinforcers could be earned only in Component 2 (30 sec in duration). Control by a stimulus-reinforcer contingency was sought during Component 1 by arranging a differential relation between Component 1 cues and schedule of reinforcement in Component 2. In Experiment 1, rate of keypecking during Component 1 varied with the presence and absence of a stimulus-reinforcer contingency. When a contingency was introduced, rate of keypecking increased during the Component 1 cue associated with the availability of reinforcement in Component 2. In Experiment 2, the stimulus-reinforcer contingency was manipulated parametrically by varying the correlation between Component 1 cues and Component 2 schedules of reinforcement. Responding in Component 1 varied as a function of strength of the stimulus-reinforcer contingency. The relatively high rates of Component 1 responding observed in Experiments 1 and 2 pose difficulties for conceptions of stimulus-reinforcer control based on probability of reinforcement. In these two experiments, the stimulus-associated probabilities of reinforcement in Component 1 were invariant at zero. An alternate dimension of stimulus-reinforcer control was explored in Experiment 3, in which Component 1 cues were differentially associated with delay to reinforcement in Component 2, while probability of reinforcement was held constant across components. When the stimulus-reinforcer contingency was in force, rate of responding in Component 1 varied inversely with delay to reinforcement in Component 2. In a quantitative analysis of data from Experiments 2 and 3, relative rate of responding during Component 1 was strongly correlated with two measures of relative delay to reinforcement.     

Response strength and temporal control in fixed-interval schedules

Pigeons responded in a two-component peak procedure in which the components differed in terms of reinforcement magnitude (Experiment 1), immediacy (Experiment 2), or probability (Experiment 3). The prediction of behavioral momentum theory that responding in the relatively richer component should be more resistant to change was tested by (1) presenting response-independent food in the intervals between components according to a variable-time (VT) schedule, (2) prefeeding, and (3) extinction. In all the experiments, peak location in baseline occurred earlier, relative to the schedule value in the richer component. Peak response rate was more resistant to change in the richer component during the VT and prefeeding tests, and change in peak rate was more sensitive to differential reinforcement than change in overall response rate. Changes in measures of performance on peak trials during the disruptor tests were partially consistent with predictions of the behavioral theory of timing. The results suggest that peak response rate provides a more sensitive index of resistance to change for fixed-interval schedules than does overall response rate and that reinforcement strengthens both peak responding and temporal control.