Ontogeny of persistence: Immediate extinction effects in preweanling and weanling rats

In Experiment I rats were trained for 21÷2 days under partial (PRF) or continuous reinforcement (CRF) conditions starting at 18, 22, 28, or 36 days of age and were then subjected to immediate extinction. At all ages there was a strong partial reinforcement extinction effect (PREE), and absolute size of PREE was greatest in the youngest rats. Rate of extinction increased as a function of age following both CRF and PRF. In Experiment II the youngest and oldest age groups of Experiment I were run under the two reward conditions of Experiment I and in a third condition, PRF with number of rewards rather than trials equated to CRF (PRF-R). The PRF-R and PRF groups were not different in extinction, and both were more persistent than CRF. The youngest rats were again more persistent than the oldest, particularly after PRF training. In Experiment III it was shown that the well-known paradoxical effect, greater reward in CRF acquisition leads to faster extinction, operates in our youngest and oldest animals, but is more pronounced in the oldest. The results are discussed in terms of whether they require different explanations than those often applied to extinction data from adult rats.     

Durability of the partial reinforcement and partial delay of reinforcement extinction effects after minimal acquisition training

Four groups of 10 rats each were given six acquisition trials (Phase 1) under continuous reinforcement (CR), partial reinforcement (PR), constant delay (CD), or partial delay of reinforcement (PD) conditions. In Phase 2, all Ss were given 18 nonreinforced trials, followed by 12 continuously reinforced trials in Phase 3. In Phase 4, all Ss were given 12 more extinction trials. A constant 24-h ITI was observed throughout the experiment. A strong partial reinforcement extinction effect (PREE) was obtained in both Phases 2 and 4. Only a temporary partial delay of reinforcement effect (PDRE) was observed, which was restricted to the first nine trials of the first extinction phase. No constant delay of reinforcement effect (CDRE) was observed in either extinction phase. The results were discussed in terms of both frustration and sequential theories.     

Effects of sequences of sucrose reward magnitudes with short ITIs in rats

Two experiments assessed the role of aftereffect learning in rats rewarded with sucrose solutions. In Experiment 1, rats were trained in a single straight runway for two trials on each of 18 days, each trial terminating with either large (20% scurose) or small (3% sucrose) reward. The ITI was 3–5 min. The sequence of daily rewards for each of four groups was small-small (SS), small-large, (SL), large-small (LS), or large-large (LL). Response patterning and a simultaneous negative contrast effect were observed in LS and SL relative to the consistently rewarded controls. During 10 massed extinction trials, resistance to extinction was greatest for Group SL, followed in order by Groups SS, LL, and LS. Experiment 2 examined single alternation of large and small rewards administered for 10 trials on each of 31 days with an ITI of 60 sec. Reward for one group was 20% or 3% sucrose while another received 1 or 10 45-mg Noyes pellets. Appropriate patterning developed only in the food-pellet rewarded animals. The overall results suggest that sucrose rewards may produce high-amplitude and long-duration aftereffects which interfere with learning in designs employing several massed daily trials, but which may facilitate learning—relative to food-pellet rewards—with longer intertrial intervals and fewer daily trials.     

Pigeons’ memory for event duration: Intertrial interval and delay effects

The effects of within-session variations in the intertriai interval (ITI) and delay on pigeons’ memory for event duration were studied in delayed symbolic matching-to-sample tasks. Pigeons were trained to peck one color following a long (8 sec) sample and another color following a short (2 sec) sample. In the first three experiments, the baseline conditions included a 10-sec delay (retention interval) and a 45-sec ITI. During testing, the delay was varied from 0 to 20 sec, and the ITI that preceded the trial was varied from 5 to 90 sec. When the ITI and delay were manipulated separately (Experiments 1 and 2), the pigeons displayed a choose-short tendency when the delay was longer than 10 sec or when the ITI was longer than 45 sec, and a choose-long tendency when either the delay or the ITI was shorter than these baseline values. These effects occurred whether the sample was food access or light. When the ITI and delay were manipulated together, the pigeons showed a large choose-long error tendency when the short delay was tested together with a short ITI, and no systematic error tendency when the short delay was tested together with a longer ITI. A very large choose-short error tendency emerged on trials with a long delay and a long ITI; a reduced choose-short tendency was present when the long delay was presented together with a short ITI. In Experiment 4, the baseline conditions were a 0-sec delay and a 45-sec ITI. In this case variations in the ITI had a smaller and unidirectional effect: the pigeons showed a choose-long error tendency when the ITI was decreased, but no effect of ITI increases. Two hypotheses were proposed and discussed: (1) that pigeons judge sample durations relative to a background time composed of the ITI and delay, and (2) that the delay and ITI effects might arise from a combination of subjective shortening and proactive effects of samples from previous trials.     

Pigeons’ memory for empty time intervals marked by visual or auditory stimuli

In Experiment 1, pigeons were trained to discriminate the duration (2 or 8 sec) of an empty interval separated by two 1325-Hz tone markers by responding to red and green comparison stimuli. During delay testing, a choose-short bias occurred at 1 sec, but a robust choose-long bias occurred at 9 sec. Responding in the absence of tone markers indicated that the pigeons were attending to the markers and not simply timing the total trial duration. The birds were then trained to match short (2-sec) or long (8-sec) empty intervals marked by light to blue/yellow comparisons. For both visual and auditory markers, delay testing produced a choose-short bias at 1 sec and a choose-long bias at 9 sec. In Experiment 2, the pigeons were shifted from a fixed to variable intertrial intervals (ITI) within sessions. On trials with tone markers, the duration of both the empty interval and the preceding ITI affected choice responding. On trials with light markers, only the duration of the empty interval influenced choice responding. Subsequent delay testing in the context of variable ITIs replicated the memory biases previously obtained. In Experiment 3, performance was assessed at various delay intervals on trials in which either the first or the second marker was omitted. The data from these omission tests indicated that the first marker initiated timing but that the second marker sometimes initiated the timing of a new interval. Explanations of these effects in terms of the internal clock model of timing are discussed, and a simple quantitative model of the delay interval data is tested.     

Resistance to extinction: contingency termination and generalization decrement

We present an algebraic model of resistance to extinction that is consistent with research on resistance to change. The model assumes that response strength is a power function of reinforcer rate and that extinction involves two additive, decremental processes: (1) the termination of the reinforcement contingency and (2) generalization decrement resulting from reinforcer omission. The model was supported by three experiments. In Experiment 1, 4 pigeons were trained on two-component multiple variable-interval (VI) 60-sec, VI 240-sec schedules. In two conditions, resistance to change was tested by terminating the response-reinforcer contingency and presenting response-independent reinforcers at the same rate as in training. In two further conditions, resistance to change was tested by prefeeding and by extinction. In Experiment 2, 6 pigeons were trained on two-component multiple VI 150-sec schedules with 8-sec or 2-sec reinforcers, and resistance to change was tested by terminating the response-reinforcer contingency in three conditions. In two of those conditions, brief delays were interposed between responses and response-independent reinforcers. In both Experiments 1 and 2, response rate was more resistant to change in the richer component, except for extinction in Experiment 1. In Experiment 3, 8 pigeons were trained on multiple VI 30-sec, VI 120-sec schedules. During extinction, half of the presentations of each component were accompanied by a novel stimulus to produce generalization decrement. The extinction data of Experiments 1 and 3 were well described by our model. The value of the exponent relating response strength and reinforcement was similar in all three experiments.     

Acquisition and extinction of runway performance under escape,avoidance, and partial-avoidance procedures

The acquisition and extinction of locomotor responses of rats in a straight alley were examined for groups trained under escape, partial-avoidance, and avoidance procedures. During acquisition, one group (escape) received a 0-sec delay between being dropped into the alley and the onset of shock; two groups (partial avoidance) had 0.5- and 1-sec delays; and two groups (avoidance) had delays of 2 and 4 sec. On the final day of acquisition, the partial-avoidance rats displayed higher running speeds than either the escape- or avoidance-trained animals. The 4-sec avoidance group was consistently slower than all other groups. Speeds for all groups decreased during extinction, with rate of decline showing some relation to terminal acquisition level. Relative group performance levels proved to be consistent with a simple arithmetic model based on the assumption that changes in running speeds affect the aversiveness of the situation by altering US duration, CS duration, and effective US length.     

Effects of intertrial interval and exteroceptive feedback duration on discriminative avoidance acquisition in the gerbil

A series of studies of shuttlebox-avoidance learning in the gerbil evaluated the efficacy of an exteroceptive feedback stimulus (FS). Experiment 1 assessed the relative effectiveness of a FS at 30- and 90-sec intertriai intervals (ITIs), and found that the FS and warning signal termination contingencies were additive sources of avoidance reinforcement; i.e., they produced “supernormal acquisition” at the short ITI, but not at the 90-sec ITI. The effectiveness of a FS at the 30-sec ITI was further explored in Experiments 2 and 3, in which FS duration was varied in delayed and trace avoidance conditioning, respectively. In both studies, a FS facilitated acquisition but FS duration was not a critical determinant of performance. These results were interpreted in terms of an expectancy account of the informational value of a FS, and the problem of experimentally distinguishing between cognitive and inhibition-of-fear accounts of avoidance learning was discussed.     

Reductions in resistance to extinction and spontaneous recovery as a function of changes in transportational and contextual stimuli

Thirty rats received 10 sessions of baseline training in which leverpressing was reinforced according to a variable-interval (VI) 60-sec schedule. Twenty-four of the subjects were then assigned to one of four groups that received five sessions of extinction, with groups being differentiated in a 2 by 2 factorial design on the basis of: (1) changes in stimuli accompanying transportation of subjects from home cages to the laboratory and placement in the apparatus, and/or (2) changes in contextual stimuli within the apparatus. During the sixth session of extinction, the transportational and contextual stimuli previously associated with baseline training were reinstated. The remaining six rats experienced changes in both transportational and contextual stimuli but were maintained on the VI 60-sec schedule of reinforcement. Changes in either transportational or contextual stimuli reduced resistance to extinction and spontaneous recovery, and substantial increments in responding occurred upon reinstatement of the transportational and contextual stimuli associated with baseline training. Evidence for summation of the two sources of stimulus change was obtained. Changes in transportational and contextual stimuli produced only a brief disruption in responding when reinforcement of leverpressing was continued.     

Partial reinforcement effect: The expectancy of reward on nonreward trials

In order to determine the importance of the development of expectancy of reward prior to partial reward trials; rats were given 20 continuously reinforced trials prior to 20 partially reinforced trials (CRF-PRF) and compared to Ss given only 20 partially reinforced trials (PRF). Control groups received 20 or 40 continuously reinforced trials (CRF-20, CRF-40) to determine the effect of differing numbers of acquisition trials. Results showed that terminal acquisition differences were minimal in the run segment of the alley and that Group CRF-PRF was more resistant to extinction than Group PRF, and both were more resistant to extinction than the CRF-20 and CRF-40 groups, which did not differ from each other. These results were interpreted as supporting the notion that the expectancy of reward on nonreward trials during partial reinforcement acquisition is a determiner of the magnitude of the partial reinforcement extinction effect.     

Stimulus control of foodcup approach following fixed ratio reinforcement

Rats were trained on fixed ratio (FR) schedules requiring either 5 or 10 leverpresses to produce reinforcement and an intertrial interval (ITI). Half of the Ss at each ratio requirement were extinguished on an FR 5 and half on an FR 10 schedule of ITI presentation. Fewer foodcup approaches were made on the FR 10 than on the FR 5 extinction schedule, regardless of acquisition FR. Leverpresses per approach were fewer on the FR 5 than on the FR 10 extinction schedule and were fewer following FR 5 than following FR 10 acquisition. Data suggested the existence of interoceptive as well as exteroceptive stimulus control of foodcup approach and were discussed in terms of their implications for a response-unit account of extinction.     

Effects of varied and partial reward on discrete-trial patterning of rats

Rats were trained in the discrete-trial operant apparatus with single alternation of large reward and small reward or large reward and nonreward. followed by either transfer or extinction. The results showed that both groups acquired appropriate response patterning, that patterning is conditioned, and that the stimuli which control patterning are derived from reward conditions on immediately preceding trials. A modification of the sequential hypothesis of instrumental learning was proposed to account for the results.     

Preference and resistance to change in concurrent variable-interval schedules

Pigeons were trained on a multiple schedule in which separate concurrent schedules were presented in the two components of the schedule. During one component, concurrent variable-interval 40-sec variableinterval 80-sec schedules operated. In the second component, concurrent variable-interval 40-sec variableinterval 20-sec schedules operated. After stable baseline performance was obtained in both components, extinction probe choice tests were presented to assess preference between the variable-interval 40-sec schedules from the two components. The variable-interval 40-sec schedule paired with the variableinterval 80-sec schedule was preferred over the variable-interval 40-sec schedule paired with the variableinterval 20-sec schedule. The subjects were also exposed to several resistance-to-change manipulations: (1) prefeeding prior to the experimental session, (2) a free-food schedule added to timeout periods separating components, and (3) extinction. The results indicated that preference and resistance to change do not necessarily covary.     

Conditioning of the nictitating membrane response of the rabbit (Oryctolagus cuniculus) as a function of length and degree of variation of intertrial interval

The study involved three experiments. The first, a parametric investigation of nictitating membrane conditioning with eight constant intertrial intervals (ITIs) between 5 and 120 sec, orthogonal to interstimulus intervals (ISIs) of 250 and 750 msec plus three temporal conditioning control groups, revealed that performance improved rapidly with increasing ITI but stabilized at relatively low ITI values. At 750-msec ISI, a decrement in performance was found at 60-sec ITI. Experiment II, using constant ITIs of 45–75 sec in 5-sec steps, at 750-msec ISI confirmed the trend toward a performance decrement around 60 sec, although the trend was weak and highly variable. Experiment III evaluated the differences in performance between constant and variable ITI, using three ITI values and three conditions of variation at each value. Findings were discussed in terms of differences in conditioning resulting from both length and degree of variation of ITI and some subtle effects which may emerge only when constant ITIs are used.     

Serial discrimination reversal learning in pigeons as a function of intertrial interval and delay of reinforcement

Pigeons learned a series of reversals of a simultaneous red-green visual discrimination. Delay of reinforcement (0 vs. 2 sec) and intertrial interval (ITI; 4 vs. 40 sec) were varied across blocks of reversals. Learning was faster with 0-sec than with 2-sec delays for both ITI values and faster with 4-sec ITIs than with 40-sec ITIs for both delays. Improvement in learning across successive reversals was evident throughout the experiment, furthermore, even after more than 120 reversals. The potent effects of small differences in reinforcement delay provide evidence for associative accounts and appear to be incompatible with accounts of choice that attempt to encompass the effects of temporal parameters in terms of animals’ timing of temporal intervals.     

Resistance to discrimination and subsequent resistance to extinction as a function of the sequence of partial S+ reward in differential conditioning

The effects of transitions from nonrewarded (N) to rewarded (R) trials (N-R transitions) on discriminative behavior in differential conditioning and subsequent resistance to extinction were investigated in two experiments. In Experiment 1, groups given N-R transitions within S+ were more resistant to discrimination (ran fast in S?) and extinction than were groups given a partial reinforcement (PRF) schedule in S+ devoid of N-R transitions. Experiment 2 indicated that N-R transitions that occur when an N trial in S? is followed by an R trial in S+ are as effective in increasing resistance to discrimination, but not resistance to extinction, as are N-R transitions that occur within S+. The sequential effects obtained here were highly similar to those in conventional PRF and support the view that differential conditioning and PRF are highly interrelated phenomena. The results are discussed in terms of the extension of sequential theory to differential conditioning and the importance of internal reward-produced cues in discrimination learning.     

The effects of intertrial and feature-target intervals on operant serial feature-positive discrimination learning

The effects of intertrial interval (ITI) and feature-target interval (FTI) on the nature of learning in discrete-trial operant serial feature-positive (feature → target+ / target?) discrimination training were examined in two experiments with rats. Discrimination performance was acquired more rapidly with longer ITIs and shorter FTIs. In contrast, transfer to a separately trained target was greater with shorter ITIs regardless of FTI. Persistence of discrimination performance after feature extinction was greater with longer ITIs. Only the last of these performance measures showed evidence for invariance with constant ITI/FTI ratios. The results are discussed in the context of theories of occasion setting.     

Representation strength in pigeon short-term memory: Effect of delay training

An attempt was madeto manipulate the strength of internal stimulus representations by exposing pigeons to brief delays between sample offset and comparison onset in a delayed conditional discrimination. In Experiment 1, pigeons were first trained on delayed conditional discrimination with either short (0.5-sec) delays or no delays. When delays were increased by 2.0 sec, birds trained with a delay performed at a higher level than did birds trained with no delays. In Experiment 2, subjects were first trained on a delayed simple discrimination. Following a circle stimulus, responses to a white key were reinforced; however, following a dot stimulus, responses to the white key were not reinforced. The pigeons were then trained on a delayed conditional discrimination involving hue samples and line-orientation comparisons with differential outcomes. Choice of vertical following red yielded food; choice of horizontal following green yielded no food. Mixed delays were then introduced to birds in Group Delay, whereas birds in the control group received overtraining. When tested on a delayed simple discrimination with hue stimuli (red and green initial stimuli followed by white response stimulus), pigeons in Group Delay tended to perform at a higher level than did birds in the control group (i.e., although the birds in both groups responded more following red than following green, birds in Group Delay did this to a greater extent than did birds in the control group). Thus, experience with delays appears to strengthen stimulus representations established during training.     

Instrumental escape learning in the rat at 24-hour intertriai interval: Effects of magnitude and schedule of reinforcement

Three experiments investigated the effects of magnitude and schedule of reinforcement and level of training in instrumental escape learning at a 24-h intertriai interval. In Experiment I, two magnitudes of reinforcement were factorially combined with two schedules of reinforcement (CRF and PRF). Under PRF, large reward produced greater resistance to extinction than did small reward, while the reverse was true under CRF. In Experiment II, two levels of acquisition training were factorially combined with three schedules of reinforcement (CRF, single-alternation, and nonalternated PRF). Patterned running was observed late in acquisition in the single-alternation extended-training condition. Resistance to extinction was greater for the nonalternated PRF condition than for the single-alternation condition following extended acquisition, and the reverse was true following limited acquisition. Experiment III confirmed the extinction findings of Experiment II. The results of all three experiments supported an analysis of escape learning at spaced trials in terms of Capaldi’s (1967) sequential theory.     

The role of trial tracking in rats’ working memory

The experiments reported in the present study tested whether decreasing intertrial intervals (ITIs) intensifies the disruptive effects of increasing retention intervals (RIs) in a delayed conditional discrimination by decreasing the animal’s trial tracking accuracy (Cohen & Armstrong, 1996; Cohen & Roberts, 1996). Rats responded on a fixed ratio (FR) 1 or fixed interval (FI) 10-sec reinforcement schedule at a second light or tone stimulus, S2, when the first light or tone stimulus, S1, had signaled an FI 10-sec or FR 1 schedule, respectively. RIs between S1 and S2 were increased from 3 to 24 sec and never exceeded ITIs that were reduced from 24 to 6 sec. For some rats, the trials were separated from each other by extending the lever at S1 and retracting it at the end of S2 (ITI lever-retracted group). For other, control rats, the lever remained extended throughout the session (lever-extended group, Experiment 1) or was extended and retracted with the onset and offset of each stimulus (RI/ITI lever-retracted group, Experiment 2). The rats under all trial conditions learned to delay leverpressing on the FI 10-sec schedule. Latency to begin leverpressing on the FI 10-sec schedule declined as RIs were increased, but this effect was attenuated in the ITI lever-retracted groups in both experiments, as would be predicted by thetrial tracking hypothesis. Decreasing ITIs from 24 to 6 sec intensified the disruptive effects of increasing RIs from 3 to 6 sec in the RI/ITI lever-retracted group (Experiment 2), as would be predicted by the trial tracking hypothesis.