Antecedent reinforcement schedule training and operant response reinstatement in rats

Rats were trained on reinforcement schedules which generated high or low response rates. After extinguishing responding by eliminating food-reinforcement delivery, response-independent food presentations reinstated responding. Higher response rates occurred if the schedule preceding extinction controlled high response rates, suggesting that discriminative stimulus properties of the reinforcer were a function of antecedent training schedules.     

Classical conditioning of the rabbit nictitating membrane response: Effects of reinforcement schedule on response maintenance and resistance to extinction

Two experiments were conducted to assess the effects of the introduction of schedules of partial reinforcement (PRF), subsequent to continuous reinforcement training, on the maintenance and resistance to extinction of the rabbit’s nictitating membrane CR. Substantial response levels were maintained by schedules of reinforcement as lean as 15%, and the performance decrements, when observed to be reliable, could not be localized to the immediate effects of one, two, or three consecutive nonreinforced trials by the examination of conditional response probabilities. Moreover, reliable PRF extinction effects were obtained. The relevance of these findings to the purported empirical divergence of PRF effects on classical and instrumental conditioning were discussed.     

Signal properties of reinforcement and reinforcement omission on a multiple fixed-ratio schedule

A series of experiments used food-deprived pigeons to examine several parameters of reinforcement omission in an attempt to control changes of keypeck response measures on a subsequent schedule. In Experiments 1 and 2, the pigeons were tested with a multiple fixed-ratio schedule on which reinforcement was occasionally omitted at the completion of the first component. The duration of the delay occurring in lieu of reinforcement was systematically varied. In Experiment 3, the stimulus that signaled the second component of the schedule was altered to appear either more or less similar to the stimulus that signaled the first component. Two principal results are reported: (1) Response latency decreased and, to a much lesser extent, terminal response rate increased as the delay occurring in lieu of reinforcement decreased; and (2) both latency decrease and response-rate increase were enhanced by a second component stimulus which was similar to the first. The results are evaluated in terms of Amsel’s frustration theory and an analysis by Staddon which suggests that reinforcement inhibits responding. The data appear to support Staddon’s argument that rate increases and latency decreases following reinforcement omission are largely a function of an attenuation of the inhibitory influence of reinforcement, an effect that is enhanced by stimulus generalization. Accordingly, it is proposed that an animal’s response to reinforcement omission is determined by a stimulus complex that minimally includes the omission event and component cues.     

Effects of locus of a brief stimulus in a second-order schedule with pigeons

On the completion by pigeons of four equal fixed intervals on one key, a light on a second key signaled that one peck on that key would be followed by food. In condition A, a brief stimulus of a further color was produced on the first key by the pecks that ended the first three (but not the fourth) fixed intervals. In condition B, no brief stimuli occurred at the end of the first three fixed intervals (tandem schedule). In condition C, the unpaired brief stimulus was presented on the second key after the pecks on the first key that ended the first three fixed intervals. An ABACA reversal design was used. Postreinforcement pauses were longer in condition B (tandem) than in condition A, an effect similar to that reported in similar conventional one-key second-order schedules. Postreinforcement pauses in condition C, with the brief stimulus on the second key, were also longer than in condition A, with the brief stimulus on the first key, although similar pauses were observed after the brief stimuli in both conditions. The locus of the brief stimulus appears to affect the control it exerts over behavior in a second-order schedule.     

Latent discrimination learning under a regular schedule of partial reinforcement

In this investigation, which employed rats in a runway, discriminative responding consisted of faster running on the reinforced than on the nonreinforced trials of either the 4NR or R4N schedule, both schedules containing fixed, repeated sequences of nonreinforced and reinforced trials. Under the 4NR schedule, four nonreinforced trials preceded a reinforced trial each day, and under the R4N schedule, a reinforced trial was followed by four nonreinforced trials each day. The major finding obtained was that under the 4NR schedule, discriminative responding was improved very substantially by a shift to extinction. Rats maintained on the 4NR schedule did not show improved discriminative responding, nor did discriminative responding improve in extinction following training under either the R4N schedule or a schedule of consistent reinforcement. Latent discrimination learning was defined as discriminative responding which fails to reflect adequately the amount of discrimination learning accomplished. The present findings demonstrate latent discrimination learning for regular schedules of partial reinforcement, something already demonstrated for brightness differential conditioning and possibly DRL schedules, as well.     

Tests of the conditioned reinforcement value of sequential stimuli in pigeons

Egger and Miller (1962) hypothesized that the conditioned reinforcing value of stimuli depends on their information value. Egger and Miller and others have tested this hypothesis by comparing the conditioned reinforcing value of S1 and S2 following S1-S2-reward training. However, none of these experiments have controlled for differential generalization of conditioned reinforcement value from training to comparison tests. That is, the S1 cue pattern during the conditioned reinforcement tests has been very similar to the S1 cue pattern of training, while the training and test S2 cue patterns have been quite dissimilar. In Experiment 1, pigeons in a procedure unconfounded by differential generalization produced S2 reliably more frequently than S1, and pigeons in a confounded procedure produced S1 somewhat more frequently than S2. A significant groups × stimuli interaction was attributed to differential stimulus generalization from training to test for S1 and S2 in the confounded condition. In Experiment 2, pigeons in an unconfounded procedure again produced S2 reliably more frequently under a different testing procedure. The results are interpreted as demonstrating that, following S1-S2-food training trials, S2 is the more effective conditioned reinforcer in unconfounded conditions. A reconceptualization of the information hypothesis is shown to be consistent with these results.     

The effects of reinforcement frequency on performance in a multiple fixed-ratio schedule

A barpress analog to the double-alley runway was sought by varying percentage reward in the first of two consecutive FR 18s. Groups of six rats each were given 0% 50%. or 100% reinforcement upon completion of the first FR 18: after a 5-sec midtnal imterval, the second FR 18 was administered on a separate lever and all groups received CRF reward upon its completion. Group 50 Ss performed faster after nonreward than after reward. Group 50 Ss performed faster after nonreward than did 0% Ss. A measure of midtnal behavior revealed a difference between groups in orienting to the bars. When all groups were shifted to a 50% first component schedule (Phase II), there were no statistically reliable effects of prior reinforcement history on rewarded or nonrewarded responding. The Phase 1 results were taken to demonstrate a frustration effect similar to that of the double alley     

Control of pigeons’ keypecking topography by a schedule of alternating food and water reward

Food- and water-deprived pigeons keypecked for food or water reinforcement on alternate trials. Under one condition, explicit stimuli on the key provided information about the trial outcome; under another condition, only the alternation schedule provided this information. Latency and/or response rate differences between food- and water-rewarded trials emerged during both conditions. Response topography also differed on food- and water-rewarded trials. These differences, as revealed by duration and force measurements of the keypeck and by human ratings of the pecking responses as being water- or food-related, were anticipatory in nature. These results not only extend previous work on reward alternation and reward-specific response topographies, but also have implications for theories of animal memory. In particular, these results are amenable to memory models that assume that an animal “codes” information that later must be recalled.     

Discriminative stimuli that follow the absence of reinforcement are preferred by pigeons over those that follow reinforcement

Clement, Feltus, Kaiser, and Zentall (2000) found that when pigeons have to work to obtain a discriminative stimulus that is followed by reinforcement, they prefer a discriminative stimulus that requires greater effort over one that requires less effort. The authors suggested that such a preference results from the greater change in hedonic value that occurs between the more aversive event and the onset of the stimulus that signals reinforcement, a contrast effect. It was hypothesized that any stimulus that follows a relatively more aversive event would be preferred over a stimulus that follows a relatively less aversive event. In the present experiment, the authors tested the counterintuitive prediction of that theory, that pigeons should prefer a discriminative stimulus that follows the absence of reinforcement over a discriminative stimulus that follows reinforcement. Results supported the theory.     

Effects of schedule and delay of reinforcement on acquisition speed

The effects of schedule of reinforcement (partial vs. consistent) and delay of reward (0 to 20 sec) on running in rats were examined in two investigations. The effects of delay depended upon schedule of reinforcement; acquisition speed decreased as delay increased under consistent reinforcement, a common finding, while acquisition speed was independent of delay under partial reinforcement, a new finding. The partial-reinforcement acquisition effect or PRAE is defined as faster acquisition speed under partial than under consistent reinforcement. Because running speed was independent of delay under partial reinforcement, but decreased as delay increased under consistent reinforcement, the PRAE increased as delay of reinforcement increased.     

Serial discrimination reversal learning in pigeons as a function of intertrial interval and delay of reinforcement

Pigeons learned a series of reversals of a simultaneous red-green visual discrimination. Delay of reinforcement (0 vs. 2 sec) and intertrial interval (ITI; 4 vs. 40 sec) were varied across blocks of reversals. Learning was faster with 0-sec than with 2-sec delays for both ITI values and faster with 4-sec ITIs than with 40-sec ITIs for both delays. Improvement in learning across successive reversals was evident throughout the experiment, furthermore, even after more than 120 reversals. The potent effects of small differences in reinforcement delay provide evidence for associative accounts and appear to be incompatible with accounts of choice that attempt to encompass the effects of temporal parameters in terms of animals’ timing of temporal intervals.     

Generalization of fixed-interval response patterning in pigeons

Three pigeons were trained on a multiple schedule in which keypecking was reinforced on a fixed-interval (FI) schedule in the presence of a vertical line and on a variable-interval (VI) schedule in the presence of a horizontal line. Response rates in the former component were positively accelerated (the “FI scallop”), whereas response rates in the second component were relatively constant. In a subsequent generalization test on the line-tilt continuum, the index of curvature decreased systematically as the line was rotated from vertical to horizontal in 30-deg steps. Frequency distributions of indices of curvature suggested that intermediate average index values for intermediate generalization test stimuli were mainly the result of intermediate response patterns in the individual test stimulus presentation, rather than the averaging of VI-like patterns during some stimulus presentations with FI-like patterns during other stimulus presentations.     

Fluid intelligence and discriminative operant learning of reinforcement contingencies in a fixed ratio 3 schedule

Although intelligence has traditionally been identified as the ability to learn (Peterson, 1925), this relationship has been questioned in simple operant learning tasks (Spielberger, 1962). Nevertheless, recent pieces of research have demonstrated a strong and significant correlation between associative learning measures and intelligence ( [Kaufman et al., 2009] and [Williams et al., 2008] ). The present paper aims to examine the relationship between intelligence and learning, complying with the experimental conditions of discriminative conditioning in a relatively complex free operant task. To that end, a sample of 1600 subjects applying for an ab initio Air Traffic Control training course was tested. The task is expected to allow identifying the specific influence of intelligence on the discrimination between stimulus dimensions. The results show how the relation among learning and intelligence strengthen on those trials which are critical in terms of discrimination.     

Effect on subsequent fixed-interval schedule performance of prior exposure to ratio and interval schedules of reinforcement

In three experiments, we examined the effect on the patterns of responding noted on fixed interval (FI) schedules of prior exposure to a range of interval and ratio schedules. Rats leverpressed for food reinforcement on random ratio (RR), random interval (RI), or variable interval (VI) schedules prior to transfer to FI schedules. In Experiment 1, prior exposure to an RR schedule retarded the development of typical FI patterns of responding. Exposure to a yoked RI schedule produced even greater retardation of typical FI performance. This effect was replicated in Experiment 2, using a within-subjects design. Rats responded on a multiple RR-RI schedule prior to a multiple FI-FI schedule. Typical FI performance emerged more slowly in the component previously associated with the RI than with that associated with the RR. In Experiment 3, exposure to an RR schedule retarded the development of FI performance to a greater extent than did exposure to a VR schedule. The latter schedule was programmed to allow the possibility that inhibitory control would develop after reinforcement. These results confirm that ratio schedules independently result in the disruption of FI responding. This effect was not long lasting and cannot be used plausibly to explain species differences in responding to FI schedules. However, it does suggest that temporal control—as manifested by the transfer of inhibitory control from one schedule to another—could facilitate movement between interval schedules.     

The nature of the response in simon discriminations by pigeons

Spence (1952) postulated that under some conditions, responding in simple discriminations is controlled by compounds or patterns consisting of the nominal discriminative cue, plus its spatial position. Stimulus control by such compounds was said to develop when no single cue or element was systematically reinforced more than any other. This analysis has been applied to good effect in understanding some peculiar accuracy results obtained from pigeons performing Simon discrimination tasks. This article describes how Spence's cue-position analysis provides a better account of pigeons' performance in this task than do nominal cue-only and configurational views of the functional discriminative stimuli. Adding a value transfer assumption improves the ability of the cue-position hypothesis to account for the accuracy data.     

Enhancement of pigeons’ conditional discrimination performance by expectancies of reinforcement and nonreinforcement

Prior work has shown that when the separate correct responses of a conditional discrimination are followed by different reinforcing outcomes, performance is enhanced relative to that obtained under the conventional, single-reinforcer procedure. Four experiments with pigeons yielded the analogous finding when the different outcomes were reinforcement and explicit nonreinforcement. Controls indicated that the results could not be attributed to the effects of intermittent reinforcement, to possible differences in cue duration, or to a variety of potential sources of conditioned reinforcement. An interpretation in terms of expectancy learning is proposed.     

Potentiation of responding on a VR schedule by a stimulus correlated with reinforcement: Effects of diffuse and localized signals

In Experiment 1, rats were trained to leverpress on a variable ratio (VR) 30 schedule with a 500-msec delay between the reinforced response and food delivery. Subjects that experienced a signal during the delay responded faster than did control subjects that received the stimulus un-correlated with reinforcement. Higher response rates were obtained when the stimulus used to signal reinforcement was auditory rather than visual. Experiments 2 and 3 compared the effects of signaling reinforcement with either a localized or a diffuse light on responding maintained by VR schedules of reinforcement. Elevated response rates were observed with the diffuse stimulus, but the localized stimulus failed to produce such potentiation. Experiment 3 also examined the conditioned reinforcing power of localized and diffuse visual stimuli. These results are discussed with reference to (1) theories of selective association and sign tracking and (2) their implications for current theories of signaling reinforcement.