Reward sequence and reinforcement level as determinants of S− behavior in differential conditioning

In two differential conditioning experiments, groups of 10 rats each differed with respect to average reward and schedule of reward received in S+. Nonreward (N) occurred on all S? trials. In both experiments, extinction of responding to S? (resistance to discrimination) was extensively regulated by reward sequence and was largely independent of average reward. In Experiment 1, resistance to discrimination was a function of transitions from N to rewarded (R) trials (N-R transitions). In Experiment 2, resistance to discrimination was increased by large reward on the R trial of N-R transitions and decreased by large reward on the R trial of R-N transitions. These schedule effects on resistance to discrimination parallel the effects of comparable schedules on resistance to extinction following partial reinforcement. The results are discussed in terms of sequential theory, reinforcement level theory, and their implications for various schedule manipulations that have previously shown S? behavior to be inversely related to average reward in S+.     

Resistance to discrimination and subsequent resistance to extinction as a function of the sequence of partial S+ reward in differential conditioning

The effects of transitions from nonrewarded (N) to rewarded (R) trials (N-R transitions) on discriminative behavior in differential conditioning and subsequent resistance to extinction were investigated in two experiments. In Experiment 1, groups given N-R transitions within S+ were more resistant to discrimination (ran fast in S?) and extinction than were groups given a partial reinforcement (PRF) schedule in S+ devoid of N-R transitions. Experiment 2 indicated that N-R transitions that occur when an N trial in S? is followed by an R trial in S+ are as effective in increasing resistance to discrimination, but not resistance to extinction, as are N-R transitions that occur within S+. The sequential effects obtained here were highly similar to those in conventional PRF and support the view that differential conditioning and PRF are highly interrelated phenomena. The results are discussed in terms of the extension of sequential theory to differential conditioning and the importance of internal reward-produced cues in discrimination learning.     

Antecedent reinforcement schedule training and operant response reinstatement in rats

Rats were trained on reinforcement schedules which generated high or low response rates. After extinguishing responding by eliminating food-reinforcement delivery, response-independent food presentations reinstated responding. Higher response rates occurred if the schedule preceding extinction controlled high response rates, suggesting that discriminative stimulus properties of the reinforcer were a function of antecedent training schedules.     

Reacquisition following extinction in appetitive conditioning

In four experiments utilizing an appetitive conditioning preparation, reacquisition of conditioned responding was found to occur both rapidly and slowly following extinction. In Experiment 1, acquisition of responding to a tone that had been conditioned and extinguished occurred more rapidly than acquisition in either a group that received equivalent exposure to the food unconditioned stimulus or a “rest” control group that received only exposure to the apparatus in the first two phases. However, reacquisition was impaired relative to acquisition in a “learning-experienced” group that had previously received conditioning and extinction with a different stimulus. Experiments 2 and 3 produced similar results, but also found that high responding during reacquisition was confined to trials that followed reinforced, rather than nonreinforced, trials. Experiment 4, in which very few initial conditioning trials were used, produced reacquisition that was slow compared with both learning-experienced and rest controls. The results are consistent with a role for sequential learning: Reacquisition is rapid when animals have learned that reinforced trials signal other reinforced trials.     

Durability of the partial reinforcement and partial delay of reinforcement extinction effects after minimal acquisition training

Four groups of 10 rats each were given six acquisition trials (Phase 1) under continuous reinforcement (CR), partial reinforcement (PR), constant delay (CD), or partial delay of reinforcement (PD) conditions. In Phase 2, all Ss were given 18 nonreinforced trials, followed by 12 continuously reinforced trials in Phase 3. In Phase 4, all Ss were given 12 more extinction trials. A constant 24-h ITI was observed throughout the experiment. A strong partial reinforcement extinction effect (PREE) was obtained in both Phases 2 and 4. Only a temporary partial delay of reinforcement effect (PDRE) was observed, which was restricted to the first nine trials of the first extinction phase. No constant delay of reinforcement effect (CDRE) was observed in either extinction phase. The results were discussed in terms of both frustration and sequential theories.     

Stimulus control of topographically tagged responding

Pigeons’ responses on an operant key were reinforced according to either multiple variable-interval variable-interval or multiple variable-interval extinction schedules. The multiple-schedule components were signaled by line-tilt stimuli on a second key (signal key). Signal-key responses never produced reinforcement, and operant-key responses were not reinforced if they followed within 1 sec of a signal-key response. Behavioral contrast was not observed on the operant key, although there was a small, but reliable, increase in signal-key responding in the variable-interval component of the multiple variable-interval extinction condition. Generalization tests were interspersed between sessions of multiple variable-interval extinction training. Generalization gradients along the line-tilt dimension exhibited peak shift for both operant-key and signal-key responding following intradimensional (line tilt) discrimination training. Line-tilt generalization gradients following interdimensional discrimination did not exhibit peak shift. Gradients following intradimensional discrimination were sharper than gradients following interdimensional discrimination for both operant-key and signal-key responding. It was concluded that dimensional stimulus control of topographically tagged responding maintained by the stimulusreinforcer relation parallels that maintained by the response-reinforcer relation.     

Narrowing down the conditions for extinction of Pavlovian feature-positive discriminations in humans

The aim of this study was to delineate the minimal conditions for extinction of Pavlovian modulation in humans. Previous experiments at our lab showed that, after X ? A+/A?C acquisition training, X?C trials did not extinguish differential X ? A+/A?C responding, while X ? A?C trials did. Additionally, X ? A?C extinction training seemed only to extinguish differential X ? A+/A?C responding, while leaving differential responding on a concurrently trained Y ? B+/B?C discrimination intact. It thus seemed that the X ? A+/A?C discrimination can only be extinguished by X ? A?C extinction trials. (Rescorla, Journal of Experimental Psychology: Animal Behavior Processes 12, 16?C24, 1986), on the other hand, found that the minimal conditions for extinction were broader in pigeons: Namely, he found that an acquired X ? A+/A?C discrimination could be extinguished by presenting the original feature X in combination with a different target (B) that was minimally trained as an exciter. We thus wanted to examine whether this was also the case in humans. We found that nonreinforced X ? B?C presentations did not abolish discriminative X ? A/A responding when target B was a nonreinforced stimulus. Nonreinforced X ? B?C trials did extinguish the X ? A+/A?C discrimination when target B had previously been trained as a target for modulation (X ? B+/B?C or Y ? B+/B?C training) or as a reinforced exciter (B+). Our results thusf parallel and extend those in nonhuman animals (Rescorla, Journal of Experimental Psychology: Animal Behavior Processes 12, 16?C24, 1986).     

Summation and configuration in patterning schedules with the rat and rabbit

Discrimination between a tone + light compound and its components in positive and negative patterning schedules was examined. In the positive schedule, reinforced compound presentations (C+) were intermixed with unreinforced component presentations (T?, L?). In the negative schedule, the compound was unreinforced (C?) and the components were reinforced (T+, L+). In Experiment 1, appetitive conditioning of rats’ anticipatory magazine responses was used, and in Experiment 2, aversive conditioning of the rabbit’s nictitating membrane response was used. Both experiments revealed that the positive patterning schedule consistently produced rapid acquisition of appropriate discriminative responding. The results of the negative patterning schedule were more complex. Specifically, the results of Experiment 1 demonstrated that naive rats initially showed rapid acquisition of the negative patterning discrimination. However, schedule reversals revealed that experience with the positive patterning schedule virtually abolished subsequent acquisition of discriminative responding under the negative patterning schedule. The results of Experiment 2 revealed that naive rabbits showed very slow acquisition of discriminative responding under the negative patterning schedule. The results are discussed in relation to the unique-stimulus hypothesis, a contextual encoding hypothesis, and a configural hypothesis.     

The ontogeny of the feature-positive effect in young chicks

The wide generality of the feature-positive effect (FPE) has caused speculation that the FPE may represent innate biases in the stimulus control of discriminative responding. There is little experimental evidence to date ragarding this possibility. In the present study, 1- or 4-day-old chicks were trained on a feature-positive (FP) or feature-negative (FN) discrimination with heat reinforcement. After the acquisition phase, these subjects received extinction training followed by a reacquisition phase. The FP performance was superior to the FN performance in both age groups. Extinction resulted in improved discrimination performance in both the FP and FN conditions. Unmasking of FN learning by the extinction treatment suggests that the FPE represents a deficit in performance, rather than an inability to learn the FN task. These data demonstrate that adult-like performance on feature discriminations is evident as early as the first day post-hatch.     

Partial reinforcement delayed extinction effect after regularly alternating reward training

When extinction is delayed very long, the superior resistance to extinction of the random schedule group relative to the alternating schedule group disappears (partial reinforcement delayed extinction effect, PRDE). Two experiments assessed the effects of reinforcement/nonreinforcement on Trial 1 on the PRDE. Following extended partial reinforcement acquisition training in a runway, rats received extinction training after a short (1-day) or long (23-day) retention interval. The schedules used in Experiment 1 were: a single-alternation (SA) schedule beginning each day with a rewarded (r) trial, for Group r-SA; an SA schedule beginning with a nonrewarded (n) trial, for Group n-SA; and a random (Rd) schedule, for Group Rd. The schedules and group names used in Experiment 2 were r-SA, Rd, and r-Rd. The results were that (1) rats given r-SA schedules yielded considerable resistance under delayed extinction, (2) those given Rd and r-Rd schedules showed a decline in resistance to extinction over a long retention interval, (3) those given the n-SA schedule showed relatively low resistance at both retention intervals, although retention deficit was not greater than in the case of the Rd schedule, and thus, (4) the PRDE was found in both experiments, although only weakly in Experiment 1. The results indicated that a regularly alternating reward pattern was a more important determinant than was type of reward on Trial 1 for the PRDE. The PRDE due to differential retention deficits among schedules is discussed on the basis of dual-process associative sequential mechanisms and cognitive rule-encoding mechanisms.     

Overshadowing by food of stimulus control by a keylight in a two-element serial compound

Pigeons were given successive discrimination training in which pecking during a choice period when the key was white was either reinforced or not, depending upon the prior presence or absence of a discriminative stimulus, which was a two-element serial compound. The compound consisted of a keylight and food, with food presented second or first in a forward or backward pairing for different groups of pigeons. In Experiment 1, the sequence was an S+ indicating reinforced trials, while in Experiment 2, the sequence was an S? indicating nonreinforced trials. Following acquisition of discriminated operant behavior, a sequence generalization test was administered during which all possible orders of the two stimuli were presented on test trials prior to the onset of the choice period. The results showed that food overshadowed stimulus control by the color of the light on the key on the sequence-generalization test, independently of whether food was presented first or second during training and independently of whether food was associated with reinforcement or nonreinforcement. The similarity of results for the two experiments suggests that overshadowing occurs independently of whether the compound is a discriminative stimulus for reinforcement or nonreinforcement. Simultaneous presentation of elements of a compound stimulus is not necessary for overshadowing because the phenomenon was captured with sequentially presented stimuli.     

Primary frustration differences following brief partial-reinforcement acquisition under varying magnitudes of reward

In Experiment I, rats which had received six partially reinforced runway acquisition trials, with a reward magnitude of 60 sec access to wet mash on rewarded trials, showed less persistent responding over highly massed extinction trials than subjects which had received the same acquisition schedule but reward magnitudes of either 1 or 10 45-mg pellets. In Experiment II, rats which had received six partially reinforced placements into one compartment of a two-compartment box, with 60 sec access to mash on rewarded placements, jumped a hurdle faster to escape nonreward than subjects which had received the same reward schedule but 10 45-mg pellets on rewarded trials. The data supported a primary frustration analysis for reward-magnitude manipulations within brief partial-reinforcement schedules.     

“counting” by pigeons: Discrimination of the number of biologically relevant sequential events

Numerical competence has been studied in animals under a variety of conditions, but only a few experiments have reported animals’ ability to detect absolute number. Capaldi and Miller (1988) tested rats’ ability to detect absolute number by using biologically important events—the number of reinforced runs followed by a nonreinforced run—and found that the rats ran significantly slower on the nonreinforced run. In the present experiments, we used a similar procedure. Pigeons were given a sequence of trials in which responding on the first three trials ended in reinforcement but responding on the fourth trial did not (RRRN). When the response requirement on each trial was a single peck (Experiment 1), we found no significant increase in latency to peck on the fourth trial. When the response requirement was increased to 10 pecks (Experiment 2), however, the time to complete the peck requirement was significantly longer on the nonreinforced trial than on the reinforced trials. Tests for control by time, number of responses, and amount of food consumed indicated that the pigeons were using primarily the number of reinforcements obtained in each sequence as a cue for nonreinforcement. This procedure represents a sensitive and efficient method for studying numerical competence in animals.     

Quantitative analysis of local-level resurgence

Resurgence is the recurrence of a previously reinforced and then extinguished behavior induced by the extinction of another more recently reinforced behavior. Resurgence provides insight into behavioral processes relevant to treatment relapse of a range of problem behaviors. Resurgence is typically studied across three phases: (1) reinforcement of a target response, (2) extinction of the target and concurrent reinforcement of an alternative response, and (3) extinction of the alternative response, resulting in the recurrence of target responding. Because each phase typically occurs successively and spans multiple sessions, extended time frames separate the training and resurgence of target responding. This study assessed resurgence more dynamically and throughout ongoing training in 6 pigeons. Baseline entailed 50-s trials of a free-operant psychophysical procedure, resembling Phases 1 and 2 of typical resurgence procedures. During the first 25 s, we reinforced target (left-key) responding but not alternative (right-key) responding; contingencies reversed during the second 25 s. Target and alternative responding followed the baseline reinforcement contingencies, with alternative responding replacing target responding across the 50 s. We observed resurgence of target responding during signaled and unsignaled probes that extended trial durations an additional 100 s in extinction. Furthermore, resurgence was greater and/or sooner when probes were signaled, suggesting an important role of discriminating transitions to extinction in resurgence. The data were well described by an extension of a stimulus-control model of discrimination that assumes resurgence is the result of generalization of obtained reinforcers across space and time. Therefore, the present findings introduce novel methods and quantitative analyses for assessing behavioral processes underlying resurgence.     

Behavioral contrast and type of reward: Role of elicited response topography

Groups of pigeons were exposed to multiple variable-interval variable-interval and multiple variable-interval extinction schedules of either food or water reinforcement for keypecking. Discriminative stimuli associated with component schedules were located either on the operant key or on a second “signal” key. When the stimuli were projected on the operant key, positive contrast appeared during discrimination conditions with either food or water as the reinforcer. When the stimuli were projected on the signal key, overall responding to the operant and signal keys showed contrast with food, but negative induction with water as the reinforcer. In the latter condition, the signal for the variable-interval shcedule of water reinforcement elicited a variety of water-related behavior, only some of which was directed at the signal. Thus, the type of reward and location of discriminative stimuli interacted to determine the presence or absence of behavioral contrast effects. In large part, these results support and extend the autoshaping view of contrast.     

A role of stimulus compounds in eliciting responses: Relatively spaced extinction following massed acquisition

One of two schedules of rewarded (R) and nonrewarded (N) trials (RNR vs. RRN) was combined factorially with intertrial interval (ITI) in acquisition (1 vs. 45 min), with extinction occurring at a 45-min ITI. The RNR schedule produced greater resistance to extinction than the RRN schedule regardless of acquisition ITI. The shift in ITI from acquisition to extinction reduced resistance to extinction slightly in the RNR group but not in the RRN group. These findings suggest that there are cues common to 1-min and 45-min ITIs. It was suggested that these ITI-associated cues enter into compound with memories to control instrumental responding.     

Conditioned suppression and discriminative control of behavior

Rats were trained on a multiple variable-interval extinction schedule of reinforcement. The effects of a stimulus which preceded an unavoidable shock were assessed when it was superimposed on both components of the schedule or on VI components only. In general, VI responding was suppressed during the preshock stimulus. There was no evidence for any increase in responding during extinction components either generally or differentially during the preshock stimulus. These findings fail to support an earlier suggestion that a preshock stimulus may impair discrimination performances.     

Replacement of event-generated memories of nonreinforcement with signal-generated memories of reinforcement during partial reinforcement training: Effect on resistance to extinction

Event-generated memory refers to the memory of a reinforcement (R) or nonreinforcement (N) event from an immediately preceding trial;signal-generated memory refers to the memory of a temporally remote R or N, retrieval of which is generated by presentation of a signal with which the memory is associated (Haggbloom, 1988). In each of three experiments, Group Signal-R received runway discrimination training in Phase 1 to establish a stimulus as a signal for R, and partial reinforcement training in Phase 2. An extinction test measured learning about the memory of nonreward (S^N)—learning that occurs when S^N is retrieved on R trials that follow N trials. In Group Signal-H, those R trials were accompanied by the signal for R, a treatment we hypothesized would generate retrieval of the memory of reinforcement (S^R) so that signal-generated S^R would replace event-generated S^N as the operative memory, thereby eliminating the increased resistance to extinction normally produced by PRF training. In each experiment, Group Signal-R was less resistant to extinction than was a control group conditioned to respond to-event-generated S^N. Extinction was as rapid in Group Signal-R as it was in a consistent reinforcement control group (Experiment 1) and in a group given intertrial reinforcements to interfere with learning about S^N (Experiment 3). Experiment 2 tested two alternative interpretations of the failure to learn about S^N in Group Signal-R. Those alternatives were found to be less viable than the hypothesis that the signal for R actively recruited retrieval of a competing memory.     

Conjunctive schedules of reinforcement IV: Effects on the pattern of responding of changes in requirement at reinforcement

The effects of different schedule requirements at reinforcement on patterns of responding by pigeons were assessed under conjunctive schedules with comparable response-number requirements, Under one conjunctive schedule (conjunctive fixed-interval fixed-ratio schedule), a response was reinforced after a 6-min interval had elapsedand a specific minimum number of responses had been emitted, Under a second conjunctive schedule, a response was reinforced after the 6-min fixed interval and upon completion of a tandem schedule requirement (conjunctive fixed-interval tandem schedule), This schedule retained the same required minimum number of responses as the first conjunctive schedule, but responses were never reinforced according to a fixed-ratio schedule; the tandem schedule was comprised of a fixed-ratio and a small (.1 to 10.0 sec) fixed-interval schedule, Under the conjunctive fixed-interval fixed-ratio schedule, responding was characterized by an initial pause, an abrupt transition to a high response rate, and a second transition to a lower rate that prevailed or slightly increased up to reinforcement, Under the conjunctive fixed-interval tandem schedule, pauses were extended, response rates were lower, and the initial high rate of responding was generally absent, The above effects depended upon the size of the fixed interval of the tandem schedule, The distinct pattern of responding generated by conjunctive fixed-interval fixed-ratio schedules depends upon occasional reinforcement of fixed-ratio responding and not merely on the addition of a minimum number of required responses.     

Rates of responding in the pigeon generated by simple and complex schedules which provide the same rates of reinforcement

Four pigeons pecked for food reinforcement on variable interval 1-min schedules and on the variable-interval 1-min components of multiple, concurrent, and pseudoconcurrent schedules. The pseudoconcurrent schedule provided only one schedule of reinforcement; but, any reinforcer could be collected by responding on either of two keys. The rate of responding generated by the variable interval schedule was not greater than the rates of responding generated by the components of the complex schedules. But, the rate of reinforcement obtained from the variable interval schedule was greater than the rates of reinforcement obtained from the components of the multiple schedule. These results may contradict the equation proposed by Herrnstein (1970). The equation predicts that the rate of responding generated by a schedule of reinforcement will be greater when the schedule appears alone, than when it appears as one component of a complex schedule.