Temporal integration and instrumental conditioned reinforcement

Stimuli associated with primary reinforcement for instrumental behavior are widely believed to acquire the capacity to function as conditioned reinforcers via Pavlovian conditioning. Some Pavlovian conditioning studies suggest that animals learn the important temporal relations between stimuli and integrate such temporal information over separate experiences to form a temporal map. The present experiment examined whether Pavlovian conditioning can establish a positive instrumental conditioned reinforcer through such temporal integration. Two groups of rats received either delay or trace appetitive conditioning in which a neutral stimulus predicted response-independent food deliveries (CS1→US). Both groups then experienced one session of backward second-order conditioning of the training CS1 and a novel CS2 (CS1–CS2 pairing). Finally, the ability of CS2 to function as a conditioned reinforcer for a new instrumental response (leverpressing) was assessed. Consistent with the previous demonstrations of temporal integration in fear conditioning, a CS2 previously trained in a trace-conditioning protocol served as a better instrumental conditioned reinforcer after backward second-order conditioning than did a CS2 previously trained in a delay protocol. These results suggest that an instrumental conditioned reinforcer can be established via temporal integration and raise challenges for existing quantitative accounts of instrumental conditioned reinforcement.     

A Pavlovian analysis of food-attraction conditioning in the snailHelix aspersa

Following brief pairing of an odor with a feeding experience (food-attraction conditioning), snails will lower their tentacles when subsequently presented with that odor alone. Three experiments investigated the possible behavioral mechanism mediating food-attraction conditioning in the snailHelix aspersa. It is suggested that food-attraction conditioning is an example of Pavlovian conditioning. In this case, the odor (conditioned stimulus) is paired with oral stimulation (unconditioned stimulus), which elicits lowering of the tentacles (unconditioned response). Following conditioning, the odor comes to elicit lowering of the tentacles (conditioned response). Experiment 1 ruled out nonassociative effects, such as habituation and sensitization, using an unpaired control group. Experiment 2 provided further evidence against a role for habituation of neophobia, through the demonstration of extinction following conditioning. In Experiment 3, an omission procedure was used to rule out the possible role of instrumental contingencies.     

Signaling and affective functions in Pavlovian conditioning

The present study employed a Pavlovian-instrumental-transfer paradigm to investigate the role of conditioned fear in appetitive discrimination learning. Each of three Pavlovian training procedures was used to establish a conditioned fear excitor (CS+), a “neutral” CS (CSo), and a conditioned fear inhibitor (CS?). Then, the CSs were administered to rats in the three groups contingent upon the rewarded response in a difficult visual discrimination. In addition, half of each group received shock punishment for each incorrect response. Relative to CSo, CS+ facilitated performance in contrast to the usual interfering effect of conditioned suppressors; conversely, CS? retarded performance even when its reinforcing action (fear inhibition) was potentiated by punishment for the incorrect response. These results, together with other findings showing a reversed outcome when the CSs are administered for the incorrect response, indicate that Pavlovian conditioning comprises both general signaling and affective functions, the former reflecting a basic “expectancy” or nominal type of cognitive processing in the rat.     

Acquisition and extinction of differential responses to signals paired with shock or shock omission in goldfish: Evidence for truly discriminated avoidance learning

Goldfish trained to discriminate between signals paired with shock (S?) and signals paired with shock omission (S+) with a linear presentation procedure, originally learned (OL) to control the signal state of a shuttle box and showed a decided preference for the S+ signal. In Experiment 1, following OL, groups had one OL signal replaced (S+ or S?), both signals replaced (S+ and S?), or the OL signals reversed (S+ and S? reversed) and were then tested in a transfer training procedure. In transfer, groups with one signal replaced maintained discriminated performance at OL levels; the S+ replaced group was slightly superior to the S? replaced group on the first day of transfer. With both OL signals replaced, discrimination dropped to chance performance levels, whereas, with OL signal shock pairing reversed, discrimination performance dropped below chance levels. In Experiment 2, following OL, extinction procedures consisted of turning off the shocker (0% shock) or of shocking 100% or a random 25% of the trials. A fourth extinction procedure (R,) retained the trial start response-dependent shock-omission contingency, but shock differentiating the S+ and S? signals was eliminated entirely. Extinction of the S+/S? discrimination was measured both during extinction training per se and with reversal retraining of the S+/S? discrimination later. Groups for which the OL S+ was paired with shock during extinction extinguished on both measures, but groups for which the OL S? was paired with shock omission did not extinguish, especially as shown by the reversal test procedure. Theoretical implications and the implications for several conditioning procedures are discussed.     

One-trial simultaneous and backward fear conditioning as reflected in conditioned suppression of licking in rats

In three experiments, groups of albino rats received one strictly simultaneous pairing of a 4-sec auditory conditioned stimulus (CS) and a 4-sec 1-mA shock unconditioned stimulus (US). Other groups received a backward pairing, in which the US began before the CS, or a forward pairing, in which the CS began before the US. Control groups received only the US or received both the CS and the US but widely separated in time. Later, the CS was presented while the rats licked a drinking tube for water, and CS-elicited suppression of licking was taken as an index of the Pavlovian conditioned response (CR). It was found that groups receiving a single forward or a single simultaneous pairing suppressed more than groups that had received a backward pairing; and the backward groups, in turn, suppressed more than the control groups. It appears, then, that excitatory fear conditioning, as reflected in conditioned suppression of licking in rats, can be produced in a single trial by both backward and simultaneous conditioning procedures.     

Extinction and food-reinforced inhibition of conditioned salivation in dogs

The present experiment compared two methods of eliminating a classically conditioned response in dogs, extinction and reinforcement of nonsalivation, using both a within- and between-subjects experimental design. Eighteen dogs were trained for 16 days in Phase I, 16 days in Phase II, and 8 days in Phase III. In Phase I, each subject received classical conditioning training to two stimuli. In Phase II, Group 1 received extinction training to one stimulus and reinforcement of nonsalivation to the other stimulus. Group 2 received continued classical conditioning training to one stimulus and reinforcement on nonsalivation training to the other. Group 3 received continued classical conditioning training to one stimulus and extinction training to the other. In both the within- and between-subjects comparisons, responding to the stimulus associated with extinction was eliminated faster than responding to the stimulus associated with reinforcement of nonsalivation.     

Effect of N-R transitions during partial reinforcement pretraining on subsequent resistance to discrimination

Three groups of 12 rats received 25 pretraining trials to each future discriminandum employed in a subsequent differential brightness conditioning problem. Groups NR and RN received partial reinforcement (PRF) pretraining either with or without, respectively, transitions from nonrewarded to rewarded trials (N-R transitions). Group CRF received consistent reinforcement during pretraining. A fourth group (n=12), Group NP, received no pretraining. During discrimination learning, one-half of the rats in each group received all their daily S+ trials preceding their daily S? trials (+? sequence); the remainder of the rats received an intermixed sequence of trials to S+ and S? (+?+ sequence). Discrimination learning was faster under the +? sequence than under the +?+ condition, and discrimination learning was retarded in Group NR relative to the other three groups, which did not differ from one another, under both the +? and +?+ discrimination sequence conditions. The results are discussed with Reference to previous experiments demonstrating N-R transition effects on discrimination learning, a theoretical extension of sequential theory to discrimination learning, and the effects of nondifferential reinforcement prior to discrimination learning on learned irrelevance.     

Changes in inhibition during differential eyeblink conditioning with increased training

Three experiments examined inhibitory learning in rats, using Pavlovian and differential inhibitory eyeblink conditioning procedures. Experiment 1 was designed to compare summation and retardation effects following Pavlovian conditioned inhibition (A1/XA) or differential inhibition (A1/X) training using auditory and visual conditioned stimuli (CSs). After ten 100-trial sessions of training, both Pavlovian conditioned inhibition and differential inhibition produced a retardation effect. However, a summation effect was obtained only for rats given Pavlovian conditioned inhibition training. Experiment 2 showed that increasing differential inhibition training to twenty 100-trial sessions produced summation and retardation effects. In Experiment 3, rats were trained with either ten or twenty 100-trial sessions of intramodal inhibitory training with two tone CSs (2 kHz vs. 8 kHz). Summation and retardation effects were obtained after only 20 sessions of differential conditioning. The findings indicate that extensive training is needed to establish conditioned inhibition with intermodal or intramodal differential conditioning.     

The influence of the information value provided by prior-cuing treatment on the reactivation of memory in preweanling rats

In two experiments, the influence of exposure to a CS? on the acquisition and retention of a conditioned odor aversion was examined. Preweanling rats were given exposure to the CS? either prior to (CS?/CS+) or following (CS + /CS?) the pairing of a second odor (the CS+) with footshock. The results of Experiment 1 indicated that subjects in both of the treatment conditions acquired aversions of comparable strength to the odor paired with footshock and that retention of the odor aversion was not affected by order of stimulus presentation during conditioning. Experiment 2 indicated, however, that the effectiveness of pretest exposure to various elements of the conditioning episode in reactivation of the memory for conditioningwas dependent on the order of stimulus presentation during conditioning. This differential effectiveness of the various reactivation treatments is discussed in terms of their relationship to the associative “status” of the stimuli present during conditioning and in terms of the information provided to the animal by the reactivation treatment.     

Processing of complex auditory stimuli (tunes) by rats and monkeys (Cebus apella)

The degree to which rats and monkeys base their discriminations of complex auditory stimuli (“tunes”) on frequency contours rather than on local features was investigated. In Experiment 1, groups of rats and monkeys trained with tunes as S+ and S? acquired a simple operant discrimination no faster than groups that received the same notes of each tune but in a new random order on each trial; neither did the groups differ on two transfer tests devised to detect learning of frequency contour in the tune-trained animals. Acquisition in the tune-trained and random-notes groups seemed to be based on the overall frequency difference between S+ and S?, which was about 1.5 octaves. In Experiment 2, S+ and S? were similar to each other with regard to overall frequency and individual notes, the most salient differentiating characteristic of the tunes being their tonal pattern. The tune-trained groups were clearly superior to the random-notes animals in acquisition, and an initial transfer test suggested that the former might have learned the discrimination on the basis of frequency contour. However, the detailed transfer tests of Experiment 3 strongly suggested that the tune-trained rats and monkeys based their discriminations primarily on local cues rather than on frequency contour. Based on the results of Experiment 4, the data of an earlier study that suggested frequency contour learning in monkeys and rats were reinterpreted in terms of control by local cues.     

Two-choice,observational learning and reversal in the rat: S-S versus S-R effects

In Experiment 1, male rats were trained to press both bars in a two-choice apparatus and were then given observational training of a go/no-go discrimination in which the observed operation of two inaccessible, dissimilar bars by a hidden experimenter constituted S+ and S?. After discrimination was established, individual rats were permitted access to the two bars. Six of the seven rats consistently pressed the S+ bar on 10 test trials, but failed to reverse bar preference after observational training was reversed. In Experiment 2, nine naive males received the same observational training as in Experiment 1, but without any pretraining to press either bar. All rats pressed the S+ bar on initial test and did so consistently throughout the 10 trials. Six of these rats received reversal training of the go/no-go discrimination after the 10 test trials. As in Experiment 1, all rats failed to press the new S+ bar. However, five of six rats in another group, which received reversal trainingprior to any test trials, did reverse and press the new S+ bar. In Experiment 3, controls for possible confounding effects of overtraining trials were conducted. These manipulations had no effect; the rats tested before reversal still failed to press the S+ bar, and the rats reversed before testing all reversed or pressed the most recent S+ bar. That is, S-R learning predominated over S-S learning if active, though unreinforced, responding to a particular bar intervened. In contrast, however, a cognitive (S-S) interpretation of directed response learning was supported by the results of Experiment 4, in which the rats that learned the go/no-go discrimination without responding (only by auditory and light cues) failed to press the S+ bar consistently.     

Positive and negative transfer of conditioned aversive stimuli to a conditioned appetitive excitor as a function of aversive US intensity

Rats were given Pavlovian aversive (Av) conditioning in which a weak (0.5-sec, 0.7-mA) or a strong (0.5-sec, 1.4-mA) footshock unconditioned stimulus (US) was presented alone or in a positive, zero, or negative correlation with a flashing-light conditioned stimulus (AvCS+, AvCSo, or AvCS?, respectively). Thereafter, the subjects received Pavlovian appetitive (Ap) conditioning in which the flashing-light CS was positively correlated in a forward order with the delivery of a food US. As anticipated, for subjects that had been trained with the strong shock US, the AvCS+ retarded and the AvCS? facilitated Ap conditioning relative to both the AvCSo and the novel-CS treatments, which did not differ in effect. However, the exact opposite prevailed for the AvCS+ and AvCS? subjects that had been trained with the weak shock US. On the basis of these and other data, we propose an “ABC” model of transfer that stresses the importance of affective, behavioral, and cognitive factors in accounting for the divergent results that are obtained in both Av-to-Ap and Ap-to-Av transfer paradigms.     

Bidirectional heart rate responses in rats associated with excitatory and inhibitory stimuli

Following classical conditioning to a shock-reinforced tone CS (T₁+), heart rate (HR) of three groups of rats was examined in response to a new reinforced tone (T₂+) and a nonreinforced light (L?) CS given during conditioned inhibition (T₂+ vs. T₁L?), discrimination conditioning (T₂+ vs. L?), or explicitly unpaired (T₂/L? vs. US alone) procedures. Decelerative HR reactions occurred to the reinforced T₂+ and T₂+ CSs. To the respective nonreinforced CSs, the conditioned inhibition group displayed diminished but sizable HR deceleration, the discrimination group showed near-zero responding, and the explicitly upaired group showed HR acceleration. Subsequent reversal conditioning to L was retarded in the conditioned inhibition and explicitly unpaired groups relative to the discrimination group. Group differences on combined-cue [T₂/L?) trials were not found. Both the HR responses during inhibitory training and the reversal-conditioning impairments suggest that inhibition may have been established to L? in the conditioned inhibition and explicitly unpaired groups.     

Fear-motivated vicious-circle behavior maintained through secondary punishment

Two experiments attempted to establish vicious-circle behavior through fear motivation combined with secondary punishment. In Experiment 1, rats were trained with two CSs, a tone and a buzzer, paired with shock in different contexts. Secondary punishment based on delay and trace conditioning procedures facilitated running in fear-motivated rats, relative to four control groups. In Experiment 2, rats were given pairings of a tone CS with shock, and a buzzer CS with a drop into a water tank. Fear-motivated rats which received secondary punishment during either 33% or 100% of test trials exhibited self-punitive running relative to a nonpunished (0%) group and a backward-conditioning control group. Results indicate that “all secondary” vicious-circle behavior can be established through Pavlovian conditioning, thus supporting a conditioned fear interpretation.     

Stimulus competition in the absence of compound conditioning

Most associative theories have assumed that stimulus competition occurs only between conditioned stimuli (CSs) that are trained in compound. The present research investigated the possibility of competition between two CSs that were individually paired to the same unconditioned stimulus (US). We used human subjects in an anticipatory suppression analogue to Pavlovian conditioning. Experiment 1 showed that X+ training followed by A+ training resulted in impaired responding to X. This did not occur when A+ training preceded X+ training. Experiment 2 replicated the basic effect and showed that it did not occur when the Phase 2 training consisted of A? instead of A+ nor when the A+ pairings occurred in a second context. Experiment 3 showed that A+ pairings occurring in a second context could still produce the effect when X was tested in the context in which the A+ pairings had occurred, but not when X was tested in a context different from that used for A+ training. Collectively, these results suggest that individually trained CSs may compete with each other when one of those CSs is more strongly activated by the test context than the other one.     

On the nature of CS and US representations in Pavlovian learning

A significant problem in the study of Pavlovian conditioning is characterizing the nature of the representations of events that enter into learning. This issue has been explored extensively with regard to the question of what features of the unconditioned stimulus enter into learning, but considerably less work has been directed to the question of characterizing the nature of the conditioned stimulus. This article introduces a multilayered connectionist network approach to understanding how “perceptual” or “conceptual” representations of the conditioned stimulus might emerge from conditioning and participate in various learning phenomena. The model is applied to acquired equivalence/distinctiveness of cue effects, as well as a variety of conditional discrimination learning tasks (patterning, biconditional, ambiguous occasion setting, feature discriminations). In addition, studies that have examined what aspects of the unconditioned stimulus enter into learning are also reviewed. Ultimately, it is concluded that adopting a multilayered connectionist network perspective of Pavlovian learning provides us with a richer way in which to view basic learning processes, but a number of key theoretical problems remain to be solved, particularly as they relate to the integration of what we know about the nature of the representations of conditioned and unconditioned stimuli.     

Contextual control of pavlovian feature-positive and feature-negative discriminations

Three pigeons were trained with a Pavlovian serial feature-positive (F-P) discrimination task in a light context, in which the houselight was on, and with a Pavlovian serial feature-negative (F-N) discrimination task in a dark context, in which the houselight was off. Three other pigeons were trained with the F-P task in the dark context and the F-N task in the light context. These two contextual conditions were changed randomly trial by trial. The former birds learned the tasks within 60 sessions, by responding exclusively to the target keylight after the feature tone in the light context and by responding exclusively to the target not preceded by the feature in the dark context. Two of the latter birds required separate training of the F-P and the F-N tasks to acquire the discrimination: responding exclusively to the target after the feature in the dark context and responding exclusively to the target not preceded by the feature in the light context. The third bird, however, failed to learn the discrimination even with separate training. These results indicate that the four-term contingency (the context-feature-target-food relationship) controlled the birds’ behavior in the Pavlovian setting. The insertion of a temporal gap between the feature and the target impaired the F-N discrimination, although it had little effect on the F-P discrimination.     

Outcome-specific conditioned inhibition in Pavlovian backward conditioning

In the present experiments, the outcome specificity of learning was explored in an appetitive Pavlovian backward conditioning procedure with rats. The rats initially were administered Pavlovian backward training with two qualitatively different unconditioned stimulus conditioned-stimulus (US-CS) pairs of stimuli (e.g., pellet --> noise or sucrose --> light), and then the effects of this training were assessed in Pavlovian-to-instrumental transfer (Experiment 1) and retardation-of-learning (Experiment 2) tests. In the transfer test, it was shown that during the last 10-sec interval, the CSs selectively reduced the rate of the instrumental responses with which they shared a US, relative to the instrumental responses with which they did not share a US. The opposite result was obtained when the USs (in the absence of the CSs) were presented noncontingently. In the retardation test, conditioned magazine approach, responding to the CSs was acquired more slowly when the stimulus-outcome combinations in the backward and the forward conditioning phases were the same, as compared with when they were reversed. These results are collectively in accord with the view that Pavlovian backward conditioning can result in the formation of outcome-specific inhibitory associations. Alternative views of backward conditioning are also examined.     

Classical conditioning in paramecia

SingleParamecium caudatum were conditioned by pairing ac-generated electric shock (US) with a vibratory stimulus (CS) produced by an auditory speaker. Naive paramecia subjected to shock reliably exhibited a backwards jerk and axial spinning similar to the avoiding reaction described by Jennings in 1904. Such responses did not occur initially to CS alone, but increasingly appeared during the CS period preceding shock pairing (delayed conditioning paradigm). Control subjects given the CS and UCS at the same intervals, but explicitly unpaired, did not show a sustained increase of responses to the CS alone. Short-term memory was demonstrated by subjects first conditioned and then presented CS alone during extinction. These subjects were readily reconditioned. Paramecia trained and stored for 24 h showed reliable memory savings as compared to stored control subjects. Other paramecia were differentially conditioned by training with two CSs. Following the recommendations of Rescorla (1967), a procedure was designed for truly random presentation of the CS and UCS as an additional control for pseudoconditioning. Single paramecia were conditioned with intervals between CSs randomly ranging from 8 to 32 sec. Control subjects received the same number of CSs and UCSs, which were administered independently and randomly during the same total session duration. Thus, CS and UCS were occasionally paired for control subjects. The responses to CS in the conditioned group were anticipatory conditional responses due to the pairing contingency and not wholly due to pseudoconditioning.     

Numerosity differences and effects of stimulus density on pigeons’ discrimination performance

Two experiments are described in which pigeons were trained in a simultaneous conditioning procedure to discriminate small arrays of dots that differed in numerosity. The birds successfully learned to choose the array of each pair that contained fewer dots when these choices were reinforced and choices of the array with more dots led to timeout. For the majority of numerosity values tested, discrimination performance for a fixed S+ value was better when the numerical difference between S+ and S-values was larger rather than smaller. This effect was seen in the first experiment when the numerical difference value was shifted between training trials and novel test trials. In the second experiment, too, performance level depended on the size of the numerosity difference when the birds were concurrently trained with two difference values that varied across trials within sessions. However, discrimination accuracy was influenced secondarily by variations in the density, or interdot spacing, of the stimulus arrays. In order to explain the latter finding, it is suggested that a tendency to “scan” a lowdensity array incompletely might alter the probability of accepting it as the smaller numerosity (S+) stimulus. This would increase error rates with S? arrays in which the dots are more widely spaced.