Positive and negative transfer of conditioned aversive stimuli to a conditioned appetitive excitor as a function of aversive US intensity

Rats were given Pavlovian aversive (Av) conditioning in which a weak (0.5-sec, 0.7-mA) or a strong (0.5-sec, 1.4-mA) footshock unconditioned stimulus (US) was presented alone or in a positive, zero, or negative correlation with a flashing-light conditioned stimulus (AvCS+, AvCSo, or AvCS?, respectively). Thereafter, the subjects received Pavlovian appetitive (Ap) conditioning in which the flashing-light CS was positively correlated in a forward order with the delivery of a food US. As anticipated, for subjects that had been trained with the strong shock US, the AvCS+ retarded and the AvCS? facilitated Ap conditioning relative to both the AvCSo and the novel-CS treatments, which did not differ in effect. However, the exact opposite prevailed for the AvCS+ and AvCS? subjects that had been trained with the weak shock US. On the basis of these and other data, we propose an “ABC” model of transfer that stresses the importance of affective, behavioral, and cognitive factors in accounting for the divergent results that are obtained in both Av-to-Ap and Ap-to-Av transfer paradigms.     

Outcome-specific conditioned inhibition in Pavlovian backward conditioning

In the present experiments, the outcome specificity of learning was explored in an appetitive Pavlovian backward conditioning procedure with rats. The rats initially were administered Pavlovian backward training with two qualitatively different unconditioned stimulus conditioned-stimulus (US-CS) pairs of stimuli (e.g., pellet --> noise or sucrose --> light), and then the effects of this training were assessed in Pavlovian-to-instrumental transfer (Experiment 1) and retardation-of-learning (Experiment 2) tests. In the transfer test, it was shown that during the last 10-sec interval, the CSs selectively reduced the rate of the instrumental responses with which they shared a US, relative to the instrumental responses with which they did not share a US. The opposite result was obtained when the USs (in the absence of the CSs) were presented noncontingently. In the retardation test, conditioned magazine approach, responding to the CSs was acquired more slowly when the stimulus-outcome combinations in the backward and the forward conditioning phases were the same, as compared with when they were reversed. These results are collectively in accord with the view that Pavlovian backward conditioning can result in the formation of outcome-specific inhibitory associations. Alternative views of backward conditioning are also examined.     

Control of conditional discrimination performance by CS-evoked event representations

The ability of visual CSs previously paired with flavored substances to substitute for those substances as conditional discriminative cues was examined in two Pavlovian appetitive conditioning experiments using rat subjects. In Experiment 1, a visual stimulus was first paired with the delivery of a sucrose solution. Then the rats were trained in conditional discrimination tasks in which sucrose delivery alone served as a conditional cue signaling whether or not a subsequent tone would be reinforced with food pellets. Subjects rapidly acquired discriminative performance to the tones, especially in a feature-negative condition in which sucrose delivery signaled when the tone would not be reinforced. In a subsequent test in which neither food nor sucrose was delivered, presentation of the visual CS also controlled discriminative performance to subsequently presented tones. Experiment 2 showed the ability of a visual CS to substitute for a flavored substance as a conditional cue to be highly stimulus specific. Experiment 2 also showed that a flavored substance was less effective as a conditional cue when it was made to be expected by preceding it with a previously associated visual signal than when it was made to be surprising by preceding it with a visual stimulus signaling another flavored liquid. These results indicate that CS-evoked representations of events can substitute for those events themselves in the control of previously established conditional discrimination performance.     

Signaling the omission of a response-contingent outcome reduces discriminative control

The effect of training a positive discriminative stimulus (S+ ) as a signal for the nonreinforcement of an instrumental response (S?) on the ability of that stimulus to evoke its original instrumental response was examined in three experiments using rats. In all three experiments, two different stimuli were established as S+s for different response-outcome relations. In Experiment 1, an S+ was less effective in controlling its original response after it had undergone training as an S? for a new response that earned the same outcome than it was after training as an S? for a response that earned a different outcome. Experiment 2 established that this effect was not mediated by Pavlovian inhibitory conditioning produced by the negative correlation between the S+ and the outcome during S? training. Simply arranging a negative correlation between S+ and the outcome whose occurrence it had previously signaled did not impair the ability of that S+ to elicit its original response. In Experiment 3, the response-evoking properties of an S+ were found to be undermined by using the S+ as a signal for the simple extinction of a new response trained with the same outcome, but not with a different outcome. These results suggest that positive discriminative stimuli use their associations with the outcomes earned in their presence to control the responses that earned those outcomes.     

Stimulus competition in the absence of compound conditioning

Most associative theories have assumed that stimulus competition occurs only between conditioned stimuli (CSs) that are trained in compound. The present research investigated the possibility of competition between two CSs that were individually paired to the same unconditioned stimulus (US). We used human subjects in an anticipatory suppression analogue to Pavlovian conditioning. Experiment 1 showed that X+ training followed by A+ training resulted in impaired responding to X. This did not occur when A+ training preceded X+ training. Experiment 2 replicated the basic effect and showed that it did not occur when the Phase 2 training consisted of A? instead of A+ nor when the A+ pairings occurred in a second context. Experiment 3 showed that A+ pairings occurring in a second context could still produce the effect when X was tested in the context in which the A+ pairings had occurred, but not when X was tested in a context different from that used for A+ training. Collectively, these results suggest that individually trained CSs may compete with each other when one of those CSs is more strongly activated by the test context than the other one.     

Negative discriminative stimuli provide information about the identity of omitted response-contingent outcomes

Three experiments using rats examined whether a signal for the nonreinforcement of an instrumental response (S-) provided information about the identity of the omitted outcome. In all three experiments, one stimulus was established as a signal for the nonreinforcement of a response that earned food pellets and another stimulus signaled the nonreinforcement of a response that earned liquid sucrose. Experiment 1 found that each S-suppressed another instrumental response trained with the same outcome significantly more than a response trained with a different outcome. Using a variant of this transfer design, Experiment 2 demonstrated that an S- was slower to develop discriminative control over a new response reinforced in its presence with the same outcome compared with an outcome different from the one whose omission-the S- had previously signaled. Experiment 3 examined transfer of the S- stimuli to a response trained with two outcomes, one of which had subsequently been devalued. Performance of this response significantly increased in the presence of a signal for the omission of the devalued outcome, but decreased in the presence of a signal for the omission of the valued outcome. These results suggest that S-s do provide information about the identity of omitted response-contingent outcomes.     

Learning about qualitatively different outcomes during a blocking procedure

In four experiments, rat subjects were used in appetitive Pavlovian magazine-approach and instrumental conditioning procedures. Experiments 1 and 2 found successful blocking of the Pavlovian conditioning of X when it was reinforced in an AX compound after prior conditioning of A. This occurred whether the outcome following AX was the same as or qualitatively different from what followed A. Experiment 3 repeated those findings but also used a transfer procedure to identify the individual associations between X and outcomes. Stimulus X developed an association with the outcome following AX when that outcome differed from that following A alone but not when it was the same as that following A alone. Experiment 4 repeated that pattern of observations for the case of an X that was an instrumental discriminative stimulus. These results suggest that different associative structures may result from a qualitatively changed and unchanged outcome in a blocking experiment. The results are related to comparable findings for the case of overexpectation.     

Dissociation of the effect of reinforcer type and response strength on the force of a conditioned response

A dissociation between the effect of reinforcer type and response strength on the force of the pigeon’s keypeck response was shown in three experiments. In Experiment 1, pigeons were trained to peck two conditioned stimuli, one paired with water and another paired with grain. The pigeons made more forceful pecks for grain than for water and also showed a tendency, albeit an unreliable one, to respond on a higher percentage of food trials than water trials. In Experiment 2, the pigeons from Experiment 1 were satiated with either food or water and were then presented with the two conditioned stimuli in an extinction test. It was found that, regardless of the drive state, the pigeons made more forceful pecks to the stimulus that predicted food than to the stimulus that predicted water. In the thirsty group, however, this difference in force was not accompanied by a difference in the percentage of trials with a response. In Experiment 3, pigeons trained with a single reinforcer pecked more often on instrumentally reinforced trials than on Pavlovian conditioning trials, but there was no difference in the force of the pecks. Taken together, these results imply that differences in response strength cannot account for the difference between the force of food- and water-reinforced pecks. Instead, stimulus-substitution theory may provide the best account of the topography of the two types of pecks.     

Temporal integration and instrumental conditioned reinforcement

Stimuli associated with primary reinforcement for instrumental behavior are widely believed to acquire the capacity to function as conditioned reinforcers via Pavlovian conditioning. Some Pavlovian conditioning studies suggest that animals learn the important temporal relations between stimuli and integrate such temporal information over separate experiences to form a temporal map. The present experiment examined whether Pavlovian conditioning can establish a positive instrumental conditioned reinforcer through such temporal integration. Two groups of rats received either delay or trace appetitive conditioning in which a neutral stimulus predicted response-independent food deliveries (CS1→US). Both groups then experienced one session of backward second-order conditioning of the training CS1 and a novel CS2 (CS1–CS2 pairing). Finally, the ability of CS2 to function as a conditioned reinforcer for a new instrumental response (leverpressing) was assessed. Consistent with the previous demonstrations of temporal integration in fear conditioning, a CS2 previously trained in a trace-conditioning protocol served as a better instrumental conditioned reinforcer after backward second-order conditioning than did a CS2 previously trained in a delay protocol. These results suggest that an instrumental conditioned reinforcer can be established via temporal integration and raise challenges for existing quantitative accounts of instrumental conditioned reinforcement.     

Dominance,the bidirectional hypothesis,and Pavlovian backward conditioning in the US-US paradigm

Based upon considerations raised by Soviet research, the role of relative stimulus intensity, or dominance, in the unconditioned stimulus-unconditioned stimulus (US-US) paradigm was investigated under circumstances presumed favorable to the backward conditioned response (CR). Using the classically conditioned forelimb response of the cat, a brief shock (USD delivered to one forepaw preceded a shock (US2) to the opposite forepaw in paired conditioning fashion; subjects in the control group received explicitly unpaired presentations of the stimuli. Conditioning in both the forward and backward directions was evaluated by the appearance of contralateral CRs on test trials to each of the USs. In Experiment 1, a ratio of the intensities between US1 and US2 of 100:80 was used to create a relative dominance in favor of the backward CR. In addition, to evaluate the suggestion that the appearance of the backward CR is retarded in the Pavlovian paradigm, overtraining was provided to a forward conditioning criterion of 200%. In Experiment 2, the cats were exposed to successive reductions in the intensity of US2 to verify manipulations of dominance reportedly involved in the reactivation of a latent backward CR. Although forward conditioning was readily established to USl, there was no evidence of back-ward conditioning to US2 under any of the conditions.     

Modulatory mechanisms in Pavlovian conditioning

Pavlovian conditioning has traditionally been thought to involve the acquisition of excitatory and inhibitory associations between a conditioned stimulus (CS) and an unconditioned stimulus (US). Recent research, however, has encouraged the view that Pavlovian learning may also encompass a higher order modulatory mechanism, in which animals use information about another stimulus to control responding to the CS. Positive modulators signal a positive relationship between the CS and the US, whereas negative modulators signal that the CS-US relationship is not in force. In both cases, the modulatory control appears to be orthogonal to the modulator’s direct associations with the US. This article reviews and evaluates the literature on this Pavlovian modulatory mechanism.     

Summation and overexpectation with qualitatively different outcomes

In five experiments, a Pavlovian appetitive conditioning preparation was used with rats to explore the interaction of associations with different, but equivalently valued, outcomes. Experiment 1 demonstrated summation in magazine responding when two stimuli previously associated with different outcomes were combined. Experiments 2A and 2B found that pairing that stimulus compound with one of the outcomes led to a decrease in performance (overexpectation) for both of the stimulus elements. Experiments 3A and 3B confirmed that result but demonstrated the continued presence of the original stimulus-outcome associations. The results are consonant with a view in which replacing one outcome with another leads to an associative structure containing both stimulus-outcome associations and an outcome-independent depressive process.     

Divergence of conditioned eyeblink and conditioned fear in backward Pavlovian training

Two experiments of Pavlovian conditioning with rabbits evaluated the effects of initiating or continuing a conditioned stimulus (CS) after a paraorbital unconditioned stimulus (US). In Experiment 1, backward pairings, in which a CS came on after the US, produced a CS that appeared inhibitory on a measure of eyeblink conditioning but excitatory on a potentiated-startle measure of conditioned fear. In Experiment 2, extending the duration of a CS that came on prior to the US, so that it continued after the US, decreased eyeblink conditioned responses, whereas it increased conditioned fear. The data from the two experiments confirm and extend those of Tait and Saladin (1986), supporting the suppositions of AESOP (Wagner & Brandon, 1989) that conditioned eyeblink and conditioned fear can be dissociated under various temporal relationships between the CS and US.     

Response variability in pigeons in a pavlovian task

Instrumental response variation is inversely related to reward probability. Gharib, Derby, and Roberts (2001) theorized that individuals behave more variably when their expectation of reward is low. They postulate that this behavioral rule assists the discovery of alternative actions when a target response is unlikely to be reinforced. This suggests that response variability may be unaffected in a situation in which an animal’s behavior is inconsequential to outcome delivery. We trained 6 pigeons in a within-subjects Pavlovian autoshaping procedure. On any given trial, the pigeons were presented with one of six colored discs on a touchscreen; each stimulus was associated with a particular probability of food, ranging from 100% to 0.6%. Pecking was more variable with low probabilities of food delivery, thus extending the rule relating variability and expectation to a Pavlovian situation.     

Summation: Assessment of a configural theory

In five experiments, rats were given Pavlovian pairings of auditory and visual stimuli with delivery of food pellets. Experiment 1 found greater responding to an AB compound after training with the individual A and B stimuli, compared with responding both to the A and B elements and to a separately trained CD compound. Experiment 2 found this enhanced responding to depend on the associative strengths of A and B. In Experiment 3, responding was greater to a CD compound than to the other compounds after an AB-, AD+, BC+ training procedure. In Experiment 4, responding to an AB compound was greater than that to the elements after A was reinforced on a 100% schedule and B on a 50% schedule. In Experiment 5, responding to an AC compound was greater than that to either A or C after an AB+, CD+, A-training procedure. A configural theory, such as that proposed by Pearce (1987), anticipates summation in none of these procedures, unless the conditioned context is assumed to have a salience greater than zero. In order to predict summation in Experiments 3, 4, and 5, a context salience greater than that of the elements must be assumed. However, such an assumption also anticipates that extinction of a 100% stimulus should eliminate responding to a 50% stimulus. The results of Experiment 3 contradicted that prediction. These results conform better to the expectations of elemental models of conditioning.     

Conditioned inhibitory effects of discriminated Pavlovian training with food in rats depend on interactions of search modes, related repertoires, and response measures

Like other accounts of conditioned inhibition, behavior systems predicts (and Experiment 1 shows) that during summation and retardation tests, presentation of a negative conditioned stimulus (a CS−) created by discriminative Pavlovian food conditioning will interfere with a focal search response, such as nosing in the feeder. Unlike most other views, behavior systems predicts (and Experiment 2 shows) that the same CS− can potentiate a general search response, like attending to a moving artificial prey stimulus. Contacting the prey stimulus in extinction increased over baseline when a CS- but not a CS Novel preceded it. Experiment 3 showed this effect was not due to unconditioned qualities of the CS−. It appears that the effects of a discriminative CS-depend on the interaction of the training contingency with search modes related to the unconditioned stimulus (US), their perceptual-motor repertoires and environmental support, and the choice of response measure.     

Sign tracking in domesticated quail with one trial a day: Generality across CS and US parameters

When the conditioned stimulus (CS) is located some distance from the unconditioned stimulus (US), pairings of the CS and US can yield either conditioned approach to the CS (sign tracking) or conditioned approach to the US (goal tracking). However, goal tracking is the more common outcome, and, because of that, goal tracking has come to serve as a “standard” measure of associative learning in several laboratories. In contrast, in previous studies of sexual conditioning with domesticated quail, only sign tracking was observed. In the present study, quail continued to show sign tracking rather than goal tracking whether or not the US was highly localized (Experiment 1), whether food or a sexual US was used (Experiment 2), whether the CS was mobile or immobile (Experiment 3), and whether the CS was 91 or 233 cm from the US compartment (Experiment 4). The present findings encourage caution in the routine use of goal tracking as a measure of learning. The possible mechanisms of goal tracking are discussed in terms of occasion setting and behavior systems theory.     

Context effects on conditioning,extinction, and reinstatement in an appetitive conditioning preparation

Three experiments with rat subjects examined the effects of contextual stimuli on performance in appetitive conditioning. A 10-sec tone conditioned stimulus (CS) was paired with a food-pellet unconditioned stimulus (US); conditioning was indexed by the observation of headjerking, a response of the rat to auditory stimuli associated with food. In Experiment 1, a context switch following initial conditioning did not affect conditioned responding to the tone; however, when the response was extinguished in the different context, a return to the original conditioning context “renewed” extinguished responding. These results were replicated in Experiments 2 and 3 after equating exposure to the two contexts (Experiment 2) and massing the conditioning and extinction trials (Experiment 3). The results of Experiment 1 also demonstrated that separate exposure to the US following extinction reinstates extinguished responding to the tone; this effect was further shown to depend at least partly on presenting the US in the context in which testing is to occur (Experiments 2 and 3). Overall, the results are consistent with previous data from aversive conditioning procedures. In either appetitive or aversive conditioning, the context may be especially important in affecting performance after extinction.     

Context excitation and modulation of conditioned sexual behavior

Three experiments were conducted to investigate direct and modulatory influences of context in the conditioned sexual behavior of male Japanese quail. A preference test procedure was used to assess the acquisition of contextual excitation. In Experiment 1, following direct context-unconditioned stimulus (US) pairings, male quail shifted their contextual preference from an initially preferred context to one in which they received copulatory opportunity with a female quail (US). Unpaired control group subjects did not demonstrate this shift in preference. This place preference procedure was used in Experiments 2 and 3 to assess contextual excitation when context was trained in the presence of a discrete conditioned stimulus (CS). Experiment 2 provided evidence that context can modulate responding to a discrete CS. In Experiment 3, we varied the spatial contiguity between the context and the US. Some subjects received the US directly in the training context, whereas other subjects received the US in an alternate context. Contextual excitation was evident only in subjects that received the former. Thus, there is a dissociation between the modulatory and excitatory properties of context in sexual conditioning that may depend on the context-US spatial contiguity.     

Transfer between Pavlovian facilitators and instrumental discriminative stimuli

Two experiments assessed the degree to which Pavlovian facilitators were interchangeable with instrumental discriminative stimuli (S^ds). In Experiment 1, rats were trained in a Pavlovian paradigm in which one stimulus (i.e., a facilitator) signaled the reinforcement of another stimulus (i.e., a target). Next, the rats were given instrumental discrimination training in which an S^d signaled the reinforcement of barpressing. A transfer test then assessed the capacity of the Pavlovian facilitator to promote barpressing. The results showed that the facilitator promoted significant barpressing, both when it was presented alone and when it was presented in compound with the S^d. Reliable transfer was not obtained with a “pseudofacilitator” control stimulus that, during training, was uninformative about the reinforcement of its target. Experiment 2 showed that a stimulus trained as an instrumental S^d reliably augmented responding to a stimulus previously trained as a target in a Pavlovian facilitation paradigm. A “pseudo-S^d” that, during training, was uninformative about the reinforcement of barpressing failed to promote such transfer. These results show that Pavlovian facilitators and instrumental S^ds are interchangeable to a significant degree, and suggest that facilitators and S^ds may act via similar mechanisms.