Instrumental responses become associated with reinforcers that differ in one feature

In three experiments, thirsty rats were trained to make several instrumental responses whose outcomes differed in which of two relatively inconsequential flavor features they contained. In Experiment 1, one of the features was subsequently devalued by pairing it with lithium chloride; in Experiment 2, it was enhanced in value by pairing it with sucrose. In both experiments, differences in the value of the features resulted in parallel differences in the likelihood of the responses during a subsequent extinction test. In Experiment 3, the animals chose between these responses in the presence of discriminative stimuli that had signaled the occurrence of these different features following another response. The stimuli selectively augmented the likelihood of the response with which they shared training by the same-flavored consequence. These results indicate that rats can separately encode features that differ along one dimension, both in the association between an instrumental response and its outcome, and in the association between a discriminative stimulus and that outcome.     

On the nature of CS and US representations in Pavlovian learning

A significant problem in the study of Pavlovian conditioning is characterizing the nature of the representations of events that enter into learning. This issue has been explored extensively with regard to the question of what features of the unconditioned stimulus enter into learning, but considerably less work has been directed to the question of characterizing the nature of the conditioned stimulus. This article introduces a multilayered connectionist network approach to understanding how “perceptual” or “conceptual” representations of the conditioned stimulus might emerge from conditioning and participate in various learning phenomena. The model is applied to acquired equivalence/distinctiveness of cue effects, as well as a variety of conditional discrimination learning tasks (patterning, biconditional, ambiguous occasion setting, feature discriminations). In addition, studies that have examined what aspects of the unconditioned stimulus enter into learning are also reviewed. Ultimately, it is concluded that adopting a multilayered connectionist network perspective of Pavlovian learning provides us with a richer way in which to view basic learning processes, but a number of key theoretical problems remain to be solved, particularly as they relate to the integration of what we know about the nature of the representations of conditioned and unconditioned stimuli.     

Transfer of instrumental control mediated by a devalued outcome

In three instrumental learning experiments, rat subjects were used to explore transfer of an instrumental discriminative stimulus to a new response. The stimulus increased the likelihood of that response to the degree that the stimulus and response had a history of association with the same outcome. Moreover, devaluation of the outcome by pairing with lithium chloride had no detrimental effects on its ability to mediate transfer to a new response. This result helps one choose among various accounts of transfer.     

Signaling the omission of a response-contingent outcome reduces discriminative control

The effect of training a positive discriminative stimulus (S+ ) as a signal for the nonreinforcement of an instrumental response (S?) on the ability of that stimulus to evoke its original instrumental response was examined in three experiments using rats. In all three experiments, two different stimuli were established as S+s for different response-outcome relations. In Experiment 1, an S+ was less effective in controlling its original response after it had undergone training as an S? for a new response that earned the same outcome than it was after training as an S? for a response that earned a different outcome. Experiment 2 established that this effect was not mediated by Pavlovian inhibitory conditioning produced by the negative correlation between the S+ and the outcome during S? training. Simply arranging a negative correlation between S+ and the outcome whose occurrence it had previously signaled did not impair the ability of that S+ to elicit its original response. In Experiment 3, the response-evoking properties of an S+ were found to be undermined by using the S+ as a signal for the simple extinction of a new response trained with the same outcome, but not with a different outcome. These results suggest that positive discriminative stimuli use their associations with the outcomes earned in their presence to control the responses that earned those outcomes.     

Conditioned inhibitory effects of discriminated Pavlovian training with food in rats depend on interactions of search modes, related repertoires, and response measures

Like other accounts of conditioned inhibition, behavior systems predicts (and Experiment 1 shows) that during summation and retardation tests, presentation of a negative conditioned stimulus (a CS−) created by discriminative Pavlovian food conditioning will interfere with a focal search response, such as nosing in the feeder. Unlike most other views, behavior systems predicts (and Experiment 2 shows) that the same CS− can potentiate a general search response, like attending to a moving artificial prey stimulus. Contacting the prey stimulus in extinction increased over baseline when a CS- but not a CS Novel preceded it. Experiment 3 showed this effect was not due to unconditioned qualities of the CS−. It appears that the effects of a discriminative CS-depend on the interaction of the training contingency with search modes related to the unconditioned stimulus (US), their perceptual-motor repertoires and environmental support, and the choice of response measure.     

A differential-outcome effect in pigeons using spatial hedonically nondifferential outcomes

We examined the extent to which nonhedonically different differential outcomes involving feeder location control pigeons’ comparison choices in matching to sample. In Experiment 1, we showed that differential feeder location outcomes associated with each of two samples can facilitate delayed-matching accuracy. In Experiment 2, we found positive transfer following training on two matching tasks with differential feeder location outcomes when samples from one task were replaced by samples from the other task. In Experiment 3, we found that when differential-outcome expectations could no longer serve as the cues for comparison choice, sample stimuli continued to exert some control over choice of comparisons. The results indicate that differential outcomes (involving feeder location) that presumably do not differ in hedonic value are sufficient to control comparison choice. Thus, the differential hedonic value of the outcome elicited by the sample does not appear to be a requirement of the differential-outcome effect. Furthermore, these differential outcomes appear to augment matching accuracy, but they do not eliminate control by the samples.     

Releaser-induced recognition learning by amphibians and reptiles

This review evaluates the bases of the widely held assumption that amphibians and reptiles possess impoverished learning abilities. Procedurally defined forms of learning (instrumental conditioning, Pavlovian conditioning and autoshaping, imprinting, and social learning) are reviewed, as well as evidence for the involvement of learning in various behavioral phenomena, including aversive stimulus and predator avoidance, noxious and palatable food recognition, conditioned aversion formation, search image, conspecific recognition, habitat recognition, and cultural transmission of stimulus recognition. The evidence reviewed suggests that amphibian and reptilian learning, for the most part, consists of a releasing-stimulus-induced redirection of innately organized released responses. Amphibians and reptiles appear to learn what stimulus to respond to rather than how to respond to a particular stimulus.     

Blocking and enhancement of fear conditioning by appetitive CSs

Prior research on Pavlovian-to-instrumental transfer has shown that when a CS previously associated with shock (AvCS+) is presented contingent upon a choice response to a discriminative stimulus for food reinforcement, it facilitates discrimination learning. Conversely, a response-contingent CS previously associated with the absence of shock (AvCS?) retards discrimination learning. To evaluate whether these findings reflect across-reinforcement blocking and enhancement effects, two experiments investigated the effects of appetitively conditioned stimuli on fear conditioning to a novel stimulus that was serially compounded with the appetitive CS during conditioned-emotional-response (CER) training. Although there were no differential effects of the appetitive CSs in CER acquisition, Experiment 1, using a relatively weak shock US, showed that a CS previously associated with food (ApCS+) retarded CER extinction to the novel stimulus, in evidence of enhanced fear conditioning to that stimulus. In addition, Experiment 2, using a stronger shock US, showed that a CS previously associated with the absence of food (ApCS?) facilitated CER extinction to the novel stimulus, in evidence of weaker fear conditioning to that stimulus. These results parallel traditional blocking effects and indicate not only that an ApCS+ and an ApCS? are functionally similar to AvCSs of opposite sign, but that their functional similarity is mediated by common central emotional states.     

Selective reinstatement of instrumental performance depends on the discriminative stimulus properties of the mediating outcome

We conducted three experiments to investigate the associative structure underlying the reinstatement of instrumental performance after extinction. In each experiment, rats were initially rewarded on two responses with different outcomes. At test, both responses were extinguished in order to assess the impact of a single noncontingent outcome delivery on response selection. Experiment 1 found evidence of outcome-selective reinstatement (i.e., more responses were performed on the lever that was trained with the reinstating outcome than on the other lever). Experiment 2 demonstrated that the outcome’s capacity to reinstate performance was not affected by a reduction in its motivational value. Experiment 3 found evidence that the reinstating outcome selectively retrieved the response it signaled rather than the response it followed during training. Together, these findings are consistent with the view that instrumental reinstatement depends on the discriminative stimulus properties of the reinstating outcome.     

An associative analysis of instrumental biconditional discrimination learning

Three different techniques were employed to analyze the associative structures mediating performance on an instrumental biconditional discrimination. In all three experiments, rats were trained concurrently on two tasks in which different stimuli signaled which one of two responses would be followed by reward. In each task, one response was rewarded in one stimulus and the other response was rewarded in the other stimulus. Correct responses earned pellets in one task and sucrose in the other task. The transfer procedure was used in Experiment 1A to identify whether or not an association developed between a biconditional discriminative stimulus and its instrumental outcome. Evidence was obtained that a biconditional cue elevated preferentially a new response trained with the same outcome. Experiments 1B and 3 examined the potential contribution of this stimulus-outcome association to biconditional performance by training the biconditional cues as signals (S-s) for the nonreinforcement of a different response. There was no evidence that this operation interfered with the ability of a biconditional cue to control performance of its correct response. In Experiments 1B and 2, the value of the instrumental outcome was reduced in an attempt to assess the contribution of stimulus-response associations to performance on the biconditional discrimination. The results of Experiments 1B and 2 reveal that correct responses were depressed following devaluation of the outcome used to train them, suggesting that learning about the response-outcome relation occurs. The implications of these results for binary and hierarchical models of instrumental learning are discussed.     

Evidence for the hierarchical structure of instrumental learning

In three experiments using rats, we examined the role of a discriminative stimulus (S) in governing the relation between a response (R) and an outcome (O) in an appetitive instrumental learning paradigm. In each experiment, we attempted to distinguish between a simple S-O association and a hierarchical relation in which S is associated with the R-O association. We used three variations on discriminative training procedures and three different assessment techniques-for revealing the hierarchical structure. In Experiment 1, we employed a training procedure in which S signaled a change in the R-O relation but no change in the likelihood of O. Although such an arrangement should not produce an excitatory S-O association, it nevertheless generated an S that controlled responding and transferred that control to other responses. In Experiment 2, we used a discrimination procedure in which two Ss each had the same two Rs and Os occur in their presence but each S signaled that a different R-O combination would be in effect. This design provided the opportunity for equivalent pairwise associations among S, R, and O but unique hierarchical relations. The subjects learned the hierarchical structure, as revealed by the specific depressive effect of a subsequent lithium-chloride-induced devaluation of O on responding only in the presence of the S in which that response had led to that outcome. In Experiment 3, one S signaled two different R-O outcomes. Then, two new stimuli were presented with the original S; the R-O relations were retained in the presence of one of the added stimuli but were rearranged in the presence of the other. The added S came to control less responding when it was redundant with respect to the R-O relations than when it was informative. Although all of the results were of modest size and each has an alternative interpretation, together they provide converging evidence for the hierarchical role of S in controlling an R-O association.     

Second-order conditioning of the pigeon’s keypeck

Pigeons learned to peck a keylight (S2) when it was paired with a stimulus (S1) that already evoked keypecking. Control procedures showed that S2 acquired control over responding because it was paired with S1 and because S1 had a conditioning history, thereby supporting the claim that S2 was a second-order conditioned stimulus. Second-order conditioning occurred as rapidly when S1 was a keylight as when it was a tone. Test procedures showed that after second-order conditioning, responding to S2 was markedly debilitated by the extinction of responding to S1, indicating that the ability of S2 to evoke a response importantly depends upon the continued ability of S1 to do so. Our demonstration that directed motor action in the pigeon is susceptible to second-order conditioning suggests a new interpretation of conditioned reinforcement in instrumental learning. Our demonstration that the effectiveness of S2 depends upon the continued effectiveness of S1 indicates that S-S associations are formed in this version of the second-order conditioning experiment.     

Associative foundation of causal learning in rats

Are humans unique in their ability to interpret exogenous events as causes? We addressed this question by observing the behavior of rats for indications of causal learning. Within an operant motor–sensory preconditioning paradigm, associative surgical techniques revealed that rats attempted to control an outcome (i.e., a potential effect) by manipulating a potential exogenous cause (i.e., an intervention). Rats were able to generate an innocuous auditory stimulus. This stimulus was then paired with an aversive stimulus. The animals subsequently avoided potential generation of the predictive cue, but not if the aversive stimulus was subsequently devalued or the predictive cue was extinguished (Exp. 1). In Experiment 2, we demonstrated that the aversive stimulus we used was in fact aversive, that it was subject to devaluation, that the cue–aversive stimulus pairings did make the cue a conditioned stimulus, and that the cue was subject to extinction. In Experiments 3 and 4, we established that the decrease in leverpressing observed in Experiment 1 was goal-directed instrumental behavior rather than purely a product of Pavlovian conditioning. To the extent that interventions suggest causal reasoning, it appears that causal reasoning can be based on associations between contiguous exogenous events. Thus, contiguity appears capable of establishing causal relationships between exogenous events. Our results challenge the widely held view that causal learning is uniquely human, and suggest that causal learning is explicable in an associative framework.     

Negative discriminative stimuli provide information about the identity of omitted response-contingent outcomes

Three experiments using rats examined whether a signal for the nonreinforcement of an instrumental response (S-) provided information about the identity of the omitted outcome. In all three experiments, one stimulus was established as a signal for the nonreinforcement of a response that earned food pellets and another stimulus signaled the nonreinforcement of a response that earned liquid sucrose. Experiment 1 found that each S-suppressed another instrumental response trained with the same outcome significantly more than a response trained with a different outcome. Using a variant of this transfer design, Experiment 2 demonstrated that an S- was slower to develop discriminative control over a new response reinforced in its presence with the same outcome compared with an outcome different from the one whose omission-the S- had previously signaled. Experiment 3 examined transfer of the S- stimuli to a response trained with two outcomes, one of which had subsequently been devalued. Performance of this response significantly increased in the presence of a signal for the omission of the devalued outcome, but decreased in the presence of a signal for the omission of the valued outcome. These results suggest that S-s do provide information about the identity of omitted response-contingent outcomes.     

Learning about qualitatively different outcomes during a blocking procedure

In four experiments, rat subjects were used in appetitive Pavlovian magazine-approach and instrumental conditioning procedures. Experiments 1 and 2 found successful blocking of the Pavlovian conditioning of X when it was reinforced in an AX compound after prior conditioning of A. This occurred whether the outcome following AX was the same as or qualitatively different from what followed A. Experiment 3 repeated those findings but also used a transfer procedure to identify the individual associations between X and outcomes. Stimulus X developed an association with the outcome following AX when that outcome differed from that following A alone but not when it was the same as that following A alone. Experiment 4 repeated that pattern of observations for the case of an X that was an instrumental discriminative stimulus. These results suggest that different associative structures may result from a qualitatively changed and unchanged outcome in a blocking experiment. The results are related to comparable findings for the case of overexpectation.     

Outcome-specific conditioned inhibition in Pavlovian backward conditioning

In the present experiments, the outcome specificity of learning was explored in an appetitive Pavlovian backward conditioning procedure with rats. The rats initially were administered Pavlovian backward training with two qualitatively different unconditioned stimulus conditioned-stimulus (US-CS) pairs of stimuli (e.g., pellet --> noise or sucrose --> light), and then the effects of this training were assessed in Pavlovian-to-instrumental transfer (Experiment 1) and retardation-of-learning (Experiment 2) tests. In the transfer test, it was shown that during the last 10-sec interval, the CSs selectively reduced the rate of the instrumental responses with which they shared a US, relative to the instrumental responses with which they did not share a US. The opposite result was obtained when the USs (in the absence of the CSs) were presented noncontingently. In the retardation test, conditioned magazine approach, responding to the CSs was acquired more slowly when the stimulus-outcome combinations in the backward and the forward conditioning phases were the same, as compared with when they were reversed. These results are collectively in accord with the view that Pavlovian backward conditioning can result in the formation of outcome-specific inhibitory associations. Alternative views of backward conditioning are also examined.     

Reversal from blocking in humans as a result of posttraining extinction of the blocking stimulus

In a blocking procedure, conditioned stimulus (CS) A is paired with the unconditioned stimulus (US) in Phase 1, and a compound of CSs A and X is then paired with the US in Phase 2. The usual result of such a treatment is that X elicits less conditioned responding than if the A-US pairings of Phase 1 had not occurred. Obtaining blocking with human participants has proven difficult, especially if a behavioral task is used or if the control group experiences reinforcement of a CS different from the blocking CS in Phase 1. In the present series, in which human participants and a behavioral measure of learning were used, we provide evidence of blocking, using the above described control condition. Most important, we demonstrate that extinction of the blocking CS (A) following blocking treatment reverses the blocking deficit (i.e., increases responding to X). These results are at odds with traditional associative theories of learning, but they support current associative theories that predict that posttraining manipulations of the competing stimulus can result in a reversal of stimulus competition phenomena.     

Transfer between Pavlovian facilitators and instrumental discriminative stimuli

Two experiments assessed the degree to which Pavlovian facilitators were interchangeable with instrumental discriminative stimuli (S^ds). In Experiment 1, rats were trained in a Pavlovian paradigm in which one stimulus (i.e., a facilitator) signaled the reinforcement of another stimulus (i.e., a target). Next, the rats were given instrumental discrimination training in which an S^d signaled the reinforcement of barpressing. A transfer test then assessed the capacity of the Pavlovian facilitator to promote barpressing. The results showed that the facilitator promoted significant barpressing, both when it was presented alone and when it was presented in compound with the S^d. Reliable transfer was not obtained with a “pseudofacilitator” control stimulus that, during training, was uninformative about the reinforcement of its target. Experiment 2 showed that a stimulus trained as an instrumental S^d reliably augmented responding to a stimulus previously trained as a target in a Pavlovian facilitation paradigm. A “pseudo-S^d” that, during training, was uninformative about the reinforcement of barpressing failed to promote such transfer. These results show that Pavlovian facilitators and instrumental S^ds are interchangeable to a significant degree, and suggest that facilitators and S^ds may act via similar mechanisms.     

Overexpectation in appetitive Pavlovian and instrumental conditioning

In three experiments that used appetitive preparations with rats, we examined the effects of reinforcing a compound consisting of two previously reinforced stimuli on subsequent responding to those stimuli. Experiment 1 showed that a Pavlovian conditioned stimulus given this treatment evoked fewer magazine entries when presented alone than did a reinforced stimulus that did not receive the compound treatment. Experiment 2 examined inhibition of delay and generalization decrement accounts for the results of Experiment 1. Experiment 3 extended this finding to an instrumental learning paradigm.     

Stimuli associated with the cancellation of food and its cues enhance eating but display negative incentive value

Initially neutral conditioned stimuli paired with food often acquire motivating properties, including serving as secondary reinforcers, enhancing instrumental responding in Pavlovian-instrumental transfer procedures, and potentiating food consumption under conditions of food satiation. Interestingly, cues associated with the cancellation of food and food cues may also potentiate food consumption (e.g., Galarce and Holland, 2009), despite their apparent negative correlations with food delivery. In three experiments with rats, we investigated conditions under which potentiation of feeding by such “interruption stimuIi” (ISs) develops, and some aspects of the content of that learning. Although in all three experiments ISs enhanced food consumption beyond control levels, they were found to act as conditioned inhibitors for anticipatory food cup entry (Experiment 1), to serve as conditioned punishers of instrumental responding (Experiment 2), and to suppress instrumental lever press responding in a Pavlovian instrumental transfer procedure (Experiment 3). Furthermore, when given concurrent choice between different foods, an IS enhanced consumption of the food whose interruption it had previously signaled, but when given a choice between performing two instrumental responses, the IS shifted rats’ choice away from the response that had previously yielded the food whose interruption had been signaled by IS (Experiment 3). Thus, the effects of an IS on appetitive responses were opposite to its effects on consummatory responding. Implications for our understanding of learned incentive motivation and the control of overeating are discussed.