Signal-directed behavior in the rat: Interactions between the nature of the CS and the nature of the UCS

Two experiments are described, which involved the investigation of interactions between the nature of the conditioned stimulus (CS) and the nature of the unconditioned stimulus (UCS) in producing signal-centered behavior. In Experiment 1, rats received response-independent heat reinforcement in a cold environment. For some groups, this heat UCS was signaled by presentations of a standard aluminum retractable lever; for other groups, it was signaled by a retractable lever covered in acrylic fur (furry lever CS). Only the subjects that received the furry lever CS paired with heat exhibited differential CS-contact behavior, when compared with unpaired, aluminum lever, and warm control subjects. In Experiment 2, hungry rats received pairings of either an aluminum or a furry lever with food (UCS). When compared with unpaired controls, only the subjects that received the aluminum lever paired with food showed differential signal-directed behavior; the subjects receiving the furry lever CS did not show differential contact with the CS, but instead exhibited differential food tray entry behavior during CS presentation. In the two studies, the signal-directed behavior exhibited by subjects resembled either thermoregulatory or feeding behaviors characteristic of rats. The results suggest that signal-directed behavior is determined by a complex interaction between the ecological relevance of the CS and the nature of the UCS—an interaction that can best be described in terms of a behavior systems model of conditioned responding.     

Spatiotemporal characteristics of serial CSs and their relation to search modes and response form

In four experiments, we examined how the spatiotemporal proximity to food of the two elements of a serial conditioned stimulus (CS) influenced the pattern of CS-directed versus food-site-directed behavior in rats. Experiment 1 showed that only temporal proximity affected responding when the serial CS consisted of two successive 4-sec presentations of either a spatially near or a spatially far lever (NN or FF). However, Experiment 2 showed that behavior depended markedly on whether rats received a near followed by a far lever (NF) or a far followed by a near lever (FN). Experiment 3 showed that the effects of Experiment 2 could be changed by increasing the duration of the second CS element, and Experiment 4 showed that these changes were not related to previous training. We concluded that behavior produced by the spatiotemporal qualities of the lever elements can be attributed to a mapping between the temporal qualities of the CS elements and an underlying sequence of search modes related to finding food.     

Applying Konorski’s model of classical conditioning to signal-centered behavior in the rat: Some functional similarities between hunger CRs and sign-tracking

Two experiments were conducted to investigate functional similarities between “hunger CRs” of Konorski’s (1967) model of appetitive classical conditioning and sign-tracking behavior in rats. Konorski’s model predicts that hunger CRs will be facilitated (1) when a nonrein-forced stimulus similar to the reinforced CS is introduced, and (2) when some CS presentations are unexpectedly nonreinforced. In Experiment 1, hungry rats acquired a leverpress response to a retractable lever that was paired with response-independent food. Following this training, a second lever was introduced whose presentation was not followed by food. The effect of the presence of this second lever was to facilitate responding to the original lever. In Experiment 2, single-lever autoshaping training was followed by a shift from 100% pairing of the lever with food to only 50% of the lever presentations being followed by food. The introduction of partial reinforcement produced an immediate and durable increase in leverpressing. The findings of both experiments are consistent with predictions from Konorski’s model of classical conditioning if sign-tracking is considered as a “hunger CR.”     

Summation: Assessment of a configural theory

In five experiments, rats were given Pavlovian pairings of auditory and visual stimuli with delivery of food pellets. Experiment 1 found greater responding to an AB compound after training with the individual A and B stimuli, compared with responding both to the A and B elements and to a separately trained CD compound. Experiment 2 found this enhanced responding to depend on the associative strengths of A and B. In Experiment 3, responding was greater to a CD compound than to the other compounds after an AB-, AD+, BC+ training procedure. In Experiment 4, responding to an AB compound was greater than that to the elements after A was reinforced on a 100% schedule and B on a 50% schedule. In Experiment 5, responding to an AC compound was greater than that to either A or C after an AB+, CD+, A-training procedure. A configural theory, such as that proposed by Pearce (1987), anticipates summation in none of these procedures, unless the conditioned context is assumed to have a salience greater than zero. In order to predict summation in Experiments 3, 4, and 5, a context salience greater than that of the elements must be assumed. However, such an assumption also anticipates that extinction of a 100% stimulus should eliminate responding to a 50% stimulus. The results of Experiment 3 contradicted that prediction. These results conform better to the expectations of elemental models of conditioning.     

Sign-tracking (autoshaping) in rats: A comparison of cocaine and food as unconditioned stimuli

A series of experiments was performed to determine whether sign-tracking would occur in rats with intravenous (i.v.) cocaine as the unconditioned stimulus. In Experiment 1, a retractable lever paired with food produced strong sign-tracking, but a lever paired with one of three doses of i.v. cocaine did not elicit any approach or contact behavior. Experiment 2 demonstrated that doses of cocaine that did not elicit sign-tracking would function as a positive reinforcer for a lever contact operant. In Experiment 3, an artificialconsummatory response was added to make the cocaine reinforcement episode more behaviorally comparable to that occasioned by food. Although the rats readily performed this response when it was required to receive cocaine infusions, they still did not contact a lever that signaled the availability of these infusions. It appears that cocaine is different from other positive reinforcers (e.g., food, water, warmth, or intracranial stimulation) in that it will not produce sign-tracking in rats.     

The learned function of food-deprivation cues: A role for conditioned modulation

Rats trained in one context to use stimuli arising from food deprivation as discriminative signals for shock were tested in other contexts to assess the basis of conditioned responding (i.e., freezing or behavioral immobility). In Experiment 1, discriminative control by 24-h food-deprivation cues failed to promote transfer responding in a test context that had no association with shock. This indicated that food deprivation cues had little direct excitatory power. However, transfer of behavioral control by 24-h food-deprivation cues was obtained in a context paired with shock only when the rats were 19 h water deprived. This finding agrees with the idea that food-deprivation cues become conditioned modulators of the capacity of external stimuli to activate their association with an unconditioned stimulus. In Experiment 2, rats trained to use 24-h food-deprivation cues as signals for shock exhibited significantly greater transfer performance when the transfer context had undergone partial extinction relative to when the transfer context had undergone only simple excitatory training. This finding with deprivation cues and transfer contexts (1) paralleled earlier results obtained with discrete (auditory and visual) conditioned modulators and transfer targets, and (2) posed difficulties for associative summation and generalization interpretations of transfer performance.     

Context sensitivity of conditioned suppression following preexposure to the conditioned stimulus

Conditioned suppression in rats is often unaffected when the context (or set of background stimuli) is changed following conditioning. This suggests that responding to the conditioned stimulus (CS) can be relatively insensitive to the context in which the CS is presented. In two experiments, we examined whether sensitivity to contextual stimuli is affected by preexposure to the CS. In Experiment 1, when the CS was novel at the outset of conditioning, conditioned suppression was not affected when the context was changed following conditioning. However, when the CS had been preexposed, responding was weaker when extinction occurred outside of the conditioning context. In Experiment 2, responding was again sensitive to the test context, regardless of whether preexposure occurred in the conditioning context or in an alternate context. These results suggest that the extent to which responding is sensitive to context can depend on the conditioning history of the CS.     

Training reinforcement rates,resistance to extinction,and the role of context in reinstatement

Behavior reduced as a consequence of extinction or intervention can relapse. According to behavioral momentum theory, the extent to which behavior persists and relapses once it has been eliminated depends on the relative training reinforcement rate among discriminative stimuli. In addition, studies of context renewal reveal that relapse depends on the similarity between the training stimulus context and the test stimulus context following disruption by extinction. In the present experiments with pigeons, we arranged different reinforcement rates in the presence of distinct discriminative stimuli across components of a multiple schedule. Following extinction, we attempted to reinstate responding in the presence of those target components with response-independent food presentations. Importantly, we arranged the reinstating food presentations either within the target components or in separate components, either paired with extinction (Experiment 1) or reinforcement (Experiment 2) during baseline. Reinstatement increased with greater training reinforcement rates when the reinstating food presentations were arranged in the target components and the separate components paired with reinforcement during training. Reinstatement was smaller and was not systematically related to training reinforcement rates in the target components when reinstating food presentation occurred in separate components paired with extinction. These findings suggest that relapse depends on the history of reinforcement associated with the discriminative stimuli in which the relapse-inducing event occurs.     

Context effects on conditioning,extinction, and reinstatement in an appetitive conditioning preparation

Three experiments with rat subjects examined the effects of contextual stimuli on performance in appetitive conditioning. A 10-sec tone conditioned stimulus (CS) was paired with a food-pellet unconditioned stimulus (US); conditioning was indexed by the observation of headjerking, a response of the rat to auditory stimuli associated with food. In Experiment 1, a context switch following initial conditioning did not affect conditioned responding to the tone; however, when the response was extinguished in the different context, a return to the original conditioning context “renewed” extinguished responding. These results were replicated in Experiments 2 and 3 after equating exposure to the two contexts (Experiment 2) and massing the conditioning and extinction trials (Experiment 3). The results of Experiment 1 also demonstrated that separate exposure to the US following extinction reinstates extinguished responding to the tone; this effect was further shown to depend at least partly on presenting the US in the context in which testing is to occur (Experiments 2 and 3). Overall, the results are consistent with previous data from aversive conditioning procedures. In either appetitive or aversive conditioning, the context may be especially important in affecting performance after extinction.     

Inhibitory effects of omission training

In Experiment 1, pigeons were trained to peck red or blue keys for food reinforcement at variable intervals, while food was contingent on withholding key pecks in the presence of a vertical line (omission training). When the line was briefly superimposed on red or blue in a compound test, responding was reduced. When the orientation of the line was varied during extinction, generalization gradients were variable but often had most responding at or near vertical. In Experiment 2, pigeons were trained in a discrete trials procedure that made food contingent upon pecking in the presence of triangle, and upon the absence of pecking in the presence of red (omission training). Food was never given on green-key trials (extinction). When red or green backgrounds were presented with the triangle in a compound test, responding was reduced similarly in the presence of both key colors. Subsequent resistance to auto-shaping was also similar for red and green. These data, taken together with reports in the literature, suggest that the inhibitory effects of omission training are quite similar to those of extinction. Thus, the crucial condition for obtaining inhibitory effects is not a negative stimulus-reinforcer correlation, as in extinction, but simply the establishment of low rates of responding to the inhibitory stimulus.     

Signaling a change in cue-outcome relations in human associative learning

In three experiments, we assessed the role of signals for changes in the consequences of cues as a potential account of the renewal effect. Experiment 1 showed recovery of responding following extinction when acquisition, extinction, and test phases occurred in different contexts. In addition, extinction treatment in multiple contexts attenuated context-induced response recovery. In Experiment 2, we used presentations of an extraneous stimulus (ES), instead of context shifts, and found that responding recovered from extinction only when the ES was presented both between acquisition and extinction and between extinction and test. In Experiment 3, we used a reversal learning design in which, during training, two cues were first paired with different outcomes, then paired with the alternative outcomes, and finally paired again with the original outcomes. In this experiment, presentation, just prior to testing, of an ES that had previously been presented between the different phases produced an expectation of reversal in the meaning of the cues.     

Potentiation of responding on a VR schedule by a stimulus correlated with reinforcement: Effects of diffuse and localized signals

In Experiment 1, rats were trained to leverpress on a variable ratio (VR) 30 schedule with a 500-msec delay between the reinforced response and food delivery. Subjects that experienced a signal during the delay responded faster than did control subjects that received the stimulus un-correlated with reinforcement. Higher response rates were obtained when the stimulus used to signal reinforcement was auditory rather than visual. Experiments 2 and 3 compared the effects of signaling reinforcement with either a localized or a diffuse light on responding maintained by VR schedules of reinforcement. Elevated response rates were observed with the diffuse stimulus, but the localized stimulus failed to produce such potentiation. Experiment 3 also examined the conditioned reinforcing power of localized and diffuse visual stimuli. These results are discussed with reference to (1) theories of selective association and sign tracking and (2) their implications for current theories of signaling reinforcement.     

Effect of intertrial interval on variable-interval discrete-trial barpressing

In seven experiments, an effect of the intertriai interval (ITI) duration on barpressing by rats was studied. A stimulus signaled a 15-sec variable-interval trial. The first response after the interval elapsed turned the stimulus off and was rewarded with food. Trials were separated by long (about 300 sec) or short (about 10 sec) ITIs. A within-subjects design established that response rate on trials after long ITIs was lower than that after short ITIs (Experiments 1 and 3–7). The effect was not cumulative (the effect of one and five consecutive short ITIs was the same). Response rate after short and long ITIs was the same when a between-subjects design was used (Experiment 2). Response rate was higher after 160-sec ITIs than after 300-sec ITIs, suggesting that the ITI duration at which all longer ITIs are treated the same (i.e., the upper limit) is greater than 160 sec (Experiment 3). When food, the trial stimulus, a novel stimulus, or a familiar stimulus never paired with food, was presented 10 sec before the next trial during some of the long ITIs, response rate on the next trial was similar to that found after 10-sec ITIs (Experiments 4–6). This similarity suggested that these events could mark the start of the ITI. However, the familiar stimulus did so only when it reliably predicted that the next trial would occur after a short interval. The effect of ITI duration on responding was apparently attributable to response latency. Response latency was greater after long ITIs, but once responding began, it was similar after long and short ITIs (Experiment 7).     

Alcohol as a cue for extinction: State dependency produced by conditioned inhibition

A conditioned-emotional-response (CER) paradigm was used in two experiments to evaluate the effects of alcohol on extinction. In both experiments, rats received tone-shock pairings without alcohol and then received extinction trials either with or without alcohol (injections of saline, .75 or 1.5 g/kg ethanol). The high dose of alcohol suppressed baseline barpressing for food reward, but there was only weak evidence (Experiment 2) that it enhanced responding during the tone early in extinction. In both experiments, extinction of CER under the high alcohol dose was found to be state dependent, that is, post-extinction tests after saline injection showed a reinstatement of suppression to the tone. Experiment 2 indicated that this effect could be attributed to alcohol’s becoming a conditioned inhibitory stimulus as a result of its association with extinction. This supports the general suggestion that situational stimuli normally become inhibitory during the course of simple extinction and may have implications for the role that state-dependent learning plays in drug dependence.     

The influence of context-reinforcer associations on instrumental performance

In each of two experiments, rats were trained to press the lever in a Skinner box, food reinforcement being available on a variable-interval 60-sec schedule (VI 60). There followed an “exposure phase” for which the levers were removed from the boxes, and then a final test with the levers replaced to assess the effects of the intervening treatment on instrumental responding. Experiment 1 showed that simple exposure to the box reduced the vigor of instrumental performance in comparison with a condition in which food was made available during the exposure phase. Animals which received no exposure treatment also showed a relatively high rate of response. Experiment 2 demonstrated that an exposure treatment in which the occurrence of food is signaled by a light stimulus also leads to a decline in instrumental responding. These results are held to support the notion that associations between the context and the reinforcer serve to energize appetitive instrumental behavior.     

Selective reinstatement of instrumental performance depends on the discriminative stimulus properties of the mediating outcome

We conducted three experiments to investigate the associative structure underlying the reinstatement of instrumental performance after extinction. In each experiment, rats were initially rewarded on two responses with different outcomes. At test, both responses were extinguished in order to assess the impact of a single noncontingent outcome delivery on response selection. Experiment 1 found evidence of outcome-selective reinstatement (i.e., more responses were performed on the lever that was trained with the reinstating outcome than on the other lever). Experiment 2 demonstrated that the outcome’s capacity to reinstate performance was not affected by a reduction in its motivational value. Experiment 3 found evidence that the reinstating outcome selectively retrieved the response it signaled rather than the response it followed during training. Together, these findings are consistent with the view that instrumental reinstatement depends on the discriminative stimulus properties of the reinstating outcome.     

Reacquisition following extinction in appetitive conditioning

In four experiments utilizing an appetitive conditioning preparation, reacquisition of conditioned responding was found to occur both rapidly and slowly following extinction. In Experiment 1, acquisition of responding to a tone that had been conditioned and extinguished occurred more rapidly than acquisition in either a group that received equivalent exposure to the food unconditioned stimulus or a “rest” control group that received only exposure to the apparatus in the first two phases. However, reacquisition was impaired relative to acquisition in a “learning-experienced” group that had previously received conditioning and extinction with a different stimulus. Experiments 2 and 3 produced similar results, but also found that high responding during reacquisition was confined to trials that followed reinforced, rather than nonreinforced, trials. Experiment 4, in which very few initial conditioning trials were used, produced reacquisition that was slow compared with both learning-experienced and rest controls. The results are consistent with a role for sequential learning: Reacquisition is rapid when animals have learned that reinforced trials signal other reinforced trials.     

Instrumental response topographies of rats

Experiments 1, 2, and 3 showed that food-deprived rats responding for food pellets made significantly more long-duration leverpresses than water-deprived rats responding for water drops. These experiments further showed that this difference in instrumental response topography is long-lived, and depends neither upon idiosyncrasies of the experimental chamber nor upon severity of deprivation conditions. In Experiment 4, food-deprived rats responding for food pellets made significantly more long-duration leverpresses than did either food- or water-deprived rats responding for sucrose solution. Human judges in Experiment 5 were able to correctly identify instrumental leverpress responses by rats as being for food or water based solely on previous viewings of other rats drinking water or eating food pellets. It appears that instrumental response topographies in rats vary depending principally upon the reinforcer received, and that these instrumental response topographies resemble consummatory response topographies.     

Secondary extinction in Pavlovian fear conditioning

Pavlov (1927/1960) reported that following the conditioning of several stimuli, extinction of one conditioned stimulus (CS) attenuated responding to others that had not undergone direct extinction. However, this secondary extinction effect has not been widely replicated in the contemporary literature. In three conditioned suppression experiments with rats, we further explored the phenomenon. In Experiment 1, we asked whether secondary extinction is more likely to occur with target CSs that have themselves undergone some prior extinction. A robust secondary extinction effect was obtained with a nonextinguished target CS. Experiment 2 showed that extinction of one CS was sufficient to reduce renewal of a second CS when it was tested in a neutral (nonextinction) context. In Experiment 3, secondary extinction was observed in groups that initially received intermixed conditioning trials with the target and nontarget CSs, but not in groups that received conditioning of the two CSs in separate sessions. The results are consistent with the hypothesis that CSs must be associated with a common temporal context during conditioning for secondary extinction to occur.     

Control of conditional discrimination performance by CS-evoked event representations

The ability of visual CSs previously paired with flavored substances to substitute for those substances as conditional discriminative cues was examined in two Pavlovian appetitive conditioning experiments using rat subjects. In Experiment 1, a visual stimulus was first paired with the delivery of a sucrose solution. Then the rats were trained in conditional discrimination tasks in which sucrose delivery alone served as a conditional cue signaling whether or not a subsequent tone would be reinforced with food pellets. Subjects rapidly acquired discriminative performance to the tones, especially in a feature-negative condition in which sucrose delivery signaled when the tone would not be reinforced. In a subsequent test in which neither food nor sucrose was delivered, presentation of the visual CS also controlled discriminative performance to subsequently presented tones. Experiment 2 showed the ability of a visual CS to substitute for a flavored substance as a conditional cue to be highly stimulus specific. Experiment 2 also showed that a flavored substance was less effective as a conditional cue when it was made to be expected by preceding it with a previously associated visual signal than when it was made to be surprising by preceding it with a visual stimulus signaling another flavored liquid. These results indicate that CS-evoked representations of events can substitute for those events themselves in the control of previously established conditional discrimination performance.