Tests of the conditioned reinforcement value of sequential stimuli in pigeons

Egger and Miller (1962) hypothesized that the conditioned reinforcing value of stimuli depends on their information value. Egger and Miller and others have tested this hypothesis by comparing the conditioned reinforcing value of S1 and S2 following S1-S2-reward training. However, none of these experiments have controlled for differential generalization of conditioned reinforcement value from training to comparison tests. That is, the S1 cue pattern during the conditioned reinforcement tests has been very similar to the S1 cue pattern of training, while the training and test S2 cue patterns have been quite dissimilar. In Experiment 1, pigeons in a procedure unconfounded by differential generalization produced S2 reliably more frequently than S1, and pigeons in a confounded procedure produced S1 somewhat more frequently than S2. A significant groups × stimuli interaction was attributed to differential stimulus generalization from training to test for S1 and S2 in the confounded condition. In Experiment 2, pigeons in an unconfounded procedure again produced S2 reliably more frequently under a different testing procedure. The results are interpreted as demonstrating that, following S1-S2-food training trials, S2 is the more effective conditioned reinforcer in unconfounded conditions. A reconceptualization of the information hypothesis is shown to be consistent with these results.     

Context effects on conditioning,extinction, and reinstatement in an appetitive conditioning preparation

Three experiments with rat subjects examined the effects of contextual stimuli on performance in appetitive conditioning. A 10-sec tone conditioned stimulus (CS) was paired with a food-pellet unconditioned stimulus (US); conditioning was indexed by the observation of headjerking, a response of the rat to auditory stimuli associated with food. In Experiment 1, a context switch following initial conditioning did not affect conditioned responding to the tone; however, when the response was extinguished in the different context, a return to the original conditioning context “renewed” extinguished responding. These results were replicated in Experiments 2 and 3 after equating exposure to the two contexts (Experiment 2) and massing the conditioning and extinction trials (Experiment 3). The results of Experiment 1 also demonstrated that separate exposure to the US following extinction reinstates extinguished responding to the tone; this effect was further shown to depend at least partly on presenting the US in the context in which testing is to occur (Experiments 2 and 3). Overall, the results are consistent with previous data from aversive conditioning procedures. In either appetitive or aversive conditioning, the context may be especially important in affecting performance after extinction.     

When more is less: Extending training of the blocking association following compound training attenuates the blocking effect

Three conditioned lick suppression experiments with rats were performed to assess the influence, following compound training of two stimuli (A and X) with the same outcome (AX-O trials), of extending training of the blocking association (i.e., A-O) on responding to the target stimulus (X) at test. In Experiment 1, backward blocking was attenuated when the blocking association was extensively trained. Experiment 2 showed that forward blocking was also attenuated by extensive further training of the blocking association following the AX-O trials. Experiment 3 contrasted candidate explanations of the results of Experiments 1 and 2 and demonstrated that these results are consistent with the framework of the extended comparator hypothesis (Denniston, Savastano, & Miller, 2001).     

Microanalysis of variable-interval performance during stimulus compounding

Additive summation is observed when more responses are emitted to the simultaneous presentation (tone-plus-light) of independently conditioned stimuli (tone and light) than to either stimulus presented alone. The current experiment sought to determine if this increased rate during tone-plus-light was a function of a new modal interresponse time (IRT) or a differential mixing of pauses with a modal IRT characteristic of the responding in tone and light alone. Three rats were trained on a three-component multiple schedule where tone and light were each associated with a variable-interval 30-sec schedule while a variable-interval 100-sec operated in the simultaneous absence of these stimuli, tone-off and light-out. Baseline response rates were 2–4 times as high in tone or light as in their absence. In testing, more responses were emitted to tone-plus-light than to tone or light by all animals, but the modal IRT was in the 0.2–0.4-sec IRT bin for all test conditions. Tone-plus-light controlled fewer long IRT values and more responses in the short modal category than tone or light alone. These results support the response mixing hypothesis of stimulus control; i.e., no “new” behavior was observed during the novel combination of stimulus elements, only a mixture of previously reinforced behavior patterns in different proportions.     

Alterations in the memory code for temporal events induced by differential outcome expectancies in pigeons

The effect of differential outcome expectancies on memory for temporal and nontemporal information was examined. Pigeons were trained to match short (2-sec) and long (8-sec) sample durations to red and green comparison stimuli, and vertical and horizontal lines to vertical and horizontal comparison stimuli. In Experiment 1, one differential outcome (DO) group received food for correct choices on short-sample trials, whereas another received food for correct choices on long-sample trials. On line-orientation trials, half of each DO group received food for correct responses following vertical samples, whereas the other half received food for correct responses following horizontal samples. Overall retention was greater in the DO groups than in a nondifferential (NDO) group that received either food or no food for correct responses on a random half of all trials. Furthermore, although the NDO group displayed a choose-short bias for temporal samples, both DO groups displayed equivalent biases to select the comparison stimulus associated with food. In Experiment 2, differential outcome expectancies were extinguished off-baseline. Subsequently, in the first nondifferential outcome test session, the. DO groups performed less, accurately than the NDO group. These findings indicate that temporal samples are not retrospectively and analogically coded when they are differentially associated with food and no food. Instead, they are remembered in terms of the corresponding outcome expectancies.     

Effect of intertrial interval on variable-interval discrete-trial barpressing

In seven experiments, an effect of the intertriai interval (ITI) duration on barpressing by rats was studied. A stimulus signaled a 15-sec variable-interval trial. The first response after the interval elapsed turned the stimulus off and was rewarded with food. Trials were separated by long (about 300 sec) or short (about 10 sec) ITIs. A within-subjects design established that response rate on trials after long ITIs was lower than that after short ITIs (Experiments 1 and 3–7). The effect was not cumulative (the effect of one and five consecutive short ITIs was the same). Response rate after short and long ITIs was the same when a between-subjects design was used (Experiment 2). Response rate was higher after 160-sec ITIs than after 300-sec ITIs, suggesting that the ITI duration at which all longer ITIs are treated the same (i.e., the upper limit) is greater than 160 sec (Experiment 3). When food, the trial stimulus, a novel stimulus, or a familiar stimulus never paired with food, was presented 10 sec before the next trial during some of the long ITIs, response rate on the next trial was similar to that found after 10-sec ITIs (Experiments 4–6). This similarity suggested that these events could mark the start of the ITI. However, the familiar stimulus did so only when it reliably predicted that the next trial would occur after a short interval. The effect of ITI duration on responding was apparently attributable to response latency. Response latency was greater after long ITIs, but once responding began, it was similar after long and short ITIs (Experiment 7).     

Temporal integration in Pavlovian appetitive conditioning in rats

We used an appetitive sensory preconditioning procedure to investigate temporal integration in rats in two experiments. In Phase 1, rats were presented with simultaneous compound trials on which a 10-sec conditioned stimulus (CS) X was embedded within a 60-sec CS A. In Group Early, CS X occurred during the early portion of CS A, whereas in Group Late, CS X occurred during the latter portion of CS A. In Phase 2, CS X was paired simultaneously with sucrose. On a subsequent test with CS A, the rate of magazine entries peaked during the early portions of the stimulus in Group Early and during the latter portions of the stimulus in Group Late (Experiments 1 and 2). Similar response peaks were not observed on tests with a control stimulus that had been presented in compound with a stimulus that did not signal reward (Experiment 2).     

Fear-potentiated startle using three conditioned stimulus modalities

The capacities of three different conditioned stimulus modalities (light, noise, and airflow produced by a fan) to produce fear-potentiated startle were evaluated. Previous experiments have shown that following either light-shock or noise-shock pairings, both the light and noise conditioned stimuli acquire the ability to potentiate the acoustically elicited startle response in rats (the so-calledfear-potentiated startle effect). In Experiment 1, the ability of airflow produced by a fan to act as a conditioned stimulus was investigated. Rats were given either paired or impaired fan-shock training followed by a test for fear-potentiated startle. The fan conditioned stimulus potentiated startle only in the group given explicit fan-shock pairings. In Experiment 2, we evaluated the discriminability of the three conditioned stimulus modalities. Rats were given light, noise, or fan-shock pairings and were subsequently tested for fear-potentiated startle with the trained conditioned stimulus as well as the two remaining novel conditioned stimuli. Only the trained conditioned stimulus potentiated startle. These results show that fear-potentiated startle can be produced with three discriminable conditioned stimulus modalities, allowing the future use of fear-potentiated startle in the investigation of higher order conditioning phenomena.     

Changes in inhibition during differential eyeblink conditioning with increased training

Three experiments examined inhibitory learning in rats, using Pavlovian and differential inhibitory eyeblink conditioning procedures. Experiment 1 was designed to compare summation and retardation effects following Pavlovian conditioned inhibition (A1/XA) or differential inhibition (A1/X) training using auditory and visual conditioned stimuli (CSs). After ten 100-trial sessions of training, both Pavlovian conditioned inhibition and differential inhibition produced a retardation effect. However, a summation effect was obtained only for rats given Pavlovian conditioned inhibition training. Experiment 2 showed that increasing differential inhibition training to twenty 100-trial sessions produced summation and retardation effects. In Experiment 3, rats were trained with either ten or twenty 100-trial sessions of intramodal inhibitory training with two tone CSs (2 kHz vs. 8 kHz). Summation and retardation effects were obtained after only 20 sessions of differential conditioning. The findings indicate that extensive training is needed to establish conditioned inhibition with intermodal or intramodal differential conditioning.     

Temporal encoding in trace conditioning

Conditioned lick suppression in rats was used to explore the role of timing in trace conditioning. In Experiment 1, two groups of rats were exposed to pairings of a CS (CS1) with a US, under conditions in which the interstimulus interval (ISI) that separated CS1 offset and US onset was either 0 or 5 sec. Two additional groups were also exposed to the same CS1→US pairings with either a 0 or a 5-sec ISI, and then received “backward” second-order conditioning in which CS1 was immediately followed by a novel CS2 (i.e., CS1→CS2). A trace conditioning deficit was observed in that the CS1 conditioned with the 5-sec gap supported less excitatory responding than the CS1 conditioned with the 0-sec gap. However, CS2 elicited more conditioned responding in the group trained with the 5-sec CS1-US gap than in the group trained with the 0-sec CS1-US gap. Thus, the CS1-US interval had inverse effects on first- and second-order conditioned responding. Experiment 2 was conducted as a sensory preconditioning analogue to Experiment 1. In Experiment 2, rats received the CS1?CS2 pairings prior to the CS1→US pairings (in which CS1 was again conditioned with either a 0 or a 5-sec ISI). Experiment 2 showed a dissociation between first- and second-order conditioned responding similar to that observed in Experiment 1. These outcomes are not compatible with the view that differences in responding to CSs conditioned with different ISIs are mediated exclusively by differences in associative value. The results are discussed in the framework of the temporal coding hypothesis, according to which temporal relationships between events are encoded in elementary associations.     

Target-absent controls in blocking experiments with rats

In three between-groups blocking experiments with rats, two concurrent and one forward, several common control procedures were employed: Reinforced trials with the putative blocking stimulus were either omitted entirely (Kamin control), replaced by unsignaled reinforcements (Wagner control), or replaced by reinforced trials with a different stimulus (C1 control). In each experiment, parallel treatments with the target stimulus absent during training served to examine the possibility that differential responding in tests with the target stimulus might be traced solely to differential exposure to the nontarget stimuli. In Experiment 1, responding by a concurrent blocking group during the test was no different than responding by a Kamin control group, and responding by a Wagner control group was greater than that of either of the other groups—a pattern of results, mirrored in the performance of the target-absent groups, that could be attributed to the elevation of contextual excitation by unsignaled reinforcement. In Experiment 2, responding in the test by a concurrent blocking group was no different than that by a C1 control group. In Experiment 3, a finding of less responding by a forward blocking group than by a C1 control group when the target stimulus was present during training, but not when it was absent, provided plausible evidence of blocking.     

Spatiotemporal characteristics of serial CSs and their relation to search modes and response form

In four experiments, we examined how the spatiotemporal proximity to food of the two elements of a serial conditioned stimulus (CS) influenced the pattern of CS-directed versus food-site-directed behavior in rats. Experiment 1 showed that only temporal proximity affected responding when the serial CS consisted of two successive 4-sec presentations of either a spatially near or a spatially far lever (NN or FF). However, Experiment 2 showed that behavior depended markedly on whether rats received a near followed by a far lever (NF) or a far followed by a near lever (FN). Experiment 3 showed that the effects of Experiment 2 could be changed by increasing the duration of the second CS element, and Experiment 4 showed that these changes were not related to previous training. We concluded that behavior produced by the spatiotemporal qualities of the lever elements can be attributed to a mapping between the temporal qualities of the CS elements and an underlying sequence of search modes related to finding food.     

Positive and negative patterning in human classical skin conductance response conditioning

Three experiments on classical differential conditioning of the human skin conductance response to elemental and compound stimuli are reported. Subjects in Experiment 1 received both positive and negative patterning training, followed by either positive or negative patterning transfer tests on new stimuli. In positive patterning, a compound of two stimuli is reinforced and its elements are nonreinforced. In negative patterning, the elements are reinforced and the compound is nonreinforced. Subjects in Experiments 2 and 3 received either positive or negative patterning during training, followed by transfer tests on new stimuli. In Experiment 2, the transfer series began with new elements, after which their compound was presented; in Experiment 3, the new compound was presented first in the transfer series, and then the separate elements were administered. All three experiments provided evidence of the acquisition of positive patterning, while negative patterning was found only in Experiments 2 and 3. Positive patterning transferred to new stimuli, indicating that it was not attributable solely to summation of sub-threshold excitation conditioned to the elements on reinforced compound trials. This finding, coupled with the negative patterning found in Experiments 2 and 3, provided support for the unique cue hypothesis. It was concluded that the assumed unique cue constituted a learned “rule,” and that the actual elemental stimuli were neither perceptually nor otherwise modified during the conditioning process.     

Background stimuli as a reminder after spontaneous forgetting: Role of duration of cuing and cuing-test interval

With a relatively complex maze, reliable forgetting is seen clearly when the training-to-test interval is 25 days. This forgetting is demonstrated by longer time to run the maze and by an increase in the number of errors and retracings from the last training trials to the first test trial. In this case, forgetting is a lapse, not a loss, since performance attains the last training trial level at a subsequent test. Furthermore, a reminder—a 90-sec exposure to background stimuli in the experimental room just prior to the test trial—that does not in itself contain sufficient information to facilitate performance in naive animals, significantly improves maze performance in rats that have “forgotten,” even on the first test trial. Two additional experiments were aimed at assessing the role of time and duration of pretest cuing. In the first experiment, the animals were presented the reminder (90 sec in duration) at different times before the test trial. The results show that this reminder significantly alleviates forgetting only when presented just prior to the test, and is less effective when given 1 or 24 h before the test. In the second experiment, contextual cues, which were presented just prior to testing in all experimental groups, varied in duration. The results showed that (1) animals given the reminder treatment for only 10 sec perform at the same level as controls; (2) cuing for 30 sec and especially for 90 sec alleviates forgetting; and (3) a longer exposure to background stimuli (300 sec) leads to intermediate levels of performance, probably due to a partial extinction of the cue value of these stimuli.     

Dissociation of the effect of reinforcer type and response strength on the force of a conditioned response

A dissociation between the effect of reinforcer type and response strength on the force of the pigeon’s keypeck response was shown in three experiments. In Experiment 1, pigeons were trained to peck two conditioned stimuli, one paired with water and another paired with grain. The pigeons made more forceful pecks for grain than for water and also showed a tendency, albeit an unreliable one, to respond on a higher percentage of food trials than water trials. In Experiment 2, the pigeons from Experiment 1 were satiated with either food or water and were then presented with the two conditioned stimuli in an extinction test. It was found that, regardless of the drive state, the pigeons made more forceful pecks to the stimulus that predicted food than to the stimulus that predicted water. In the thirsty group, however, this difference in force was not accompanied by a difference in the percentage of trials with a response. In Experiment 3, pigeons trained with a single reinforcer pecked more often on instrumentally reinforced trials than on Pavlovian conditioning trials, but there was no difference in the force of the pecks. Taken together, these results imply that differences in response strength cannot account for the difference between the force of food- and water-reinforced pecks. Instead, stimulus-substitution theory may provide the best account of the topography of the two types of pecks.     

Conditioning trial duration affects ethanol-induced conditioned place preference in mice

The present experiments were designed to determine the effect of conditioning trial duration on strength of ethanol-induced conditioned place preference in mice. In a counterbalanced, differential conditioning procedure, DBAI2J mice received four pairings of a distinctive tactile (floor) stimulus with injection of ethanol (2 g/kg); a different floor stimulus was paired with saline. Different groups were exposed to the floor stimuli for 5, 15, or 30 min after injection. Conditioned place preference was inversely related to trial duration, with mice in the 5-, 15-, and 30-mmn groups, spending 83%, 74%, and 66% of their time, respectively, on the ethanol-paired floor during a choice test. This outcome was replicated in a second experiment, which also showed that context familiarity can influence conditioned place preference. In general, these findings suggest that ethanol’s rewarding effect is greatest shortly after injection.     

Outcome-specific conditioned inhibition in Pavlovian backward conditioning

In the present experiments, the outcome specificity of learning was explored in an appetitive Pavlovian backward conditioning procedure with rats. The rats initially were administered Pavlovian backward training with two qualitatively different unconditioned stimulus conditioned-stimulus (US-CS) pairs of stimuli (e.g., pellet --> noise or sucrose --> light), and then the effects of this training were assessed in Pavlovian-to-instrumental transfer (Experiment 1) and retardation-of-learning (Experiment 2) tests. In the transfer test, it was shown that during the last 10-sec interval, the CSs selectively reduced the rate of the instrumental responses with which they shared a US, relative to the instrumental responses with which they did not share a US. The opposite result was obtained when the USs (in the absence of the CSs) were presented noncontingently. In the retardation test, conditioned magazine approach, responding to the CSs was acquired more slowly when the stimulus-outcome combinations in the backward and the forward conditioning phases were the same, as compared with when they were reversed. These results are collectively in accord with the view that Pavlovian backward conditioning can result in the formation of outcome-specific inhibitory associations. Alternative views of backward conditioning are also examined.     

Directed forgetting in monkeys

Based on several recent demonstrations of a directed forgetting effect in pigeons, three experiments were carried out in an attempt to demonstrate directed forgetting in three squirrel monkeys. During initial training with a delayed matching-to-sample procedure, retention tests were always given for sample stimuli followed by remember cues (R-cues) and were always omitted for sample stimuli followed by forget cues (F-cues). Retention of F-cued items was tested on probe trials after initial training. The first two experiments examined the effects of R- and F-cues on memory for slide-projected pictures, with different pictures used on each trial of a session. In Experiment 1, a complex design was used in which one or two sample pictures were presented on each trial; when two pictures were presented, both could be R-cued or F-cued, or one could be R-cued and the other F-cued. A simpler design was used in Experiment 2, with only single pictures presented as sample stimuli and half the trials within a session R-cued and the other half F-cued. In both of these experiments, no differential retention of R- and F-cued stimuli was found, even at a retention interval as long as 16 sec. In Experiment 3, a series of studies was performed to test for directed forgetting when only two sample stimuli were used repeatedly throughout training and testing. With two pictures as sample stimuli, clear evidence of directed forgetting was found in Experiment 3b. It is suggested that the directed forgetting effect may arise only when a small set of sample stimuli is used.     

Systematic errors in pigeons’ memory for event duration: Interaction between training and test delay

Pigeons trained to choose different stimuli following short- and long-duration signals make disproportionately more “short” choices (i.e., “choose-short errors”) following an increase in the retention interval and more “choose-long errors” following a decrease in this delay. The present experiment provided a systematic investigation of how these selective errors depend on the relationship between the training delay and the test delay. Pigeons were first trained with a 0-sec delay between the signal (2- or 8-sec food presentations) and the choice stimuli (red- and blue-lit keys). On subsequent test trials with 5- and 10-sec delays, choose-short errors predominated. Next, the birds were trained with a constant 10-sec delay and then tested with shorter or longer delays on some trials. The birds now responded accurately and without selective errors at the 10-sec training delay, but made choose-long errors at shorter delays and choose-short errors at longer delays. Finally, the birds were trained with a constant 20-sec delay and then tested with shorter and longer delays. Choose-long errors again appeared at shorter test delays, choose-short errors at longer test delays, and no differential errors at the 20-sec training delay. The selectivity of these errors generally increased with the absolute difference between the training and test delay. Theoretical implications of these results are discussed.     

A behavior systems view of responding to probe stimuli during an interfood clock

Two experiments used a behavior systems approach to relate the form of responses during an interfood clock to the temporal distance of the individual clock stimuli to food. Stimuli proximate to food should better control a focal search mode and related responses, whereas stimuli temporally distant from food should better control a general search mode and related responses. Experiment 1 conditioned two groups of rats with a sequence of four equal-length 12-sec clock stimuli that terminated with food and then tested for the conditioning of a general search mode by presenting an unconditioned moving probe stimulus (either a rolling ball bearing or a rotating mechanical door) during each of the clock stimuli. Consistent with a behavior systems view, contact with the ball bearing was markedly greater during a clock stimulus distant from food. The absence of similar differential contact of the door across the clock stimuli showed that the effect was specific to the ball bearing rather than a general response to stimulus dimensions of movement and sound. Experiment 2 showed that the general search mode was controlled by the clock stimulus rather than the passage of time.