Tests of the conditioned reinforcement value of sequential stimuli in pigeons

Egger and Miller (1962) hypothesized that the conditioned reinforcing value of stimuli depends on their information value. Egger and Miller and others have tested this hypothesis by comparing the conditioned reinforcing value of S1 and S2 following S1-S2-reward training. However, none of these experiments have controlled for differential generalization of conditioned reinforcement value from training to comparison tests. That is, the S1 cue pattern during the conditioned reinforcement tests has been very similar to the S1 cue pattern of training, while the training and test S2 cue patterns have been quite dissimilar. In Experiment 1, pigeons in a procedure unconfounded by differential generalization produced S2 reliably more frequently than S1, and pigeons in a confounded procedure produced S1 somewhat more frequently than S2. A significant groups × stimuli interaction was attributed to differential stimulus generalization from training to test for S1 and S2 in the confounded condition. In Experiment 2, pigeons in an unconfounded procedure again produced S2 reliably more frequently under a different testing procedure. The results are interpreted as demonstrating that, following S1-S2-food training trials, S2 is the more effective conditioned reinforcer in unconfounded conditions. A reconceptualization of the information hypothesis is shown to be consistent with these results.     

Integration of cardiac responses to serial stimuli after Pavlovian conditioning in rats

The expression of cardiac responses to sequences of two sounds was studied in restrained rats following discriminative trace or delay conditioning. Stimuli paired with a tail shock 10 sec later (CS1) elicited conditioned bradycardia. Unpaired or neutral stimuli (CS0) elicited mostly tachycardia. Rats did not learn to suppress responding to nonreinforced sequences with an interval of 6 sec between sounds. Responses to the second stimulus were significantly augmented following a CS1 stimulus, but not following a CS0 stimulus. Real-time summation of simple responses provided a more complete and quantitative prediction of dual responses than did resetting or facilitation. These results extend the time range over which summation may be observed from less than 2 sec to at least 16 sec. They appear to be inconsistent with models involving competition between unitary representations of stimuli in short-term memory and suggest the existence of multiple stimulus traces with independent time courses.     

Facilitation of appetitive conditioning with naturalistic conditioned stimuli: CS and US factors

Adding limited female cues to a conditioned stimulus (CS) facilitates conditioned male sexual responding. In two experiments, we examined the mechanisms of this facilitation effect. The color of the female cues on the CS was varied in Experiment 1. Similarity between the CS plumage color and the color of the live female (the unconditioned stimulus [US]) could only partially account for the results. The extent to which the facilitation effect represents a specialization of sexual behavior was examined in Experiment 2 by comparing conditioning with either food or copulation as the US. The CSs with female cues elicited more approach and grab responses regardless of which US was used. However, uniquely sexual conditioned responses (mounts and cloacal contacts) were enhanced only when sexual reinforcement served as the US. These findings suggest that the facilitation effect of female cues represents a general feature of appetitive behavior systems.     

The order of presentation of conditioned stimuli during extinction

Three experiments examined the effects of prior extinction with nontarget CSs on a target CS’s rate of extinction and amount of associative loss. After fear was conditioned to several CSs, rats received extinction training with the target CS either before or after extinction with nontarget CSs. In general, neither rate of extinction nor amount of associative change, as measured by reinstatement and relearning, were significantly affected by order of extinction. The present results do not support Rescorla’s interpretation of extinction.     

Behavior patterns in pigeons during autoshaping with an incremental conditioned stimulus

Pigeons were trained on a variant of the autoshaping procedure in which a keylight stimulus of increasing brightness was used to signal the passing of a 30-sec interfood interval (IFI). Key-pecking developed in all subjects within the first session (65 trials). Within trials, pecking began midway through the IFI, increased throughout the remainder, and decreased just before food delivery. Other behavioral stereotypies were also recorded: Low light levels were associated with a retreat to the rear of the test chamber, and medium light levels (during the midportion of the IFI) were associated with high rates of pacing toward and away from the food source. Probe trials revealed that pecking, pacing, and retreat were all under strong stimulus control; that is, when the light was held constant at its lowest or highest brightness, or when the brightness ramp was presented in reverse order, the behavior pattern almost invariably remained tied to stimulus brightness. Results are discussed in terms of associative and nonassociative sources of the form and sequential characteristics of the behavior.     

Effects of context exposure during conditioning on conditioned taste aversions

Rats were used in a conditioned taste aversion procedure in order to examine the effects of context exposure duration during the conditioning sessions on conditioned responding. One flavor was paired with lithium chloride during a long session in one context, whereas another flavor was conditioned during a short session in another context. Testing occurred in the home cage. The results showed that conditioning during short sessions produced strong conditioned taste aversions. Conditioning during long sessions produced strong conditioned taste aversions when the conditioned-stimulus-unconditionedstimulus (CS-US) pairing occurred at the end of the lengthy session. Other results showed that context-US associations were formed during the short duration sessions and that these associations supported conditioned responding to the CS trained in that context. The results are discussed with respect to the different influences that contextual cues can exert on conditioned responding.     

Trial number and compound stimuli temporal relationship as joint determinants of second-order conditioning and conditioned inhibition

Two conditioned lick suppression experiments with rats used feature-negative training (A-footshock trials intermixed with nonreinforced XA presentations) to analyze the role of the number of XA compound presentations and the temporal relationship of the elements within the compound (simultaneous or serial) as determinants of the resulting behavioral control. Second-order conditioning (i.e., excitatory behavioral control by X) was observed to decline as the number of XA compound trials was increased. This decline was more rapid if X and A were presented simultaneously, as opposed to serially (i.e., X before A; Experiment 1). Conditioned inhibition to X, as assessed by a summation test (Experiment 1) and a retardation test (Experiment 2), increased with the number of XA trials and did so more quickly for simultaneous than for serial pairings of X and A. The results help to clarify previously discrepant findings regarding factors that promote excitation versus inhibition with this protocol.     

Real-time factors in the rabbit’s nictitating membrane response to pulsed and serial conditioned stimuli

Conditioning theories and recent real-time models commonly postulate that a reinforcer is signaled by a series of stimuli. In both Pavlovian and operant procedures, serial stimuli have been shown to control the likelihood and timing of responses over intervals of seconds and minutes. The present experiments were conducted to determine whether serial stimuli exercise similar effects over stimulus-reinforcer intervals in the order of hundreds of milliseconds. Such intervals typify those used in conditioning of the rabbit nictitating membrane response. A sequence of four tone pulses (50–100 msec) was used as the CS to assess the effectiveness of serial stimuli. After training with this CS, tests were conducted in which one or more of the pulses were removed. These perturbations of the sequence of stimuli over a 400-msec interval produced large deficits in CR likelihood and smaller alterations in CR timing. The results are discussed with respect to their implications for current real-time models of conditioning, and particularly with respect to their assumptions about the source of internal stimuli, rules for learning, and rules for generating CRs.     

Control of pigeons’ keypecking by the left-right arrangement of stimuli

Control of pigeons’ keypecking by conditionalities in the spatial arrangement of two element stimuli (designated A and B) was investigated. In Experiment 1, reinforcement for keypecking was made conditional upon the left-right location of A and B: Reinforcement was available when A was on the left and B was on the right (AB), but not on BA, AA, or BB trials. The pigeons successfully discriminated the rewarded AB configuration, but only after a stage in which a particular element in a particular location (e.g., A on left) primarily controlled pecking. Experiments 2 and 3 systematically replicated these findings and included controls to discount discrimination of the AB compound on the basis of the temporal order (e.g., A followed by B) rather than the spatial configuration of the elements. During a generalization test in Experiment 4, the elements were presented singly either in the left (AX, BX) or right (XA, XB) positions. As would be expected had the animals learned “A on the left, B on the right is rewarded,” responding on AX trials exceeded that on XA trials, and responding on XB trials exceeded that on BX trials.     

Brain mechanisms for changes in processing of conditioned stimuli in Pavlovian conditioning: Implications for behavior theory

This article is a review of the results of a series of experiments designed to identify brain systems involved in appetitive conditioning of rats. It discusses some of their implications for behavioral theories of learning, especially those that concern changes in processing of conditioned stimuli (CSs). Evidence is presented which suggests that separable brain circuits are involved in (1) the production of CS-dependent conditioned orienting responses, (2) the enhancement of CS associability produced when expectancies about upcoming events are violated, (3) the reduction of CS associability produced when stimuli are consistent predictors of other events or are presented without consequence, and (4) the abilities of CSs to serve as reinforcers for second-order conditioning and to be sensitive to postconditioning changes in the value of the unconditioned stimulus (US). Finally, none of these circuits seems critical for normal acquisition of the most common indicator of Pavlovian conditioning, US-dependent conditioned responses (CRs). Although the independence of brain pathways does not demand independence of behavioral function, clustering of behavioral phenomena on anatomical grounds may provide useful guides for constructing behavior theories.     

Contextual stimuli and memory retrieval in pigeons

Pigeons were given free-operant successive discrimination training in which, on alternate days, two different interdimensional problems were employed (color positive and line angle negative; a different color negative and a different line angle positive). Between days, these problems could be construed as intradimensional ones. For one group, training was conducted on each problem in the presence of the same ambient (contextual) stimuli, while for a second group each problem was trained in a different context. For a third group, these two contexts were randomly related to the problems. Postdiscrimination stimulus generalization gradients showed that peak shifts were obtained in both the same context and the random context groups, but no peak shifts were found for the group which learned each problem in a different context. Results were consistent with Spear’s (1973) treatment of animal memory which attributes retention test performance to the operation of a “context-elicited” retrieval process. An alternative uniprocess conditioning account of these and similar data was also discussed.     

Concept learning by pigeons: Matching-to-sample with trial-unique video picture stimuli

Pigeons were trained to match-to-sample with several new methodologies: a large number of stimuli, computer-drawn color picture stimuli, responses monitored by a computer touch screen, stimuli presented horizontally from the floor, and grain reinforcement delivered onto the picture stimuli. Following acquisition, matching-to-sample concept learning was assessed by transfer to novel stimuli on the first exposure to pairs of novel stimuli. One group (trial-unique), trained with 152 different pictures presented once daily, showed excellent transfer (80% correct). Transfer and baseline performances were equivalent, indicating that the matching-to-sample concept had been learned. A second group (2-stimulus), trained with only two different pictures, showed no evidence of transfer. These results are discussed in terms of the effect of numbers of exemplars on previous failures to find concept learning in pigeons, and the implications of the positive finding from this experiment on abstract concept learning and evolutionary cognitive development.     

The role of test stimuli in matching to compound samples by pigeons

Two pigeons matched to sample in a three-key operant conditioning chamber. In Experiment I, two different kinds of samples were presented on the center key.Element samples were members of one of two sample sets — colors (a red or blue disk) or lines (a vertical or horizontal orientation of a set of white lines). These samples were followed by their respective sample sets on the side keys as comparison stimuli.Compound samples consisted of a set of lines superimposed on a colored disk. Following these samples, either sample set could appear as comparison stimuli. Matching to compound samples was less accurate than matching to element samples. One interpretation is that sharing of attention among elements of a compound sample weakened stimulus control by each element. A different interpretation is that an element sample controlled matching better because it was physically identical to a comparison stimulus whereas a compound sample was not. Experiments II–IV evaluated this “generalization decrement” alternative by testing element- vs. compound sample control with both element and compound comparison stimuli. Irrelevant elements were added to form compound comparison stimuli, some of which were physically identical to a preceding compound sample, but never identical to an element sample. In all experiments, the addition of irrelevant elements of comparison stimuli reduced sample control. However, the generalization decrement hypothesis failed to predict how differences in performance maintained by element and compound samples were affected by different tests of sample control. Matching accuracy appeared to be independently determined by the number of elements in a sample and whether irrelevant elements were present during tests of sample control.     

Simultaneous processing of visual and spatial stimuli in pigeons

Pigeons were trained to symbolically match comparison stimuli to either visual sample stimuli presented on a center key or to spatial sample stimuli presented on side keys. Tests were carried out in which visual and spatial cues were simultaneously presented in compound and short-term memory was probed for either visual or spatial information. Symmetrical interference with the matching of visual and spatial components of compounds was found when the visual and spatial cues were presented on separate keys. However, when visual and spatial cues were superimposed on the same side key, no interference was observed relative to element control tests. Discussion of these findings focuses on accounts in terms of limited processing capacity, coding decrement, and receptor orientation mechanisms.     

Reinstatement of postresponse stimuli prior to reward in delayed-reward discrimination learning by pigeons

Delayed-reward learning in pigeons was examined using a simultaneous red-green visual discrimination task in which the conditions during the delay interval were varied between groups. The nondifferential group received training in which the stimulus present during the 1-min delay was the same following a peck on the correct and incorrect colors. The other three groups received 1-min delay training in which different stimuli occurred in the delay interval following correct and incorrect choices. The differential group received continuous, differential stimuli during the delay. The reinstatement group received the differential stimuli in the 10 sec immediately following the choice and during the last 10 sec of the delay. The reversedcue group was treated in the same way, except that the 10-sec delay stimulus immediately following an incorrect response was also presented for 10 sec prior to reward on correct choices, and the stimulus following a correct response also occurred 10 sec before nonreward on incorrect choices. Nondifferential birds failed to learn the discrimination, while differential and reinstatement birds learned it readily. The reversed-cue birds learned to choose the incorrect stimulus. Differential and reinstatement birds showed no decrement in performance when the delay was increased to 2 min. These findings suggest that similarity of prereward and postresponse delay stimuli controls choice responding in long-delay learning, a finding compatible with both memorial and conditioned reinforcement interpretations.     

Outcome-specific conditioned inhibition in Pavlovian backward conditioning

In the present experiments, the outcome specificity of learning was explored in an appetitive Pavlovian backward conditioning procedure with rats. The rats initially were administered Pavlovian backward training with two qualitatively different unconditioned stimulus conditioned-stimulus (US-CS) pairs of stimuli (e.g., pellet --> noise or sucrose --> light), and then the effects of this training were assessed in Pavlovian-to-instrumental transfer (Experiment 1) and retardation-of-learning (Experiment 2) tests. In the transfer test, it was shown that during the last 10-sec interval, the CSs selectively reduced the rate of the instrumental responses with which they shared a US, relative to the instrumental responses with which they did not share a US. The opposite result was obtained when the USs (in the absence of the CSs) were presented noncontingently. In the retardation test, conditioned magazine approach, responding to the CSs was acquired more slowly when the stimulus-outcome combinations in the backward and the forward conditioning phases were the same, as compared with when they were reversed. These results are collectively in accord with the view that Pavlovian backward conditioning can result in the formation of outcome-specific inhibitory associations. Alternative views of backward conditioning are also examined.     

Short-term memory for visual and auditory stimuli in pigeons

Separate groups of pigeons were trained to perform symbolic delayed matching to sample with auditory and visual sample stimuli. For animals in the auditory group, ambient tones that varied in frequency served as sample stimuli; for animals in the visual group, ambient red and green lights served as sample stimuli. In both cases, the sample stimuli were mapped onto the yellow and blue comparison stimuli presented on left and right pecking keys. In Experiments 1 and 2, it was found that visual and auditory delayed matching were affected in the same ways by several temporal variables: delay, length of exposure to the sample stimulus, and intertrial interval. In Experiments 3, 4A, and 4B, a houselight presented during the delay interval strongly interfered with retention in both visual and auditory groups, but white noise presented during the delay had little effect in either group. These results seem to be more in line with a prospective memory model, in which visual and auditory sample stimuli are coded into the same instructional memories, than with a model based on concepts of retrospective memory and modality specificity.     

Intertrial unconditioned stimuli differentially impact trace conditioning

Three experiments assessed how appetitive conditioning in rats changes over the duration of a trace conditioned stimulus (CS) when unsignaled unconditioned stimuli (USs) are introduced into the intertrial interval. In Experiment 1, a target US occurred at a fixed time either shortly before (embedded), shortly after (trace), or at the same time (delay) as the offset of a 120-s CS. During the CS, responding was most suppressed by intertrial USs in the trace group, less so in the delay group, and least in the embedded group. Unreinforced probe trials revealed a bell-shaped curve centered on the normal US arrival time during the trace interval, suggesting that temporally specific learning occurred both with and without intertrial USs. Experiments 2a and 2b confirmed that the bulk of the trace CS became inhibitory when intertrial USs were scheduled, as measured by summation and retardation tests, even though CS offset evoked a temporally precise conditioned response. Thus, an inhibitory CS may give rise to new stimuli specifically linked to its termination, which are excitatory. A modification to the microstimulus temporal difference model is offered to account for the data.     

Differential second-order aversive conditioning using contextual stimuli

Two experiments were conducted to determine if contextual stimuli used as S₂ in a higher-order differential conditioning procedure would control the performance of rats. Discrete stimuli were first paired with footshock in a separate training context. During second-order training, a shock-associated discrete stimulus was presented in one of two discriminable observation chambers. Over 4 days of training, subjects engaged in more freezing in the context associated with an excitatory discrete S₁, relative to a context in which no discrete stimulus, or a stimulus that had been explicitly unpaired with shock delivery, was presented. After acquisition of the second-order discrimination, animals were returned to the original training context where they received a “signaled inflation” treatment designed to change the current value of S₁, and the US. This postconditioning manipulation did not selectively affect performance of defensive freezing or conditional analgesia in S₂.     

The insensitivity of schedule-induced polydipsia to conditioned taste aversions: Effect of amount consumed during conditioning

In Experiment 1, animals poisoned following schedule-induced or prandial-induced saccharin consumption subsequently showed identical aversions to saccharin when tested under water deprivation. In Experiment 2, animals conditioned to avoid saccharin to similar levels under water deprivation were differentially affected when saccharin was subsequently presented on the baselines of schedule-induced and prandial-induced drinking. Together, these data indicate that the differential effects observed on schedule-induced and prandial-induced drinking when animals are poisoned following consumption under these two schedules do not reflect the differential acquisition of taste aversions, but instead reflect the differential tendencies to drink induced by the spaced and massed feedings.