Differential outcome expectancies and directed forgetting effects in pigeons

Pigeons were trained in a two-choice delayed matching-to-sample task with red and green hues. A brief postsample cue (a vertical or horizontal line) signaled whether the comparison stimuli would be presented or omitted on each trial. Comparison stimuli were always presented following the remember (R) cue, but never following the forget (F) cue or no-cue trials. One group of birds, the differential outcome (DO) group, received reinforcement with a probability of 1.0 for correct responses following one sample stimulus and a probability of 0.2 for correct responses following the other sample stimulus. The nondifferential outcome (NDO) group received reinforcement with a probability of 0.6 for correct responses to either stimulus. The effect of postsample cues was greater for the DO group than for the NDO group. Relative to the NDO group, the DO group displayed higher accuracy on R-cue trials and lower accuracy on F- and no-cue trials. Both tendencies contributed to the enhanced cue effectiveness obtained in the DO group. The results indicate that outcome expectancies are subject to maintenance rehearsal, which comes under the control of postsample R and F cues. They also suggest that maintenance rehearsal may be easier to sustain under DO conditions than under NDO conditions when a memory test is anticipated, but that it may be easier to terminate maintenance rehearsal under DO conditions when a memory test is not anticipated. The results are inconsistent with the assumption that the rehearsal of outcome expectancies is automatic.     

Some temporal factors affecting conditional discrimination

Pigeons acquired a serial conditional discrimination in which the onset of one of two colors (the instructional cue) on the center key preceded the onset of a white light (the trial cue) on one of two side keys. An autoshaping preparation was employed, in which food was delivered depending upon the color-side combination. Five groups of birds were studied at instructional cue durations of either 30 or 60 sec, and trial cue durations of 3, 6, or 12 sec. These temporal parameters allowed for different ratios of the instructional stimulus duration (I) to the trial stimulus duration (T), while keeping the absolute duration of the instructional stimulus constant, and for different absolute durations of the instructional stimulus, while keeping the I/T ratio constant. These manipulations were studied with either a 30 or a 60-sec cycle (the interval between the onset of the intertriai interval and the offset of the trial cue), thus permitting examination of the cycle duration to trial duration ratios as well. The results showed that the larger the value of I relative to that of T, the greater the final level of accuracy; this implicates the I/T ratio as a controlling variable. In contrast, the larger the cycle duration (C) relative to T, the greater the rate of responding to the trial stimulus, which is consistent with previous findings in autoshaping studies. These results suggest that whereas the C/T ratio directly influences response rate, the I/T ratio affects accuracy in a serial conditional discrimination.     

Intrinsic reinforcing properties of putatively neutral stimuli in an instrumental two-lever discrimination task

Four experiments examined the influence of a stimulus presented after one response in a two-lever choice task. In Experiment 1, food-deprived rats trained on a concurrent variable-interval extinction schedule responded more often on the extinction lever when such responding periodically produced a visual stimulus than when it did not. In Experiments 2 and 3, a similar signal-induced enhancement effect was found even when food was delivered randomly with respect to responding on both levers or when no food was presented. In Experiment 4, a response-contingent visual stimulus elevated responding to the lever on which it was presented, but an auditory cue suppressed responding. These findings indicate that visual stimuli may possess intrinsically reinforcing properties for rats.     

Control of conditional discrimination performance by CS-evoked event representations

The ability of visual CSs previously paired with flavored substances to substitute for those substances as conditional discriminative cues was examined in two Pavlovian appetitive conditioning experiments using rat subjects. In Experiment 1, a visual stimulus was first paired with the delivery of a sucrose solution. Then the rats were trained in conditional discrimination tasks in which sucrose delivery alone served as a conditional cue signaling whether or not a subsequent tone would be reinforced with food pellets. Subjects rapidly acquired discriminative performance to the tones, especially in a feature-negative condition in which sucrose delivery signaled when the tone would not be reinforced. In a subsequent test in which neither food nor sucrose was delivered, presentation of the visual CS also controlled discriminative performance to subsequently presented tones. Experiment 2 showed the ability of a visual CS to substitute for a flavored substance as a conditional cue to be highly stimulus specific. Experiment 2 also showed that a flavored substance was less effective as a conditional cue when it was made to be expected by preceding it with a previously associated visual signal than when it was made to be surprising by preceding it with a visual stimulus signaling another flavored liquid. These results indicate that CS-evoked representations of events can substitute for those events themselves in the control of previously established conditional discrimination performance.     

A bowed serial-position function in habituation of sequential stimuli

The effect of serial position was examined in a study of habituation of the vasoconstrictive orienting response in rabbits. In a procedure analogous to a recognition memory test, subjects received multiple exposures to an invariant list of stimuli before their response was assessed to an isolated item that had consistently occupied either the first, middle, or last position in the list. A reliably bowed serial-position function was obtained, with the least habituation (poorest retention) occurring for the item that had occupied the middle position in the list. The finding of both a primacy and a recency effect parallels the most general finding in studies of human memory and encourages a compatible account of habituation in infrahuman subjects. The result was discussed in terms of the opportunities for differential rehearsal of list items and contextual cues at different positions in the list.     

Establishing a forget cue in pigeons using the intratrial interference procedure

Pigeons were trained in an intratrial interference preparation in which a horizontal or vertical line was presented for 1 sec immediately following termination of a sample (red or green). Two samples were presented successively on interference trials. Choice of the comparison corresponding to the second (target) sample was designated correct and was reinforced, and choice of the comparison corresponding to the first (interfering) sample was designated incorrect and was not reinforced. Control trials involved the presentation of a single, target sample. A horizontal line was presented upon termination of an interfering sample, and a vertical line was presented upon termination of a target sample. The results of three experiments led to the conclusion that the horizontal line acquired and capacity to reduce postperceptual processing (rehearsal) of information derived from an immediately preceding sample stimulus. These findings include (1) convergence of accuracy on control and interference trials as training progressed, (2) a reduction in accuracy on control and especially on interference trials when the correlation between sample type (interfering or target) and cue type (horizontal or vertical) was reduced to zero, (3) higher accuracy (i.e., less interference) when the horizontal rather than the vertical line followed the interfering sample, and (4) higher accuracy on single-sample trials when the vertical rather than the horizontal line followed sample presentation.     

The effects of illumination on the acquisition of delayed matching-to-sample

Illumination effects during steady-state performance of discrimination tasks in animals have been well documented, whereas research on illumination effects during acquisition has been largely ignored. Exceptions to this rule are Wasserman’s (1973) autoshaping experiments and Maki’s (1979) successive discrimination experiment. The present experiment investigated the effects of illumination changes on acquisition of a conditional discrimination—delayed matching-to-sample (DMTS). Pigeons were used in a between-groups design which factorially varied house-light illumination, on or off, during the presentations of DMTS stimuli, the delay interval, and the intertrial interval (ITI). DMTS performance over five blocks of sessions was the dependent variable. The major result was the three-way interaction of sessions, the intertrial interval, and the DMTS stimuli. Constant illumination resulted in the highest discrimination ratios over the last four blocks of sessions. A constant dark condition did not differ from a condition with dark ITIs and illuminated stimulus presentations or from a condition with illuminated ITIs and dark stimulus presentations. The proffered explanation of these data emphasizes the disruptive effects of stimulus changes and Wasserman’s (1973) cue localization hypothesis. The loci of the stimulus change and cue localization effect are suggested to be either at the beginning of a trial or at the end of a trial. A pretrial account emphasizes the role of stimulus changes on the encoding of the sample stimulus, and a posttrial account emphasizes the role of stimulus changes during consolidation processing.     

Directed forgetting effects in pigeons: Remember cues initiate rehearsal

Pigeons were trained in a two-choice delayed matching-to-sample task with red and green hues. A brief postsample cue (a vertical or horizontal line) signaled whether the comparison stimuli would be presented or omitted on each trial. Comparison stimuli were always presented following the remember-cue (R-cue) trial, but never following the forget-cue (F-cue) and no-cue trials. In Experiment 1, matching accuracy on F-cue and no-cue trials was equivalent and was considerably inferior to accuracy on R-cue trials. In Experiment 2, the placement of the postsample cue was manipulated. Matching accuracy decreased as the R cue was delayed in the retention interval, but performance in the F-cue condition was not affected. These data indicate that the no-cue condition can function as an implicit F cue and that the R cue can function to initiate and maintain rehearsal.     

Discriminated avoidance learning and reversal by goldfish in a shuttlebox using a linear presentation procedure

Goldfish, trained in a shuttlebox with cues and shock controlled by a linear presentation procedure, learned to control prevailing cue states and shock, and thus to discriminatively avoid shock presentation. The linear presentation procedure, adapted for the shuttlebox from the sequence of cue presentation occurring in the Y-maze, utilized three cues on each trial, a trial-start (TS) cue, a shock-paired (S?) cue, and an unpaired (S+) cue. At trial onset (TS cue), the goldfish had 10 sec to respond. The first response produced, via response-contingent programming, either the shock-paired (S?) or unpaired (S+) cue, and subsequent responses produced alternation of S+ and S?. The shock was omitted only if the S+ cue state prevailed 10 sec after trial onset. AU other cue states (TS and S?) were paired with shock. Goldfish learned to respond and control the prevailing cue state so that the S+ cue prevailed at 10 sec posttrial onset for a variety of different color-cue combinations and also learned to reverse their originally learned cue preference when the color cues were reversed. The linear presentation procedure represents an alternative discrimination learning procedure that appears to be free of the interpretational problems encountered in training goldfish in the shuttlebox apparatus with other one- and two-stimulus procedures.     

Flexible coding of temporal information by pigeons: Event durations as remember and forget cues for temporal samples

The ability of pigeons to use event durations as remember (R) and forget (F) cues for temporal samples was examined. Pigeons were required to indicate whether a houselight sample stimulus was short (2 sec) or long (6 sec) by pecking a red or a green comparison stimulus. After training with a constant 10-sec delay interval, temporal cues (illumination of the center key) were presented 2 sec after the offset of the temporal samples. For one group, a short (2-sec) temporal cue served as the R cue and a long (6-3ec) temporal cue served as the F cue. This was reversed for a second group of birds. During training, comparison stimuli were always presented following the temporal R cue, but never following the temporal F cue. Tests for the effectiveness of the temporal R and F cues showed that F cues were equally effective in reducing matching accuracy in both groups of birds. It was concluded that pigeons used the duration of the cue to determine whether or not to rehearse the memory code for the temporal sample.     

Blocking in autoshaped lever-pressing procedures with rats

Rats will approach and contact a lever whose insertion into the chamber signals response-independent food delivery. This “autoshaping” or “sign-tracking” phenomenon has recently attracted considerable attention as a platform for studying individual differences in impulsivity, drug sensitization, and other traits associated with vulnerability to drug addiction. Here, we examined two basic stimulus selection phenomena—blocking and overshadowing—in the autoshaped lever pressing of rats. Blocking and overshadowing were decidedly asymmetrical. Previously reinforced lever-extension conditioned stimuli (CSs) completely blocked conditioning to auditory cues (Exps. 1 and 2), and previously nonreinforced lever-extension CSs overshadowed conditioning to auditory cues. By contrast, conditioning to lever-extension CSs was not blocked by either auditory (Exp. 3) or lever-insertion (Exp. 4) cues, and was not overshadowed by auditory cues. Conditioning to a lever-insertion cue was somewhat overshadowed by the presence of another lever, especially in terms of food cup behavior displayed after lever withdrawal. We discuss several frameworks in which the apparent immunity of autoshaped lever pressing to blocking might be understood. Given evidence that different brain systems are engaged when different kinds of cues are paired with food delivery, it is worth considering the possibility that interactions among them in learning and performance may follow different rules. In particular, it is intriguing to speculate that the roles of simple cue–reinforcer contiguity, as well as of individual and aggregate reinforcer prediction errors, may differ across stimulus classes.     

Some determinants of second-order conditioning

In a Pavlovian conditioning situation, an initially neutral stimulus may be made excitatory by nonreinforced presentations in compound with an established conditioned excitor [i.e., second-order conditioning (SOC)]. The established excitor may be either a punctate cue or the training context. In four conditioned suppression experiments using rats, we investigated whether SOC phenomena parallel other cue interaction effects. In Experiment 1, we found that the response potential of a target stimulus was directly related to the intertrial interval when SOC was mediated by a punctate cue, and inversely related to the intertrial interval when SOC was mediated by the training context. Experiment 2 demonstrated that punctate- and context-mediated SOC are oppositely affected by posttraining context extinction, and Experiments 3 and 4 demonstrated that context- and punctate-mediated SOC are differentially affected by conditioned stimulus (Experiment 3) and unconditioned stimulus (Experiment 4) preexposure treatments. These findings parallel phenomena in conditioned inhibition and cue competition situations.     

Addition versus deletion as a signal

Subjects typically show superior discriminative performance when a distinguishing feature appears on reinforced rather than nonreinforced trials. The phenomenon is usually attributed to the relative predictiveness of the reinforcer by different stimulus elements. However, stimulus addition may be more effective than stimulus deletion as a signal. By removing the standard intertriai intervals, we made addition and deletion equally predictive of the reinforcer in four operant experiments involving between- and within-subject comparisons. Pigeons consistently performed better on operant discriminations when the addition rather than deletion of an auditory or visual stimulus served as the cue for food. This general finding persisted despite manipulation of the relative duration and localizability of the signal. Thus mere presence as opposed to absence plays a role in the feature-positive superiority, an outcome that may reflect a fundamental, biologically based difference between addition and deletion as effective signals of reinforcement.     

Associative foundation of causal learning in rats

Are humans unique in their ability to interpret exogenous events as causes? We addressed this question by observing the behavior of rats for indications of causal learning. Within an operant motor–sensory preconditioning paradigm, associative surgical techniques revealed that rats attempted to control an outcome (i.e., a potential effect) by manipulating a potential exogenous cause (i.e., an intervention). Rats were able to generate an innocuous auditory stimulus. This stimulus was then paired with an aversive stimulus. The animals subsequently avoided potential generation of the predictive cue, but not if the aversive stimulus was subsequently devalued or the predictive cue was extinguished (Exp. 1). In Experiment 2, we demonstrated that the aversive stimulus we used was in fact aversive, that it was subject to devaluation, that the cue–aversive stimulus pairings did make the cue a conditioned stimulus, and that the cue was subject to extinction. In Experiments 3 and 4, we established that the decrease in leverpressing observed in Experiment 1 was goal-directed instrumental behavior rather than purely a product of Pavlovian conditioning. To the extent that interventions suggest causal reasoning, it appears that causal reasoning can be based on associations between contiguous exogenous events. Thus, contiguity appears capable of establishing causal relationships between exogenous events. Our results challenge the widely held view that causal learning is uniquely human, and suggest that causal learning is explicable in an associative framework.     

Further evidence that conditioned inhibition is not the mechanism of an extinction cue’s effect: A reinforced cue prevents spontaneous recovery

Previous research has demonstrated that discrete cues presented during extinction of a conditioned response (extinction cues) subsequently reduce spontaneous recovery and the renewal effect. In order to assess whether an extinction cue’s effect is due to conditioned inhibition, the effect of pairing an extinction cue with food on the cue’s capacity to modulate spontaneous recovery was investigated in an appetitive conditioning experiment with rats. Conditioned inhibitors paired with the unconditioned stimulus lose their potential to inhibit responding. The food-paired extinction cue did not lose its potential to reduce spontaneous recovery. In fact, it reduced spontaneous recovery more than did an extinction cue that was not paired with food. The results indicate that extinction cues attenuate postextinction recovery of conditioned responding through a mechanism other than conditioned inhibition. The cue’s action appears to be similar to that of serial negative occasion setters. Theories of spontaneous recovery and the relationship of extinction cues to other modulatory stimuli are discussed.     

Blocking of the sexual conditioning of differentially effective conditioned stimulus objects

The blocking phenomenon was investigated in the sexual response system of male Japanese quail. Access to a live female quail served as the unconditioned stimulus (US). The same audiovisual cue served as the pretrained stimulus in all of the experiments. Following asymptotic conditioning of the audiovisual cue, a second conditioned stimulus (CS2) was added. In Experiment 1, CS2 was a rectangular wood block that had little or no resemblance to a female quail and could not support copulatory behavior. In Experiment 2, CS2 was a terrycloth object that had no quail parts but could support copulatory behavior, and, in Experiment 3, CS2 was a terrycloth object that had a taxidermically prepared head of a female quail added. The terrycloth-only object supported more rapid conditioning than did the wood block, but the blocking effect was obtained with both kinds of stimuli. Approach responding to the terrycloth + head object required pairing it with copulatory opportunity, and the terrycloth + head object supported at least as rapid conditioning as did the terrycloth-only object. However, responding to the terrycloth + head object was not blocked by the pretrained audiovisual cue. These results indicate that the blocking effect occurs in sexual conditioning even with stimulus objects that can support copulation. However, the addition of species-typical head cues to an object makes that object such a powerful stimulus that conditioned approach responding to it cannot be blocked by a previously conditioned arbitrary audiovisual cue.     

Attention as an acquisition and performance variable (AAPV)

This article introduces a new model of Pavlovian conditioning, attention as an acquisition and performance variable (AAPV), which, like several other so-called attentional models, emphasizes the role of variation of cue salience, together with associative strength, in accounting for conditioning phenomena. AAPV is primarily (but not exclusively) a performance-focused model in that it assumes not only that both the saliences and associative strengths of cue representations change during acquisition, but also that they are both influential at the time of test in determining responding. Different weights are given to the representations’ associative strengths according to the representations’ respective saliences at test. The model also treats the representation of a stimulus that is directly activated by presentation of that stimulus as distinct from the representation of the same stimulus that is activated by presenting a companion of the stimulus. Additionally, extinction is viewed as resulting from a decrease in the salience of the cue’s representation, rather than a decrease in associative strength. Simulations of several Pavlovian phenomena are presented in order to illustrate the model and assess its robustness.     

Effects of trial duration on overall and momentary rates of maintained autoshaped keypecking: Choice and single-stimulus trials

Pigeons were exposed to differentially cued autoshaping trials in which conditioned stimuli were followed by food after 6 or 14 sec. Average and momentary rates of keypecking were examined on two types of unreinforced test trials: single-stimulus probe trials and simultaneous choice trials, each 40 sec in duration. Rates averaged over the 40-sec test trials did not favor the cue associated with the shorter delay to food (the short-delay cue) on either type of test trial; however, average rates prior to the scheduled time of food delivery were reliably higher for the short-delay cue on choice trials. Momentary rates of keypecking during choice trials varied as a function of both cue and elapsed time from trial onset. At short elapsed trial times, rate of pecking was higher for the short-delay cue, with this difference reversing at longer times. A reversal of the programmed relation between key color and delay to food presentation for 5 birds confirmed the generality of these findings. Implications of these data for models of Pavlovian conditioning and for methods of assessing conditioned response strength are discussed.     

The effect of redundant contextual stimuli on autoshaping the pigeon’s keypeck

Three experiments investigated the effect of contextual and trial stimulus lighting conditions on keypeck autoshaping in pigeons. White illumination of a response key before food presentation readily produced keypecking in a brightly lit chamber but failed to do so in a chamber without house illumination (Experiments I and III). Keypecking in a darkened cubicle progressively increased and the facilitatory effect of a houselight decreased as the keylight stimulus was varied from a color change (Experiment II) to a feature change (Experiment III). These findings support a “cue localization” hypothesis of autoshaping. according to which reinforcement signals select specific behaviors for expression and direct these behaviors toward the source of stimulation. This account was extended to superstitious and operant conditioning situations.     

Temporal determinants of occasion setting in feature-positive discriminations

Three experiments examined the acquisition and transfer of Pavlovian feature-positive discriminations (XA+, A?) in rat subjects. To identify the nature of the associations formed in those discriminations, the form of the conditioned responses (CRs) was examined. If the feature,X, and common element, A, cues started and ended together onXA compoud trials, associations betweenX and the food unconditioned stimulus (US) were acquired. If the onsets and/or terminations ofX preceded those of A,X acquired the ability to set the occasion for responding to A, that is, A evoked CRs only onXA compound trials. The acquisition of occasion setting was favored when (1) the onset of X preceded that of A, (2) the interval betweenX and A and/or the US was relatively long, and (3) the termination ofX occurred prior to the onset of A. The occasion-setting power ofX was fairly specific to A:X did not modulate responding evoked by another cue that had been first trained and then extinguished or by a cue that had been paired with the US only a few times. However,X did enhance responding to a cue that had been a common element in another, identical feature-positive discrimination. That transfer was somewhat greater if theX and A terminated together than ifX terminated prior to the onset of A. Implications for theories of stimulus control in Pavlovian conditioning are discussed.