Blocking of first- and second-order autoshaping in pigeons

In Experiment 1, the development of autoshaped pecking to a keylight signaling food was blocked if the keylight was presented only in conjunction with another stimulus already established as a signal for food, even though the blocking stimulus (either an overhead light or a train of clicks) never elicited pecking itself. In Experiment 2, pigeons came to peck a white keylight which signaled the presentation of a red keylight which had earlier been established as a first-order signal for food, but this second-order autoshaping was blocked if the white keylight was presented only in conjunction with the houselight or clicker which had previously signaled the presentation of the first-order stimulus. Second-order autoshaping was thus blocked in the same way as was first-order autoshaping.     

Transfer of inhibition and facilitation mediated by the original target stimulus

An autoshaping paradigm with pigeon subjects was used to study two-types-of modulation: facilitation and inhibition. In each paradigm, a diffuse stimulus signaled when a keylight would be reinforced. Transfer of that diffuse stimulus to a different keylight was assessed after the response-evoking power of the original keylight had been altered in the presence of a different modulator. That alteration of the original target produced a specific change in the transfer of modulators trained with that target. The results are interpreted as suggesting that transfer to another target stimulus is partly mediated by the association of the original target with the reinforcer. They are inconsistent with at least one version of a configurai cue interpretation.     

The effect of redundant contextual stimuli on autoshaping the pigeon’s keypeck

Three experiments investigated the effect of contextual and trial stimulus lighting conditions on keypeck autoshaping in pigeons. White illumination of a response key before food presentation readily produced keypecking in a brightly lit chamber but failed to do so in a chamber without house illumination (Experiments I and III). Keypecking in a darkened cubicle progressively increased and the facilitatory effect of a houselight decreased as the keylight stimulus was varied from a color change (Experiment II) to a feature change (Experiment III). These findings support a “cue localization” hypothesis of autoshaping. according to which reinforcement signals select specific behaviors for expression and direct these behaviors toward the source of stimulation. This account was extended to superstitious and operant conditioning situations.     

Acquired equivalence of cues in pigeon autoshaping: Effects of training with common consequences and with common antecedents

In three experiments, we sought evidence for the acquired equivalence of cues in pigeons trained in an autoshaping paradigm. In Experiment 1, presentations of each of a pair of cues (different keylight stimuli) preceded a common consequence (a different keylight stimulus). The pattern of response then established by further training given to one member of the pair was found to generalize preferentially to the other, demonstrating equivalence between cues that had shared a common consequence. The same test procedure was used in Experiment 2, but with a training procedure in which each cue of a pair was preceded by a given stimulus. This too resulted in enhanced generalization between members of the pair, showing that equivalence can be established when cues have been experienced along with a common antecedent. Both training procedures were combined in Experiment 3 to confirm the reliability of the effects previously obtained. The discussion is focused on ways in which the associative explanation offered for cases of equivalence mediated by a common consequence might be extended to accommodate equivalence mediated by a common antecedent.     

Sign and goal tracking during delay and trace autoshaping in pigeons

In two experiments, pigeons were exposed to an autoshaping procedure in which a keylight was followed by food under delay or trace conditions, while measures were taken of keypecking and time spent near the key. In Experiment 1, the birds in the trace group spent less time near the key than did the delay birds. Moreover, the trace birds exhibited a pattern of withdrawal from the key during the trial. In Experiment 2, visual observations of the birds’ location and latencies to eat both indicated that the trace birds’ withdrawal from the conditioned stimulus was accompanied by goal tracking. The difference in performance between the delay and trace conditions was taken as evidence that a trace stimulus may exert control over behavior as an occasion setter.     

Second-order conditioning of the pigeon’s keypeck

Pigeons learned to peck a keylight (S2) when it was paired with a stimulus (S1) that already evoked keypecking. Control procedures showed that S2 acquired control over responding because it was paired with S1 and because S1 had a conditioning history, thereby supporting the claim that S2 was a second-order conditioned stimulus. Second-order conditioning occurred as rapidly when S1 was a keylight as when it was a tone. Test procedures showed that after second-order conditioning, responding to S2 was markedly debilitated by the extinction of responding to S1, indicating that the ability of S2 to evoke a response importantly depends upon the continued ability of S1 to do so. Our demonstration that directed motor action in the pigeon is susceptible to second-order conditioning suggests a new interpretation of conditioned reinforcement in instrumental learning. Our demonstration that the effectiveness of S2 depends upon the continued effectiveness of S1 indicates that S-S associations are formed in this version of the second-order conditioning experiment.     

The determiners of keypeck duration

In Experiment 1, pigeons were exposed to a discriminative autoshaping procedure in which one keylight was correlated with negative automaintenance and the other keylight was correlated with positive automaintenance. The negative automaintenance procedure maintained shorter keypeck durations than the positive automaintenance procedure. Keypecking tended to occur in bursts of two, three, or four pecks, and within these bursts keypeck durations reliably decreased. In Experiment 2, keypeck durations and off-keypecks were initially examined during exposure to a positive automaintenance procedure. When a negative keypeck-reinforcer contingency was introduced, keypeck durations decreased and the ratio of off-keypecks to on-keypecks increased. When a positive keypeck-reinforcer contingency was introduced, keypeck durations increased and the ratio of off-keypecks to on-keypecks decreased. These results suggest that keypeck duration is determined by the extent to which the peck is directed at the key.     

The nature and determinants of spatial retreat in the pigeon between periodic grain presentations

Pigeons were exposed to fixed-time and fixed-interval schedules that ranged from 30 to 960 sec. The probability of a subject’s location in the rear of the chamber (away from the reinforcer dispenser) peaked during the postreinforcer period, and was referenced to proportional time between reinforcers. Increasing the interreinforcer interval generally increased time in the rear. In some sessions (Experiment 1), location in the rear produced an explicit stimulus change (altered the color and intensity of lights, i.e., time-out); this change increased time spent in the rear without affecting its temporal locus or its relation to the interreinforcer interval. During Experiment 2, a keypeck (near the reinforcer site) produced the explicit stimulus change used in Experiment 1. The characteristics of keypeck time-out resembled those of movement to the rear of the chamber (with and without an explicit stimulus change), suggesting that movement away from the reinforcer site is functionally homologous to keypeck time-out.     

Dissociation of the effect of reinforcer type and response strength on the force of a conditioned response

A dissociation between the effect of reinforcer type and response strength on the force of the pigeon’s keypeck response was shown in three experiments. In Experiment 1, pigeons were trained to peck two conditioned stimuli, one paired with water and another paired with grain. The pigeons made more forceful pecks for grain than for water and also showed a tendency, albeit an unreliable one, to respond on a higher percentage of food trials than water trials. In Experiment 2, the pigeons from Experiment 1 were satiated with either food or water and were then presented with the two conditioned stimuli in an extinction test. It was found that, regardless of the drive state, the pigeons made more forceful pecks to the stimulus that predicted food than to the stimulus that predicted water. In the thirsty group, however, this difference in force was not accompanied by a difference in the percentage of trials with a response. In Experiment 3, pigeons trained with a single reinforcer pecked more often on instrumentally reinforced trials than on Pavlovian conditioning trials, but there was no difference in the force of the pecks. Taken together, these results imply that differences in response strength cannot account for the difference between the force of food- and water-reinforced pecks. Instead, stimulus-substitution theory may provide the best account of the topography of the two types of pecks.     

Consummatory response latency and the stimulus-reinforcer relation in autoshaping

Autoshaping procedures with pigeons were used to assess the susceptibility of unconditioned response (UR) activity to Pavlovian relations between stimulus and reinforcer events. Foodpeck latency (a measure of UR activity) was investigated as a function of the interval between stimulus (keylight) and reinforcer (grain) presentations, and of the stimulus-reinforcer contingency, that is, the conditional probabilities of reinforcer delivery in the presence and absence of the stimulus. Four experiments indicated that food-peck latency was sensitive to both manipulations. Generally, conditions that led to higher keypeck rates were associated with shorter latencies. Thus, UR potentiation was demonstrated. However, when the bird’s location prior to grain delivery was fixed by imposing a keypeck-reinforcer contingency, UR potentiation vanished; it then reappeared when the location constraint was removed. Visual observations supported the conclusion that food-peck latency effects were mediated by approach/withdrawal tendencies generated by the stimulus-reinforcer relation. Implications of these results for expectancy theory are discussed.     

Sign-tracking (autoshaping) in rats: A comparison of cocaine and food as unconditioned stimuli

A series of experiments was performed to determine whether sign-tracking would occur in rats with intravenous (i.v.) cocaine as the unconditioned stimulus. In Experiment 1, a retractable lever paired with food produced strong sign-tracking, but a lever paired with one of three doses of i.v. cocaine did not elicit any approach or contact behavior. Experiment 2 demonstrated that doses of cocaine that did not elicit sign-tracking would function as a positive reinforcer for a lever contact operant. In Experiment 3, an artificialconsummatory response was added to make the cocaine reinforcement episode more behaviorally comparable to that occasioned by food. Although the rats readily performed this response when it was required to receive cocaine infusions, they still did not contact a lever that signaled the availability of these infusions. It appears that cocaine is different from other positive reinforcers (e.g., food, water, warmth, or intracranial stimulation) in that it will not produce sign-tracking in rats.     

Effects of CS preexposure on inhibition of delay

Two experiments examined the effects of preexposure to a stimulus on the subsequent acquisition of conditioned suppression by rats. Variations in the level of suppression within conditioning trials were noted so thatinhibition of delay (taken here to mean less suppression at the beginning of a trial than at the end) could be detected. Inhibition of delay was observed both during the acquisition of suppression and (in Experiment 1) when suppression began to wane with continued postasymptotic training. Preexposure to the to-be-conditioned stimulus retarded acquisition of suppression and slowed the appearance of inhibition of delay both in acquisition and (in Experiment 1) in postasymptotic performance. Experiment 2 demonstrated that inhibition of delay was attenuated during conditioning that followed preexposure in which the stimulus was paired with a weak reinforcer. These results provide no support for the suggestion that preexposure to a stimulus retards later conditioning because it allows the subject to acquire information about stimulus duration that in turn fosters the development of inhibition of delay. Rather, they are compatible with the suggestion that preexposure causes the stimulus to lose associability.     

Resistance to extinction: contingency termination and generalization decrement

We present an algebraic model of resistance to extinction that is consistent with research on resistance to change. The model assumes that response strength is a power function of reinforcer rate and that extinction involves two additive, decremental processes: (1) the termination of the reinforcement contingency and (2) generalization decrement resulting from reinforcer omission. The model was supported by three experiments. In Experiment 1, 4 pigeons were trained on two-component multiple variable-interval (VI) 60-sec, VI 240-sec schedules. In two conditions, resistance to change was tested by terminating the response-reinforcer contingency and presenting response-independent reinforcers at the same rate as in training. In two further conditions, resistance to change was tested by prefeeding and by extinction. In Experiment 2, 6 pigeons were trained on two-component multiple VI 150-sec schedules with 8-sec or 2-sec reinforcers, and resistance to change was tested by terminating the response-reinforcer contingency in three conditions. In two of those conditions, brief delays were interposed between responses and response-independent reinforcers. In both Experiments 1 and 2, response rate was more resistant to change in the richer component, except for extinction in Experiment 1. In Experiment 3, 8 pigeons were trained on multiple VI 30-sec, VI 120-sec schedules. During extinction, half of the presentations of each component were accompanied by a novel stimulus to produce generalization decrement. The extinction data of Experiments 1 and 3 were well described by our model. The value of the exponent relating response strength and reinforcement was similar in all three experiments.     

Signaling functions of the second-order CS: Partial reinforcement during second-order conditioning of the pigeon’s keypeck

The signaling function of the second-order CS (S2) was manipulated in second-order autoshaping by arranging a partial reinforcement schedule. S2 was paired with a well-conditioned first-order CS (SI) on a continuous reinforcement or a 25% reinforcement schedule in different groups. Schedule of reinforcement did not influence the number of S2-S1 pairings required to establish keypecking to S2. However, in the postacquisition sessions, responding to S2 was initially weaker but persisted for many more sessions on the 25% schedule than on the 100% schedule. The data indicate that S2-S1 pairings are responsible both for the acquisition of second-order keypecking to S2 and for the subsequent conversion of S2 into an inhibitory stimulus.     

The role of signaled periods of nonreinforcement in responding on a random schedule in autoshaping

Little responding develops to a conditioned stimulus (CS) that is placed in a random relation to an unconditioned stimulus (US). However, if the USs not preceded by that CS are themselves signaled by another stimulus, then the CS does come to elicit responding. This result has been attributed (e.g., by Durlach, 1983) to the signal’s blocking of conditioning to background cues that otherwise would prevent conditioning of the CS. However, Goddard and Jenkins (1987) have suggested the alternative that signaling the USs promotes responding due to the adventitious creation of periods of signaled nonreinforcement. Two experiments were conducted to assess this alternative, involving an autoshaping preparation in pigeons. In Experiment 1, little responding to a keylight CS presented in a random relation to a food US occurred, despite the explicit presentation of a discrete noise signaling periods of no food in the intertrial interval (ITI). Experiment 2 was designed to replicate the procedure of Goddard and Jenkins, in which an auditory stimulus extended throughout the ITI of a random schedule, terminating only prior to extra USs and during the CS. Contrary to their findings, little responding developed to the target CS. However, responding did develop when the sound-free period occurred only prior to the extra USs. These results offer little support for the hypothesis that signaled periods of nonreinforcement promote responding on random schedules. However, they are consistent with the view that signaling of ITI USs acts by preventing conditioning of potentially competitive background cues.     

US duration and local trial spacing affect autoshaped responding

The acquisition and maintenance of signal-directed pecking was examined in week-old Leg-Horn chicks responding to a keylight stimulus paired with heat. In contrast with previous studies using pigeons with food as the US, both speed of acquisition and asymptotic level of keypecking were a direct function of US duration. Experiment 2 examined responding using a within-subject design to isolate the effects of trial spacing on performance during the immediate trial from the effects on performance during a following trial of fixed length. These comparisons revealed a significant effect of intertriai interval (ITI), with less responding after shorter intervals. The effect of different temporal spacing was apparent in responding on the immediate trial, but not on the following trial. These local ITI effects were better predicted by a recent autoshaping model based on relative waiting time (Jenkins, Barnes, & Barrera, 1981) than by a model based on relative US expectancy (Gibbons & Balsam, 1981). However, neither model predicted the effect of US duration. A reexamination of the US-duration literature suggested that the diversity of previous findings is consistent with the assumption that conditioned responding is an inverted U-shaped function of US duration.     

The form of timing distributions in pigeons under penalties for responding early

The peak procedure was used in two experiments. Pigeons in the penalty group in Experiment 1 were rewarded with food in the first phase for the first peck after 12.5 sec had elapsed since the onset of a keylight. In the second phase, reward was withheld if the pigeons pecked within 6.25 sec after keylight onset. Responses in time were tabulated on occasional unrewarded tests in which the keylight was left on for 37.5 sec. Under the penalty contingencies, the response distribution in time remained nearly symmetric about the peak, while the spread of the distribution narrowed, and the time of peak responding came slightly earlier. The yoke group underwent a schedule of rewards similar to that for the penalty group, but without the penalty contingencies. Their response distributions remained similar throughout. The results of Experiment 1 were replicated in Experiment 2, which showed further that the changes due to the penalty contingencies did not generalize to the use of another key on which the penalty contingencies were not in effect. The narrower spread under the penalty contingencies is explained in terms of a change in threshold for when to start responding, and more weight being given to timing versus responding in the presence of the signal per se. No explanation was found for the change in the time of peak responding.     

Appetitive-aversion interactions: Facilitation of aversive conditioning by prior appetitive training in the rat

In Experiment 1, four groups of rats received conditioned suppression training in which a tone was reinforced with shock. If the tone had been previously paired with response-independent food, aversive conditioning was slightly facilitated by comparison to control groups preexposed either to the tone randomly associated with food or to the tone and food unpaired. However, by comparison to a control which was not preexposed to the tone, animals receiving prior pairings of the tone and food showed retarded aversive conditioning. Experiment 2 replicated the facilitation in aversive conditioning after the tone had been paired with food relative to the random control condition and demonstrated that this difference occurred even if the tone and background stimuli continued to be associated with response-independent food during aversive conditioning. This result suggests that pairing a stimulus with an appetitive reinforcer reduces the retardation of aversive conditioning produced by stimulus preexposure.     

Control of pecking response form in the pigeon: Topography of ingestive behaviors and conditioned keypecks with food and water reinforcers

In Experiment 1, the form of keypecks produced in an autoshaping procedure with food or water reinforcers was compared with that of eating and drinking responses. Because the responses involve a number of different effector systems, several elements of response form were measured, including peck force and duration, gape, and eye closure. Gape was the only measure to reliably distinguish between both ingestive responses and between conditioned keypecks reinforced with food or water. With either reinforcer, keypecks had greater force than did ingestive behaviors. In Experiment 2, a transition between two forms of keypeck was produced by manipulating deprivation and reinforcer conditions. Some measures appeared to vary in a dichotomous manner between two discrete response forms; gape showed a gradual and continuous change involving the production of intermediate forms of the response. It was concluded that the control of conditioned response form involves theconstruction of the response from movements produced by several effector systems, each with potentially different sources of control.     

Associations with anticipated and obtained outcomes in instrumental learning

In four experiments, we investigated encoding of the reinforcer in instrumental learning. We contrasted the view that the reinforcer is encoded as a consequence of the response with the position that the expectation of the reinforcer serves as an antecedent stimulus for the response. In all four experiments, a response was followed by one reinforcer in the presence of a stimulus known to elicit an expectation of a different reinforcer. In Experiments 1 and 2, we found that devaluing the consequent reinforcer reduced performance of the response more than did devaluing the expected reinforcer. In Experiment 3, we found no evidence at all for a detrimental effect of devaluing the expected reinforcer. Experiment 4 showed that a stimulus associated with a reinforcer will preferentially promote a response that has the same-consequent reinforcer rather than the same-antecedent reinforcer. These results provide further support for the view that response-reinforcer associations are the crucial product in instrumental learning situations.