The effect of redundant contextual stimuli on autoshaping the pigeon’s keypeck

Three experiments investigated the effect of contextual and trial stimulus lighting conditions on keypeck autoshaping in pigeons. White illumination of a response key before food presentation readily produced keypecking in a brightly lit chamber but failed to do so in a chamber without house illumination (Experiments I and III). Keypecking in a darkened cubicle progressively increased and the facilitatory effect of a houselight decreased as the keylight stimulus was varied from a color change (Experiment II) to a feature change (Experiment III). These findings support a “cue localization” hypothesis of autoshaping. according to which reinforcement signals select specific behaviors for expression and direct these behaviors toward the source of stimulation. This account was extended to superstitious and operant conditioning situations.     

Organization of attack and other behaviors of White King pigeons exposed to intermittent water presentations

Six water-deprived pigeons were exposed to a fixed-time 90-sec water schedule with and without a conspecific target available. Target contacts and the pigeon’s location in the test chamber during the interreinforcement interval were recorded, and the results were compared with those previously obtained with food reinforcement. Prior to target introduction, water-reinforced birds spent more total time in the front near the reinforcer dispenser and less in the rear than food-reinforced birds and, unlike food-reinforced birds, exhibited postreinforcement drinking-like behaviors near the reinforcer dispenser before moving away from that area. With the target available, the level, topography, and duration of target-directed biting pecks were comparable for food- and water-reinforced pigeons. In contrast, the temporal organization of target pecks reflected the different temporal and spatial organizations of behavior prior to target introduction. For both food- and water-reinforced birds, the time between reinforcers at which a bird was spatially situated halfway between the front and rear of the chamber prior to target presentation was positively correlated with the time at which maximum target contact subsequently occurred.     

Consummatory response latency and the stimulus-reinforcer relation in autoshaping

Autoshaping procedures with pigeons were used to assess the susceptibility of unconditioned response (UR) activity to Pavlovian relations between stimulus and reinforcer events. Foodpeck latency (a measure of UR activity) was investigated as a function of the interval between stimulus (keylight) and reinforcer (grain) presentations, and of the stimulus-reinforcer contingency, that is, the conditional probabilities of reinforcer delivery in the presence and absence of the stimulus. Four experiments indicated that food-peck latency was sensitive to both manipulations. Generally, conditions that led to higher keypeck rates were associated with shorter latencies. Thus, UR potentiation was demonstrated. However, when the bird’s location prior to grain delivery was fixed by imposing a keypeck-reinforcer contingency, UR potentiation vanished; it then reappeared when the location constraint was removed. Visual observations supported the conclusion that food-peck latency effects were mediated by approach/withdrawal tendencies generated by the stimulus-reinforcer relation. Implications of these results for expectancy theory are discussed.     

The effect of rate of reinforcement and time in session on preference for variability

Pigeons pecked keys on concurrent-chains schedules that provided a variable interval 30-sec schedule in the initial link. One terminal link provided reinforcers in a fixed manner; the other provided reinforcers in a variable manner with the same arithmetic mean as the fixed alternative. In Experiment 1, the terminal links provided fixed and variable interval schedules. In Experiment 2, the terminal links provided reinforcers after a fixed or a variable delay following the response that produced them. In Experiment 3, the terminal links provided reinforcers that were fixed or variable in size. Rate of reinforcement was varied by changing the scheduled interreinforcer interval in the terminal link from 5 to 225 sec. The subjects usually preferred the variable option in Experiments 1 and 2 but differed in preference in Experiment 3. The preference for variability was usually stronger for lower (longer terminal links) than for higher (shorter terminal links) rates of reinforcement. Preference did not change systematically with time in the session. Some aspects of these results are inconsistent with explanations for the preference for variability in terms of scaling factors, scalar expectancy theory, risk-sensitive models of optimal foraging theory, and habituation to the reinforcer. Initial-link response rates also changed within sessions when the schedules provided high, but not low, rates of reinforcement. Within-session changes in responding were similar for the two initial links. These similarities imply that habituation to the reinforcer is represented differently in theories of choice than are other variables related to reinforcement.     

Control of pecking response form in the pigeon: Topography of ingestive behaviors and conditioned keypecks with food and water reinforcers

In Experiment 1, the form of keypecks produced in an autoshaping procedure with food or water reinforcers was compared with that of eating and drinking responses. Because the responses involve a number of different effector systems, several elements of response form were measured, including peck force and duration, gape, and eye closure. Gape was the only measure to reliably distinguish between both ingestive responses and between conditioned keypecks reinforced with food or water. With either reinforcer, keypecks had greater force than did ingestive behaviors. In Experiment 2, a transition between two forms of keypeck was produced by manipulating deprivation and reinforcer conditions. Some measures appeared to vary in a dichotomous manner between two discrete response forms; gape showed a gradual and continuous change involving the production of intermediate forms of the response. It was concluded that the control of conditioned response form involves theconstruction of the response from movements produced by several effector systems, each with potentially different sources of control.     

Dissociation of the effect of reinforcer type and response strength on the force of a conditioned response

A dissociation between the effect of reinforcer type and response strength on the force of the pigeon’s keypeck response was shown in three experiments. In Experiment 1, pigeons were trained to peck two conditioned stimuli, one paired with water and another paired with grain. The pigeons made more forceful pecks for grain than for water and also showed a tendency, albeit an unreliable one, to respond on a higher percentage of food trials than water trials. In Experiment 2, the pigeons from Experiment 1 were satiated with either food or water and were then presented with the two conditioned stimuli in an extinction test. It was found that, regardless of the drive state, the pigeons made more forceful pecks to the stimulus that predicted food than to the stimulus that predicted water. In the thirsty group, however, this difference in force was not accompanied by a difference in the percentage of trials with a response. In Experiment 3, pigeons trained with a single reinforcer pecked more often on instrumentally reinforced trials than on Pavlovian conditioning trials, but there was no difference in the force of the pecks. Taken together, these results imply that differences in response strength cannot account for the difference between the force of food- and water-reinforced pecks. Instead, stimulus-substitution theory may provide the best account of the topography of the two types of pecks.     

Variations of two temporal parameters in observing response procedures

Temporal parameters were varied in two different observing response procedures. In Experiment I, concurrent variable-interval chain schedules were employed. Responding on one key led to either a stimulus correlated with reinforcement or a stimulus correlated with time-out. Responding on the other key led to a stimulus which ended either in reinforcement or time-out. The duration of the delay to reinforcement or time-out was varied, the delays for all three stimuli always remaining equal in a given phase. It was found that the longer the delay, the greater the preference for the observing response. In Experiment II a procedure was employed in which birds pecked during a “trial” to produce stimuli correlated with reinforcement or time-out at the end of the trial. The duration of the trial ending in time out was varied while the positive trial duration remained constant. It was found that the longer the duration of the negative trial, the greater the strength of observing responses. The results were interpreted as supporting the hypothesis that the value of a positive stimulus is a function of time spent in stimuli correlated with nonreinforcement.

     

Motivational mechanisms and schedule-induced behavioral stereotypy

In two experiments, behavioral stereotypies elicited by scheduled presentations of food and water were compared. In Experiment 1, pigeons were exposed to a fixed-time 30-sec (FT 30-sec) schedule of food or water deliveries with a brightening keylight stimulus signaling time to the unconditioned stimulus (UCS) on each trial. Food and water presentations both produced terminal autoshaped keypecking that was similarly distributed in the trial but differed in response topography and persistence. Locomotor interim behavior was different in the two motivational conditions: With food presentations, it consisted of a “retreat” to the rear of the chamber after UCS termination, followed by “pacing” in the midportion of trials. The water schedule produced very little locomotor activity with no regular distribution in the trial. Experiment 2, using a random-time 30-sec (RT 30-sec) schedule, showed that the differences in interim locomotor behavior persisted in the absence of temporal predictability of the UCS and the keypecking terminal response. The results are taken to support Timberlake’s (1983a) behavior-system theory.     

Choice responding following multiple schedule training

Pigeons’ choice responding on 10-sec interpolated probes was studied after baseline training on multiple variable-interval variable-interval schedules of food reinforcement. Unreinforced choice following training with three different relative reinforcement rates (Experiment 1), with a 3-ply multiple schedule (Experiment 2), and with three different relative reinforcement durations (Experiment 3) was examined. Least squares lines were fit to choice relative response rate and schedule relative response rate as functions of training relative reinforcement rate; choice slope was significantly greater than schedule slope in all three experiments. This result is counter to the prediction of Herrnstein’s (1970) theory that these slopes should not differ. Luce’s (1959) theory also failed to account for the data. It was concluded that choice responding was controlled by both approach to the stimulus associated with the smaller mean interreinforcer interval or the longer duration, and avoidance of the other stimulus.     

Extensive training,partial reinforcement,and temporal gaps do not affect S-S learning in second-order autoshaping

A second-order autoshaping procedure was used to examine the effects of three variables on the amount of information that could be learned about the stimulus properties of a reinforcer. All three experiments paired several keylight S2s with different keylight S1s and then carried out discriminative autoshaping with those S1s. Learning about the stimulus properties of S1 was inferred from changes in the response to its paired S2 when that S1 was changed in value. The sensitivity of S2 to changes in S1 was investigated as a function of number of S2-S1 pairings (Experiment 1), partial reinforcement (Experiment 2), and temporal distance between S2 and S1 (Experiment 3). Each experiment found evidence of a selective change in responding to an S2 as a function of the treatment of its S1. However, the amount of that change was not affected by any of the three variables studied. Those results imply that, within the ranges used here, none of these variables changes the degree of learning about the stimulus properties of a reinforcer.     

Associations with anticipated and obtained outcomes in instrumental learning

In four experiments, we investigated encoding of the reinforcer in instrumental learning. We contrasted the view that the reinforcer is encoded as a consequence of the response with the position that the expectation of the reinforcer serves as an antecedent stimulus for the response. In all four experiments, a response was followed by one reinforcer in the presence of a stimulus known to elicit an expectation of a different reinforcer. In Experiments 1 and 2, we found that devaluing the consequent reinforcer reduced performance of the response more than did devaluing the expected reinforcer. In Experiment 3, we found no evidence at all for a detrimental effect of devaluing the expected reinforcer. Experiment 4 showed that a stimulus associated with a reinforcer will preferentially promote a response that has the same-consequent reinforcer rather than the same-antecedent reinforcer. These results provide further support for the view that response-reinforcer associations are the crucial product in instrumental learning situations.     

Grasping in the pigeon (Columba livid): Stimulus control during conditioned and consummatory responses

Control of beak opening (gape) and peck location was examined in pigeons. Feeding pecks showed accurate guidance that positioned the seed between the beaks. At the moment of contact with the seed, gape was proportional to seed diameter, although pecks with gape less than seed diameter were more frequent following an increase in seed size during a meal. There were no substantial differences between pigeons trained to keypeck with autoshaping and those trained with operant conditioning procedures. With either procedure, water reinforcement produced keypecks with the beak closed; seed reinforcers of different sizes produced means for gape proportional to the seed diameters. Black or white circular stimuli of different sizes projected as conditioning signals had little influence upon gape, but a greater percentage of responses was directed to white stimuli. These results indicate that visual stimuli elicit and orient the peck, whereas the adjustment of gape also involves the somatosensory stimuli provided during previous experience with a particular reinforcer or food type.     

Behavior patterns in pigeons during autoshaping with an incremental conditioned stimulus

Pigeons were trained on a variant of the autoshaping procedure in which a keylight stimulus of increasing brightness was used to signal the passing of a 30-sec interfood interval (IFI). Key-pecking developed in all subjects within the first session (65 trials). Within trials, pecking began midway through the IFI, increased throughout the remainder, and decreased just before food delivery. Other behavioral stereotypies were also recorded: Low light levels were associated with a retreat to the rear of the test chamber, and medium light levels (during the midportion of the IFI) were associated with high rates of pacing toward and away from the food source. Probe trials revealed that pecking, pacing, and retreat were all under strong stimulus control; that is, when the light was held constant at its lowest or highest brightness, or when the brightness ramp was presented in reverse order, the behavior pattern almost invariably remained tied to stimulus brightness. Results are discussed in terms of associative and nonassociative sources of the form and sequential characteristics of the behavior.     

The determiners of keypeck duration

In Experiment 1, pigeons were exposed to a discriminative autoshaping procedure in which one keylight was correlated with negative automaintenance and the other keylight was correlated with positive automaintenance. The negative automaintenance procedure maintained shorter keypeck durations than the positive automaintenance procedure. Keypecking tended to occur in bursts of two, three, or four pecks, and within these bursts keypeck durations reliably decreased. In Experiment 2, keypeck durations and off-keypecks were initially examined during exposure to a positive automaintenance procedure. When a negative keypeck-reinforcer contingency was introduced, keypeck durations decreased and the ratio of off-keypecks to on-keypecks increased. When a positive keypeck-reinforcer contingency was introduced, keypeck durations increased and the ratio of off-keypecks to on-keypecks decreased. These results suggest that keypeck duration is determined by the extent to which the peck is directed at the key.     

Selective reinstatement of stimulus-outcome associations

In two experiments, the possibility of outcome-selective reinstatement of conditioned responding was examined. Evidence for outcome-selective reinstatement of previously extinguished appetitively conditioned magazine responses by rats was observed in both Pavlovian (Experiment 1) and discriminated instrumental conditioning (Experiment 2) procedures. In both experiments, stimulus-elicited magazine responses occurred more in the presence of a stimulus whose reinforcer was reinstated than they did in the presence of another stimulus whose reinforcer was not reinstated. This effect was observed after both brief and extensive amounts of extinction. Outcome-selective reinstatement of instrumental leverpressing, however, was not observed, although nonselective reinstatement of magazine responding and leverpressing was obtained in Experiment 2. Overall, the data from these studies challenge existing theories of reinstatement, and they provide additional evidence of the importance of outcome-specific processes in the control of learned performance.     

The effects of changeover delays of fixed or variable duration on concurrent variable-interval performance in pigeons

The effects of changeover delays of fixed or variable duration on concurrent variable-interval performance in pigeons were investigated in a series of three experiments. Experiment 1 compared the effects of a fixed, variable, or variable signaled changeover delay on interchangeover times and responding during and after the changeover delay. The duration of the changeover delays was systematically varied in Experiment 2, and the relative reinforcement frequencies were manipulated in Experiment 3. Interchangeover times were found to be shorter when changeover delays of variable duration were compared with those of fixed duration. Changeover delays of fixed duration produced higher response rates during the changeover delay than after the changeover delay had elapsed; changeover delays of variable duration produced such differences to a lesser extent. It was concluded that the changeover delay in concurrent variable-interval schedules of reinforcement functionally acts as a delay period to the next opportunity for reinforcement, possibly serving as a conditioned reinforcer for the behavior preceding it (the interchangeover time) and as a discriminative stimulus for the behavior in its presence (response rates during the delay).     

Temporal discrimination using different feature-target intervals in classical conditioning of the rabbit’s nictitating membrane response

In a typical conditional discrimination, a target stimulus (X) is reinforced during one feature cue (A→X+), but not during another feature cue (B→X−). The present experiments used only a single “feature” cue (a 66-sec tone). On half of the trials, the target stimulus (a 400-msec light) was paired with the reinforcer when the feature-target interval was one duration (e.g., 5 sec). On the remaining trials, the interval was different (e.g., 45 sec), and the target stimulus was presented without the reinforcer. All the animals acquired this temporal discrimination, and subsequent testing with other feature-target intervals yielded generalization-like gradients. These results provide solid evidence that each portion of a feature cue is encoded in a distinctive fashion. Had temporal encoding not occurred, the feature cue would have been just as ambiguous a predictor of the reinforcer as was the target stimulus, and discrimination would not have been possible. The integration of real-time temporal encoding mechanisms into models of conditional discrimination is discussed.     

Resistance to extinction: contingency termination and generalization decrement

We present an algebraic model of resistance to extinction that is consistent with research on resistance to change. The model assumes that response strength is a power function of reinforcer rate and that extinction involves two additive, decremental processes: (1) the termination of the reinforcement contingency and (2) generalization decrement resulting from reinforcer omission. The model was supported by three experiments. In Experiment 1, 4 pigeons were trained on two-component multiple variable-interval (VI) 60-sec, VI 240-sec schedules. In two conditions, resistance to change was tested by terminating the response-reinforcer contingency and presenting response-independent reinforcers at the same rate as in training. In two further conditions, resistance to change was tested by prefeeding and by extinction. In Experiment 2, 6 pigeons were trained on two-component multiple VI 150-sec schedules with 8-sec or 2-sec reinforcers, and resistance to change was tested by terminating the response-reinforcer contingency in three conditions. In two of those conditions, brief delays were interposed between responses and response-independent reinforcers. In both Experiments 1 and 2, response rate was more resistant to change in the richer component, except for extinction in Experiment 1. In Experiment 3, 8 pigeons were trained on multiple VI 30-sec, VI 120-sec schedules. During extinction, half of the presentations of each component were accompanied by a novel stimulus to produce generalization decrement. The extinction data of Experiments 1 and 3 were well described by our model. The value of the exponent relating response strength and reinforcement was similar in all three experiments.     

Directed approach responses and positive conditioned suppression in the rat

Three experiments evaluated an alternative to accounts of positive conditioned suppression that stress central (i.e., motivational or emotional) states. This “competing-response” interpretation was tested by analyzing directed movements that develop in rats during a visual or an auditory stimulus (CS) that signals an appetitive reinforcer (US) in a situation where the subject is also emitting an instrumental response for food. In each experiment, positive conditioned suppression (i.e., a reduction in the rate of such instrumental responding during CS presentations) was accompanied by responses directed toward the CS source and/or the US-delivery site. In Experiment 1, a diffuse (auditory) CS signaled a US delivered at some specific place in the chamber and rats approached the US-delivery site during CS. In Experiments 2 and 3, the spatial proximity of a localized visual CS and US-delivery site determined whether CS-directed or US-directed behavior predominated during the CS. The results suggest that the topographies of conditioned responses on any positive conditioned suppression procedure depend upon the spatial arrangements of features that elicit and support these behaviors. They further suggest that the identification of these features and their spatial arrangements is necessary for the analysis of appetitive classical-instrumental interactions.     

Does contextual change determine long-term forgetting?

A series of experiments were run to test the hypothesis that “spontaneous forgetting” could result from subtle contextual changes. The first experiment demonstrated that when Sprague-Dawley male rats are trained in a runway alley with a food reinforcer, retention performance is dramatically affected by a change in the pattern of the walls of the training apparatus when testing takes place 1, 3, or 5 days following training and not after 1 week. Experiment 2 demonstrated that this performance deficit cannot be alleviated by any of the three selected cuing treatments, whereas “spontaneous forgetting” (resulting from a training-to-test interval of 14 days) can be. These data indicate that the detrimental effect of contextual change reduces over time, and that such an effect cannot be interpreted in terms of retrieval failure. These results lend strong support to the argument that the disruptive contextual change effect is basically different from the disruptive effect that results from an extended training-to-test interval.