Consummatory response latency and the stimulus-reinforcer relation in autoshaping

Autoshaping procedures with pigeons were used to assess the susceptibility of unconditioned response (UR) activity to Pavlovian relations between stimulus and reinforcer events. Foodpeck latency (a measure of UR activity) was investigated as a function of the interval between stimulus (keylight) and reinforcer (grain) presentations, and of the stimulus-reinforcer contingency, that is, the conditional probabilities of reinforcer delivery in the presence and absence of the stimulus. Four experiments indicated that food-peck latency was sensitive to both manipulations. Generally, conditions that led to higher keypeck rates were associated with shorter latencies. Thus, UR potentiation was demonstrated. However, when the bird’s location prior to grain delivery was fixed by imposing a keypeck-reinforcer contingency, UR potentiation vanished; it then reappeared when the location constraint was removed. Visual observations supported the conclusion that food-peck latency effects were mediated by approach/withdrawal tendencies generated by the stimulus-reinforcer relation. Implications of these results for expectancy theory are discussed.     

The control of keypecks during automaintenance by prekeypeck omission training

Measures of pigeons’ prekeypecking (pecking in front of the response key) during automaintenance sessions with keypeck omission contingencies indicate that prekeypecks occur frequently and are often followed by grain delivery. When the omission procedure is extended to prevent food delivery following any trial on which prekeypecks occur within 2 in. of the response key, keypecking is not maintained. These results are taken to suggest that the automaintenance keypeck is part of a sequence of approach behaviors, including prekeypecks. The persistence of keypecking during automaintenance schedules appears to result from the adventitious reinforcement of prekeypecks close to the response key, and the effectiveness of the omission procedure seems to depend upon the extent of the approach behaviors which result in omission.     

The effect of redundant contextual stimuli on autoshaping the pigeon’s keypeck

Three experiments investigated the effect of contextual and trial stimulus lighting conditions on keypeck autoshaping in pigeons. White illumination of a response key before food presentation readily produced keypecking in a brightly lit chamber but failed to do so in a chamber without house illumination (Experiments I and III). Keypecking in a darkened cubicle progressively increased and the facilitatory effect of a houselight decreased as the keylight stimulus was varied from a color change (Experiment II) to a feature change (Experiment III). These findings support a “cue localization” hypothesis of autoshaping. according to which reinforcement signals select specific behaviors for expression and direct these behaviors toward the source of stimulation. This account was extended to superstitious and operant conditioning situations.     

Dissociation of the effect of reinforcer type and response strength on the force of a conditioned response

A dissociation between the effect of reinforcer type and response strength on the force of the pigeon’s keypeck response was shown in three experiments. In Experiment 1, pigeons were trained to peck two conditioned stimuli, one paired with water and another paired with grain. The pigeons made more forceful pecks for grain than for water and also showed a tendency, albeit an unreliable one, to respond on a higher percentage of food trials than water trials. In Experiment 2, the pigeons from Experiment 1 were satiated with either food or water and were then presented with the two conditioned stimuli in an extinction test. It was found that, regardless of the drive state, the pigeons made more forceful pecks to the stimulus that predicted food than to the stimulus that predicted water. In the thirsty group, however, this difference in force was not accompanied by a difference in the percentage of trials with a response. In Experiment 3, pigeons trained with a single reinforcer pecked more often on instrumentally reinforced trials than on Pavlovian conditioning trials, but there was no difference in the force of the pecks. Taken together, these results imply that differences in response strength cannot account for the difference between the force of food- and water-reinforced pecks. Instead, stimulus-substitution theory may provide the best account of the topography of the two types of pecks.     

The effect of sample and comparison ratio schedules on delayed matching to sample in the pigeon

In Experiment 1, three food-deprived pigeons received trials that began with red or green illumination of the center pecking key. Two or four pecks on this sample key turned it off and initiated a 0- to 10-sec delay. Following the delay, the two outer comparison keys were illuminated, one with red and one with green light. In one condition, a single peck on either of these keys turned the other key off and produced either grain reinforcement (if the comparison that was pecked matched the preceding sample) or the intertrial interval (if it did not match). In other conditions, 3 or 15 additional pecks were required to produce reinforcement or the intertrial interval. The frequency of pecking the matching comparison stimulus (matching accuracy) decreased as the delay increased, increased as the sample ratio was increased, and decreased as the comparison ratio was increased. The results of Experiment 2 suggested that higher comparison ratios adversely affect matching accuracy primarily by delaying reinforcement for choosing the correct comparison. The results of Experiment 3, in which delay of reinforcement for choosing the matching comparison was manipulated, confirmed that delayed reinforcement decreases matching accuracy.     

Specification of the stimulus-reinforcer relation in multiple schedules: Delay and probability of reinforcement

Control of pigeons’ keypecking by a stimulus-reinforcer contingency was investigated in the context of a four-component multiple schedule. In each of three experiments, pigeons were exposed to a schedule consisting of two two-component sequences. Discriminative stimuli identifying each sequence were present only in Component 1, which was 4, 6, or 8 sec in duration, while reinforcers could be earned only in Component 2 (30 sec in duration). Control by a stimulus-reinforcer contingency was sought during Component 1 by arranging a differential relation between Component 1 cues and schedule of reinforcement in Component 2. In Experiment 1, rate of keypecking during Component 1 varied with the presence and absence of a stimulus-reinforcer contingency. When a contingency was introduced, rate of keypecking increased during the Component 1 cue associated with the availability of reinforcement in Component 2. In Experiment 2, the stimulus-reinforcer contingency was manipulated parametrically by varying the correlation between Component 1 cues and Component 2 schedules of reinforcement. Responding in Component 1 varied as a function of strength of the stimulus-reinforcer contingency. The relatively high rates of Component 1 responding observed in Experiments 1 and 2 pose difficulties for conceptions of stimulus-reinforcer control based on probability of reinforcement. In these two experiments, the stimulus-associated probabilities of reinforcement in Component 1 were invariant at zero. An alternate dimension of stimulus-reinforcer control was explored in Experiment 3, in which Component 1 cues were differentially associated with delay to reinforcement in Component 2, while probability of reinforcement was held constant across components. When the stimulus-reinforcer contingency was in force, rate of responding in Component 1 varied inversely with delay to reinforcement in Component 2. In a quantitative analysis of data from Experiments 2 and 3, relative rate of responding during Component 1 was strongly correlated with two measures of relative delay to reinforcement.     

Grasping in the pigeon (Columba livid): Stimulus control during conditioned and consummatory responses

Control of beak opening (gape) and peck location was examined in pigeons. Feeding pecks showed accurate guidance that positioned the seed between the beaks. At the moment of contact with the seed, gape was proportional to seed diameter, although pecks with gape less than seed diameter were more frequent following an increase in seed size during a meal. There were no substantial differences between pigeons trained to keypeck with autoshaping and those trained with operant conditioning procedures. With either procedure, water reinforcement produced keypecks with the beak closed; seed reinforcers of different sizes produced means for gape proportional to the seed diameters. Black or white circular stimuli of different sizes projected as conditioning signals had little influence upon gape, but a greater percentage of responses was directed to white stimuli. These results indicate that visual stimuli elicit and orient the peck, whereas the adjustment of gape also involves the somatosensory stimuli provided during previous experience with a particular reinforcer or food type.     

Blocking of first- and second-order autoshaping in pigeons

In Experiment 1, the development of autoshaped pecking to a keylight signaling food was blocked if the keylight was presented only in conjunction with another stimulus already established as a signal for food, even though the blocking stimulus (either an overhead light or a train of clicks) never elicited pecking itself. In Experiment 2, pigeons came to peck a white keylight which signaled the presentation of a red keylight which had earlier been established as a first-order signal for food, but this second-order autoshaping was blocked if the white keylight was presented only in conjunction with the houselight or clicker which had previously signaled the presentation of the first-order stimulus. Second-order autoshaping was thus blocked in the same way as was first-order autoshaping.     

The nature and determinants of spatial retreat in the pigeon between periodic grain presentations

Pigeons were exposed to fixed-time and fixed-interval schedules that ranged from 30 to 960 sec. The probability of a subject’s location in the rear of the chamber (away from the reinforcer dispenser) peaked during the postreinforcer period, and was referenced to proportional time between reinforcers. Increasing the interreinforcer interval generally increased time in the rear. In some sessions (Experiment 1), location in the rear produced an explicit stimulus change (altered the color and intensity of lights, i.e., time-out); this change increased time spent in the rear without affecting its temporal locus or its relation to the interreinforcer interval. During Experiment 2, a keypeck (near the reinforcer site) produced the explicit stimulus change used in Experiment 1. The characteristics of keypeck time-out resembled those of movement to the rear of the chamber (with and without an explicit stimulus change), suggesting that movement away from the reinforcer site is functionally homologous to keypeck time-out.     

Maintenance of keypecking in pigeons by a food avoidance but not by a shock avoidance contingency

Response key illuminations were followed by food delivery or shock, and keypecks were programmed to prevent the occurrence of whichever stimulus was scheduled. At high shock intensity, pigeons did not peck: at low shock intensity, pigeons pecked in about half of the trials. When different key colors signaled food and shock trials, pigeons pecked on food trials, thus preventing food delivery, but not on shock trials, thus failing to avoid shock delivery. That pecks occurred despite the fact that they avoided food but did not occur when they avoided shock is taken as evidence that the keypeck is frequently governed by biological predispositions, and not by its consequences.     

The form of timing distributions in pigeons under penalties for responding early

The peak procedure was used in two experiments. Pigeons in the penalty group in Experiment 1 were rewarded with food in the first phase for the first peck after 12.5 sec had elapsed since the onset of a keylight. In the second phase, reward was withheld if the pigeons pecked within 6.25 sec after keylight onset. Responses in time were tabulated on occasional unrewarded tests in which the keylight was left on for 37.5 sec. Under the penalty contingencies, the response distribution in time remained nearly symmetric about the peak, while the spread of the distribution narrowed, and the time of peak responding came slightly earlier. The yoke group underwent a schedule of rewards similar to that for the penalty group, but without the penalty contingencies. Their response distributions remained similar throughout. The results of Experiment 1 were replicated in Experiment 2, which showed further that the changes due to the penalty contingencies did not generalize to the use of another key on which the penalty contingencies were not in effect. The narrower spread under the penalty contingencies is explained in terms of a change in threshold for when to start responding, and more weight being given to timing versus responding in the presence of the signal per se. No explanation was found for the change in the time of peak responding.     

Contrast during multiple schedules with different component response requirements

Positive and negative behavioral contrast were examined when pigeons were required to keypeck in one component and treadle press in the other component of a series of multiple schedules. The experimental conditions were constructed so that the positive and negative contrast groups were exposed to complementary conditions. Positive keypeck and negative treadle-press contrast occurred in all subjects. Positive treadle-press contrast seemed to depend on the order of schedule presentation. Only one subject exhibited strong negative keypeck contrast. The results indicate that symmetrical conditions are not sufficient to produce positive and negative contrast for a given response when topographically different responses are used in the components of a multiple schedule. The results are globally consistent with the competition theory of behavioral contrast.     

Instrumental responding by rats on free-operant schedules with components that schedule response-dependent reinforcer omission: Implications for optimization theories

In two experiments, we assessed whether rats optimize the number of reinforcers per response. In Experiment 1, one group of rats was trained to leverpress for food reinforcement on a simple variable-interval (VI) 60-sec schedule. For another group, a negative fixed-ratio component was imposed on the VI schedule to produce a conjoint contingency in which reinforcement became available on the VI schedule but was omitted when the ratio criterion was satisfied. In Experiment 2, one group of rats responded on a VI schedule and also received response-independent food. For another group, responding above a certain rate canceled the response-independent food. Despite the negative contingency experienced by the groups in Experiments 1 and 2, responding was maintained at a level at which the number of obtained reinforcers was reduced substantially below the maximum number possible. In addition, in both experiments, the groups that experienced the negative contingency responded at a lower rate than did a yoked control group that experienced the same frequency of reinforcement but lacked the negative component. These results demonstrate that although rats do not optimize their behavior with respect to reinforcement, they are nevertheless sensitive to some aspect of the instrumental contingency in operation.     

The role of primary reinforcement and overt movements in autoshaping in the pigeon

Following sessions of free grain delivery, a transparent shield was placed over the magazine, which made food unavailable. Different groups of pigeons then “observed” positive, zero, or negative correlations between the keylight and inaccessible grain. Keypecks were rare in all groups. Next, the shield was removed, and a transfer-test was given in which all subjects were exposed to keylight presentations followed by available grain. The previously positive group pecked sooner and more frequently than the others. A second experiment, which yielded similar results, excluded the possibility that approaches to the keylight during the observation phase had mediated learning in the first experiment. These findings were discussed in relationship to operant and Pavlovian analyses of autoshaping.     

“counting” by pigeons: Discrimination of the number of biologically relevant sequential events

Numerical competence has been studied in animals under a variety of conditions, but only a few experiments have reported animals’ ability to detect absolute number. Capaldi and Miller (1988) tested rats’ ability to detect absolute number by using biologically important events—the number of reinforced runs followed by a nonreinforced run—and found that the rats ran significantly slower on the nonreinforced run. In the present experiments, we used a similar procedure. Pigeons were given a sequence of trials in which responding on the first three trials ended in reinforcement but responding on the fourth trial did not (RRRN). When the response requirement on each trial was a single peck (Experiment 1), we found no significant increase in latency to peck on the fourth trial. When the response requirement was increased to 10 pecks (Experiment 2), however, the time to complete the peck requirement was significantly longer on the nonreinforced trial than on the reinforced trials. Tests for control by time, number of responses, and amount of food consumed indicated that the pigeons were using primarily the number of reinforcements obtained in each sequence as a cue for nonreinforcement. This procedure represents a sensitive and efficient method for studying numerical competence in animals.     

Inhibitory effects of omission training

In Experiment 1, pigeons were trained to peck red or blue keys for food reinforcement at variable intervals, while food was contingent on withholding key pecks in the presence of a vertical line (omission training). When the line was briefly superimposed on red or blue in a compound test, responding was reduced. When the orientation of the line was varied during extinction, generalization gradients were variable but often had most responding at or near vertical. In Experiment 2, pigeons were trained in a discrete trials procedure that made food contingent upon pecking in the presence of triangle, and upon the absence of pecking in the presence of red (omission training). Food was never given on green-key trials (extinction). When red or green backgrounds were presented with the triangle in a compound test, responding was reduced similarly in the presence of both key colors. Subsequent resistance to auto-shaping was also similar for red and green. These data, taken together with reports in the literature, suggest that the inhibitory effects of omission training are quite similar to those of extinction. Thus, the crucial condition for obtaining inhibitory effects is not a negative stimulus-reinforcer correlation, as in extinction, but simply the establishment of low rates of responding to the inhibitory stimulus.     

Preference for 50% reinforcement over 75% reinforcement by pigeons

When pigeons are given a choice between an initial-link alternative that results in either a terminal-link stimulus correlated with 100% reinforcement or a stimulus correlated with 0% reinforcement (overall 50% reinforcement) and another initial-link alternative that always results in a terminal-link stimulus correlated with 100% reinforcement, some pigeons show a preference for the initial-link alternative correlated with 50% reinforcement. Using this procedure, in Experiment 1, we found a relatively modest preference for 100% over 50% reinforcement. In Experiment 2, we decreased the reinforcement density for the second initial-link alternative to 75% and found a significant preference for the 50% reinforcement initial-link alternative. It may be that this “maladaptive” behavior results from a positive contrast between the expectation of reinforcement correlated with the 50% reinforcement initial-link alternative and the terminal-link stimulus correlated with 100% reinforcement. But apparently, the complementary negative contrast does not develop between the expectation of reinforcement correlated with the 50% reinforcement initial-link alternative and the terminal-link stimulus correlated with 0% reinforcement that often follow. Such paradoxical choice may account for certain human appetitive risk-taking behavior (e.g., gambling) as well.     

The belief revision model: asymmetrical effects of noncontingency on human covariation learning

A noncontingent experience affects the subsequent detection of positive and negative contingencies between the same events. Experiments 1 and 2 showed that such preexposure can produce both an impairment in the detection of subsequent positive contingency and a facilitation of a negative one, independent of the level of contingency during the contingent phase. Experiment 3 raised difficulties for a model that assumes that associations to the context can explain this asymmetrical effect. Experiment 4 suggested that the different weights usually assigned to the different types of trials when computing the contingency between events can change as a result of a noncontingent experience with the same events. This change supports an account of the asymmetrical effect by a belief revision model based on a mechanism that updates the weights of the different trial types as a function of previous experience. More generally, the belief revision model is a statistical (i.e., nonassociative) model of learning that is capable of accounting for trial-order effects, which have long posed problems for statistical models.     

Extensive training,partial reinforcement,and temporal gaps do not affect S-S learning in second-order autoshaping

A second-order autoshaping procedure was used to examine the effects of three variables on the amount of information that could be learned about the stimulus properties of a reinforcer. All three experiments paired several keylight S2s with different keylight S1s and then carried out discriminative autoshaping with those S1s. Learning about the stimulus properties of S1 was inferred from changes in the response to its paired S2 when that S1 was changed in value. The sensitivity of S2 to changes in S1 was investigated as a function of number of S2-S1 pairings (Experiment 1), partial reinforcement (Experiment 2), and temporal distance between S2 and S1 (Experiment 3). Each experiment found evidence of a selective change in responding to an S2 as a function of the treatment of its S1. However, the amount of that change was not affected by any of the three variables studied. Those results imply that, within the ranges used here, none of these variables changes the degree of learning about the stimulus properties of a reinforcer.     

Acquired equivalence of cues in pigeon autoshaping: Effects of training with common consequences and with common antecedents

In three experiments, we sought evidence for the acquired equivalence of cues in pigeons trained in an autoshaping paradigm. In Experiment 1, presentations of each of a pair of cues (different keylight stimuli) preceded a common consequence (a different keylight stimulus). The pattern of response then established by further training given to one member of the pair was found to generalize preferentially to the other, demonstrating equivalence between cues that had shared a common consequence. The same test procedure was used in Experiment 2, but with a training procedure in which each cue of a pair was preceded by a given stimulus. This too resulted in enhanced generalization between members of the pair, showing that equivalence can be established when cues have been experienced along with a common antecedent. Both training procedures were combined in Experiment 3 to confirm the reliability of the effects previously obtained. The discussion is focused on ways in which the associative explanation offered for cases of equivalence mediated by a common consequence might be extended to accommodate equivalence mediated by a common antecedent.