Some tests of the additivity (autoshaping) theory of behavioral contrast

Two experiments were performed to determine whether the location of the discriminative stimuli affects the amount of positive behavioral contrast exhibited during discrimination learning by pigeons. When signals for reinforcement and nonreinforcement of keypecking were situated directly on the response key (different line tilts), pecking rates during the positive stimulus were higher than when the source of the signals was located elsewhere (changes in chamber illumination or auditory click frequency). These results are in general agreement with the additivity theory of behavioral contrast, which attributes contrast to the combined effects of stimulus-reinforcer and response-reinforcer correlations on behavior directed at signals of reinforcement. Some shortcomings of the theory were discussed, and the notion that behavioral contrast is a basic, unitary phenomenon was criticized.     

Information effects on the response-reinforcer association

Pigeons were trained on a discrete-trial delayed reinforcement procedure with respect to one response key that was periodically illuminated. In some conditions, a second response key, or a tone, both previously paired with reinforcement, was interpolated in the delay-of-reinforcement interval. In comparison to a control condition with neither stimulus in the delay interval, the interpolated stimulus attenuated (blocked) the amount of responding that was maintained by the delayed reinforcement contingency. The degree of blocking was unaffected by whether the interpolated stimulus was the tone or keylight, in spite of the fact that the keylight evoked responding and the tone did not. A second study showed that the blocking effects involved the response-reinforcer association in that blocking occurred when the delayed reinforcement was response-dependent but did not occur when reinforcement was response-independent. The results thus show that response-reinforcer associations are affected by informational variables in the same manner as has been shown for stimulus-reinforcer associations. They also demonstrate that preexisting stimulus-reinforcer associations can block response-reinforcer associations, thus suggesting that both types of association depend upon the same associative process.     

Additive summation by stimulus compounding irrespective of behavioral contrast during discrimination training: An investigation with positive reinforcement and avoidance schedules

The similarity in the discrimination training leading to behavioral contrast and that preceding tests producing response enhancement to combined discriminative stimuli suggested that the two phenomena might be related. This was investigated by determining if contrast indiscrimination training was necessary for this outcome of stimulus compounding. Responding to tone, light, and to the simultaneous absence of tone and light (T + L) was maintained during baseline training by food reinforcement in Experiment I and by shock avoidance in Experiment II. During subsequent discrimination training, responding was reduced in T + L by programming nonreinforcement in Experiment I and safety or response-punishment in Experiment II. In the first experiment, one rat exhibited positive behavioral contrast, i.e., tone and light rates increased while his T + L rate decreased. In Experiment II, rats punished in T + L showed contrast in tone and light, this being the first demonstration of punishment contrast on an avoidance baseline with rats. The discrimination acquisition data are discussed in the light of current explanations of contrast by Gamzu and Schwartz (1973) and Terrace (1972). During stimulus compounding tests, all subjects in both experiments emitted more responses to tone-plus-light than to tone or light (additive summation). An analysis of the terminal training baselines suggests that the factors producing these test results seem unrelated to whether or not contrast occurred during discrimination training. It was concluded that the stimulus compounding test reveals the operation of the terminal baseline response associations and reinforcement associations conditioned on these multicomponent free-operant schedules of reinforcement.

     

Multiple-schedule interactions and discrimination

Pigeons received variable-interval (VI) reinforcement for keypecking during randomized presentations of seven line-orientation stimuli forming a continuum ranging from horizontal (0 deg) to vertical (90 deg). Each line presentation lasted for 30 sec and was preceded and followed by 30-sec time-outs. After responding stabilized, only responding in the two extreme stimuli (0 and 90 deg) was reinforced. As discrimination training proceeded, strong behavioral contrast and dimensional contrast effects appeared. However, only marginal local effects (local contrast and local dimensional effects), exerted by one line-orientation component upon a second, appeared, indicating that behavioral and dimensional contrast may be independent of parallel local effects. An attempt was made to apply Blough’s (1975) quantitative model of operant generalization and discrimination to the present discrimination procedure. However, this model did not predict the generalization gradient shape that was experimentally obtained. This experiment also yielded two serendipitous findings: (1) Positive behavioral contrast appeared in an extinction-related stimulus (time-out) when other stimuli were switched from reinforcement to extinction (hitherto, positive behavioral contrast had been observed only in responding to a reinforcementrelated stimulus when other stimuli were switched from reinforcement to extinction), and (2) final responding was higher in the presence of an extinction stimulus that had always been an extinction stimulus than it was in the presence of other extinction stimuli that had previously been paired with VI reinforcement.     

Contextual cues and the retrieval of competing memories of goal events

In Experiment 1, four groups of rats were initially trained on a discrimination which established a stimulus as a signal for reinforcement. That signal was then presented during subsequent partial reinforcement training in a way that could potentially interfere with retrieval of the memory of nonreinforcement (S^N) on the preceding trial either because (1) thestorage and retrieval contexts for S^N were different (retrieval failure hypothesis), or (2) the memory of reinforcement produced by the signal acted as a competing memory (competing memory hypothesis). Experiment 1 supported the competing memory hypothesis. In Experiment 2, we investigated the effect of stimulus change on the capacity of the context to retrieve a competing memory of a temporally remote reinforcement event with which the context was strongly associated. Retrieval of a competing memory was impaired by differences between the storage and retrieval contexts in a manner analogous to the effect of context on retrieval of a reinforcement event memory from an immediately preceding trial.     

Variations of two temporal parameters in observing response procedures

Temporal parameters were varied in two different observing response procedures. In Experiment I, concurrent variable-interval chain schedules were employed. Responding on one key led to either a stimulus correlated with reinforcement or a stimulus correlated with time-out. Responding on the other key led to a stimulus which ended either in reinforcement or time-out. The duration of the delay to reinforcement or time-out was varied, the delays for all three stimuli always remaining equal in a given phase. It was found that the longer the delay, the greater the preference for the observing response. In Experiment II a procedure was employed in which birds pecked during a “trial” to produce stimuli correlated with reinforcement or time-out at the end of the trial. The duration of the trial ending in time out was varied while the positive trial duration remained constant. It was found that the longer the duration of the negative trial, the greater the strength of observing responses. The results were interpreted as supporting the hypothesis that the value of a positive stimulus is a function of time spent in stimuli correlated with nonreinforcement.

     

Overshadowing by food of stimulus control by a keylight in a two-element serial compound

Pigeons were given successive discrimination training in which pecking during a choice period when the key was white was either reinforced or not, depending upon the prior presence or absence of a discriminative stimulus, which was a two-element serial compound. The compound consisted of a keylight and food, with food presented second or first in a forward or backward pairing for different groups of pigeons. In Experiment 1, the sequence was an S+ indicating reinforced trials, while in Experiment 2, the sequence was an S? indicating nonreinforced trials. Following acquisition of discriminated operant behavior, a sequence generalization test was administered during which all possible orders of the two stimuli were presented on test trials prior to the onset of the choice period. The results showed that food overshadowed stimulus control by the color of the light on the key on the sequence-generalization test, independently of whether food was presented first or second during training and independently of whether food was associated with reinforcement or nonreinforcement. The similarity of results for the two experiments suggests that overshadowing occurs independently of whether the compound is a discriminative stimulus for reinforcement or nonreinforcement. Simultaneous presentation of elements of a compound stimulus is not necessary for overshadowing because the phenomenon was captured with sequentially presented stimuli.     

Specification of the stimulus-reinforcer relation in multiple schedules: Delay and probability of reinforcement

Control of pigeons’ keypecking by a stimulus-reinforcer contingency was investigated in the context of a four-component multiple schedule. In each of three experiments, pigeons were exposed to a schedule consisting of two two-component sequences. Discriminative stimuli identifying each sequence were present only in Component 1, which was 4, 6, or 8 sec in duration, while reinforcers could be earned only in Component 2 (30 sec in duration). Control by a stimulus-reinforcer contingency was sought during Component 1 by arranging a differential relation between Component 1 cues and schedule of reinforcement in Component 2. In Experiment 1, rate of keypecking during Component 1 varied with the presence and absence of a stimulus-reinforcer contingency. When a contingency was introduced, rate of keypecking increased during the Component 1 cue associated with the availability of reinforcement in Component 2. In Experiment 2, the stimulus-reinforcer contingency was manipulated parametrically by varying the correlation between Component 1 cues and Component 2 schedules of reinforcement. Responding in Component 1 varied as a function of strength of the stimulus-reinforcer contingency. The relatively high rates of Component 1 responding observed in Experiments 1 and 2 pose difficulties for conceptions of stimulus-reinforcer control based on probability of reinforcement. In these two experiments, the stimulus-associated probabilities of reinforcement in Component 1 were invariant at zero. An alternate dimension of stimulus-reinforcer control was explored in Experiment 3, in which Component 1 cues were differentially associated with delay to reinforcement in Component 2, while probability of reinforcement was held constant across components. When the stimulus-reinforcer contingency was in force, rate of responding in Component 1 varied inversely with delay to reinforcement in Component 2. In a quantitative analysis of data from Experiments 2 and 3, relative rate of responding during Component 1 was strongly correlated with two measures of relative delay to reinforcement.     

Compound conditioning in honeybees: Blocking tests of the independence assumption

Six experiments with color-odor compounds failed to produce convincing evidence of blocking in honeybees even when the possibility of masking by within-compound association could be discounted. The parsimonious assumption that the components of a compound stimulus gain and lose associative strength independently with reinforcement and nonreinforcement of the compounds (which the experiments were designed to challenge) remains tenable for color-odor compounds, although perhaps not for intramodal compounds.     

Addition versus deletion as a signal

Subjects typically show superior discriminative performance when a distinguishing feature appears on reinforced rather than nonreinforced trials. The phenomenon is usually attributed to the relative predictiveness of the reinforcer by different stimulus elements. However, stimulus addition may be more effective than stimulus deletion as a signal. By removing the standard intertriai intervals, we made addition and deletion equally predictive of the reinforcer in four operant experiments involving between- and within-subject comparisons. Pigeons consistently performed better on operant discriminations when the addition rather than deletion of an auditory or visual stimulus served as the cue for food. This general finding persisted despite manipulation of the relative duration and localizability of the signal. Thus mere presence as opposed to absence plays a role in the feature-positive superiority, an outcome that may reflect a fundamental, biologically based difference between addition and deletion as effective signals of reinforcement.     

Consummatory response latency and the stimulus-reinforcer relation in autoshaping

Autoshaping procedures with pigeons were used to assess the susceptibility of unconditioned response (UR) activity to Pavlovian relations between stimulus and reinforcer events. Foodpeck latency (a measure of UR activity) was investigated as a function of the interval between stimulus (keylight) and reinforcer (grain) presentations, and of the stimulus-reinforcer contingency, that is, the conditional probabilities of reinforcer delivery in the presence and absence of the stimulus. Four experiments indicated that food-peck latency was sensitive to both manipulations. Generally, conditions that led to higher keypeck rates were associated with shorter latencies. Thus, UR potentiation was demonstrated. However, when the bird’s location prior to grain delivery was fixed by imposing a keypeck-reinforcer contingency, UR potentiation vanished; it then reappeared when the location constraint was removed. Visual observations supported the conclusion that food-peck latency effects were mediated by approach/withdrawal tendencies generated by the stimulus-reinforcer relation. Implications of these results for expectancy theory are discussed.     

Signal-food contingency and signal frequency in a continuous trials auto-shaping paradigm

Five groups of pigeons were studied in an auto-shaping procedure which programmed two types of trials represented by hues on the response key. Each signal was separated by a brief intertriai interval. Three groups were studied with a positive correlation between one of the signals and food (contingent groups). They differed with respect to the frequency with which the positive signal appeared. Two noncontingent groups were studied in which the correlation between the signals and food was eliminated by programming food with the same probability following either signal. One noncontingent group had a high density of reinforcement produced by adding reinforcement in the other signal, at the same rate as programmed in the positive signal for the contingent groups. The other noncontingent group experienced the same number of reinforcements in the session as the contingent group with the least frequent positive trial, but these reinforcements were distributed with equal probability across the signals. Birds in the contingent groups with intermediate or infrequent positive signals all acquired reliable pecking, with acquisition most rapid for the infrequent signal. Maintained responding covaried with the speed of acquisition. No birds in the noncontingent groups showed reliable responding. Birds in the contingent group with a frequent positive signal (approximately 3/4 of the session), also showed no reliable pecking. This result suggests that more than one noncontingent group is informative for assessing the role of differential reinforcement probability in the acquisition of auto-shaped keypecking. In particular, a noncontingent group which controls for the frequency of reinforced trials is an appropriate reference group.

     

The free food (contrafreeloading) phenomenon: A review and analysis

Animals will perform an operant response to obtain food when abundant free food is available. These data have implications for current learning theories, especially in terms of the motivational variables associated with such behavior. The present paper reviews the literature and provides an analysis that suggests that responding for food in the presence of free food is importantly controlled by stimulus change attendant upon response-dependent food presentation. This apparent stimulus-reinforcer effect on behavior is compared to that observed in other areas of animal learning research that include preference between schedules of response-dependent and response-independent reinforcement, preference between schedules of signaled and unsignaled reinforcement, autoshaping and automaintenance, and self-reinforcement in animals.     

Replacement of event-generated memories of nonreinforcement with signal-generated memories of reinforcement during partial reinforcement training: Effect on resistance to extinction

Event-generated memory refers to the memory of a reinforcement (R) or nonreinforcement (N) event from an immediately preceding trial;signal-generated memory refers to the memory of a temporally remote R or N, retrieval of which is generated by presentation of a signal with which the memory is associated (Haggbloom, 1988). In each of three experiments, Group Signal-R received runway discrimination training in Phase 1 to establish a stimulus as a signal for R, and partial reinforcement training in Phase 2. An extinction test measured learning about the memory of nonreward (S^N)—learning that occurs when S^N is retrieved on R trials that follow N trials. In Group Signal-H, those R trials were accompanied by the signal for R, a treatment we hypothesized would generate retrieval of the memory of reinforcement (S^R) so that signal-generated S^R would replace event-generated S^N as the operative memory, thereby eliminating the increased resistance to extinction normally produced by PRF training. In each experiment, Group Signal-R was less resistant to extinction than was a control group conditioned to respond to-event-generated S^N. Extinction was as rapid in Group Signal-R as it was in a consistent reinforcement control group (Experiment 1) and in a group given intertrial reinforcements to interfere with learning about S^N (Experiment 3). Experiment 2 tested two alternative interpretations of the failure to learn about S^N in Group Signal-R. Those alternatives were found to be less viable than the hypothesis that the signal for R actively recruited retrieval of a competing memory.     

Negative discriminative stimuli provide information about the identity of omitted response-contingent outcomes

Three experiments using rats examined whether a signal for the nonreinforcement of an instrumental response (S-) provided information about the identity of the omitted outcome. In all three experiments, one stimulus was established as a signal for the nonreinforcement of a response that earned food pellets and another stimulus signaled the nonreinforcement of a response that earned liquid sucrose. Experiment 1 found that each S-suppressed another instrumental response trained with the same outcome significantly more than a response trained with a different outcome. Using a variant of this transfer design, Experiment 2 demonstrated that an S- was slower to develop discriminative control over a new response reinforced in its presence with the same outcome compared with an outcome different from the one whose omission-the S- had previously signaled. Experiment 3 examined transfer of the S- stimuli to a response trained with two outcomes, one of which had subsequently been devalued. Performance of this response significantly increased in the presence of a signal for the omission of the devalued outcome, but decreased in the presence of a signal for the omission of the valued outcome. These results suggest that S-s do provide information about the identity of omitted response-contingent outcomes.     

Discriminative stimuli that follow the absence of reinforcement are preferred by pigeons over those that follow reinforcement

Clement, Feltus, Kaiser, and Zentall (2000) found that when pigeons have to work to obtain a discriminative stimulus that is followed by reinforcement, they prefer a discriminative stimulus that requires greater effort over one that requires less effort. The authors suggested that such a preference results from the greater change in hedonic value that occurs between the more aversive event and the onset of the stimulus that signals reinforcement, a contrast effect. It was hypothesized that any stimulus that follows a relatively more aversive event would be preferred over a stimulus that follows a relatively less aversive event. In the present experiment, the authors tested the counterintuitive prediction of that theory, that pigeons should prefer a discriminative stimulus that follows the absence of reinforcement over a discriminative stimulus that follows reinforcement. Results supported the theory.     

The maintenance of matching behavior when contrast occurs in multiple concurrent concurrent schedules of reinforcement

Pigeons were trained on a multiple schedule of reinforcement in which each component was a concurrent schedule. The concurrent schedules were programmed by the changeover-key procedure. The primary purpose was to determine if the relative behavior allocated to two response alternatives is affected when absolute changes in these behaviors occur; i.e., to determine if matching is affected when positive behavioral contrast occurs. Results showed that (1) relative behavior in the unaltered component of the multiple schedule is not disrupted when positive contrast occurs in that component, (2) positive contrast occurred when the overall frequency of reinforcement in the reinforcement-correlated component(s) was high, but not when it was low, (3) changeover behavior was susceptible to positive contrast effects, and (4) changeover contrast and food-key contrast are independent phenomena.     

The isolation of stimulus-reinforcer associations established with multiple schedules

Multiple schedules established stimulus-reinforcer (S-S^R) associations on baselines in which equal response rates and patterning were maintained in all components. Subsequently, stimuli associated with an increase in reinforcement but no change in ongoing response rate were compounded. For one experimental group, free-operant avoidance (FOA) was programmed in tone and in light while variable-interval (VI) food reinforcement was effective in their simultaneous absence (T + L). The opposite stimulus-schedule combinations were programmed for the other. Both groups remained in their VI components 85% of the session on schedule preference tests, and on a stimulus compounding test emitted approximately 1.5 times as many responses to tone-plus-light (T + L) as to tone or light alone. This is the first report of additive summation to combined discriminative stimuli associated with only an increase in reinforcement. Nondifferentially trained controls who had the same contingency effective in tone, light, and T + L-VI or FOA—showed neither preference among schedule components or summation during stimulus compounding, indicating that nonassociative stimulus factors made no contribution to either resultant in the experimental animals. Evidence supporting an algebraic combination of response and reinforcement associations is presented, and functional similarities between transfer-of-control studies and the stimulus compounding tests of the experimental groups in the present experiment are discussed.     

Behavioral contrast and type of reward: Role of elicited response topography

Groups of pigeons were exposed to multiple variable-interval variable-interval and multiple variable-interval extinction schedules of either food or water reinforcement for keypecking. Discriminative stimuli associated with component schedules were located either on the operant key or on a second “signal” key. When the stimuli were projected on the operant key, positive contrast appeared during discrimination conditions with either food or water as the reinforcer. When the stimuli were projected on the signal key, overall responding to the operant and signal keys showed contrast with food, but negative induction with water as the reinforcer. In the latter condition, the signal for the variable-interval shcedule of water reinforcement elicited a variety of water-related behavior, only some of which was directed at the signal. Thus, the type of reward and location of discriminative stimuli interacted to determine the presence or absence of behavioral contrast effects. In large part, these results support and extend the autoshaping view of contrast.     

Effects of several extinction treatments upon the integrity of Pavlovian stimulus-outcome associations

Pavlovian-to-instrumental transfer tests were used to assess the sensitivity of Pavlovian stimulus-outcome (S-O) associations to various extinction treatments in four appetitive conditioning experiments with rats. In Experiment 1, simple nonreinforcement of a stimulus was shown to have little impact on the ability of that stimulus to display outcome-specific transfer of control. Extinguishing a stimulus by pairing the stimulus with an alternative reinforcement in Experiment 2 also had no detectable effect on the S-O associations as assessed with the outcome-specific transfer measure. The third and fourth experiments, respectively, examined the impact of postconditioning exposures to random and explicitly unpaired S-O contingencies upon previously learned S-O associations. These treatments, as well, had no detectable harmful effects upon the integrity of the S-O associations. In contrast to the consistent failures of various extinction treatments to influence the ability of stimuli to display outcome-specific transfer, these treatments often did reduce the strength of conditioned responding initially trained to these stimuli. These results support the view that extinction entails the preservation of S-O associations as well as the parallel development of inhibitory stimulus-response associations. Other notions of extinction are also discussed.