The isolation of stimulus-reinforcer associations established with multiple schedules

Multiple schedules established stimulus-reinforcer (S-S^R) associations on baselines in which equal response rates and patterning were maintained in all components. Subsequently, stimuli associated with an increase in reinforcement but no change in ongoing response rate were compounded. For one experimental group, free-operant avoidance (FOA) was programmed in tone and in light while variable-interval (VI) food reinforcement was effective in their simultaneous absence (T + L). The opposite stimulus-schedule combinations were programmed for the other. Both groups remained in their VI components 85% of the session on schedule preference tests, and on a stimulus compounding test emitted approximately 1.5 times as many responses to tone-plus-light (T + L) as to tone or light alone. This is the first report of additive summation to combined discriminative stimuli associated with only an increase in reinforcement. Nondifferentially trained controls who had the same contingency effective in tone, light, and T + L-VI or FOA—showed neither preference among schedule components or summation during stimulus compounding, indicating that nonassociative stimulus factors made no contribution to either resultant in the experimental animals. Evidence supporting an algebraic combination of response and reinforcement associations is presented, and functional similarities between transfer-of-control studies and the stimulus compounding tests of the experimental groups in the present experiment are discussed.     

Integrating control generated by positive and negative reinforcement on an operant baseline: Appetitive-aversive interactions

The interaction of opposing motivational states was measured within a design in which rats barpressed for food in one stimulus and to avoid shock in another. Tone and light discriminative stimuli (Sds) were counterbalanced over incentive conditions. Extinction eliminated responding when neither Sd was present. To minimize the influence of competing peripheral operants or reinforcer-elicited behaviors during appetitive-aversive interactions, contingency parameters were manipulated to generate similar rates and patterns of barpressing in both Sds and stimulus-compounding tests were administered in extinction. On these tests, rates in tone, light, and tone plus light (T+L) were equivalent. In contrast, when the same reinforcer (i.e., food or shock avoidance) maintained comparable training rates in tone and in light, in testing, T+L controlled double the rates of the single stimuli—strong additive summation. These results were strikingly similar to those of single-incentive experiments concerned with the contribution of excitatory and inhibitory incentive states to the results of stimulus compounding. Simultaneously presenting two Sds whose implicit stimulus-reinforcer (S-S^r) contingencies were arranged to make them, respectively, conditioned appetitive and aversive exciters (present experiment) produced test results comparable to those of two Sds whose implicit S-S^r contingencies were arranged to make them both conditioned inhibitors. Reciprocal antagonism between these two motive states more than neutralizes them. It appears to produce a negative (i.e., an inhibitory) motive state.     

Failure to block control by a relevant stimulus

Pigeons were trained to depress a treadle in the presence of a discriminative stimulus, either a tone or illumination of red houselights, in order to obtain access to grain or avoid electric shock. In avoidance training, the auditory discriminative stimulus yielded faster acquisition than did the visual one. In appetitive training, the visual discriminative stimulus yielded faster acquisition than the auditory one. Experiments 2 and 3 used these stimuli in Kamin’s (1969) blocking design. In Experiment 2, when the pigeons were trained to depress a treadle in the presence of tone to obtain grain and then red light was added as the redundant stimulus, the light acquired stimulus control over treadlepressing; blocking was not observed. In Experiment 3, when the pigeons were trained to depress a treadle in the presence of red light to avoid electric shock and then tone was added as the redundant stimulus, the tone acquired stimulus control over treadle-pressing. Again, blocking was not observed. The implications of these results for several models of stimulus control are discussed.     

Behavioral contrast and inhibitory stimulus control

Two experiments examined the presumed relationship between behavioral contrast and inhibitory stimulus control. In Experiment I, pigeons were exposed to mult VI 1-min VI 1-min or mult VI 5-min VI 5-min during baseline training prior to mult VI 1-min VI 5-min discrimination training. Half of the subjects received a timeout (TO) component during baseline in order to reduce the degree of contrast during discrimination training. Only 3 of 8 subjects receiving the TO showed contrast while all other subjects showed various degrees of contrast. Postdiscrimination generalization gradients indicated excitatory rather than inhibitory control by the stimulus associated with the VI 5-min schedule. During baseline training in Experiment II, responding to all the generalization stimuli was reinforced. In addition, some subjects received the TO stimulus. The subjects were next exposed to mult VI 1-min EXT, mult VI 1-min VI 5-min, or just the VI 5-min component. Generalization gradients indicated inhibitory control by the stimulus associated with EXT or VI 5-min for 19 of 20 subjects even though some subjects did not show contrast. These results question the presumed relationship between behavioral contrast and inhibitory stimulus control.     

Choice responding following multiple schedule training

Pigeons’ choice responding on 10-sec interpolated probes was studied after baseline training on multiple variable-interval variable-interval schedules of food reinforcement. Unreinforced choice following training with three different relative reinforcement rates (Experiment 1), with a 3-ply multiple schedule (Experiment 2), and with three different relative reinforcement durations (Experiment 3) was examined. Least squares lines were fit to choice relative response rate and schedule relative response rate as functions of training relative reinforcement rate; choice slope was significantly greater than schedule slope in all three experiments. This result is counter to the prediction of Herrnstein’s (1970) theory that these slopes should not differ. Luce’s (1959) theory also failed to account for the data. It was concluded that choice responding was controlled by both approach to the stimulus associated with the smaller mean interreinforcer interval or the longer duration, and avoidance of the other stimulus.     

Preweanling and adult rats treat conditioned light-tone combinations differently

Ontogenetic differences in processing light-tone compounds were discovered in preweanling (17-day-old) and adult (60–80-day-old) rats. Suppression of general activity was used as an index of the magnitude of conditioned fear following a single training session in which a CS+ was paired with mild footshock. In Experiment 1, rats were trained on discriminations in which the CS? consisted of a light and the CS+ was either a tone alone (simple discrimination) or a light-tone compound (simultaneous feature-positive discrimination). Adults and preweanlings given each type of discrimination were then tested for fear of the CS? and a target stimulus (tone alone or light-tone compound). Adults in all groups displayed greater fear of the target than of the CS?. Preweanlings, however, discriminated the CS? from the target only when the target was the same as the original CS+. Experiment 2 revealed that age-related differences in conventional stimulus generalization is not a likely explanation for the pattern of results found in Experiment 1. Experiment 3 revealed age-related differences in expressed fear of a serial feature-positive discrimination; adults, but not preweanlings, showed greater fear of the compound than of the CS?. Alternative interpretations of the results from these experiments are discussed, and the general conclusion is that adults appear more inclined to process elements of a compound stimulus selectively, whereas preweanlings seem more likely to process the compound unselectively, with roughly equivalent processing of each element.     

Multiple-schedule interactions and discrimination

Pigeons received variable-interval (VI) reinforcement for keypecking during randomized presentations of seven line-orientation stimuli forming a continuum ranging from horizontal (0 deg) to vertical (90 deg). Each line presentation lasted for 30 sec and was preceded and followed by 30-sec time-outs. After responding stabilized, only responding in the two extreme stimuli (0 and 90 deg) was reinforced. As discrimination training proceeded, strong behavioral contrast and dimensional contrast effects appeared. However, only marginal local effects (local contrast and local dimensional effects), exerted by one line-orientation component upon a second, appeared, indicating that behavioral and dimensional contrast may be independent of parallel local effects. An attempt was made to apply Blough’s (1975) quantitative model of operant generalization and discrimination to the present discrimination procedure. However, this model did not predict the generalization gradient shape that was experimentally obtained. This experiment also yielded two serendipitous findings: (1) Positive behavioral contrast appeared in an extinction-related stimulus (time-out) when other stimuli were switched from reinforcement to extinction (hitherto, positive behavioral contrast had been observed only in responding to a reinforcementrelated stimulus when other stimuli were switched from reinforcement to extinction), and (2) final responding was higher in the presence of an extinction stimulus that had always been an extinction stimulus than it was in the presence of other extinction stimuli that had previously been paired with VI reinforcement.     

Covariation in conditioned response strength between stimuli trained in compound

Conditioned lick suppression by water-deprived rats was used to elaborate on recent evidence that the attenuated conditioned response elicited by an overshadowed stimulus may be enhanced by extinction of the overshadowing stimulus with which it had been trained in simultaneous compound. Using a modified serial stimulus arrangement in which a light coexisted with the last half of a tone that terminated with footshock, it was found in Experiment 1 that the tone overshadowed the light. Extinction of the tone-shock association resulted in a virtually complete recovery of the response to the overshadowed light. Using this serial overshadowing procedure, the possibility that the strength of a conditioned response to an element trained in compound covaries as a function of the strength of the response to the other element was tested in Experiment 2. Following overshadowing training similar to that of Experiment 1, independent reinforcement of the overshadowed light, that is, associative inflation, was found to have no deleterious effect on the response to the overshadowing tone. This suggests that the effects of postconditioning extinction and inflation of one element do not have symmetrical effects upon responding to the other element. The results of Experiment 2 were replicated in Experiment 3 using a simultaneous compound stimulus as opposed to the serial compound of the previous studies. These results are discussed in terms of various associative and cognitive models of learning and performance.     

Overshadowing by food of stimulus control by a keylight in a two-element serial compound

Pigeons were given successive discrimination training in which pecking during a choice period when the key was white was either reinforced or not, depending upon the prior presence or absence of a discriminative stimulus, which was a two-element serial compound. The compound consisted of a keylight and food, with food presented second or first in a forward or backward pairing for different groups of pigeons. In Experiment 1, the sequence was an S+ indicating reinforced trials, while in Experiment 2, the sequence was an S? indicating nonreinforced trials. Following acquisition of discriminated operant behavior, a sequence generalization test was administered during which all possible orders of the two stimuli were presented on test trials prior to the onset of the choice period. The results showed that food overshadowed stimulus control by the color of the light on the key on the sequence-generalization test, independently of whether food was presented first or second during training and independently of whether food was associated with reinforcement or nonreinforcement. The similarity of results for the two experiments suggests that overshadowing occurs independently of whether the compound is a discriminative stimulus for reinforcement or nonreinforcement. Simultaneous presentation of elements of a compound stimulus is not necessary for overshadowing because the phenomenon was captured with sequentially presented stimuli.     

Effects of memory reactivation treatments on postdiscrimination generalization performance in pigeons

In Experiment 1, it was shown that generalization testing following successive discrimination training between two closely spaced wavelengths results in a sharp gradient with a peak of responding shifted from S+ so as to be further removed from S?. Testing after a 24-h delay resulted in a flatter gradient with greater peak (and area) shift. A 5-min pretest exposure to S+, reinforced or unreinforced, or to S? (unreinforced) reinstated immediate test performance; free reinforcement with no discriminative stimulus present had no such effect. Experiment 2 replicated the flattening of generalization gradients and enhanced peak shift in delayed testing. Free feeding in a pretest treatment with a distinctive food uniquely associated with the wavelength discrimination problem failed to reinstate immediate test performance. Experiment 3 tested the hypothesis that free feeding failed as a reactivation treatment because it did not engender keypecking. Subjects were trained to peck a vertical line stimulus before being given wavelength discrimination training. Again, the enhanced peak shift and greater flattening with delayed wavelength generalization testing was found. A pretest exposure to the vertical line stimulus elicited pecking but had no effect on subsequent wavelength generalization. Thus, only a reactivation treatment that included one of the discrimination training stimuli was effective in producing delayed test performance comparable to that obtained in an immediate test.     

Stimulus specificity of concurrent recovery in the rabbit nictitating membrane response

Three experiments demonstrated that, following the extinction of an established conditioned stimulus (CS; e.g., tone), the pairing of an orthogonal stimulus from another modality (e.g., light) with the unconditioned stimulus (US) results in strong recovery of responding to the extinguished CS. This recovery occurred to about an equal degree regardless of whether or not initial training contained unambiguous stimulus—reinforcer relationships—that is, consistent CS—US pairings—or some degree of ambiguity, including intramodal discrimination training, partial reinforcement, or even cross-modal discrimination training (tone vs. light). Experiments 1 and 2 demonstrated that this recovery of responding was largely specific to the extinguished CS, but moderate generalization to other stimuli from the same modality did appear. The results are discussed with reference to alternative mechanisms applicable to learning-dependent generalization between otherwise distinct CSs. These models assume that such generalization is mediated by either a shared response, shared reinforcer, shared context, or shared hidden units within a layered neural network. A specific layered network is proposed to explain the present results as well as other types of savings seen previously in conditioning of the rabbit nictitating membrane response.     

Training reinforcement rates,resistance to extinction,and the role of context in reinstatement

Behavior reduced as a consequence of extinction or intervention can relapse. According to behavioral momentum theory, the extent to which behavior persists and relapses once it has been eliminated depends on the relative training reinforcement rate among discriminative stimuli. In addition, studies of context renewal reveal that relapse depends on the similarity between the training stimulus context and the test stimulus context following disruption by extinction. In the present experiments with pigeons, we arranged different reinforcement rates in the presence of distinct discriminative stimuli across components of a multiple schedule. Following extinction, we attempted to reinstate responding in the presence of those target components with response-independent food presentations. Importantly, we arranged the reinstating food presentations either within the target components or in separate components, either paired with extinction (Experiment 1) or reinforcement (Experiment 2) during baseline. Reinstatement increased with greater training reinforcement rates when the reinstating food presentations were arranged in the target components and the separate components paired with reinforcement during training. Reinstatement was smaller and was not systematically related to training reinforcement rates in the target components when reinstating food presentation occurred in separate components paired with extinction. These findings suggest that relapse depends on the history of reinforcement associated with the discriminative stimuli in which the relapse-inducing event occurs.     

Information effects on the response-reinforcer association

Pigeons were trained on a discrete-trial delayed reinforcement procedure with respect to one response key that was periodically illuminated. In some conditions, a second response key, or a tone, both previously paired with reinforcement, was interpolated in the delay-of-reinforcement interval. In comparison to a control condition with neither stimulus in the delay interval, the interpolated stimulus attenuated (blocked) the amount of responding that was maintained by the delayed reinforcement contingency. The degree of blocking was unaffected by whether the interpolated stimulus was the tone or keylight, in spite of the fact that the keylight evoked responding and the tone did not. A second study showed that the blocking effects involved the response-reinforcer association in that blocking occurred when the delayed reinforcement was response-dependent but did not occur when reinforcement was response-independent. The results thus show that response-reinforcer associations are affected by informational variables in the same manner as has been shown for stimulus-reinforcer associations. They also demonstrate that preexisting stimulus-reinforcer associations can block response-reinforcer associations, thus suggesting that both types of association depend upon the same associative process.     

Conditioning of simultaneous and serial feature-positive discriminations

Three experiments with rat subjects examined the development of simultaneous and serial feature-positive discriminations in appetitive conditioning. In Experiment 1, reinforced presentations of a simultaneous light-tone compound were intermixed with nonreinforced presentations of either the light or the tone. The compound stimulus acquired conditioned behaviors of a form characteristic of the predictive feature alone; the element common to reinforced and nonreinforced trials did not evoke conditioned behavior. In Experiment 2, reinforced presentations of a serial light-trace-tone compound were intermixed with nonreinforced tone-alone presentations. The light feature stimulus acquired conditioned behaviors characteristic of visual CSs. The tone stimulus, common to reinforced and nonreinforced trials, evoked conditioned behaviors characteristic of auditory CSs, but only when preceded by the light. In Experiment 3, variations in the interval between the light and tone on reinforced trials had little effect on responding to the light CS but substantially altered the pattern of responding to the tone CS. These results suggested that simultaneous and serial feature-positive discriminations may be solved differently. Performance in simultaneous feature-positive discriminations may be determined solely by associations between the feature stimulus and the reinforcer, but performance in serial discriminations may also involve the acquisition of a conditional cue function to the feature.     

Go-no go avoidance discriminations in rats with simple “go” and compound “no go” signals: Stimulus modality and stimulus intensity

Four groups of 16 rats were trained in a go-no go discrimination with symmetrical negative reinforcement (active-passive avoidance task). A 2 by 2 design was used with stimulus combinations as one factor [light-go, noise/light-no go, (L⁺ NL^?) vs noise-go, light/noise-no go (N⁺ LN^?)] and noise intensity as the other factor (90 dB, high, vs 70 dB, low). The two N⁺ LN^? groups learned both active and passive avoidance responses much more rapidly than the L⁺ NL^? groups. In later phases of training, high noise intensity facilitated passive avoidance in the L⁺ NL^? condition and exerted a slight effect in the opposite direction in the other condition. While the differences in active avoidance learning were the same as in previous work on light and noise CSs, the interactions between stimuli of different modalities appeared to be more important than stimulus modality or stimulus intensity per se in determining rate of passive avoidance learning.     

Effects of stimulus alternations on extinguishing a discrimination

Rats were trained in a light ON vs light OFF discrimination in operant chambers with food reinforcement. Following acquisition, extinction under conditions of no alternations between S+ and S? and under conditions of numerous alternations between S+ and S? were examined. In Experiment I, extinction in S+ or S? alone produced less responding to S+ and more responding to S? than extinction with regular alternations between S+ and S?. In Experiment II, 9, 39, or 79 alternations in extinction between S+ and S? produced no differences in responding. These results indicate that during extinction of a discrimination there is (a) sharpening of differential performance, (b) a difference between multiple- and single-stimulus procedures, and (c) little effect of different numbers of alternations.     

Differential response gradients to a Pavlovian safety signal following active avoidance training in young and adult rats

Following 300 training trials in two-way shuttle avoidance signaled by a tone (CS+), two groups each of weanling and adult rats were given Pavlovian discrimination training in which the CS+ was followed by inescapable shock, and a more intense tone (CS—) signaled no shock. An additional group at each age level received both tones paired randomly with shock or no shock. Subsequent generalization tests along the frequency dimension indicated that both pups and adults tested at the CS+ intensity showed similar gradients of frequency control. Gradients for the adults tested at the CS — intensity tended to be inverted, with least responding at CS—, a result not found in the young subjects. The results were considered in light of Pavlovian extradimensional influences on the control of avoidance behavior.     

Summation and configuration between and within sensory modalities in classical conditioning of the rabbit

Three experiments examined “atomistic” and “configurai” processes in stimulus compounding using the rabbit’s conditioned nictitating membrane response. Two conditioned stimuli (CSs) were trained separately and then tested together in a compound. Animals trained with CSs from different modalities—namely, tone and light—showed summation in both acquisition and extinction. That is, the probability of a response to the compound could be predicted by the statistical sum of responding to the CSs. In contrast, animals trained with CSs from the auditory modality, tone and noise, showed a level of responding to the tone + noise compound that was the same as that of the CSs, well under the level predicted by the statistical sum of responding to the CSs. In conclusion, atomistic processes appear to predominate in cross-modal compounding. Configurai processes may occur during compounding within the auditory modality, but atomistic alternatives—namely, common elements and selective attention hypotheses—may be able to explain the results.     

Contextual cues and the retrieval of competing memories of goal events

In Experiment 1, four groups of rats were initially trained on a discrimination which established a stimulus as a signal for reinforcement. That signal was then presented during subsequent partial reinforcement training in a way that could potentially interfere with retrieval of the memory of nonreinforcement (S^N) on the preceding trial either because (1) thestorage and retrieval contexts for S^N were different (retrieval failure hypothesis), or (2) the memory of reinforcement produced by the signal acted as a competing memory (competing memory hypothesis). Experiment 1 supported the competing memory hypothesis. In Experiment 2, we investigated the effect of stimulus change on the capacity of the context to retrieve a competing memory of a temporally remote reinforcement event with which the context was strongly associated. Retrieval of a competing memory was impaired by differences between the storage and retrieval contexts in a manner analogous to the effect of context on retrieval of a reinforcement event memory from an immediately preceding trial.     

Extradimensional transfer in the discriminative learning of goldfish

In Experiment I, one group of goldfish (TD) was trained to discriminate blue and green while a second group (PD) was exposed to the same colors in a “pseudodiscrimination,” after which both groups were reinforced for response to a tone. The TD group subsequently showed a sharper auditory discrimination gradient than the PD group and performed better in a differentially reinforced tone discrimination. The former PD animals then were given true discrimination training and the former TD animals pseudodiscrimination training with the colors, after which the first group showed better tone discrimination than the second. These results are analogous to those found in pigeons and rats. In Experiment II, goldfish which were trained in an easy color discrimination and shifted to a more difficult tone discrimination performed better than a control group trained from the outset with the tones. This result suggests that the dimensional specificity of the processes responsible for “transfer along a continuum” cannot safely be assumed in the absence of appropriate controls.