Transfer between Pavlovian facilitators and instrumental discriminative stimuli

Two experiments assessed the degree to which Pavlovian facilitators were interchangeable with instrumental discriminative stimuli (S^ds). In Experiment 1, rats were trained in a Pavlovian paradigm in which one stimulus (i.e., a facilitator) signaled the reinforcement of another stimulus (i.e., a target). Next, the rats were given instrumental discrimination training in which an S^d signaled the reinforcement of barpressing. A transfer test then assessed the capacity of the Pavlovian facilitator to promote barpressing. The results showed that the facilitator promoted significant barpressing, both when it was presented alone and when it was presented in compound with the S^d. Reliable transfer was not obtained with a “pseudofacilitator” control stimulus that, during training, was uninformative about the reinforcement of its target. Experiment 2 showed that a stimulus trained as an instrumental S^d reliably augmented responding to a stimulus previously trained as a target in a Pavlovian facilitation paradigm. A “pseudo-S^d” that, during training, was uninformative about the reinforcement of barpressing failed to promote such transfer. These results show that Pavlovian facilitators and instrumental S^ds are interchangeable to a significant degree, and suggest that facilitators and S^ds may act via similar mechanisms.     

The isolation of stimulus-reinforcer associations established with multiple schedules

Multiple schedules established stimulus-reinforcer (S-S^R) associations on baselines in which equal response rates and patterning were maintained in all components. Subsequently, stimuli associated with an increase in reinforcement but no change in ongoing response rate were compounded. For one experimental group, free-operant avoidance (FOA) was programmed in tone and in light while variable-interval (VI) food reinforcement was effective in their simultaneous absence (T + L). The opposite stimulus-schedule combinations were programmed for the other. Both groups remained in their VI components 85% of the session on schedule preference tests, and on a stimulus compounding test emitted approximately 1.5 times as many responses to tone-plus-light (T + L) as to tone or light alone. This is the first report of additive summation to combined discriminative stimuli associated with only an increase in reinforcement. Nondifferentially trained controls who had the same contingency effective in tone, light, and T + L-VI or FOA—showed neither preference among schedule components or summation during stimulus compounding, indicating that nonassociative stimulus factors made no contribution to either resultant in the experimental animals. Evidence supporting an algebraic combination of response and reinforcement associations is presented, and functional similarities between transfer-of-control studies and the stimulus compounding tests of the experimental groups in the present experiment are discussed.     

Replacement of event-generated memories of nonreinforcement with signal-generated memories of reinforcement during partial reinforcement training: Effect on resistance to extinction

Event-generated memory refers to the memory of a reinforcement (R) or nonreinforcement (N) event from an immediately preceding trial;signal-generated memory refers to the memory of a temporally remote R or N, retrieval of which is generated by presentation of a signal with which the memory is associated (Haggbloom, 1988). In each of three experiments, Group Signal-R received runway discrimination training in Phase 1 to establish a stimulus as a signal for R, and partial reinforcement training in Phase 2. An extinction test measured learning about the memory of nonreward (S^N)—learning that occurs when S^N is retrieved on R trials that follow N trials. In Group Signal-H, those R trials were accompanied by the signal for R, a treatment we hypothesized would generate retrieval of the memory of reinforcement (S^R) so that signal-generated S^R would replace event-generated S^N as the operative memory, thereby eliminating the increased resistance to extinction normally produced by PRF training. In each experiment, Group Signal-R was less resistant to extinction than was a control group conditioned to respond to-event-generated S^N. Extinction was as rapid in Group Signal-R as it was in a consistent reinforcement control group (Experiment 1) and in a group given intertrial reinforcements to interfere with learning about S^N (Experiment 3). Experiment 2 tested two alternative interpretations of the failure to learn about S^N in Group Signal-R. Those alternatives were found to be less viable than the hypothesis that the signal for R actively recruited retrieval of a competing memory.     

Oddity of visual patterns conceptualized by pigeons

Pigeons were trained to learn an instrumental oddity-from-sample discrimination involving visual forms. One group, the “few examples” group, dealt with 5 patterns in 40 different combinations. Another group, the “many examples” group, dealt with 20 patterns in 160 different combinations. After both groups had reached asymptotic performance and had learned to operate under partial reinforcement conditions, they were tested for transfer under extinction conditions with two different groups of 5 novel patterns, each in 40 combinations. All animals showed significant above chance transfer to both of these novel stimulus sets. Transfer performance with test stimuli of similar geometric design to training stimuli was better than performance with stimuli of markedly different design. The transfer performance of the “many examples” group was marginally better than that of the “few examples” group, even though the latter’s performance on the training stimuli was better throughout. It is concluded that pigeons can learn to employ an oddity concept and that this may be promoted by the use of many training exemplars. Furthermore, it is inferred that pigeons may normally use a mixture of strategies to solve oddity and identity problems.     

Variations of two temporal parameters in observing response procedures

Temporal parameters were varied in two different observing response procedures. In Experiment I, concurrent variable-interval chain schedules were employed. Responding on one key led to either a stimulus correlated with reinforcement or a stimulus correlated with time-out. Responding on the other key led to a stimulus which ended either in reinforcement or time-out. The duration of the delay to reinforcement or time-out was varied, the delays for all three stimuli always remaining equal in a given phase. It was found that the longer the delay, the greater the preference for the observing response. In Experiment II a procedure was employed in which birds pecked during a “trial” to produce stimuli correlated with reinforcement or time-out at the end of the trial. The duration of the trial ending in time out was varied while the positive trial duration remained constant. It was found that the longer the duration of the negative trial, the greater the strength of observing responses. The results were interpreted as supporting the hypothesis that the value of a positive stimulus is a function of time spent in stimuli correlated with nonreinforcement.

     

Differential second-order aversive conditioning using contextual stimuli

Two experiments were conducted to determine if contextual stimuli used as S₂ in a higher-order differential conditioning procedure would control the performance of rats. Discrete stimuli were first paired with footshock in a separate training context. During second-order training, a shock-associated discrete stimulus was presented in one of two discriminable observation chambers. Over 4 days of training, subjects engaged in more freezing in the context associated with an excitatory discrete S₁, relative to a context in which no discrete stimulus, or a stimulus that had been explicitly unpaired with shock delivery, was presented. After acquisition of the second-order discrimination, animals were returned to the original training context where they received a “signaled inflation” treatment designed to change the current value of S₁, and the US. This postconditioning manipulation did not selectively affect performance of defensive freezing or conditional analgesia in S₂.     

The effects of stimulus similarity on different types of behavioral contrast

Pigeons were trained on multiple schedules with component stimuli of different degrees of similarity. In Experiment 1, a two-component schedule was used in which the two stimuli were either two line orientations or a line orientation versus a diffuse color. Reinforcement rate was varied in one component to determine the effects of stimulus similarity on different aspects of behavioral contrast. Contrast in terms of average response rates (molar contrast) was larger with less similar stimuli. Local contrast effects at the beginning of the component were larger for more similar stimuli, but these effects were more variable and did not attain statistical significance. Independent of the level of molar contrast, the local pattern of schedule interaction differed for the two levels of similarity: with more similar stimuli, the maximum degree of interaction occurred at the beginning of the components and then decreased; with less similar stimuli, the degree of interaction increased throughout the components and was at its maximum near their end. In Experiment 2, the same three stimuli were used while reinforcement rate in the middle component of a three-component sequence was varied; this isolated the effects of the preceding schedule from those of the following schedule. Contrast effects were generally greater in the target component preceding the variable schedule, and these were enhanced by less similar stimuli. Contrast in the target component following the variable schedule was manifested primarily in terms of the behavior at the beginning of the component, and these effects were inconsistently related to stimulus similarity. The functional separation of the effects of stimulus similarity on the different locations of contrast suggest that “anticipatory contrast” and “local contrast” depend upon different mechanisms, thus excluding any account of contrast solely in terms of relative rate of reinforcement.     

Value transfer in a simultaneous discrimination by pigeons: The value of the S+ is not specific to the simultaneous discrimination context

During simultaneous discrimination training, there is evidence that some of the value of the S⁺ transfers to the S^?. When the value of the S+ is altered outside the context of the simultaneous discrimination, two very different predictions are made concerning its effect on its S^?, depending on whether one views the S⁺ as an occasion setter or as a stimulus capable of transferring value. In four experiments, pigeons were trained with two similar simultaneous discriminations, A⁺B^? and C⁺D^?, and two single-stimulus trial types, A and C, (in which A always had greater nominal value than C). According to value transfer theory, on test trials, B should always be preferred over D, because B and D should be affected by the net values of A and C, respectively. According to an occasion setting account, however, D should be preferred over B because the presence of D signals a higher probability of reinforcement for responding to C than when C is alone, and/or the presence of B signals a lower probability of reinforcement for responding to A than when A is alone. In all four experiments, the pigeons preferred B over D, a result consistent with value transfer theory. Thus, an S^? can acquire value from an S⁺ even when that value is conditioned in a “context” different from that of the simultaneous discrimination.     

The influence of context-reinforcer associations on instrumental performance

In each of two experiments, rats were trained to press the lever in a Skinner box, food reinforcement being available on a variable-interval 60-sec schedule (VI 60). There followed an “exposure phase” for which the levers were removed from the boxes, and then a final test with the levers replaced to assess the effects of the intervening treatment on instrumental responding. Experiment 1 showed that simple exposure to the box reduced the vigor of instrumental performance in comparison with a condition in which food was made available during the exposure phase. Animals which received no exposure treatment also showed a relatively high rate of response. Experiment 2 demonstrated that an exposure treatment in which the occurrence of food is signaled by a light stimulus also leads to a decline in instrumental responding. These results are held to support the notion that associations between the context and the reinforcer serve to energize appetitive instrumental behavior.     

Effects of changeover contingencies on auditory stimulus control of two responses

Rats were exposed to a procedure in which auditory stimuli signaled which of two levers was associated with a variable-interval 60-sec schedule of food presentation. Presses on the lever that was not associated with the variable-interval schedule (“errors”) postponed availability of reinforcement on the other lever by either a fixed number of responses or a fixed amount of time. Increasing the number of responses by which “errors” postponed food availability enhanced the level of stimulus control, and. alter a relatively high degree of control had been achieved, reduction of the requirement had no effect. Control experiments ruled out extended exposure to the discrimination procedure as a factor in the increase in stimulus control and suggested that the time of introduction of a changeover contingent is an important determinant of its effect.     

Gradient form and sequential effects during generalization testing in extinction

Four naive pigeons were given six generalization tests in extinction after periods of pretraining in which S+ appeared with food reinforcement and S? appeared in extinction. An analysis of sequential effects among presentations of test stimuli showed that the overall gradient was influenced differently by stimuli at the extremes of the continuum of test stimuli and by S+ and adjacent stimuli. Gradients consisting of responding in each stimulus when it was preceded by an extreme stimulus tended to peak at S+, while gradients produced when each stimulus was preceded by S+ or an adjacent stimulus tended to show a peak shift. This was true whether the overall gradient showed a peak shift or not. Two naive subjects were added and four additional tests were given after pretraining in which unequal frequencies of reinforcement accompanied both S+ and S?. Results of all 10 tests show that sequential effects occur during generalization testing in extinction and that these “local dimensional effects” are unlike local contrast. These stimulus-specific sequential effects may greatly influence overall gradient form.     

Contextual cues and the retrieval of competing memories of goal events

In Experiment 1, four groups of rats were initially trained on a discrimination which established a stimulus as a signal for reinforcement. That signal was then presented during subsequent partial reinforcement training in a way that could potentially interfere with retrieval of the memory of nonreinforcement (S^N) on the preceding trial either because (1) thestorage and retrieval contexts for S^N were different (retrieval failure hypothesis), or (2) the memory of reinforcement produced by the signal acted as a competing memory (competing memory hypothesis). Experiment 1 supported the competing memory hypothesis. In Experiment 2, we investigated the effect of stimulus change on the capacity of the context to retrieve a competing memory of a temporally remote reinforcement event with which the context was strongly associated. Retrieval of a competing memory was impaired by differences between the storage and retrieval contexts in a manner analogous to the effect of context on retrieval of a reinforcement event memory from an immediately preceding trial.     

Temporal integration and instrumental conditioned reinforcement

Stimuli associated with primary reinforcement for instrumental behavior are widely believed to acquire the capacity to function as conditioned reinforcers via Pavlovian conditioning. Some Pavlovian conditioning studies suggest that animals learn the important temporal relations between stimuli and integrate such temporal information over separate experiences to form a temporal map. The present experiment examined whether Pavlovian conditioning can establish a positive instrumental conditioned reinforcer through such temporal integration. Two groups of rats received either delay or trace appetitive conditioning in which a neutral stimulus predicted response-independent food deliveries (CS1→US). Both groups then experienced one session of backward second-order conditioning of the training CS1 and a novel CS2 (CS1–CS2 pairing). Finally, the ability of CS2 to function as a conditioned reinforcer for a new instrumental response (leverpressing) was assessed. Consistent with the previous demonstrations of temporal integration in fear conditioning, a CS2 previously trained in a trace-conditioning protocol served as a better instrumental conditioned reinforcer after backward second-order conditioning than did a CS2 previously trained in a delay protocol. These results suggest that an instrumental conditioned reinforcer can be established via temporal integration and raise challenges for existing quantitative accounts of instrumental conditioned reinforcement.     

Memories of reward events and expectancies of reward events may work in tandem

A common assumption is that expectancies of reward events in instrumental tasks are established on the basis of Pavlovian conditioning. According to the tandem hypothesis, tested in the four runway investigations reported here employing rats, memories of reward events may serve as the conditioned stimuli eliciting expectancies. In Experiments 1–3, rats were trained under a schedule of partial reward (P), which did not produce increased resistance to extinction, and subsequently shifted to consistent reward (C). According to the tandem hypothesis, the shift to the C schedule should result in increased resistance to extinction if, as hypothesized, under the P schedule the memory of reward, S^R, came to elicit the expectancy of nonreward,_EN. This hypothesis was confirmed under a variety of conditions. It was shown that increased resistance to extinction could not be attributed to the P schedule alone, to the rats receiving two schedules, P and C, to stimuli other than S^R eliciting E_N, or to the rats forgetting reward-produced memories when expecting nonreward (Experiment 4). It was shown that the tandem hypothesis could explain the divergent findings obtained in prior studies employing a shift from P to C as well as in the present study.     

Pavlovian aversive to instrumental appetitive transfer: Evidence for across-reinforcement blocking effects

Rats received Pavlovian aversive (shock) conditioning in which white noise was established for different groups as a CS+, CSO, or CS?. Then, in an appetitive T-maze discrimination, the CSs were presented contingent upon a designated correct response for which food reinforcement was factorially varied at 0, 1, 2, or 4 pellets. Although the CS+ suppressed and the CS? facilitated speed of running in the correct arm at the start of discrimination training, these effects extinguished rapidly and did not interact with reward magnitude. Furthermore, choice learning was faciltated by the CS+ and retarded by the CS?, with these effects being comparable for the 1- to 4-pellet reinforcement conditions, but absent for the 0-pellet condition. These findings are difficult to reconcile with a transfer interpretation positing a general signaling property of the CS and are better interpreted as across-reinforcement blocking effects: By predicting a preferred outcome (safety) comparable to the preferred outcome of food reinforcement, the CS? blocks (retards) the association of reinforcement and the S^D; conversely, by predicting a nonpreferred (shock) outcome discrepant from the preferred food outcome, the CS+ “counterblocks” (enhances) the association of reinforcement and the S^D.     

Numerical discrimination by rats using sequential auditory stimuli

Three rats were trained under a discrimination procedure in which responding was reinforced only following the repeated presentation of three bursts of white noise (S+). S? consisted of presentations of either two or four bursts of noise. All animals responded significantly more in the presence of S+ and, in two cases, showed lower response rates to both “2” and “4” stimuli. Responding by the third animal revealed differentiation between S+ and the stimulus “2,” but no reliable suppression to stimulus “4.” The present instances of discriminative control by the stimulus “3” replicate Fernandes and Church’s (1982) demonstration of control by sequential auditory stimuli in the rat. Moreover, because the present procedure involves adjacent S? values both greater as well as less than S+, these results extend our knowledge of the rat’s abilities with sequential auditory stimuli: Rats are capable of making intermediate numerical discriminations based upon something other than a simple many-versus-few dichotomy.     

Impact of stimulus format and reward value on quantity discrimination in capuchin and squirrel monkeys

Quantity discrimination abilities are seen in a diverse range of species with similarities in performance patterns, suggesting common underlying cognitive mechanisms. However, methodological factors that impact performance make it difficult to draw broad phylogenetic comparisons of numerical cognition across studies. For example, some Old World monkeys selected a higher quantity stimulus more frequently when choosing between inedible (pebbles) than edible (food) stimuli. In Experiment 1 we presented brown capuchin (Cebus [Sapajus] paella) and squirrel monkeys (Saimiri sciureus) with the same two-choice quantity discrimination task in three different stimulus conditions: edible, inedible, and edible replaced (in which choice stimuli were food items that stood in for the same quantity of food items that were given as a reward). Unlike Old World monkeys, capuchins selected the higher quantity stimulus more in the edible condition and squirrel monkeys showed generally poor performance across all stimulus types. Performance patterns suggested that differences in subjective reward value might motivate differences in choice behavior between and within species. In Experiment 2 we manipulated the subjective reinforcement value of the reward by varying reward type and delay to reinforcement and found that delay to reinforcement had no impact on choice behavior, while increasing the value of the reward significantly improved performance by both species. The results of this study indicate that species presented with identical tasks may respond differently to methodological factors such as stimulus and reward types, resulting in significant differences in choice behavior that may lead to spurious suggestions of species differences in cognitive abilities.     

The overshadowing of instrumental conditioning by a stimulus that predicts reinforcement better than the response

In an experiment designed to demonstrate the overshadowing of appetitive instrumental conditioning, three groups of rats were given 10 sessions of RI (random interval) training in which reinforcement was delivered 450 msec following the response. The correlated group experienced a stimulus during the response-reinforcer delay interval, while an uncorrelated control group experienced a similar brief stimulus occasionally following responses, but these were not responses that typically produced reinforcement. A no-stimulus control group did not experience the brief response-produced stimulus. The experiment was run in two replications. The first employed a light as the critical stimulus, the second a tone. Over the 10 RI sessions, subjects in the two control conditions increased their rate of responding significantly faster than subjects in the correlated condition in both replications. This finding was interpreted as an instance of the overshadowing of the acquisition of signal value by a response because of the presence of the stimulus, which, in the correlated condition, was a more reliable predictor of reinforcement than the response. A subsequent conditioned reinforcement test confirmed that, in the correlated condition, the stimulus had, indeed, become a signal for reinforcement as a function of RI training.     

Deprivation level and choice in pigeons: a test of within-trial contrast

Within-trial contrast has been proposed as a mechanism underlying preferences for stimuli that follow relatively more aversive events over stimuli that follow less aversive events. In this study, we manipulated deprivation level to test within-trial contrast predictions. In Experiment 1, pigeons encountered two discriminative stimuli, one presented when they were deprived and the other when they were prefed. When later given a choice between the two stimuli, pigeons strongly preferred the stimulus encountered when deprived, independently of their deprivation level at test. In Experiment 2, pigeons learned two simultaneous discriminations, one when deprived and the other when prefed. Here, subsequent tests between the two S⁺ or the two S⁻ stimuli revealed no consistent preferences. These contrasting findings suggest that differential aversiveness is necessary but not sufficient to induce preferences via within-trial contrast.     

Behavioral momentum and relapse of extinguished operant responding

Previous experiments on behavioral momentum have shown that relative resistance to extinction of operant behavior in the presence of a stimulus depends on the rate of reinforcement associated with that stimulus, even if some of those reinforcers occur independently of the behavior. We present three experiments examining whether the rate of reinforcement in the presence of a stimulus similarly modulates the relative relapse of operant behavior produced by reinstatement, resurgence, and renewal paradigms. During baseline conditions, pigeons responded for food reinforcement on variable-interval 120-sec schedules in alternating periods of exposure to two stimuli arranged by a multiple schedule. Additional response-independent food presentations were also delivered in the presence of one of the multiple-schedule stimuli. Consistent with previous research, baseline response rates were lower in the presence of the stimulus with the added response-independent reinforcement, and relative resistance to extinction was greater in the presence of that stimulus. In addition, following extinction, the relative relapse of responding produced by reinstatement, resurgence, and renewal paradigms was greater in the presence of the stimulus associated with the higher rate of reinforcement. We suggest that a model of extinction from behavioral momentum theory may be useful for understanding these results.