Competing sources of stimulus value in anticipatory contrast

In previous studies of anticipatory contrast, identical target components (A and B) preceded either a lower (extinction) or a richer schedule. Higher response rates occurred during the target preceding the lower rate of reinforcement, whereas preference was in favor of the target preceding the richer schedule. In Experiment 1, the response and preference measures were positively related when additional stimuli, with no reinforcement of their own, preceded the target components. The effect of these additional stimuli was presumed to be due to their overshadowing of the Pavlovian association between the target components and their following schedules. Experiment 2 also demonstrated a consistent relation between response rate and preference in a conditioned reinforcement procedure. In the absence of a strong Pavlovian association, anticipatory contrast, like other forms of contrast in free-operant procedures, reflects an increase in the value of the target component with an unchanged reinforcement schedule.     

Pavlovian contingencies and anticipatory contrast

Pigeons were trained on a four-component multiple schedule in which two target components with identical reinforcement schedules were followed by other components with either higher or lower reinforcement rates. The Pavlovian signal properties of the target-component stimuli were varied by changes in their duration relative to the following components, and by whether the two following components were cued by the same or different stimuli, When different stimuli occurred in the following components, response rates were higher in the target component preceding the following component with the lower reinforcement rate, and these contrast effects were larger with shorter relative durations. But with nondifferential stimuli in the following components, contrast consistently occurred only with the longer durations of the target components. Moreover, several subjects with the shorter duration target stimuli had higher response rates in the target followed by the richer schedule—that is, Pavlovian conditioning occurred to the target stimuli. This interaction suggests that the processes underlying anticipatory contrast and Pavlovian conditioning are in opposition, and that the Pavlovian effect can dominate ifthe signaling properties of the target components are sufficiently enhanced.     

The effects of stimulus similarity on different types of behavioral contrast

Pigeons were trained on multiple schedules with component stimuli of different degrees of similarity. In Experiment 1, a two-component schedule was used in which the two stimuli were either two line orientations or a line orientation versus a diffuse color. Reinforcement rate was varied in one component to determine the effects of stimulus similarity on different aspects of behavioral contrast. Contrast in terms of average response rates (molar contrast) was larger with less similar stimuli. Local contrast effects at the beginning of the component were larger for more similar stimuli, but these effects were more variable and did not attain statistical significance. Independent of the level of molar contrast, the local pattern of schedule interaction differed for the two levels of similarity: with more similar stimuli, the maximum degree of interaction occurred at the beginning of the components and then decreased; with less similar stimuli, the degree of interaction increased throughout the components and was at its maximum near their end. In Experiment 2, the same three stimuli were used while reinforcement rate in the middle component of a three-component sequence was varied; this isolated the effects of the preceding schedule from those of the following schedule. Contrast effects were generally greater in the target component preceding the variable schedule, and these were enhanced by less similar stimuli. Contrast in the target component following the variable schedule was manifested primarily in terms of the behavior at the beginning of the component, and these effects were inconsistently related to stimulus similarity. The functional separation of the effects of stimulus similarity on the different locations of contrast suggest that “anticipatory contrast” and “local contrast” depend upon different mechanisms, thus excluding any account of contrast solely in terms of relative rate of reinforcement.     

Multiple determinants of transfer of evaluative function after conditioning with free-operant schedules of reinforcement

The aim of the four present experiments was to explore how different schedules of reinforcement influence schedule-induced behavior, their impact on evaluative ratings given to conditioned stimuli associated with each schedule through evaluative conditioning, and the transfer of these evaluations through derived stimulus networks. Experiment 1 compared two contrasting response reinforcement rules (variable ratio [VR], variable interval [VI]). Experiment 2 varied the response to reinforcement rule between two schedules but equated the outcome to response rate (differential reinforcement of high rate [DRH] vs. VR). Experiment 3 compared molar and molecular aspects of contingencies of reinforcement (tandem VIVR vs. tandem VRVI). Finally, Experiment 4 employed schedules that induced low rates of responding to determine whether, under these circumstances, responses were more sensitive to the molecular aspects of a schedule (differential reinforcement of low rate [DRL] vs. VI). The findings suggest that the transfer of evaluative functions is determined mainly by differences in response rate between the schedules and the molar aspects of the schedules. However, when neither schedule was based on a strong response reinforcement rule, the transfer of evaluative judgments came under the control of the molecular aspects of the schedule.     

Behavioral contrast and reinforcement value

Behavioral contrast was produced in two target components of a four-component multiple schedule by having two target stimuli followed either by a higher rate of reinforcement or by extinction. Response rate was higher in the target followed by extinction. Periodic probe trials were then presented in which the two target stimuli were presented together. Choice on these probe trials was in favor of the stimulus followed by the higher rate of reinforcement during regular training. Experiment 2 replicated this finding but with probe trials presented throughout training. Here, preference for the stimulus followed by the higher rate of reinforcement was evident early in training, substantially before the contrast effects developed. The results challenge interpretations of contrast based on the concept of relative value.     

Concurrent VR VI schedules: primacy of molar control of preference and molecular control of response rates

In the first condition in Experiment 1, 6 rats were exposed to concurrent variable ratio (VR) 30, variable interval (VI) 30-sec schedules. In the next two conditions, the subjects were exposed to concurrent VI VI schedules and concurrent tandem VI-differential-reinforcement-of-high-rate VI schedules. For the latter conditions, the overall and relative reinforcer rates equaled those in the first condition. Only minor differences appeared in time allocation (a molar measure) across conditions. However, local response rate differences (a molecular measure) appeared between schedule types consistently with the interresponse times these schedules reinforced. In Experiment 2, these findings reappeared when the prior experiment was replicated with 5 subjects, except that the VR schedule was replaced by a VI plus linear feedback schedule. These results suggest that within the context tested, the molar factor of relative reinforcement rate controls preference, whereas the molecular factor of the relation between interresponse times and reinforcer probability controls the local response rate.     

The effects of varying the interreinforcement interval on appetitive contextual conditioning

Appetitive contextual conditioning in rats and ringdoves was investigated in six experiments. In Experiment 1, differential contextual training produced greater anticipatory activity in rats in the presence of a context paired with food than it did in rats in the presence of a different context in which food was never presented. Furthermore, the rats showed a preference for the context associated with food when they were given a simultaneous choice test between contexts. In Experiment 2, rats were more active in and preferred a context associated with a variable-time 30-sec (VT30) schedule as opposed to a VT180 schedule. Experiment 3 was a between-subjects replication of the previous experiment. As expected, rats exhibited significantly more anticipatory activity in a context in which food had been presented on a VT30 schedule than they did in a context in which food had been presented on a VT180 schedule. Experiment 4 showed that anticipatory activity was a reflection of context-US associations in ringdoves, and in Experiments 5 and 6, ringdoves also exhibited an inverse relationship between the. amount of anticipatory activity and the length of the interreinforcement interval (IRI). These results reveal a relation between ERI and contextual conditioning opposite from that obtained in studies of aversive conditioning.     

The influence of brief stimuli uncorrelated with reinforcement on choice between variable-ratio schedules

Experiment 1 investigated the behavior of rats trained to leverpress on a concurrent variable ratio (VR) 30 VR-30 schedule with a brief, 500-msec, light occurring at the midpoint of the ratio on one of the levers. Higher response rates were recorded on the lever associated with this stimulus, a finding that paralleled the effect produced by inserting primary reinforcement at the midpoint (i.e., by training on a concurrent VR-30 VR-15 schedule). Similar results were found in Experiment 2 using a concurrent VR-20 VR-20 schedule with a 2-sec visual stimulus presented midway through one of the components. In addition, a brief stimulus inserted midway through the VR-20 component of a concurrent VR-20 VR-10 schedule retarded the development of a difference in response rates between the components relative to a VR-20 VR-10 group lacking the signal. In Experiment 3, multiple VR VR schedules were used. Again, the response rate was higher in the component that had the added stimulus or, for a second group of subjects, on the component with the smaller response requirement. Probe-choice trials revealed a preference for the component that generated the higher rate in both groups. Presenting a stimulus partway through a ratio appears to reduce the effect on response rate and choice of a large ratio value.     

Stimulus preference and the transitivity of preference

Four pigeons were exposed to multiple schedules with concurrent variable interval (VI) components and then tested for preference transfer. Half of the pigeons were trained on a multiple concurrent VI 20-sec, VI 40-sec/cuncurrent VI 4G-sec5 VI 80-sec schedule. The remaining pigeons were trained on a multiple concurrent VI 80-sec, VI 40-sec/concurrent VI 40-sec, VI 20-sec schedule-After stability criteria for time and response proportions were simultaneously met, four preference transfer tests were conducted with the stimuli associated with the VI 40-sec schedules. During the transfer tests, each pigeon allocated a greater proportion of responses (M=0,79) and time (M=0.82) to the stimulus associated with the VI 40-sec schedule that was paired with the VI 80-sec schedule than lo the VI 40-sec schedule stimulus paired with the VI 20-sec schedule. Absolute reinforcement rates on the two VI 40 sec schedules were approximately equal and unlikely to account for the observed preference. Nor was the preference consistent with the differences in local reinforcement rates associated with the two stimuli. Instead, the results were interpreted in terms of the differential value that stimuli acquire as a function of previous pairings with alternative schedules of reinforcement.     

The isolation of stimulus-reinforcer associations established with multiple schedules

Multiple schedules established stimulus-reinforcer (S-S^R) associations on baselines in which equal response rates and patterning were maintained in all components. Subsequently, stimuli associated with an increase in reinforcement but no change in ongoing response rate were compounded. For one experimental group, free-operant avoidance (FOA) was programmed in tone and in light while variable-interval (VI) food reinforcement was effective in their simultaneous absence (T + L). The opposite stimulus-schedule combinations were programmed for the other. Both groups remained in their VI components 85% of the session on schedule preference tests, and on a stimulus compounding test emitted approximately 1.5 times as many responses to tone-plus-light (T + L) as to tone or light alone. This is the first report of additive summation to combined discriminative stimuli associated with only an increase in reinforcement. Nondifferentially trained controls who had the same contingency effective in tone, light, and T + L-VI or FOA—showed neither preference among schedule components or summation during stimulus compounding, indicating that nonassociative stimulus factors made no contribution to either resultant in the experimental animals. Evidence supporting an algebraic combination of response and reinforcement associations is presented, and functional similarities between transfer-of-control studies and the stimulus compounding tests of the experimental groups in the present experiment are discussed.     

Changeover contingencies and choice on concurrent schedules

Pigeons were trained on a multiple concurrent schedule with two components per session. In one component, changing schedules required the completion of a small fixed ratio on the switching key (a fixed-ratio changeover, or FRCO), and in the other component, changing schedules required only one switching response but engaged a changeover delay (COD) during which keypecks were not reinforced. Response ratios overmatched reinforcer ratios under the FRCO but undermatched under the COD. There was no difference in time allocation. In addition to these molar regularities in behavior, there were characteristic differences in performance at the molecular level. These local patterns of behavior, which can be explained within the context of contingencies created by the different changeover requirements, appear to underlie differences in performance at the molar level. Obtained molar differences in performance are not compatible with the assumption that there is a “general outcome” on concurrent schedules; and explaining these molar differences in performance in terms of the local contingencies of reinforcement is contrary to the assumption that behavior is allocated as a function of molar distributions of reinforcers.     

Contrast and value: Beyond the work ethic effect. A reply to Zentall (2008)

Zentall (2008) challenges Arantes and Grace’s (2008) failure to replicate Clement, Feltus, Kaiser, and Zentall (2000) by suggesting that our results may have been due to insufficient training or to subjects’ experimental histories, and that our results are actually consistent with those of Clement et al. when examined closely. On the contrary, our pigeons received more training than Clement et al.’s did, and when an overall measure of preference on test trials was calculated, independent of the effect of initiating event that we reported, there was no evidence of preference for the stimulus preceded by the greater response requirement in our data. Although there have now been two unsuccessful attempts to replicate Clement et al.’s work ethic effect, there is evidence that in some situations, the value of a stimulus, as assessed by transfer tests, varies inversely with the context of reinforcement. It is important to look for convergent results using other procedures and transfer tests, such as resistance to change, to identify the principles that determine when contrast manipulations affect value and when they do not.     

Effects of several extinction treatments upon the integrity of Pavlovian stimulus-outcome associations

Pavlovian-to-instrumental transfer tests were used to assess the sensitivity of Pavlovian stimulus-outcome (S-O) associations to various extinction treatments in four appetitive conditioning experiments with rats. In Experiment 1, simple nonreinforcement of a stimulus was shown to have little impact on the ability of that stimulus to display outcome-specific transfer of control. Extinguishing a stimulus by pairing the stimulus with an alternative reinforcement in Experiment 2 also had no detectable effect on the S-O associations as assessed with the outcome-specific transfer measure. The third and fourth experiments, respectively, examined the impact of postconditioning exposures to random and explicitly unpaired S-O contingencies upon previously learned S-O associations. These treatments, as well, had no detectable harmful effects upon the integrity of the S-O associations. In contrast to the consistent failures of various extinction treatments to influence the ability of stimuli to display outcome-specific transfer, these treatments often did reduce the strength of conditioned responding initially trained to these stimuli. These results support the view that extinction entails the preservation of S-O associations as well as the parallel development of inhibitory stimulus-response associations. Other notions of extinction are also discussed.     

Some determinants of second-order conditioning

In a Pavlovian conditioning situation, an initially neutral stimulus may be made excitatory by nonreinforced presentations in compound with an established conditioned excitor [i.e., second-order conditioning (SOC)]. The established excitor may be either a punctate cue or the training context. In four conditioned suppression experiments using rats, we investigated whether SOC phenomena parallel other cue interaction effects. In Experiment 1, we found that the response potential of a target stimulus was directly related to the intertrial interval when SOC was mediated by a punctate cue, and inversely related to the intertrial interval when SOC was mediated by the training context. Experiment 2 demonstrated that punctate- and context-mediated SOC are oppositely affected by posttraining context extinction, and Experiments 3 and 4 demonstrated that context- and punctate-mediated SOC are differentially affected by conditioned stimulus (Experiment 3) and unconditioned stimulus (Experiment 4) preexposure treatments. These findings parallel phenomena in conditioned inhibition and cue competition situations.     

Generalization of causal efficacy judgments after evaluative learning

In three experiments, we examined the effect of response-outcome relations on human ratings of causal efficacy and demonstrated that such efficacy ratings transfer to novel situations through derived stimulus relations. Causal efficacy ratings were higher, and probability of an outcome given a response was lower, for a differential reinforcement of high rate schedule than for either a differential reinforcement of low rate schedule (Experiment 1) or a variable interval schedule (Experiment 2). In Experiment 3, we employed schedules that were equated for outcome probability and noted that ratings of causal efficacy and the rate of response were higher on a variable ratio than on a variable interval schedule. For participants in all three experiments, causal efficacy ratings transferred to the stimulus present during each schedule and generalized to novel stimuli through derived relations. The results corroborate the view that schedules are a determinant of both response rates and causal efficacy ratings. In addition, the novel demonstration of a mechanism of generalization of these ratings via derived relations has clinical implications.     

Discriminative stimuli that follow the absence of reinforcement are preferred by pigeons over those that follow reinforcement

Clement, Feltus, Kaiser, and Zentall (2000) found that when pigeons have to work to obtain a discriminative stimulus that is followed by reinforcement, they prefer a discriminative stimulus that requires greater effort over one that requires less effort. The authors suggested that such a preference results from the greater change in hedonic value that occurs between the more aversive event and the onset of the stimulus that signals reinforcement, a contrast effect. It was hypothesized that any stimulus that follows a relatively more aversive event would be preferred over a stimulus that follows a relatively less aversive event. In the present experiment, the authors tested the counterintuitive prediction of that theory, that pigeons should prefer a discriminative stimulus that follows the absence of reinforcement over a discriminative stimulus that follows reinforcement. Results supported the theory.     

Training reinforcement rates,resistance to extinction,and the role of context in reinstatement

Behavior reduced as a consequence of extinction or intervention can relapse. According to behavioral momentum theory, the extent to which behavior persists and relapses once it has been eliminated depends on the relative training reinforcement rate among discriminative stimuli. In addition, studies of context renewal reveal that relapse depends on the similarity between the training stimulus context and the test stimulus context following disruption by extinction. In the present experiments with pigeons, we arranged different reinforcement rates in the presence of distinct discriminative stimuli across components of a multiple schedule. Following extinction, we attempted to reinstate responding in the presence of those target components with response-independent food presentations. Importantly, we arranged the reinstating food presentations either within the target components or in separate components, either paired with extinction (Experiment 1) or reinforcement (Experiment 2) during baseline. Reinstatement increased with greater training reinforcement rates when the reinstating food presentations were arranged in the target components and the separate components paired with reinforcement during training. Reinstatement was smaller and was not systematically related to training reinforcement rates in the target components when reinstating food presentation occurred in separate components paired with extinction. These findings suggest that relapse depends on the history of reinforcement associated with the discriminative stimuli in which the relapse-inducing event occurs.     

Preference for the interval schedule following multiple variable-ratio yoked-variable-interval schedule training

Pigeons were studied on multiple variable-ratio yoked-variable-interval schedules in which components had equal rates of food reinforcement and appeared equally often on each of two keys. Interpolated between component changes on the final multiple schedule were 10-sec probes in which both schedule stimuli were present, one on each key. During multiple schedule training, variable-ratio response rates were greater than yoked-variable-interval rates; however, response rate differences in the components were not a function of the mean ratio value for the 40-to-320-ratio range studied. During the choice probes, subjects responded more to the stimulus associated with the interval schedule than to the one associated with the ratio schedule. It was concluded that pigeons prefer interval schedules over equal reinforcement rate ratio schedules, because the former generate fewer responses per reinforcement.     

Partial reinforcement effects on learning and extinction of place preferences in the water maze

In two experiments, two groups of rats were trained in a navigation task according to either a continuous or a partial schedule of reinforcement. In Experiment 1, animals that were given continuous reinforcement extinguished the spatial response of approaching the goal location more readily than animals given partial reinforcement—a partial reinforcement extinction effect. In Experiment 2, after partially or continuously reinforced training, animals were trained in a new task that made use of the same reinforcer according to a continuous reinforcement schedule. Animals initially given partial reinforcement performed better in the novel task than did rats initially given continuous reinforcement. These results replicate, in the spatial domain, well-known partial reinforcement phenomena typically observed in the context of Pavlovian and instrumental conditioning, suggesting that similar principles govern spatial and associative learning. The results reported support the notion that salience modulation processes play a key role in determining partial reinforcement effects.     

Effects of signaled reward in instrumental conditioning: Enhanced learning on DRL and DRH schedules of reinforcement

The effects of signaled reward were examined using DRL and DRH schedules of reinforcement. In each case, one group of rats received a brief cue between the reinforced response and the reward, and a second group received brief cues at random times. With the DRL schedule (Experiment 1), signaled reward decreased response rate, increased response efficiency (number of responses per reinforcer), and increased resistence to satiation relative to the control group. With the DRH schedule (Experiment 2), signaled reward increased response rate, efficiency, and resistance to satiation. These results refute an overshadowing explanation of the effects of signaled reward and suggest that food-correlated cues enhance learning of the reinforcement contingencies.