Certainties and mysteries in the within-trial contrast literature: A reply to Zentall (2008)

Contrary to Zentall’s (2008) hypothesis that extensive overtraining is required in order to obtain the work ethic effect in pigeons, we find no evidence of this in two separate paradigms designed to assess preference for stimuli encountered after high effort. In light of these findings, along with our previously published results and those of Arantes and Grace (2008), we maintain that the work ethic effect is not a reliable phenomenon in pigeons. On the other hand, we are uncertain about the reason(s) underlying the different preference profiles for high- versus low-deprivation stimuli as a function of successive versus simultaneous discrimination training (Vasconcelos & Urcuioli, 2008). Nevertheless, we disagree with Zentall that additional analyses of the sort he suggests support the conclusion that both training regimens yield within-trial contrast effects.     

Contrast and value: Beyond the work ethic effect. A reply to Zentall (2008)

Zentall (2008) challenges Arantes and Grace’s (2008) failure to replicate Clement, Feltus, Kaiser, and Zentall (2000) by suggesting that our results may have been due to insufficient training or to subjects’ experimental histories, and that our results are actually consistent with those of Clement et al. when examined closely. On the contrary, our pigeons received more training than Clement et al.’s did, and when an overall measure of preference on test trials was calculated, independent of the effect of initiating event that we reported, there was no evidence of preference for the stimulus preceded by the greater response requirement in our data. Although there have now been two unsuccessful attempts to replicate Clement et al.’s work ethic effect, there is evidence that in some situations, the value of a stimulus, as assessed by transfer tests, varies inversely with the context of reinforcement. It is important to look for convergent results using other procedures and transfer tests, such as resistance to change, to identify the principles that determine when contrast manipulations affect value and when they do not.     

Failure to obtain value enhancement by within-trial contrast in simultaneous and successive discriminations

The present research tested the generality of the “work ethic“ effect described by Clement, Feltus, Kaiser, and Zentall (2000). In Experiment 1, we trained 10 pigeons on a pair of either simultaneous or successive discriminations. One discrimination followed a high-effort requirement (20 pecks to the center key) and the other followed a low-effort requirement (1 peck). Contrary to Clement et al.’s results, we found that preferences between the S⁺ and S⁻ stimuli in transfer tests depended on the event that initiated the trial: Pigeons preferred the stimulus from the baseline discrimination whose initiating event was most dissimilar from that preceding the test trial. Preferences were similar but less extreme in the successive condition. In Experiment 2, we investigated whether test preferences depended on the amount of training. A total of 12 pigeons were trained on a pair of simultaneous discriminations, except that test sessions were scheduled after every three baseline sessions. Preferences increased across test sessions but were similar to those in Experiment 1. Together with Vasconcelos, Urcuioli, and Lionello-DeNolf (2007a), our study represents a second failure to replicate Clement et al.’s work ethic effect. The finding that preference depends on the event that initiates the test trial suggests that choice probes may not provide unambiguous assessments of stimulus value.     

Behavioral contrast in pigeons and rats: A comparative analysis

The effects of reinforcement rate on behavioral contrast were examined in pigeons and rats. Each species was exposed to a series of 12 multiple variable-interval schedules, divided into four 3-schedule series. Each series consisted of a standard contrast manipulation, and baseline schedules provided a different rate of reinforcement in each of the series. The functions relating reinforcement rate to the magnitude of contrast were different across species. Rats showed a U-shaped function, with reliable contrast occurring only at high reinforcement rates. Pigeons showed an inverted U-shaped function, with contrast occurring on all schedules except the schedule providing the lowest rate of reinforcement. Pigeons discriminated between schedule components better than rats did, although differences in discrimination were probably not responsible for the differences in contrast. The results suggest that behavioral contrast in rats may be a different phenomenon from behavioral contrast in pigeons. The results cannot be explained by current theories, which view contrast as the product of a single general process.     

Effects of response-independent stimuli on fixed-interval and fixed-ratio performance of rats: a model for stressful disruption of cyclical eating patterns

Binge eating is often associated with stress-induced disruption of typical eating patterns. Three experiments were performed with the aim of developing a potential model for this effect by investigating the effect of presenting response-independent stimuli on rats’ lever-pressing for food reinforcement during both fixed-interval (FI) and fixed-ratio (FR) schedules of reinforcement. In Experiment 1, a response-independent brief tone (500-ms, 105-dB, broadband, noisy signal, ranging up to 16 kHz, with spectral peaks at 3 and 500 Hz) disrupted the performance on an FI 60-s schedule. Responding with the response-independent tone was more vigorous than in the absence of the tone. This effect was replicated in Experiment 2 using a within-subject design, but no such effect was noted when a light was employed as a disrupter. In Experiment 3, a 500-ms tone, but not a light, had a similar effect on rats' performance on FR schedules. This tone-induced effect may represent a release from response-inhibition produced by an aversive event. The implications of these results for modeling binge eating are discussed.     

Competing sources of stimulus value in anticipatory contrast

In previous studies of anticipatory contrast, identical target components (A and B) preceded either a lower (extinction) or a richer schedule. Higher response rates occurred during the target preceding the lower rate of reinforcement, whereas preference was in favor of the target preceding the richer schedule. In Experiment 1, the response and preference measures were positively related when additional stimuli, with no reinforcement of their own, preceded the target components. The effect of these additional stimuli was presumed to be due to their overshadowing of the Pavlovian association between the target components and their following schedules. Experiment 2 also demonstrated a consistent relation between response rate and preference in a conditioned reinforcement procedure. In the absence of a strong Pavlovian association, anticipatory contrast, like other forms of contrast in free-operant procedures, reflects an increase in the value of the target component with an unchanged reinforcement schedule.     

Classical conditioning of the rabbit nictitating membrane response: Effects of reinforcement schedule on response maintenance and resistance to extinction

Two experiments were conducted to assess the effects of the introduction of schedules of partial reinforcement (PRF), subsequent to continuous reinforcement training, on the maintenance and resistance to extinction of the rabbit’s nictitating membrane CR. Substantial response levels were maintained by schedules of reinforcement as lean as 15%, and the performance decrements, when observed to be reliable, could not be localized to the immediate effects of one, two, or three consecutive nonreinforced trials by the examination of conditional response probabilities. Moreover, reliable PRF extinction effects were obtained. The relevance of these findings to the purported empirical divergence of PRF effects on classical and instrumental conditioning were discussed.     

Latent discrimination learning under a regular schedule of partial reinforcement

In this investigation, which employed rats in a runway, discriminative responding consisted of faster running on the reinforced than on the nonreinforced trials of either the 4NR or R4N schedule, both schedules containing fixed, repeated sequences of nonreinforced and reinforced trials. Under the 4NR schedule, four nonreinforced trials preceded a reinforced trial each day, and under the R4N schedule, a reinforced trial was followed by four nonreinforced trials each day. The major finding obtained was that under the 4NR schedule, discriminative responding was improved very substantially by a shift to extinction. Rats maintained on the 4NR schedule did not show improved discriminative responding, nor did discriminative responding improve in extinction following training under either the R4N schedule or a schedule of consistent reinforcement. Latent discrimination learning was defined as discriminative responding which fails to reflect adequately the amount of discrimination learning accomplished. The present findings demonstrate latent discrimination learning for regular schedules of partial reinforcement, something already demonstrated for brightness differential conditioning and possibly DRL schedules, as well.     

Effect on subsequent fixed-interval schedule performance of prior exposure to ratio and interval schedules of reinforcement

In three experiments, we examined the effect on the patterns of responding noted on fixed interval (FI) schedules of prior exposure to a range of interval and ratio schedules. Rats leverpressed for food reinforcement on random ratio (RR), random interval (RI), or variable interval (VI) schedules prior to transfer to FI schedules. In Experiment 1, prior exposure to an RR schedule retarded the development of typical FI patterns of responding. Exposure to a yoked RI schedule produced even greater retardation of typical FI performance. This effect was replicated in Experiment 2, using a within-subjects design. Rats responded on a multiple RR-RI schedule prior to a multiple FI-FI schedule. Typical FI performance emerged more slowly in the component previously associated with the RI than with that associated with the RR. In Experiment 3, exposure to an RR schedule retarded the development of FI performance to a greater extent than did exposure to a VR schedule. The latter schedule was programmed to allow the possibility that inhibitory control would develop after reinforcement. These results confirm that ratio schedules independently result in the disruption of FI responding. This effect was not long lasting and cannot be used plausibly to explain species differences in responding to FI schedules. However, it does suggest that temporal control—as manifested by the transfer of inhibitory control from one schedule to another—could facilitate movement between interval schedules.     

Additive summation by stimulus compounding irrespective of behavioral contrast during discrimination training: An investigation with positive reinforcement and avoidance schedules

The similarity in the discrimination training leading to behavioral contrast and that preceding tests producing response enhancement to combined discriminative stimuli suggested that the two phenomena might be related. This was investigated by determining if contrast indiscrimination training was necessary for this outcome of stimulus compounding. Responding to tone, light, and to the simultaneous absence of tone and light (T + L) was maintained during baseline training by food reinforcement in Experiment I and by shock avoidance in Experiment II. During subsequent discrimination training, responding was reduced in T + L by programming nonreinforcement in Experiment I and safety or response-punishment in Experiment II. In the first experiment, one rat exhibited positive behavioral contrast, i.e., tone and light rates increased while his T + L rate decreased. In Experiment II, rats punished in T + L showed contrast in tone and light, this being the first demonstration of punishment contrast on an avoidance baseline with rats. The discrimination acquisition data are discussed in the light of current explanations of contrast by Gamzu and Schwartz (1973) and Terrace (1972). During stimulus compounding tests, all subjects in both experiments emitted more responses to tone-plus-light than to tone or light (additive summation). An analysis of the terminal training baselines suggests that the factors producing these test results seem unrelated to whether or not contrast occurred during discrimination training. It was concluded that the stimulus compounding test reveals the operation of the terminal baseline response associations and reinforcement associations conditioned on these multicomponent free-operant schedules of reinforcement.

     

The maintenance of matching behavior when contrast occurs in multiple concurrent concurrent schedules of reinforcement

Pigeons were trained on a multiple schedule of reinforcement in which each component was a concurrent schedule. The concurrent schedules were programmed by the changeover-key procedure. The primary purpose was to determine if the relative behavior allocated to two response alternatives is affected when absolute changes in these behaviors occur; i.e., to determine if matching is affected when positive behavioral contrast occurs. Results showed that (1) relative behavior in the unaltered component of the multiple schedule is not disrupted when positive contrast occurs in that component, (2) positive contrast occurred when the overall frequency of reinforcement in the reinforcement-correlated component(s) was high, but not when it was low, (3) changeover behavior was susceptible to positive contrast effects, and (4) changeover contrast and food-key contrast are independent phenomena.     

Multiple-schedule interactions and discrimination

Pigeons received variable-interval (VI) reinforcement for keypecking during randomized presentations of seven line-orientation stimuli forming a continuum ranging from horizontal (0 deg) to vertical (90 deg). Each line presentation lasted for 30 sec and was preceded and followed by 30-sec time-outs. After responding stabilized, only responding in the two extreme stimuli (0 and 90 deg) was reinforced. As discrimination training proceeded, strong behavioral contrast and dimensional contrast effects appeared. However, only marginal local effects (local contrast and local dimensional effects), exerted by one line-orientation component upon a second, appeared, indicating that behavioral and dimensional contrast may be independent of parallel local effects. An attempt was made to apply Blough’s (1975) quantitative model of operant generalization and discrimination to the present discrimination procedure. However, this model did not predict the generalization gradient shape that was experimentally obtained. This experiment also yielded two serendipitous findings: (1) Positive behavioral contrast appeared in an extinction-related stimulus (time-out) when other stimuli were switched from reinforcement to extinction (hitherto, positive behavioral contrast had been observed only in responding to a reinforcementrelated stimulus when other stimuli were switched from reinforcement to extinction), and (2) final responding was higher in the presence of an extinction stimulus that had always been an extinction stimulus than it was in the presence of other extinction stimuli that had previously been paired with VI reinforcement.     

Discriminative stimuli that follow the absence of reinforcement are preferred by pigeons over those that follow reinforcement

Clement, Feltus, Kaiser, and Zentall (2000) found that when pigeons have to work to obtain a discriminative stimulus that is followed by reinforcement, they prefer a discriminative stimulus that requires greater effort over one that requires less effort. The authors suggested that such a preference results from the greater change in hedonic value that occurs between the more aversive event and the onset of the stimulus that signals reinforcement, a contrast effect. It was hypothesized that any stimulus that follows a relatively more aversive event would be preferred over a stimulus that follows a relatively less aversive event. In the present experiment, the authors tested the counterintuitive prediction of that theory, that pigeons should prefer a discriminative stimulus that follows the absence of reinforcement over a discriminative stimulus that follows reinforcement. Results supported the theory.     

Delayed matching to sample: Reinforcement has opposite effects on resistance to change in two related procedures

The effects of reinforcement on delayed matching to sample (DMTS) have been studied in two within-subjects procedures. In one, reinforcer magnitudes or probabilities vary from trial to trial and are signaled within trials (designated signaled DMTS trials). In the other, reinforcer probabilities are consistent for a series of trials produced by responding on variable-interval (VI) schedules within multiple-schedule components (designated multiple VI DMTS). In both procedures, forgetting functions in rich trials or components are higher than and roughly parallel to those in lean trials or components. However, during disruption, accuracy has been found to decrease more in rich than in lean signaled DMTS trials and, conversely, to decrease more in lean than in rich multiple VI DMTS components. In the present study, we compared these procedures in two groups of pigeons. In baseline, forgetting functions in rich trials or components were higher than and roughly parallel to those in lean trials or components, and were similar between the procedures. During disruption by prefeeding or extinction, accuracy decreased more in rich signaled DMTS trials, whereas accuracy decreased more in lean multiple VI DMTS components. These results replicate earlier studies and are predicted by a model of DMTS from Nevin, Davison, Odum, and Shahan (2007).     

Effects of effort and difficulty on human preference for a stimulus: Investigation of the within-trial contrast

The within-trial contrast hypothesis (WTC) provides a more parsimonious explanation for the phenomenon that humans and animals prefer outcomes that follow more effortful events to outcomes that follow less effortful events (Zentall, 2013). We conducted two WTC experiments with human adults. In Experiment 1, we manipulated the difficulty of a preceding event by varying the interresponse time and the limited-hold interval during differential reinforcement with a low response rate schedule, to examine the effect of effort on the preference for the subsequent stimuli. In Experiment 2, we attempted to identify the variables that had affected the results of Experiment 1, by manipulating time as the delay of reinforcement. The results showed preferences based on WTC only when participants made a high rate of incorrect responses in the preceding event, which was used as an index of the strength of individual effort. These results extend the findings of previous human WTC studies and suggest that the difficulty of a task could serve as an aversive event that affects the WTC effect. It is possible that an index based on performance in the preceding event would provide useful information for predicting the contrast effect.     

Behavioral contrast and type of reward: Role of elicited response topography

Groups of pigeons were exposed to multiple variable-interval variable-interval and multiple variable-interval extinction schedules of either food or water reinforcement for keypecking. Discriminative stimuli associated with component schedules were located either on the operant key or on a second “signal” key. When the stimuli were projected on the operant key, positive contrast appeared during discrimination conditions with either food or water as the reinforcer. When the stimuli were projected on the signal key, overall responding to the operant and signal keys showed contrast with food, but negative induction with water as the reinforcer. In the latter condition, the signal for the variable-interval shcedule of water reinforcement elicited a variety of water-related behavior, only some of which was directed at the signal. Thus, the type of reward and location of discriminative stimuli interacted to determine the presence or absence of behavioral contrast effects. In large part, these results support and extend the autoshaping view of contrast.     

Contrast effects in simultaneous discrimination learning

Pigeons were trained on a form discrimination that alternated across trials with an already-learned color discrimination, which was associated with different probabilities of reinforcement for different groups. The rate of acquisition of the form discrimination varied inversely with the rate of reinforcement for the color discrimination, thus demonstrating a “contrast effect” in simultaneous discrimination learning.     

The sensitivity of the pigeon’s keypeck to the differential reinforcement of long interresponse times

Keypecking of pigeons was studied under differential-reinforcement-of-low-rate (DRL) and variable-interval (VI) schedules in which the interreinforcement times on the two schedules were equated by a yoking procedure. Each schedule was available for half of every session and a change of schedule was signaled by a change of key color. The value of the DRL schedule was varied from .5 to 300 sec. Response rates were always higher in the VI schedule, but within sessions there was a sharp change in response rate coincident with the change in schedule only under lower schedule values. A group without prior training was tested with a 180-sec schedule value, and it, too, developed a higher response rate during the VI schedule, showing that the effect was not dependent on prior experience under low schedule values. In all conditions except the .5- and 1-sec values of the schedule, the mean proportion of responses emitted during the VI schedule was approximately .85 of the responses emitted during both schedules. The conclusion was that the requirement of a minimum interresponse time for reinforcement may work its effect by determining which responses may occur just prior to the reinforced response and thus receive delayed reinforcement.     

Choice, delay, probability, and conditioned reinforcement

The hyperbolic-decay model is a mathematical expression of the relation between delay and reinforcer value. The model has been used to predict choices in discrete-trial experiments on delay-amount tradeoffs, on preference for variable over fixed delays, and on probabilistic reinforcement. Experiments manipulating the presence or absence of conditioned reinforcers on trials that end without primary reinforcement have provided evidence that the hyperbolic-decay model actually predicts the strength of conditioned reinforcers rather than the strength of delayed primary reinforcers. The model states that the strength of a conditioned reinforcer is inversely related to the time spent in its presence before a primary reinforcer is delivered. A possible way to integrate the model with Grace’s (1994) contextual-choice model for concurrent-chain schedules is presented. Also discussed are unresolved difficulties in determining exactly when a stimulus will or will not serve as a conditioned reinforcer.     

The stimulus-response overshadowing phenomenon with VI versus FI schedules of reinforcement

Previous research has shown that response rates on a variable interval (VI) schedule of reinforcement decrease if a brief response-produced signal is given prior to reward. One explanation is that the signal overshadows the response because it is a better predictor of reinforcement. The S-R overshadowing effect does not occur with variable ratio (VR) schedules, however. Tarpy, Lea, and Midgley (1983) explained this fact by suggesting that the signal functions to enhance the salience of the temporal interval offset on the VI schedule (a characteristic not possessed by VR schedules), which then overshadows the response. In this experiment, the salience of the temporal interval was enhanced in another way: signaled or unsignaled reward was provided to rats responding on either a VI or fixed interval (FI) reward schedule. As predicted, rates were lowest for animals receiving signaled reinforcement on an FI schedule and highest for those receiving unsignaled reinforcement on a VI schedule.