Divided attention and the matching law: Sample duration affects sensitivity to reinforcement allocation

Previously, we have shown that changes in pigeons’ divided attention performance resulting from changes in relative reinforcement are well described by the generalized matching law. In the present experiment, we examined whether sensitivity of performance to variations in relative reinforcement would be dependent upon sample duration. Pigeons responded on a delayed matching-to-sample procedure with compound samples (color 1 line orientation) and element comparison stimuli (two colors or two line orientations). Relative reinforcement for accurate matches on the two types of comparison trials varied across conditions. Sample duration was short (i.e., 0.75 sec) for half of the trials in a session and longer (i.e., 2.25 sec) for the other half. Sensitivity of accuracy to changes in relative reinforcement was greater with the longer sample than with the shorter sample, suggesting that differential reinforcement alters the allocation of attending to the elements of compound stimuli. Continued examination of the applicability of well-established theories of goal-directed behavior to the allocation of attention may provide further insights into what is variously referred to as goal-directed, voluntary, endogenous, or top-down control of attention.     

Differential reinforcement and resistance to change of divided-attention performance

Behavioral momentum theory provides a framework for understanding how conditions of reinforcement influence instrumental response strength under conditions of disruption (i.e., resistance to change). The present experiment examined resistance to change of divided-attention performance when different overall probabilities of reinforcement were arranged across two components of a multiple schedule. Pigeons responded in a delayed-matching-to-sample procedure with compound samples (color + line orientation) and element comparisons (two colors or two line orientations). Reinforcement ratios of 1:9, 1:1, and 9:1 for accurate matches on the two types of comparison trials were examined across conditions using reinforcement probabilities (color/lines) of .9/.1, .5/.5, and .1/.9 in the rich component and .18/.02, .1/.1, and .02/.18 in the lean component. Relative accuracy with color and line comparisons was an orderly function of relative reinforcement, but this relation did not depend on the overall rate of reinforcement between components. The resistance to change of divided-attention performance was greater for both trial types in the rich component with presession feeding and extinction, but not with decreases in sample duration. These findings suggest promise for the applicability of quantitative models of operant behavior to divided-attention performance, but they highlight the need to further explore conditions impacting the resistance to change of attending.     

Independence of the amount and delay ratios in the generalized matching law

Eight pigeons chose between pairs of different sizes and delays of reinforcement scheduled according to nonindependent concurrent variable-interval variable-interval schedules. The results were best described by the generalized matching law, where the relative effects of amount and delay on preference are independent and multiplicative. Order of presentation of the conditions had a significant effect on preference that was best represented in the model by a modification of the bias parameter.     

Stimulus discriminability,contingency discriminability,and schedule performance

A detection-theory model to describe the effects of varying stimulus disparity on switching-key concurrent variable-interval schedule performance (Miller, Saunders, & Bourland, 1980) is presented. It describes the available data on stimulus disparity well. Using the additional notion of contingency discriminability, the model is then developed into an account of schedule and stimulus control that is both wider in application and conceptually clearer than the generalized matching law. A basic assumption of the new model is that subjects may not perfectly discriminate that a reinforcer followed a response of one class versus that of another, and this ability is measured as reinforcer-contingency discriminability, d_r. This idea is then applied to performance in signal-detection procedures, both with and without error reinforcement, to multiple-schedule performance, and to single-schedule performance. The model fitted the data well, and it thus constitutes a coherent and viable alternative to the generalized matching law in the procedures and conditions in which the latter has been shown to apply.     

Conservation,matching, and the variable-interval schedule

The conservation model was generalized to the variable-interval schedule by incorporating the concept of unscheduled instrumental responses, those which occur in the time before the next setup is due. Thirsty rats responded in constant-duration sessions on two 7-sec schedules that required one leverpress for 25 and 50 licks at a water tube and on a 14-sec 25-lick schedule. In accordance with the model, total licks decreased linearly as total presses increased, and the schedules facilitated leverpressing and suppressed licking relative to paired baseline levels of responding. While the matching model also gave a satisfactory fit to instrumental responding under the schedules, its two constants, representing asymptotic rate of responding and extraneous reinforcement, had anomalous values which led the model to predict that response rate would decrease as the rate of reinforcement increased, directly opposing its prediction for the constant-consumption experiments of its previous tests.     

Concurrent schedule responding as a function of body weight

Five pigeons pecked for food reinforcement on several concurrent schedules. Their body weights were varied from 80% to 110% of their free-feeding weights. A number of predictions of the equations proposed by Herrnstein (1970) were tested. As predicted, the relative rate of responding equalled the relative rate of reinforcement for all subjects, on all schedules, at all body weights. And, as predicted, the overall rates of responding on the components of a concurrent schedule were slower than the local rates of responding on the components of an identical multiple schedule. Contrary to prediction, the total rate of responding generated by the concurrent schedules did not increase with increases in the total rate of reinforcement they provided. And, contrary to prediction, the k parameter did not remain constant, and the R₀ parameter did not increase with increases in body weight. It was concluded that Herrnstein’s matching law and his interpretation of the m parameter are correct but that the interpretations of k and R₀ require further investigation.     

The maintenance of matching behavior when contrast occurs in multiple concurrent concurrent schedules of reinforcement

Pigeons were trained on a multiple schedule of reinforcement in which each component was a concurrent schedule. The concurrent schedules were programmed by the changeover-key procedure. The primary purpose was to determine if the relative behavior allocated to two response alternatives is affected when absolute changes in these behaviors occur; i.e., to determine if matching is affected when positive behavioral contrast occurs. Results showed that (1) relative behavior in the unaltered component of the multiple schedule is not disrupted when positive contrast occurs in that component, (2) positive contrast occurred when the overall frequency of reinforcement in the reinforcement-correlated component(s) was high, but not when it was low, (3) changeover behavior was susceptible to positive contrast effects, and (4) changeover contrast and food-key contrast are independent phenomena.     

Time and response matching with topographically different responses

Four pigeons were exposed to several nonindependent concurrent variable-interval schedules of reinforcement. One schedule component required a keypecking response; the other component required a treadlepressing response. The birds matched the ratio of their behavior (as measured by responses and time) between the two topographically different responses to the ratio of reinforcement in those two components. When additional foods not contingent on a keypeck or treadle-press were then added, the birds matched time spent in the components to total rates of food delivered in those components; response matching was somewhat disrupted. The matching law, developed under concurrent variable-interval schedules requiring similar responses, can thus account for choice behavior involving topographically different responses.     

Feedback functions for reinforcement: A paradigmatic experiment

A reinforcement schedule states a rule for obtaining reinforcement as a function of behavior actually emitted and perhaps as a function of additional variables. These functions are here called “feedback functions.” Behavior actually emitted is, in turn, a function of obtained reinforcement. This reciprocal interdependency was quantified for an experiment in which pigeons chose among either three or two alternatives. Shifting from three to two alternatives, and vice versa, produced changes in the distribution of responding which were approximately accounted for by equations that combined the feedback functions with the matching law for reinforced responding. These equations predicted, among other things, a violation of the constant-ratio rule of formal choice theory and an absence of simultaneous contrast effects between response alternatives reinforced on variable-interval schedules. Both predictions were approximately confirmed.     

Component duration effects in multiple schedules

Previous research has produced conflicting results regarding the effects of component duration on interactions in multiple schedules. In Experiment 1, potential sources of this conflict were evaluated. Both the effects of absolute reinforcement rate and carry-over effects (hysteresis) from a preceding condition were isolated. When 10-sec components were used, the sensitivity of relative response rate to relative reinforcement rate was affected very little by hysteresis effects and absolute reinforcement rate, but it was systematically reduced as a function of the number of prior conditions. Sensitivity to relative reinforcement rate was also substantially higher with the 10-sec components than with 2-min components. In Experiment 2, this effect of component duration was decomposed into two separate effects. Contrast effects during presentation of a target component with a constant reinforcement rate were greater the shorter the target component was itself; but they were smaller the shorter the alternative component in which reinforcement rate was varied. The latter effect was smaller and more unreliable across subjects. The existence of these two separate effects demonstrates that the usual method of studying component duration—that is, holding all components equal in duration—systematically causes underestimation of the effects of the component duration, and obscures the different processes controlling the two effects.     

Postdiscrimination generalization as a function of interresponse time

Following 20 sessions of variable-interval 20-sec reinforcement in the presence of a single 45-deg line-tilt stimulus, three pigeons were trained to discriminate between line tilts of 45 deg correlated with variable-interval 20-sec reinforcement and line tilts of 15 deg correlated with extinction. A generalization test along the line-tilt dimension was administered following a criterion discrimination performance. Gradients derived in terms of relative frequency of response as a function of line tilt indicated strong external stimulus control and exhibited clear peak shift. From the interresponse time (IRT) distributions generated for responding to each test stimulus, probability of response conditional upon IRT (IRTs/Op) was derived as a joint function of line tilt and IRT. The IRTs/Op functions for responses following IRTs in 0.2-sec-wide classes from 0.2 to 1.0 sec and for responses following IRTs in the interval of 1.0 to 2.0 sec were similar to the relative generalization gradients and also exhibited peak shift. Few IRTs were greater than 2.0 sec. External stimulus control was established over responses terminating IRTs both longer and shorter than 1.0 sec.     

The effect of sample and comparison ratio schedules on delayed matching to sample in the pigeon

In Experiment 1, three food-deprived pigeons received trials that began with red or green illumination of the center pecking key. Two or four pecks on this sample key turned it off and initiated a 0- to 10-sec delay. Following the delay, the two outer comparison keys were illuminated, one with red and one with green light. In one condition, a single peck on either of these keys turned the other key off and produced either grain reinforcement (if the comparison that was pecked matched the preceding sample) or the intertrial interval (if it did not match). In other conditions, 3 or 15 additional pecks were required to produce reinforcement or the intertrial interval. The frequency of pecking the matching comparison stimulus (matching accuracy) decreased as the delay increased, increased as the sample ratio was increased, and decreased as the comparison ratio was increased. The results of Experiment 2 suggested that higher comparison ratios adversely affect matching accuracy primarily by delaying reinforcement for choosing the correct comparison. The results of Experiment 3, in which delay of reinforcement for choosing the matching comparison was manipulated, confirmed that delayed reinforcement decreases matching accuracy.     

Pavlovian contingencies and anticipatory contrast

Pigeons were trained on a four-component multiple schedule in which two target components with identical reinforcement schedules were followed by other components with either higher or lower reinforcement rates. The Pavlovian signal properties of the target-component stimuli were varied by changes in their duration relative to the following components, and by whether the two following components were cued by the same or different stimuli, When different stimuli occurred in the following components, response rates were higher in the target component preceding the following component with the lower reinforcement rate, and these contrast effects were larger with shorter relative durations. But with nondifferential stimuli in the following components, contrast consistently occurred only with the longer durations of the target components. Moreover, several subjects with the shorter duration target stimuli had higher response rates in the target followed by the richer schedule—that is, Pavlovian conditioning occurred to the target stimuli. This interaction suggests that the processes underlying anticipatory contrast and Pavlovian conditioning are in opposition, and that the Pavlovian effect can dominate ifthe signaling properties of the target components are sufficiently enhanced.     

Qualitatively different reinforcers and parameters of Herrnstein’s (1970) response-strength equation

Previous research that compared the estimated parameters (i.e.,k andR _e) from Herrnstein’s (1970) hyperbolic matching law equation within the same individuals responding for qualitatively different consummatory reinforcers (i.e., water and sucrose solution) found similar asymptotic response rates (k). The present study compared these parameters within subjects responding on levers for consummatory and nonconsummatory reinforcers. Male Wistar rats responded on a lever in a running wheel on a series of tandem FR 1 VI schedules for either 0.1 ml of a 15% sucrose solution or the opportunity to run for 15 sec. Herrnstein’s hyperbola was fit to response and reinforcement rates from each session. Results showed thatk values were significantly higher for sucrose than for wheel-running reinforcement. On average,R _e was lower for sucrose than for wheel-running reinforcement, though not significantly lower. The results of the present study appear to violate the assumption of the constancy ofk in Herrnstein’s matching law analysis.     

Preference reversal and delayed reinforcement

Pigeons chose, in a two-key discrete-trial procedure, between 2- and 4-sec access to grain, with the larger amount always presented 4 sec later than the smaller. As the delay between the choice and the availability of the smaller reinforcement was varied from .01 to 12 sec, all subjects reversed preference from the small-early to the large-late reinforcement. The values of delay at which preference reversed were approximately consistent with the matching law as adapted for delayed reinforcement.     

Increasing appropriate classroom behaviour and academic performance by reinforcing correct work alone

The present study examined indirect effects of modifying appropriate classroom behaviour by itself and academic performance by itself. Of particular interest were changes in appropriate behaviour as students were reinforced for correct work. Five disruptive boys were given arithmetic assignments during daily 20-minute experimental sessions. Changes in appropriate behaviour and arithmetic performance were measured during the following phases: (a) an initial baseline phase; (b) reinforcement for appropriate behaviour alone; (c) return to baseline; (d) reinforcement for correct arithmetic work. It was found that reinforcing appropriate behaviour increased that behaviour, but produced little improvement in arithmetic performance. However, reinforcing arithmetic performance not only increased correct work, but resulted in a high rate of appropriate behaviour as well. The implications of these findings for the management of classroom behaviours were discussed.     

Specification of the stimulus-reinforcer relation in multiple schedules: Delay and probability of reinforcement

Control of pigeons’ keypecking by a stimulus-reinforcer contingency was investigated in the context of a four-component multiple schedule. In each of three experiments, pigeons were exposed to a schedule consisting of two two-component sequences. Discriminative stimuli identifying each sequence were present only in Component 1, which was 4, 6, or 8 sec in duration, while reinforcers could be earned only in Component 2 (30 sec in duration). Control by a stimulus-reinforcer contingency was sought during Component 1 by arranging a differential relation between Component 1 cues and schedule of reinforcement in Component 2. In Experiment 1, rate of keypecking during Component 1 varied with the presence and absence of a stimulus-reinforcer contingency. When a contingency was introduced, rate of keypecking increased during the Component 1 cue associated with the availability of reinforcement in Component 2. In Experiment 2, the stimulus-reinforcer contingency was manipulated parametrically by varying the correlation between Component 1 cues and Component 2 schedules of reinforcement. Responding in Component 1 varied as a function of strength of the stimulus-reinforcer contingency. The relatively high rates of Component 1 responding observed in Experiments 1 and 2 pose difficulties for conceptions of stimulus-reinforcer control based on probability of reinforcement. In these two experiments, the stimulus-associated probabilities of reinforcement in Component 1 were invariant at zero. An alternate dimension of stimulus-reinforcer control was explored in Experiment 3, in which Component 1 cues were differentially associated with delay to reinforcement in Component 2, while probability of reinforcement was held constant across components. When the stimulus-reinforcer contingency was in force, rate of responding in Component 1 varied inversely with delay to reinforcement in Component 2. In a quantitative analysis of data from Experiments 2 and 3, relative rate of responding during Component 1 was strongly correlated with two measures of relative delay to reinforcement.     

Reinforcement context and pacemaker rate in the behavioral theory of timing

In the present experiment, an attempt was made to extend the base of evidence for the assumption of the behavioral theory of timing that pacemaker rate is determined by reinforcement rate. Pigeons discriminated the first half from the second half of a 50-sec trial in a free-operant psychophysical procedure. Left-key responding was reinforced at variable intervals during the first 25 sec, and right-key responding was reinforced at variable intervals during the second 25 sec. The rate of “extraneous” reinforcers delivered at variable intervals following responses to a center key was manipulated independently of performance in the temporal discrimination. Quantitative estimates of pacemaker rate were not directly proportional to extraneous rate of reinforcement, whether extraneous reinforcers were available during the intertrial interval, the entire session, or the trial only. Instead, estimates of pacemaker rate were inversely related to the rate of extraneous reinforcement, which suggests that pacemaker rate is determined by the ratio of the rate of reinforcement for the timing response relative to other sources of reinforcement.