Behavioral variability is controlled by discriminative stimuli

Previous research has demonstrated that behavioral variability can be modified by reinforcers contingent on it, but there has been no convincing evidence of discriminative stimulus control over such variability. We therefore rewarded 20 rats for variable response sequences in the presence of one stimulus and provided equal rewards independently of sequence variability in the presence of a second stimulus. We found that sequence variability was significantly higher during the first stimulus than during the second, with the greatest difference occurring immediately following onset of the stimuli. Removing the discriminative stimuli caused levels of variability to converge. These experiments provide strong evidence that behavioral variability can be controlled by discriminative stimuli, which may be important for general theories of operant behavior and their applications.     

Behavioral momentum and relapse of extinguished operant responding

Previous experiments on behavioral momentum have shown that relative resistance to extinction of operant behavior in the presence of a stimulus depends on the rate of reinforcement associated with that stimulus, even if some of those reinforcers occur independently of the behavior. We present three experiments examining whether the rate of reinforcement in the presence of a stimulus similarly modulates the relative relapse of operant behavior produced by reinstatement, resurgence, and renewal paradigms. During baseline conditions, pigeons responded for food reinforcement on variable-interval 120-sec schedules in alternating periods of exposure to two stimuli arranged by a multiple schedule. Additional response-independent food presentations were also delivered in the presence of one of the multiple-schedule stimuli. Consistent with previous research, baseline response rates were lower in the presence of the stimulus with the added response-independent reinforcement, and relative resistance to extinction was greater in the presence of that stimulus. In addition, following extinction, the relative relapse of responding produced by reinstatement, resurgence, and renewal paradigms was greater in the presence of the stimulus associated with the higher rate of reinforcement. We suggest that a model of extinction from behavioral momentum theory may be useful for understanding these results.     

The role of response-reinforcer correlation in signaled reinforcement effects

In three experiments, we examined the effects of signaling reinforcement during operant responding in order to illuminate the factors underlying instrumental overshadowing and potentiation effects. Specifically, we examined whether signaling reinforcement produces an enhancement and attentuation of responding when the response-reinforcer correlation is weak and strong, respectively. In Experiment 1, rats responded on variable-ratio (VR) or variable-interval (VI) schedules that were equated for the number of responses emitted per reinforcer. A signal correlated with reinforcement enhanced response rates on the VR schedule, but attenuated response rates were produced by the signal on the VI schedule. In Experiment 2, two groups of rats responded on a VI schedule while the two other groups received a conjoint VI, negative fixed-ratio schedule in which the subjects lost the availability of reinforcements if they emitted high response rates. A reinforcement signal attenuated responding for the simple VI groups but not for the animals given the negative fixed-ratio component, although the signal improved response efficiency in both groups. In Experiment 3, a poor correlation between responding and reinforcement was produced by a VI schedule onto which the delivery of response-independent food was superimposed. A signal for reinforcement initially elevated responding on this schedule, relative to an unsignaled condition; however, this pattern was reversed with further training. In sum, the present experiments provide little support for the view that signaling reinforcement enhances responding when the response-reinforcer correlation is weak and attenuates responding when this correlation is strong.     

The effect of rate of reinforcement and time in session on preference for variability

Pigeons pecked keys on concurrent-chains schedules that provided a variable interval 30-sec schedule in the initial link. One terminal link provided reinforcers in a fixed manner; the other provided reinforcers in a variable manner with the same arithmetic mean as the fixed alternative. In Experiment 1, the terminal links provided fixed and variable interval schedules. In Experiment 2, the terminal links provided reinforcers after a fixed or a variable delay following the response that produced them. In Experiment 3, the terminal links provided reinforcers that were fixed or variable in size. Rate of reinforcement was varied by changing the scheduled interreinforcer interval in the terminal link from 5 to 225 sec. The subjects usually preferred the variable option in Experiments 1 and 2 but differed in preference in Experiment 3. The preference for variability was usually stronger for lower (longer terminal links) than for higher (shorter terminal links) rates of reinforcement. Preference did not change systematically with time in the session. Some aspects of these results are inconsistent with explanations for the preference for variability in terms of scaling factors, scalar expectancy theory, risk-sensitive models of optimal foraging theory, and habituation to the reinforcer. Initial-link response rates also changed within sessions when the schedules provided high, but not low, rates of reinforcement. Within-session changes in responding were similar for the two initial links. These similarities imply that habituation to the reinforcer is represented differently in theories of choice than are other variables related to reinforcement.     

The role of reinforcement and nonreinforcement in an operant frustration effect

Male albino rats were run in a discrete-trial two-bar operant analog of the double alley. Completion of a FR 4 response chain on the first bar was rewarded 50% of the time for the 12 experimental Ss, but was never rewarded for the 12 control Ss. Both groups received consistent reward at the end of a FR 4 chain on the second bar. Eighty-four trials were given at a rate of four trials per day. A significantly faster rate of responding on the second bar was found following nonrewarded first-bar ratios than following rewarded first-bar ratios. This frustration effect was not attributable to response depression, since the nonrewarded performance of the experimental group exceeded that of the control group.     

Partial reinforcement in autoshaping with pigeons

The acquisition, maintenance, and extinction of autoshaped responding in pigeons were studied under partial and continuous reinforcement. Five values of probability of reinforcement, ranging from .1 to 1.0, were combined factorially with five values of intertrial interval ranging from 15 to 250 sec for different groups. The number of trials required before autoshaped responding emerged varied inversely with the duration of the intertriai interval and probability of reinforcment, but partial reinforcement did not increase the number of reinforcers before acquisition. During maintained training, partial reinforcement increased the overall rate of responding. A temporal gradient of accelerated responding over the trial duration emerged during maintenance training for partial reinforcement groups, and was evident for all groups in extinction. Partial reinforcement groups responded more than continuous reinforcement groups over an equivalent number of trials in extinction. However, this partial-reinforcment extinction effect disappeared when examined in terms of the omission of “expected” reinforcers.     

Rate of reinforcement and session duration as determinants of within-session patterns of responding

Rats pressed levers for Noyes pellets or keys for sweetened condensed milk reinforcers delivered by multiple schedules. Session length and baseline rates of reinforcement were varied in two experiments. Rates of responding increased during the early part of the session and then decreased for both responses and reinforcers, as well as for all subjects and values of the independent variables. Changes in response rates across the session sometimes exceeded 500%. Respoiise rates peaked approximately 20 min after the beginning of the session, regardless of session duration, when subjects responded on a multiple variable interval 1-min variable interval 1-min schedule. The function was flatter for longer sessions than it was for shorter sessions. The function was flatter, more symmetrical, and peaked later for lower rates of reinforcement than for higher rates of reinforcement. The function appeared early in training, and further experience moved and reduced its peak. Variables related to reinforcement exerted more control over some aspects of this function than did variables related to responding. These within-session patterns of responding may have fundamental implications for experimental design and theorizing.     

Instrumental responding by rats on free-operant schedules with components that schedule response-dependent reinforcer omission: Implications for optimization theories

In two experiments, we assessed whether rats optimize the number of reinforcers per response. In Experiment 1, one group of rats was trained to leverpress for food reinforcement on a simple variable-interval (VI) 60-sec schedule. For another group, a negative fixed-ratio component was imposed on the VI schedule to produce a conjoint contingency in which reinforcement became available on the VI schedule but was omitted when the ratio criterion was satisfied. In Experiment 2, one group of rats responded on a VI schedule and also received response-independent food. For another group, responding above a certain rate canceled the response-independent food. Despite the negative contingency experienced by the groups in Experiments 1 and 2, responding was maintained at a level at which the number of obtained reinforcers was reduced substantially below the maximum number possible. In addition, in both experiments, the groups that experienced the negative contingency responded at a lower rate than did a yoked control group that experienced the same frequency of reinforcement but lacked the negative component. These results demonstrate that although rats do not optimize their behavior with respect to reinforcement, they are nevertheless sensitive to some aspect of the instrumental contingency in operation.     

History effects on induced and operant variability

Two experiments evaluated history effects on induced and operant variability. College students typed three-digit sequences on a computer keyboard. Sequence variability was induced (by no reinforcement or variation- independent reinforcement) or reinforced (by variation- or repetition-dependent reinforcement). Conditions with induced and operant variability were presented according to a reverse between-groups design. In Experiment 1, we examined transitions from the variation or repetition contingencies to no reinforcement, and vice versa. In Experiment 2, the variation or repetition contingencies were followed or preceded by variation-independent reinforcement. The results showed that (1) a history of no reinforcement impaired operant variability learning; (2) induced variability levels were higher and lower after a history of reinforcement for variation and repetition, respectively; (3) repetition was more easily disrupted by no reinforcement and independent reinforcement than was variation; and (4) response variability and stability were a function of past and current reinforcement conditions. These results indicate that reinforcement history influences both induced and operant variability levels.     

Classwide Peer Tutoring with or without Reinforcement: effects on academic responding,content coverage,achievement, intrinsic interest and reported project experiences

ABSTRACT The present study examined the effects of Classwide Peer Tutoring (CWPT) under two conditions‐‐with reinforcement (CWPT+ R) and without reinforcement (CWPT‐‐ R)‐‐on 77 students’ spelling performance and intrinsic interest in lower secondary school Integrated Science. The students displayed educational attainments in the average range. Both CWPT approaches led to significant improvements in spelling test performance. Both groups also evaluated the programme positively. However, the CWPT+ R group made significantly greater learning gains than did the CWPT‐‐ R students. These appeared to be related to higher levels of academic responding. Despite their impressive spelling gains, students in the CWPT+ R group displayed significantly lower intrinsic interest in Integrated Science after involvement in CWPT. However, great caution should be exercised in interpreting this last result, as statistically significant effects werenot found when group differences at post‐test were examined by way of t‐test and analysis of covariance methods. In summary, the data from this study suggest: (a) that the CWPT method of instruction is useful for helping lower secondary school students, (b) that reinforcers may increase learning gains by encouraging higher rates of responding, but (c) that there is an indication, subject to caution, that these reinforcers may possibly depress intrinsic interest in an academic activity (science in the case of this study).     

The generality of within-session patterns of responding: Rate of reinforcement and session length

Rats and pigeons responded for food delivered according to multiple schedules. The session length varied from 10 to 120 min, and the programmed rate of reinforcement varied from 15 to 240 reinforcers per hour. Response rates usually changed systematically within experimental sessions. For both rats and pigeons, responding reached a peak after an approximately constant amount of time since the beginning of the session, regardless of session length. When rats, but not pigeons, served as subjects, the peak rates of responding occurred later in the session and the within-session changes were smaller for lower than for higher rates of reinforcement. The similarities between the results for rats and for pigeons when session length varied suggest that at least one of the factors that produces the within-session changes in responding is shared by the present species, responses, and reinforcers. The differences in results when rate of reinforcement varied are more difficult to interpret.     

Prediction of concurrent keypeck treadle-press responding from simple schedule performance

Four pigeons pecked keys and pressed treadles for food reinforcers delivered by several variable-interval schedules of reinforcement. Then the subjects responded on several concurrent schedules. Keypecking produced reinforcers in one component, and treadle-pressing produced reinforcers in the other. The changeover delay, which prevented reinforcement after all switches from one response to the other, was 0, 5, or 20 sec long. An equation proposed by Kerrnstein (1970) described the rates of treadle-pressing and keypecking emitted during the variable-interval schedules. The k parameter of this equation was larger for keypecking than for treadle-pressing. The R₀ parameters were not systematically different for the two responses. The rates of keypecking and treadle-pressing emitted during the components of the concurrent schedules correlated with, but were not equal to, the rates of responding predicted by Herrnstein’s equation and the subject’s simple schedule responding. The ratios of the rates of responding emitted during, and the ratios of the time spent responding on, the components of the concurrent schedules conformed to an equation proposed by Baum (1974), but not to Herrnstein’s equation.     

Blocking despite changes in reinforcer identity

Rats were trained on a discriminated operant barpressing task according to a standard blocking design. In some conditions, the reinforcer was changed between the pretraining and compound conditioning phases; for other conditions, the reinforcer remained the same across phases. In three separate experiments using both between- and within-subject designs, strong blocking effects occurred regardless of the change in the reinforcer. In a fourth experiment, a multiple schedule of reinforcement was used in which response-independent reinforcers were superimposed on the schedule of response-contingent reinforcers. The degree of response suppression caused by the free reinforcer was greater when the free reinforcers were the same as the response-contingent reinforcers than when they were different. The role played by the reinforcer identity in contingency experiments thus appears to be different from the role it plays in blocking experiments.     

A matching law analysis of the reinforcing efficacy of wheel running in rats

Previous research has demonstrated that running in a rotating wheel functions as a reinforcer for leverpressing in rats. In these studies, the pattern of responding was similar to the pattern of responding maintained by consummatory reinforcers, such as food and water. The present study investigated quantitative features of responding maintained by running. In previous experiments in which responses were reinforced according to variable-interval (VI) schedules and food and water served as the reinforcer, the equation for a rectangular hyperbola described the relationship between response rate and reinforcement rate. This experiment tested whether this quantitative regularity also applies to leverpressing maintained by the opportunity to run in a wheel. Fourteen male Wistar rats responded on levers for the opportunity to run. In each session, subjects were exposed to a series of VI schedules. An opportunity to run for 60 sec was the reinforcing consequence. Results showed that response rate was a negatively accelerated function of reinforcement rate, and the relationship between these two variables was described well by the equation for a rectangular hyperbola. To further test the similarity between running and consummatory reinforcers, the response requirement and access were manipulated. In previous experiments with food and water, these types of manipulations differentially changed the two parameters of the hyperbola. A similar pattern of results was obtained with wheel running. Thus, the equation appears to apply to running about as well as it does to consummatory reinforcers.     

Reinforcer and response specificity in appetitive transfer of control

Three experiments examine transfer from appetitive Pavlovian conditioning to appetitive instrumental responding by varying the similarity between conditions of Pavlovian reinforcement and instrumental reward. After conditioning with rats confined in a restraining device, a CS for electrical stimulation of the brain (ESB) produced substantial facilitation of operant responding for ESB, while a CS for food facilitated operant responding for food. However, no effects on rate of responding for food were seen during a CS for ESB. In a fourth experiment, four groups of rats were trained to barpress for rewarding electrical stimulation of the brain (ESB) and then given discriminative Pavlovian conditioning with ESB. The groups differed in the degree of similarity between the stimulus-response sequences present during Pavlovian conditioning and those occurring in instrumental responding. As similarity increased, so did the degree of conditioned facilitation in subsequent transfer tests. These results indicate that conditioned incentive responses or reinforcer-derived expectancies are specific to the conditions under which they develop, rather than generalized emotional or motivational responses.     

Concurrent-chain performance in transition: Effects of terminal-link duration and individual reinforcers

Pigeons responded on concurrent-chain schedules with variable-interval initial links and equal delays as terminal links. The terminal-link delays were 1 sec in some conditions and 20 sec in other conditions. The percentages of reinforcers delivered for responses on the left key were 10%, 30%, 70%, or 90%, and this percentage was switched every five to nine sessions. The rate of change in the pigeons’ response percentages after a switch was the same whether the terminal-link delays were 1 sec or 20 sec. Analysis of the effects of individual reinforcers showed that after a response on one key had been reinforced, response percentages on that key were higher for at least the next 100 responses. Small effects of individual reinforcers were evident after eight or nine additional reinforcers had been delivered. The effects of individual reinforcers were about equally large during times of transition and during periods in which overall response percentages were relatively stable.     

Stimulus control of topographically tagged responding

Pigeons’ responses on an operant key were reinforced according to either multiple variable-interval variable-interval or multiple variable-interval extinction schedules. The multiple-schedule components were signaled by line-tilt stimuli on a second key (signal key). Signal-key responses never produced reinforcement, and operant-key responses were not reinforced if they followed within 1 sec of a signal-key response. Behavioral contrast was not observed on the operant key, although there was a small, but reliable, increase in signal-key responding in the variable-interval component of the multiple variable-interval extinction condition. Generalization tests were interspersed between sessions of multiple variable-interval extinction training. Generalization gradients along the line-tilt dimension exhibited peak shift for both operant-key and signal-key responding following intradimensional (line tilt) discrimination training. Line-tilt generalization gradients following interdimensional discrimination did not exhibit peak shift. Gradients following intradimensional discrimination were sharper than gradients following interdimensional discrimination for both operant-key and signal-key responding. It was concluded that dimensional stimulus control of topographically tagged responding maintained by the stimulusreinforcer relation parallels that maintained by the response-reinforcer relation.     

Development of preference and spontaneous recovery in choice behavior with concurrent variable-interval schedules

Pigeons pecked on two response keys that delivered reinforcers on a variable-interval schedule. The proportion of reinforcers delivered by one key was constant for a few sessions and then changed, and subjects’ choice responses were recorded during these periods of transition. In Experiment 1, response proportions approached a new asymptote slightly more slowly when the switch in reinforcement proportions was more extreme. In Experiment 2, slightly faster transitions were found with higher overall rates of reinforcement. The results from the first session, after a switch in the reinforcement proportions, were generally consistent with a mathematical model that assumes that the strength of each response is increased by reinforcement and decreased by nonreinforcement. However, neither this model nor other similar models predicted the “spontaneous recovery” observed in later sessions: At the start of these sessions, response proportions reverted toward their preswitch levels. Computer simulations could mimic the spontaneous recovery by assuming that subjects store separate representations of response strength for each session, which are averaged at the start of each new session.     

Response sequence learning as a function of primary versus conditioned reinforcement

Two groups of pigeons were required to generate a fixed sequence of responses on three keys, for example, middle-left-right. One group received a small food reward (S^Food) following each correct response except the terminal one, which was followed by a large food reward. The second group received conditioned reinforcement from an overhead light (S^Light) for each correct response, with the terminal correct response followed by both S^Light and the large food reward. We manipulated length of sequence (3 or 7 responses) and duration of required interresponse interval (IRI; 1 to 9 sec). S^Light contingencies generated more accurate performances than did S^Food when sequence length was 3 responses but not when it was 7 responses. IRI duration influenced accuracy under the S^Light contingencies but not under S^Food. These results show that conditioned reinforcers sometimes generate more accurate sequence learning than do primary reinforcers, and that schedule contigencies influence which type of feedback will optimize performance. The results parallel those from the matching-to-sample and conditional discrimination literature.     

Behavioral variability as a function of response topography and reinforcement contingency

Long-Evans rats were reinforced for generating variable sequences of responses on two operanda. The current sequence of four left and right responses was required to differ from each of the previous five sequences. Variability under thisvary schedule was compared with that under ayoke control schedule where reinforcement was independent of the sequences. Three different response topographies were compared: two levers were pressed in one case, two keys pushed in another, and two wires pulled in a third. Both reinforcement contingency (vary vs. yoke) and response topography (leverpress, key push, and wire pull) significantly influenced sequence variability. As is the case for operant dimensions, behavioral variability is jointly controlled by reinforcement contingency and response topography.