The role of response-reinforcer correlation in signaled reinforcement effects

In three experiments, we examined the effects of signaling reinforcement during operant responding in order to illuminate the factors underlying instrumental overshadowing and potentiation effects. Specifically, we examined whether signaling reinforcement produces an enhancement and attentuation of responding when the response-reinforcer correlation is weak and strong, respectively. In Experiment 1, rats responded on variable-ratio (VR) or variable-interval (VI) schedules that were equated for the number of responses emitted per reinforcer. A signal correlated with reinforcement enhanced response rates on the VR schedule, but attenuated response rates were produced by the signal on the VI schedule. In Experiment 2, two groups of rats responded on a VI schedule while the two other groups received a conjoint VI, negative fixed-ratio schedule in which the subjects lost the availability of reinforcements if they emitted high response rates. A reinforcement signal attenuated responding for the simple VI groups but not for the animals given the negative fixed-ratio component, although the signal improved response efficiency in both groups. In Experiment 3, a poor correlation between responding and reinforcement was produced by a VI schedule onto which the delivery of response-independent food was superimposed. A signal for reinforcement initially elevated responding on this schedule, relative to an unsignaled condition; however, this pattern was reversed with further training. In sum, the present experiments provide little support for the view that signaling reinforcement enhances responding when the response-reinforcer correlation is weak and attenuates responding when this correlation is strong.     

Conjunctive schedules of reinforcement IV: Effects on the pattern of responding of changes in requirement at reinforcement

The effects of different schedule requirements at reinforcement on patterns of responding by pigeons were assessed under conjunctive schedules with comparable response-number requirements, Under one conjunctive schedule (conjunctive fixed-interval fixed-ratio schedule), a response was reinforced after a 6-min interval had elapsedand a specific minimum number of responses had been emitted, Under a second conjunctive schedule, a response was reinforced after the 6-min fixed interval and upon completion of a tandem schedule requirement (conjunctive fixed-interval tandem schedule), This schedule retained the same required minimum number of responses as the first conjunctive schedule, but responses were never reinforced according to a fixed-ratio schedule; the tandem schedule was comprised of a fixed-ratio and a small (.1 to 10.0 sec) fixed-interval schedule, Under the conjunctive fixed-interval fixed-ratio schedule, responding was characterized by an initial pause, an abrupt transition to a high response rate, and a second transition to a lower rate that prevailed or slightly increased up to reinforcement, Under the conjunctive fixed-interval tandem schedule, pauses were extended, response rates were lower, and the initial high rate of responding was generally absent, The above effects depended upon the size of the fixed interval of the tandem schedule, The distinct pattern of responding generated by conjunctive fixed-interval fixed-ratio schedules depends upon occasional reinforcement of fixed-ratio responding and not merely on the addition of a minimum number of required responses.     

Rate of reinforcement and session duration as determinants of within-session patterns of responding

Rats pressed levers for Noyes pellets or keys for sweetened condensed milk reinforcers delivered by multiple schedules. Session length and baseline rates of reinforcement were varied in two experiments. Rates of responding increased during the early part of the session and then decreased for both responses and reinforcers, as well as for all subjects and values of the independent variables. Changes in response rates across the session sometimes exceeded 500%. Respoiise rates peaked approximately 20 min after the beginning of the session, regardless of session duration, when subjects responded on a multiple variable interval 1-min variable interval 1-min schedule. The function was flatter for longer sessions than it was for shorter sessions. The function was flatter, more symmetrical, and peaked later for lower rates of reinforcement than for higher rates of reinforcement. The function appeared early in training, and further experience moved and reduced its peak. Variables related to reinforcement exerted more control over some aspects of this function than did variables related to responding. These within-session patterns of responding may have fundamental implications for experimental design and theorizing.     

Prediction of concurrent keypeck treadle-press responding from simple schedule performance

Four pigeons pecked keys and pressed treadles for food reinforcers delivered by several variable-interval schedules of reinforcement. Then the subjects responded on several concurrent schedules. Keypecking produced reinforcers in one component, and treadle-pressing produced reinforcers in the other. The changeover delay, which prevented reinforcement after all switches from one response to the other, was 0, 5, or 20 sec long. An equation proposed by Kerrnstein (1970) described the rates of treadle-pressing and keypecking emitted during the variable-interval schedules. The k parameter of this equation was larger for keypecking than for treadle-pressing. The R₀ parameters were not systematically different for the two responses. The rates of keypecking and treadle-pressing emitted during the components of the concurrent schedules correlated with, but were not equal to, the rates of responding predicted by Herrnstein’s equation and the subject’s simple schedule responding. The ratios of the rates of responding emitted during, and the ratios of the time spent responding on, the components of the concurrent schedules conformed to an equation proposed by Baum (1974), but not to Herrnstein’s equation.     

Quantitative analysis of local-level resurgence

Resurgence is the recurrence of a previously reinforced and then extinguished behavior induced by the extinction of another more recently reinforced behavior. Resurgence provides insight into behavioral processes relevant to treatment relapse of a range of problem behaviors. Resurgence is typically studied across three phases: (1) reinforcement of a target response, (2) extinction of the target and concurrent reinforcement of an alternative response, and (3) extinction of the alternative response, resulting in the recurrence of target responding. Because each phase typically occurs successively and spans multiple sessions, extended time frames separate the training and resurgence of target responding. This study assessed resurgence more dynamically and throughout ongoing training in 6 pigeons. Baseline entailed 50-s trials of a free-operant psychophysical procedure, resembling Phases 1 and 2 of typical resurgence procedures. During the first 25 s, we reinforced target (left-key) responding but not alternative (right-key) responding; contingencies reversed during the second 25 s. Target and alternative responding followed the baseline reinforcement contingencies, with alternative responding replacing target responding across the 50 s. We observed resurgence of target responding during signaled and unsignaled probes that extended trial durations an additional 100 s in extinction. Furthermore, resurgence was greater and/or sooner when probes were signaled, suggesting an important role of discriminating transitions to extinction in resurgence. The data were well described by an extension of a stimulus-control model of discrimination that assumes resurgence is the result of generalization of obtained reinforcers across space and time. Therefore, the present findings introduce novel methods and quantitative analyses for assessing behavioral processes underlying resurgence.     

Acquisition with partial and continuous reinforcement in pigeon autoshaping

Contemporary time accumulation models make the unique prediction that acquisition of a conditioned response will be equally rapid with partial and continuous reinforcement, if the time between conditioned stimuli is held constant. To investigate this, acquisition of conditioned responding was examined in pigeon autoshaping under conditions of 100% and 25% reinforcement, holding intertrial interval constant. Contrary to what was predicted, evidence for slowed acquisition in partially reinforced animals was observed with several response measures. However, asymptotic performance was superior with 25% reinforcement. A switching of reinforcement contingencies after initial acquisition did not immediately affect responding. After further sessions, partial reinforcement augmented responding, whereas continuous reinforcement did not, irrespective of an animal's reinforcement history. Subsequent training with a novel stimulus maintained the response patterns. These acquisition results generally support associative, rather than time accumulation, accounts of conditioning.     

Effect of separate presentation of the elements on within-compound learning in autoshaping

Five experiments used an autoshaping procedure with pigeon subjects to explore the consequences of separate-element presentation for within-compound associations. When a keylight compound had initially been reinforced, separate presentation of one element reduced responding to the other. This was true whether that separate-element presentation was itself reinforced or nonreinforced. That suggests that the detrimental effect of separate-element presentation results from an effect on the within-compound association, rather than one on the association of either element with the US. When the compound had initially been nonreinforced, a moderate level of reinforcement of one element resulted in responding to the other element. However, more extensive reinforcement reduced that responding. Moreover, an attempt to reverse the adverse consequences of extensive separate-element presentation by further exposure to the compound was not successful. These results comment on the associative mechanisms underlying performance in sensory preconditioning and related paradigms.     

A matching law analysis of the reinforcing efficacy of wheel running in rats

Previous research has demonstrated that running in a rotating wheel functions as a reinforcer for leverpressing in rats. In these studies, the pattern of responding was similar to the pattern of responding maintained by consummatory reinforcers, such as food and water. The present study investigated quantitative features of responding maintained by running. In previous experiments in which responses were reinforced according to variable-interval (VI) schedules and food and water served as the reinforcer, the equation for a rectangular hyperbola described the relationship between response rate and reinforcement rate. This experiment tested whether this quantitative regularity also applies to leverpressing maintained by the opportunity to run in a wheel. Fourteen male Wistar rats responded on levers for the opportunity to run. In each session, subjects were exposed to a series of VI schedules. An opportunity to run for 60 sec was the reinforcing consequence. Results showed that response rate was a negatively accelerated function of reinforcement rate, and the relationship between these two variables was described well by the equation for a rectangular hyperbola. To further test the similarity between running and consummatory reinforcers, the response requirement and access were manipulated. In previous experiments with food and water, these types of manipulations differentially changed the two parameters of the hyperbola. A similar pattern of results was obtained with wheel running. Thus, the equation appears to apply to running about as well as it does to consummatory reinforcers.     

Latent discrimination learning under a regular schedule of partial reinforcement

In this investigation, which employed rats in a runway, discriminative responding consisted of faster running on the reinforced than on the nonreinforced trials of either the 4NR or R4N schedule, both schedules containing fixed, repeated sequences of nonreinforced and reinforced trials. Under the 4NR schedule, four nonreinforced trials preceded a reinforced trial each day, and under the R4N schedule, a reinforced trial was followed by four nonreinforced trials each day. The major finding obtained was that under the 4NR schedule, discriminative responding was improved very substantially by a shift to extinction. Rats maintained on the 4NR schedule did not show improved discriminative responding, nor did discriminative responding improve in extinction following training under either the R4N schedule or a schedule of consistent reinforcement. Latent discrimination learning was defined as discriminative responding which fails to reflect adequately the amount of discrimination learning accomplished. The present findings demonstrate latent discrimination learning for regular schedules of partial reinforcement, something already demonstrated for brightness differential conditioning and possibly DRL schedules, as well.     

The effects of dependent and interdependent group contingencies on socially appropriate responses in classes for emotionally handicapped children

The effects of group dependent and group interdependent reinforcement contingencies on the level of socially appropriate behaviors emitted by emotionally handicapped elementary school students were examined. The investigation employed an ABAC design with a systematic replication (ACAB) across a second classroom. The findings suggested that: (a) Both group dependent and group interdependent reinforcement contingencies were effective in incieasing the level of socially appropriate behaviors. (b) For Classroom II, group interdependent reinforcement contingencies more often resulted in higher rates of responding for individual subjects than did group dependent reinforcement contingencies. (c) Group interdependent contingencies were shown to set the occasion for cooperative behavior among class members designed to help each other achieve individual but interdependent goals.     

Timing processes in the reinforcement-omission effect

A comparison was made of the effects of long-term exposure to fixed-interval reinforcement with unsignaled and with signaled reinforcement omissions. When successive fixed-interval cycles ended in reinforcement, subjects showed a clear “scalloped” response pattern. When reinforcement was omitted, but a brief signal was given in lieu of reinforcement, responding in the next cycle started earlier than it did after reinforcement. When reinforcement was omitted without exteroceptive cues, response rates peaked near the time of reinforcement and then declined to a flat but substantial level. The classicreinforcement-omission effect was observed, in that total responding was greatest after uncued omission, somewhat less after cued omission, and least after reinforcement. However, response rate plotted as a function of time showed that the uncued-omission condition had a very different function from that of the cued-omission or reinforcement conditions. A failed-discrimination account of the omission effect might accommodate the three functions if the discrimination is considered to be a temporal one. The temporal-discrimination account argues that high-rate responding reflects the accumulation of subjective time proximal to the memory of the time of reinforcement. The accumulation resets completely with food reinforcement, incompletely with cues in lieu of reinforcement, and not at all in the absence of cues.     

The sensitivity of the pigeon’s keypeck to the differential reinforcement of long interresponse times

Keypecking of pigeons was studied under differential-reinforcement-of-low-rate (DRL) and variable-interval (VI) schedules in which the interreinforcement times on the two schedules were equated by a yoking procedure. Each schedule was available for half of every session and a change of schedule was signaled by a change of key color. The value of the DRL schedule was varied from .5 to 300 sec. Response rates were always higher in the VI schedule, but within sessions there was a sharp change in response rate coincident with the change in schedule only under lower schedule values. A group without prior training was tested with a 180-sec schedule value, and it, too, developed a higher response rate during the VI schedule, showing that the effect was not dependent on prior experience under low schedule values. In all conditions except the .5- and 1-sec values of the schedule, the mean proportion of responses emitted during the VI schedule was approximately .85 of the responses emitted during both schedules. The conclusion was that the requirement of a minimum interresponse time for reinforcement may work its effect by determining which responses may occur just prior to the reinforced response and thus receive delayed reinforcement.     

Reinforcement context and pacemaker rate in the behavioral theory of timing

In the present experiment, an attempt was made to extend the base of evidence for the assumption of the behavioral theory of timing that pacemaker rate is determined by reinforcement rate. Pigeons discriminated the first half from the second half of a 50-sec trial in a free-operant psychophysical procedure. Left-key responding was reinforced at variable intervals during the first 25 sec, and right-key responding was reinforced at variable intervals during the second 25 sec. The rate of “extraneous” reinforcers delivered at variable intervals following responses to a center key was manipulated independently of performance in the temporal discrimination. Quantitative estimates of pacemaker rate were not directly proportional to extraneous rate of reinforcement, whether extraneous reinforcers were available during the intertrial interval, the entire session, or the trial only. Instead, estimates of pacemaker rate were inversely related to the rate of extraneous reinforcement, which suggests that pacemaker rate is determined by the ratio of the rate of reinforcement for the timing response relative to other sources of reinforcement.     

Interactions of numerical and temporal stimulus characteristics on the control of response location by brief flashes of light

Pigeons pecked on three keys, responses to one of which could be reinforced after 3 flashes of the houselight, to a second key after 6, and to a third key after 12. The flashes were arranged according to variable-interval schedules. Response allocation among the keys was a function of the number of flashes. When flashes were omitted, transitions occurred very late. Increasing flash duration produced a leftward shift in the transitions along a number axis. Increasing reinforcement probability produced a leftward shift, and decreasing reinforcement probability produced a rightward shift. Intermixing different flash rates within sessions separated allocations: Faster flash rates shifted the functions sooner in real time, but later in terms of flash count, and conversely for slower flash rates. A model of control by fading memories of number and time was proposed.     

Signal-food contingency and signal frequency in a continuous trials auto-shaping paradigm

Five groups of pigeons were studied in an auto-shaping procedure which programmed two types of trials represented by hues on the response key. Each signal was separated by a brief intertriai interval. Three groups were studied with a positive correlation between one of the signals and food (contingent groups). They differed with respect to the frequency with which the positive signal appeared. Two noncontingent groups were studied in which the correlation between the signals and food was eliminated by programming food with the same probability following either signal. One noncontingent group had a high density of reinforcement produced by adding reinforcement in the other signal, at the same rate as programmed in the positive signal for the contingent groups. The other noncontingent group experienced the same number of reinforcements in the session as the contingent group with the least frequent positive trial, but these reinforcements were distributed with equal probability across the signals. Birds in the contingent groups with intermediate or infrequent positive signals all acquired reliable pecking, with acquisition most rapid for the infrequent signal. Maintained responding covaried with the speed of acquisition. No birds in the noncontingent groups showed reliable responding. Birds in the contingent group with a frequent positive signal (approximately 3/4 of the session), also showed no reliable pecking. This result suggests that more than one noncontingent group is informative for assessing the role of differential reinforcement probability in the acquisition of auto-shaped keypecking. In particular, a noncontingent group which controls for the frequency of reinforced trials is an appropriate reference group.

     

Multiple determinants of transfer of evaluative function after conditioning with free-operant schedules of reinforcement

The aim of the four present experiments was to explore how different schedules of reinforcement influence schedule-induced behavior, their impact on evaluative ratings given to conditioned stimuli associated with each schedule through evaluative conditioning, and the transfer of these evaluations through derived stimulus networks. Experiment 1 compared two contrasting response reinforcement rules (variable ratio [VR], variable interval [VI]). Experiment 2 varied the response to reinforcement rule between two schedules but equated the outcome to response rate (differential reinforcement of high rate [DRH] vs. VR). Experiment 3 compared molar and molecular aspects of contingencies of reinforcement (tandem VIVR vs. tandem VRVI). Finally, Experiment 4 employed schedules that induced low rates of responding to determine whether, under these circumstances, responses were more sensitive to the molecular aspects of a schedule (differential reinforcement of low rate [DRL] vs. VI). The findings suggest that the transfer of evaluative functions is determined mainly by differences in response rate between the schedules and the molar aspects of the schedules. However, when neither schedule was based on a strong response reinforcement rule, the transfer of evaluative judgments came under the control of the molecular aspects of the schedule.     

Multiple-schedule interactions and discrimination

Pigeons received variable-interval (VI) reinforcement for keypecking during randomized presentations of seven line-orientation stimuli forming a continuum ranging from horizontal (0 deg) to vertical (90 deg). Each line presentation lasted for 30 sec and was preceded and followed by 30-sec time-outs. After responding stabilized, only responding in the two extreme stimuli (0 and 90 deg) was reinforced. As discrimination training proceeded, strong behavioral contrast and dimensional contrast effects appeared. However, only marginal local effects (local contrast and local dimensional effects), exerted by one line-orientation component upon a second, appeared, indicating that behavioral and dimensional contrast may be independent of parallel local effects. An attempt was made to apply Blough’s (1975) quantitative model of operant generalization and discrimination to the present discrimination procedure. However, this model did not predict the generalization gradient shape that was experimentally obtained. This experiment also yielded two serendipitous findings: (1) Positive behavioral contrast appeared in an extinction-related stimulus (time-out) when other stimuli were switched from reinforcement to extinction (hitherto, positive behavioral contrast had been observed only in responding to a reinforcementrelated stimulus when other stimuli were switched from reinforcement to extinction), and (2) final responding was higher in the presence of an extinction stimulus that had always been an extinction stimulus than it was in the presence of other extinction stimuli that had previously been paired with VI reinforcement.     

Multiple-pair training enhances transposition in pigeons

We studied transposition in pigeons by training them to select the smaller (or the larger) of a pair of circles. In training, different groups of pigeons were given one pair, two pairs, or three pairs of circles along the size dimension. Testing included two stimulus pairs for which, according to theoretical postdiscrimination generalization gradients, transposition should decrease from one-pair to two-pair to three-pair training. On the basis of the results of our earlier study (Lazareva, Wasserman, & Young, 2005) and contrary to these predictions, we expected that transposition should increase from one-pair to two-pair to three-pair training. We found that multiple-pair discrimination training enhanced transposition, which, on average, rose from 47% (one-pair training) to 52% (two-pair training) to 64% (three-pair training). In addition, we found that the overall similarity of the testing pair to the training pair(s) modulated the strength of relational responding. These results demonstrate that encountering multiple instances of a rule leads to stronger relational learning, even when reinforcement history predicts the opposite trend. These results also provide strong evidence against stimulus generalization as the sole determinant of relational responding in transposition.     

The determinants of random choice

The purpose of the present experiment was to describe a pattern of reinforcement sufficient to produce an unpredictable pattern of choice response by rats. On each trial two levers were inserted into an experimental chamber. If the reinforcement was always contingent upon single alternation (a simple pattern), Ss learned to alternate at significantly better than chance level; if reinforcement was contingent upon alternation on 50% of the trials (an insoluble pattern), Ss developed a position preference. To produce apparently random responding, the less preferred response (an alternation) was differentially reinforced on 75% of the trials. A simple stochastic model adequately described the results.