Factors affecting schedule-induced wood-chewing in rats: Percentage and rate of reinforcement,and operant requirement

In Experiment 1, rats were allowed to acquire either schedule-induced drinking or schedule-induced wood-chewing behavior under a fixed-interval (FI) 60-sec schedule of food reinforcement, following which food was omitted from 20% and then 50% of interreinforcement intervals. Omission of food severely disrupted induced drinking but had relatively little effect on induced wood-chewing. Experiment 2 investigated wood-chewing as a function of reinforcement rate, using a range of FI schedules from 5 to 180 sec in duration. Both the amount of chewing per session and the relative time spent chewing were bitonically related to reinforcement rate. In Experiment 3, schedule-induced chewing that had been acquired under a response-dependent schedule was found to persist under a response-independent schedule. Induced wood-chewing resembles other induced behaviors in important respects, but quantitative differences are also apparent.     

Effect on subsequent fixed-interval schedule performance of prior exposure to ratio and interval schedules of reinforcement

In three experiments, we examined the effect on the patterns of responding noted on fixed interval (FI) schedules of prior exposure to a range of interval and ratio schedules. Rats leverpressed for food reinforcement on random ratio (RR), random interval (RI), or variable interval (VI) schedules prior to transfer to FI schedules. In Experiment 1, prior exposure to an RR schedule retarded the development of typical FI patterns of responding. Exposure to a yoked RI schedule produced even greater retardation of typical FI performance. This effect was replicated in Experiment 2, using a within-subjects design. Rats responded on a multiple RR-RI schedule prior to a multiple FI-FI schedule. Typical FI performance emerged more slowly in the component previously associated with the RI than with that associated with the RR. In Experiment 3, exposure to an RR schedule retarded the development of FI performance to a greater extent than did exposure to a VR schedule. The latter schedule was programmed to allow the possibility that inhibitory control would develop after reinforcement. These results confirm that ratio schedules independently result in the disruption of FI responding. This effect was not long lasting and cannot be used plausibly to explain species differences in responding to FI schedules. However, it does suggest that temporal control—as manifested by the transfer of inhibitory control from one schedule to another—could facilitate movement between interval schedules.     

The role of trial tracking in rats’ working memory

The experiments reported in the present study tested whether decreasing intertrial intervals (ITIs) intensifies the disruptive effects of increasing retention intervals (RIs) in a delayed conditional discrimination by decreasing the animal’s trial tracking accuracy (Cohen & Armstrong, 1996; Cohen & Roberts, 1996). Rats responded on a fixed ratio (FR) 1 or fixed interval (FI) 10-sec reinforcement schedule at a second light or tone stimulus, S2, when the first light or tone stimulus, S1, had signaled an FI 10-sec or FR 1 schedule, respectively. RIs between S1 and S2 were increased from 3 to 24 sec and never exceeded ITIs that were reduced from 24 to 6 sec. For some rats, the trials were separated from each other by extending the lever at S1 and retracting it at the end of S2 (ITI lever-retracted group). For other, control rats, the lever remained extended throughout the session (lever-extended group, Experiment 1) or was extended and retracted with the onset and offset of each stimulus (RI/ITI lever-retracted group, Experiment 2). The rats under all trial conditions learned to delay leverpressing on the FI 10-sec schedule. Latency to begin leverpressing on the FI 10-sec schedule declined as RIs were increased, but this effect was attenuated in the ITI lever-retracted groups in both experiments, as would be predicted by thetrial tracking hypothesis. Decreasing ITIs from 24 to 6 sec intensified the disruptive effects of increasing RIs from 3 to 6 sec in the RI/ITI lever-retracted group (Experiment 2), as would be predicted by the trial tracking hypothesis.     

Operant response topographies of rats receiving food or water reinforcers on FR or FI reinforcement schedules

Separate groups of food- and water-deprived rats pressed a lever for food or water, respectively, on continuous reinforcement and various fixed-ratio and fixed-interval reinforcement schedules. Food-reinforced rats on continuous, FR 2-, or FI 10-sec schedules showed consistently longer mean lever contact durations per leverpress than did water-reinforced rats on the same schedules. Mean lever-contact-duration differences between food- and water-reinforced rats were greatly attenuated or disappeared under FR 4-, FR 8-, FI 20-sec, and FI 30-sec schedules of reinforcement. These results are interpreted as supporting earlier hypotheses that there are respondent components of operantly conditioned and autoshaped leverpresses, but that these respondent components weaken with partial reinforcement and the leverpress topography comes under the control of operant contingencies.     

Schedule-induced polydipsia contrast in the rat

Schedule-induced polydipsia was studied using a behavioral contrast paradigm. Food pellets were delivered to food-deprived rats on a response-independent FT 1-min schedule. Licking on a tube produced water on a MULT FR 10 FR 10, MULT FR 10 EXT, or MIXED FR 10 EXT for three rats (Experiment 1) and on a MULT VI VI, MULT VI EXT, or MIXED VI EXT schedule for three other rats (Experiment 2). On the FR schedules, rats could drink more water by increasing lick rates, but on the VI schedules the amount of drinking was fixed by the schedule parameters and was relatively unaffected by lick rates. Relative to MULT FR FR, positive polydipsia contrast was clearly demonstrated on MULT and MIXED FR EXT; but relative to MULT VI VI, contrast was not demonstrated on MULT and MIXED VI EXT. These data suggest that polydipsia contrast occurs only if increased licking permits increased drinking.     

Selected schedules of reinforcement in the black-tailed prairie dog (Cynomys ludovicianus)

Wild black-tailed prairie dogs were run on FR, FI, VR, and VI schedules for Noyes pellet reinforcement. Cumulative barpress responses, postreinforcement pause lengths, and responses per second were recorded. The highest response rates occurred in the VR schedules, with the lowest response rates coming in the FI schedules. Fixed-ratio schedules had the longest postreinforcement pauses, VI schedules had the shortest. At the upper levels of the fixed-ratio schedules (FR 90–100), the animals ceased to respond consistently. Generally, data from prairie dogs were consistent with data reported in studies from other mammalian species.     

Additive summation by stimulus compounding irrespective of behavioral contrast during discrimination training: An investigation with positive reinforcement and avoidance schedules

The similarity in the discrimination training leading to behavioral contrast and that preceding tests producing response enhancement to combined discriminative stimuli suggested that the two phenomena might be related. This was investigated by determining if contrast indiscrimination training was necessary for this outcome of stimulus compounding. Responding to tone, light, and to the simultaneous absence of tone and light (T + L) was maintained during baseline training by food reinforcement in Experiment I and by shock avoidance in Experiment II. During subsequent discrimination training, responding was reduced in T + L by programming nonreinforcement in Experiment I and safety or response-punishment in Experiment II. In the first experiment, one rat exhibited positive behavioral contrast, i.e., tone and light rates increased while his T + L rate decreased. In Experiment II, rats punished in T + L showed contrast in tone and light, this being the first demonstration of punishment contrast on an avoidance baseline with rats. The discrimination acquisition data are discussed in the light of current explanations of contrast by Gamzu and Schwartz (1973) and Terrace (1972). During stimulus compounding tests, all subjects in both experiments emitted more responses to tone-plus-light than to tone or light (additive summation). An analysis of the terminal training baselines suggests that the factors producing these test results seem unrelated to whether or not contrast occurred during discrimination training. It was concluded that the stimulus compounding test reveals the operation of the terminal baseline response associations and reinforcement associations conditioned on these multicomponent free-operant schedules of reinforcement.

     

Resistance to extinction: contingency termination and generalization decrement

We present an algebraic model of resistance to extinction that is consistent with research on resistance to change. The model assumes that response strength is a power function of reinforcer rate and that extinction involves two additive, decremental processes: (1) the termination of the reinforcement contingency and (2) generalization decrement resulting from reinforcer omission. The model was supported by three experiments. In Experiment 1, 4 pigeons were trained on two-component multiple variable-interval (VI) 60-sec, VI 240-sec schedules. In two conditions, resistance to change was tested by terminating the response-reinforcer contingency and presenting response-independent reinforcers at the same rate as in training. In two further conditions, resistance to change was tested by prefeeding and by extinction. In Experiment 2, 6 pigeons were trained on two-component multiple VI 150-sec schedules with 8-sec or 2-sec reinforcers, and resistance to change was tested by terminating the response-reinforcer contingency in three conditions. In two of those conditions, brief delays were interposed between responses and response-independent reinforcers. In both Experiments 1 and 2, response rate was more resistant to change in the richer component, except for extinction in Experiment 1. In Experiment 3, 8 pigeons were trained on multiple VI 30-sec, VI 120-sec schedules. During extinction, half of the presentations of each component were accompanied by a novel stimulus to produce generalization decrement. The extinction data of Experiments 1 and 3 were well described by our model. The value of the exponent relating response strength and reinforcement was similar in all three experiments.     

Response strength and temporal control in fixed-interval schedules

Pigeons responded in a two-component peak procedure in which the components differed in terms of reinforcement magnitude (Experiment 1), immediacy (Experiment 2), or probability (Experiment 3). The prediction of behavioral momentum theory that responding in the relatively richer component should be more resistant to change was tested by (1) presenting response-independent food in the intervals between components according to a variable-time (VT) schedule, (2) prefeeding, and (3) extinction. In all the experiments, peak location in baseline occurred earlier, relative to the schedule value in the richer component. Peak response rate was more resistant to change in the richer component during the VT and prefeeding tests, and change in peak rate was more sensitive to differential reinforcement than change in overall response rate. Changes in measures of performance on peak trials during the disruptor tests were partially consistent with predictions of the behavioral theory of timing. The results suggest that peak response rate provides a more sensitive index of resistance to change for fixed-interval schedules than does overall response rate and that reinforcement strengthens both peak responding and temporal control.     

Signaled reinforcement effects on fixed-interval performance of the rat

In three experiments using fixed-interval schedules with a 500-msec delay of reinforcement, rats receiving a localized light signal during the delay leverpressed more slowly than rats trained without the signal. In Experiment 1 these groups showed no differences in temporal patterning of responding, but in the remaining two experiments the signaled rats showed better patterning than the unsignaled rats. In Experiments 2 and 3 rats receiving a diffuse tone signal during the delay instead of a light pressed more rather than less rapidly than the unsignaled group. Their patterning was better than that of the unsignaled rats in one of these experiments. Several explanations for both the attenuation and enhancing effects of signaling reward are discussed.     

Interval timing under a behavioral microscope: Dissociating motivational and timing processes in fixed-interval performance

The distribution of latencies and interresponse times (IRTs) of rats was compared between two fixed-interval (FI) schedules of food reinforcement (FI 30 s and FI 90 s), and between two levels of food deprivation. Computational modeling revealed that latencies and IRTs were well described by mixture probability distributions embodying two-state Markov chains. Analysis of these models revealed that only a subset of latencies is sensitive to the periodicity of reinforcement, and prefeeding only reduces the size of this subset. The distribution of IRTs suggests that behavior in FI schedules is organized in bouts that lengthen and ramp up in frequency with proximity to reinforcement. Prefeeding slowed down the lengthening of bouts and increased the time between bouts. When concatenated, latency and IRT models adequately reproduced sigmoidal FI response functions. These findings suggest that behavior in FI schedules fluctuates in and out of schedule control; an account of such fluctuation suggests that timing and motivation are dissociable components of FI performance. These mixture-distribution models also provide novel insights on the motivational, associative, and timing processes expressed in FI performance. These processes may be obscured, however, when performance in timing tasks is analyzed in terms of mean response rates.     

The value hypothesis and acquisition of preference in concurrent chains

Experiment 1 compared the acquisition of initial- and terminal-link responding in concurrent chains. The terminal-link schedules were fixed interval (FI) 10 sec and FI 20 sec, but some presentations were analogous to no-food trials in the peak procedure, lasting 60 sec with no reinforcement delivery. Pigeons completed a series of reversals in which the schedules signaled by the terminal-link stimuli (red and green on the center key) were changed. Acquisition of temporal control of terminal-link responding (as measured by peak location on no-food trials) was more rapid than acquisition of preference in the initial links. Experiment 2 compared acquisition in concurrent chains, using the typical procedure in which the terminal-link schedules are changed with a novel arrangement in which the initial-link key assignments were changed while the terminal-link schedules remained the same. Acquisition of preference was faster in the latter condition, in which the terminal-link stimulus-reinforcer relations were preserved. These experiments provide the first acquisition data that support the view that initial-link preference is determined by the values of the terminal-link stimuli.     

Effect of required response force on rats’ performance on a VI+ schedule of reinforcement

Three experiments were performed to examine the effect of response force on rats’ performance on various schedules of reinforcement. Response force was manipulated by changing the weight of the lever in the operant chamber—a heavy lever for high response force and a light lever for low response force. Using a within-subjects design, Experiment 1 replicated previous findings that rats respond more quickly on variable ratio (VR) than on equivalent variable-interval-plus-linear-feedback (VI+) schedules. Experiment 2 replicated this finding but also showed that the use of a smaller response force abolished the response rate difference between the VR and VI+ schedules. Experiment 3 used a between-subjects design and showed a response rate difference between the VR and VI+ schedules with a high response force but no response rate difference with a low response force. This suggests that under conditions of low force, when the rats’ responding can continue at prolonged high rates, these subjects show little difference in their response rates between VR and VI+ schedules. These data are similar to those found for human subjects.     

Procedure for preventing response strain on random interval schedules with a linear feedback loop

An experiment examined the impact of a procedure designed to prevent response or extinction strain occurring on random interval schedules with a linear feedback loop (i.e., an RI+ schedule). Rats lever-pressed for food reinforcement on either a RI+ or a random interval (RI) schedule that was matched to the RI+ schedule in terms of reinforcement rate. Two groups of rats responded on an RI+ and two on an RI schedule matched for rate of reinforcement. One group on each schedule also received response-independent food if there had been no response for 60 s, and response-independent food continued to be delivered on an RT-60 schedule until a response was made. Rats on the RI and RI+ obtained similar rates of reinforcement and had similar reinforced inter-response times to one another. On the schedules without response-independent food, rats had similar rates of response to one another. However, while the delivery of response-independent food reduced rates of response on an RI schedule, they enhanced response rates on an RI+ schedule. These results suggest that rats can display sensitivity to the molar aspects of the free-operant contingency, when procedures are implemented to reduce the impact of factors such as extinction-strain.     

Potentiation of responding on a VR schedule by a stimulus correlated with reinforcement: Effects of diffuse and localized signals

In Experiment 1, rats were trained to leverpress on a variable ratio (VR) 30 schedule with a 500-msec delay between the reinforced response and food delivery. Subjects that experienced a signal during the delay responded faster than did control subjects that received the stimulus un-correlated with reinforcement. Higher response rates were obtained when the stimulus used to signal reinforcement was auditory rather than visual. Experiments 2 and 3 compared the effects of signaling reinforcement with either a localized or a diffuse light on responding maintained by VR schedules of reinforcement. Elevated response rates were observed with the diffuse stimulus, but the localized stimulus failed to produce such potentiation. Experiment 3 also examined the conditioned reinforcing power of localized and diffuse visual stimuli. These results are discussed with reference to (1) theories of selective association and sign tracking and (2) their implications for current theories of signaling reinforcement.     

The role of response-reinforcer correlation in signaled reinforcement effects

In three experiments, we examined the effects of signaling reinforcement during operant responding in order to illuminate the factors underlying instrumental overshadowing and potentiation effects. Specifically, we examined whether signaling reinforcement produces an enhancement and attentuation of responding when the response-reinforcer correlation is weak and strong, respectively. In Experiment 1, rats responded on variable-ratio (VR) or variable-interval (VI) schedules that were equated for the number of responses emitted per reinforcer. A signal correlated with reinforcement enhanced response rates on the VR schedule, but attenuated response rates were produced by the signal on the VI schedule. In Experiment 2, two groups of rats responded on a VI schedule while the two other groups received a conjoint VI, negative fixed-ratio schedule in which the subjects lost the availability of reinforcements if they emitted high response rates. A reinforcement signal attenuated responding for the simple VI groups but not for the animals given the negative fixed-ratio component, although the signal improved response efficiency in both groups. In Experiment 3, a poor correlation between responding and reinforcement was produced by a VI schedule onto which the delivery of response-independent food was superimposed. A signal for reinforcement initially elevated responding on this schedule, relative to an unsignaled condition; however, this pattern was reversed with further training. In sum, the present experiments provide little support for the view that signaling reinforcement enhances responding when the response-reinforcer correlation is weak and attenuates responding when this correlation is strong.     

Appetitive-aversion interactions: Facilitation of aversive conditioning by prior appetitive training in the rat

In Experiment 1, four groups of rats received conditioned suppression training in which a tone was reinforced with shock. If the tone had been previously paired with response-independent food, aversive conditioning was slightly facilitated by comparison to control groups preexposed either to the tone randomly associated with food or to the tone and food unpaired. However, by comparison to a control which was not preexposed to the tone, animals receiving prior pairings of the tone and food showed retarded aversive conditioning. Experiment 2 replicated the facilitation in aversive conditioning after the tone had been paired with food relative to the random control condition and demonstrated that this difference occurred even if the tone and background stimuli continued to be associated with response-independent food during aversive conditioning. This result suggests that pairing a stimulus with an appetitive reinforcer reduces the retardation of aversive conditioning produced by stimulus preexposure.     

Covariation in conditioned response strength between stimuli trained in compound

Conditioned lick suppression by water-deprived rats was used to elaborate on recent evidence that the attenuated conditioned response elicited by an overshadowed stimulus may be enhanced by extinction of the overshadowing stimulus with which it had been trained in simultaneous compound. Using a modified serial stimulus arrangement in which a light coexisted with the last half of a tone that terminated with footshock, it was found in Experiment 1 that the tone overshadowed the light. Extinction of the tone-shock association resulted in a virtually complete recovery of the response to the overshadowed light. Using this serial overshadowing procedure, the possibility that the strength of a conditioned response to an element trained in compound covaries as a function of the strength of the response to the other element was tested in Experiment 2. Following overshadowing training similar to that of Experiment 1, independent reinforcement of the overshadowed light, that is, associative inflation, was found to have no deleterious effect on the response to the overshadowing tone. This suggests that the effects of postconditioning extinction and inflation of one element do not have symmetrical effects upon responding to the other element. The results of Experiment 2 were replicated in Experiment 3 using a simultaneous compound stimulus as opposed to the serial compound of the previous studies. These results are discussed in terms of various associative and cognitive models of learning and performance.     

The role of predictability in preventing escape deficits following loss of control over food acquisition

The present experiment examined whether predictability of food acquisition would eliminate the impairment of subsequent escape performance that otherwise resulted from uncontrollability over food acquisition. In pretreatment, the yoked and the yoked-signal groups received response-independent food at the same times as the experimental group acquired it on an FR 5/20 lever-press schedule. However, a pellet presented for the yoked-signal group followed a 1.5-sqc tone, which served as a predictive signal of food. The naive control group received the same clumber of pellets in their home cages in this phase. Results of the escape latency in the subsequent FR 2 shuttling shock-escape test indicated that the predictability of outcome eliminated the escape deficits showed by the yoked-non signal group. This modulating effect of a predictive signal is hypothesized to be due to an overshadowing of uncontrollability by predictability.     

The stimulus-response overshadowing phenomenon with VI versus FI schedules of reinforcement

Previous research has shown that response rates on a variable interval (VI) schedule of reinforcement decrease if a brief response-produced signal is given prior to reward. One explanation is that the signal overshadows the response because it is a better predictor of reinforcement. The S-R overshadowing effect does not occur with variable ratio (VR) schedules, however. Tarpy, Lea, and Midgley (1983) explained this fact by suggesting that the signal functions to enhance the salience of the temporal interval offset on the VI schedule (a characteristic not possessed by VR schedules), which then overshadows the response. In this experiment, the salience of the temporal interval was enhanced in another way: signaled or unsignaled reward was provided to rats responding on either a VI or fixed interval (FI) reward schedule. As predicted, rates were lowest for animals receiving signaled reinforcement on an FI schedule and highest for those receiving unsignaled reinforcement on a VI schedule.