Extinction of operant behavior: An analysis based on foraging considerations

In two experiments, the frequency of food reinforcement provided by variable interval (VI) schedules prior to extinction was varied. In the first experiment, two-component multiple schedules resulted in a greater number of responses in extinction in the presence of the stimulus previously associated with the richer of the two component schedules than that previously associated with the leaner schedule. In the second experiment, different groups of animals were trained on different VI schedules. Responding in extinction was analyzed into bouts of responding showing that the number of response bouts increased and the number of responses per bout decreased with decreasing frequency of reinforcement during training. These data are compatible with an analysis of operant behavior based on an analogy to processes that presumably-occur-in naturalistic foraging situations. According to this analogy, behavior associated with search for a food source (i.e., number of response bouts) and that of procurement of food from a source (i.e., responses per bout) represent aspects of behavior that are differentially strengthened by different VI schedules. Extinction serves to reveal this differential strengthening.     

Reinforcement and the organization of behavior in golden hamsters: Sunflower seed and nest paper reinforcers

Undeprived hamsters received nest paper or sunflower seeds in their home cages, free or contingent on one of three activities. The pattern of effects of reinforcement was the same as that reported previously with food for hungry hamsters in an open field: Open rearing and scrabbling quickly increased in rate when reinforced with paper or seeds, but face washing showed little or no increase. Certain interrelations found previously between noncontingent effects of reinforcers, deprivation effects, and contingent effects were absent and therefore are not essential to the differences among these three behaviors during reinforcement. The hamsters could learn to interrupt fact washing to collect either paper or seeds, but even then they did not increase their time face washing to high levels. Raising the operant level of face washing by spraying the animals with water did not improve instrumental performance. Face washing could be classified as an involuntary activity, but this term does not provide a full account of its special properties, such as shortened bouts, during reinforcement and punishment.     

Multiple determinants of transfer of evaluative function after conditioning with free-operant schedules of reinforcement

The aim of the four present experiments was to explore how different schedules of reinforcement influence schedule-induced behavior, their impact on evaluative ratings given to conditioned stimuli associated with each schedule through evaluative conditioning, and the transfer of these evaluations through derived stimulus networks. Experiment 1 compared two contrasting response reinforcement rules (variable ratio [VR], variable interval [VI]). Experiment 2 varied the response to reinforcement rule between two schedules but equated the outcome to response rate (differential reinforcement of high rate [DRH] vs. VR). Experiment 3 compared molar and molecular aspects of contingencies of reinforcement (tandem VIVR vs. tandem VRVI). Finally, Experiment 4 employed schedules that induced low rates of responding to determine whether, under these circumstances, responses were more sensitive to the molecular aspects of a schedule (differential reinforcement of low rate [DRL] vs. VI). The findings suggest that the transfer of evaluative functions is determined mainly by differences in response rate between the schedules and the molar aspects of the schedules. However, when neither schedule was based on a strong response reinforcement rule, the transfer of evaluative judgments came under the control of the molecular aspects of the schedule.     

Defensive burying in rats and hamsters

Hooded rats and golden hamsters were shocked by one of two prods in a chamber with a sawdust-covered floor. Rats buried the prod through which they had been shocked, but hamsters displayed no burying behavior. Hamsters may not have buried the prod because they could not perform the required motor pattern. However, hamsters can carry and pile food pellets. Therefore, in a second experiment, rats and hamsters were shocked in a chamber with wooden blocks on the floor. Rats piled blocks around the prod through which they had been shocked, but hamsters did not. The third experiment established that, like rats, hamsters can associate a prod with shock in one trial, since they showed differential avoidance of a prod through which they had been shocked. Since hamsters are nonsocial and rats are social, these results are consistent with suggestions that burying sources of aversive stimulation evolved as an altruistic behavior.     

Adjunctive behavior of the rat under periodic food delivery in a 24-hour environment

The behavior of 4 rats living in complex environments was monitored 24 h per day during free-feeding baseline and under conditions of periodic access to food. Under the periodic schedules, the minimum interfood interval (IFI) was increased from 16 to 512 sec in an ascending series. Periodic food produced robust overall increases in investigation of the feeder, drinking, general activity, and rearing, but not in wheel-running. The temporal distribution of behavior within the IFI was similar across subjects and supported the hypothesis that some responses were largely time-locked to the period immediately following eating, while other responses expanded to fill the interval. However, these response differences were not adequately captured by present classification schemes. Finally, the distribution of drinking following a food pellet strongly resembled the distribution of drinking following bouts of feeding in baseline. The results suggest that adjunctive behavior stems from three sources: (1) a simple increase in the number of opportunities for expression of normal preprandial and postprandial behavior, (2) an increase in the preprandial behavior directed toward the site of expected food, and (3) an increase in the postprandial distribution of both site-directed and more general exploratory behavior. These findings suggest that adjunctive behavior is not extraneous, but is an orderly distribution of responses ordinarily related to feeding and foraging for food.     

Procedure for preventing response strain on random interval schedules with a linear feedback loop

An experiment examined the impact of a procedure designed to prevent response or extinction strain occurring on random interval schedules with a linear feedback loop (i.e., an RI+ schedule). Rats lever-pressed for food reinforcement on either a RI+ or a random interval (RI) schedule that was matched to the RI+ schedule in terms of reinforcement rate. Two groups of rats responded on an RI+ and two on an RI schedule matched for rate of reinforcement. One group on each schedule also received response-independent food if there had been no response for 60 s, and response-independent food continued to be delivered on an RT-60 schedule until a response was made. Rats on the RI and RI+ obtained similar rates of reinforcement and had similar reinforced inter-response times to one another. On the schedules without response-independent food, rats had similar rates of response to one another. However, while the delivery of response-independent food reduced rates of response on an RI schedule, they enhanced response rates on an RI+ schedule. These results suggest that rats can display sensitivity to the molar aspects of the free-operant contingency, when procedures are implemented to reduce the impact of factors such as extinction-strain.     

Time and response matching with topographically different responses

Four pigeons were exposed to several nonindependent concurrent variable-interval schedules of reinforcement. One schedule component required a keypecking response; the other component required a treadlepressing response. The birds matched the ratio of their behavior (as measured by responses and time) between the two topographically different responses to the ratio of reinforcement in those two components. When additional foods not contingent on a keypeck or treadle-press were then added, the birds matched time spent in the components to total rates of food delivered in those components; response matching was somewhat disrupted. The matching law, developed under concurrent variable-interval schedules requiring similar responses, can thus account for choice behavior involving topographically different responses.     

Tracking of the expected time to reinforcement in temporal conditioning procedures

In one experiment, the rate and pattern of responding (head entry into the food cup) under different distributions of intervals between food deliveries were examined. Separate groups of rats received fixed-time (45, 90, 180, or 360 sec), random-time (45, 90, 180, or 360 sec), or tandem fixed-time (45 or 90 sec) random-time (45 or 90 sec) schedules of reinforcement. Schedule type affected the pattern of responding as a function of time, whereas mean interval duration affected the mean rate of responding. Responses occurred in bouts with characteristics that were invariant across conditions. Packet theory, which assumes that the momentary probability of bout occurrence is negatively related to the conditional expected time remaining until the next reinforcer, accurately predicted global and local measures of responding. The success of the model advances the prediction of multiple measures of responding across different types of time-based schedules.     

Within- and between-subjects partial reinforcement effects with an autoshaped response using Japanese quail (Coturnix coturnix japonica)

Animals exposed to schedules of partial reinforcement are typically more resistant to extinction than are animals trained with continuous reinforcement. This is the partial reinforcement effect (PRE). Animals experienced with both partial and continuous schedules are often more persistent on the continuous schedule, yielding a reversed PRE. Both conventional and reversed PREs have been elusive with classical conditioning paradigms. The present experiment attempted to demonstrate between- and within-subject PREs using 50% and 100% autoshaping schedules. Presence or absence of a PRE depended on the behavioral measures used. Marked terminal group differences in acquisition produced a between-subjects PRE with absolute response levels but not with rate-of-change measures. Within subjects, only choice trial comparisons were sensitive enough to differentiate the two schedules. Acquisition data were inconsistent with most of the classical conditioning PRE literature, but consistent with results reported in the autoshaping literature. These discrepancies may reflect the operant-classical interaction in autoshaping.     

Preference for the interval schedule following multiple variable-ratio yoked-variable-interval schedule training

Pigeons were studied on multiple variable-ratio yoked-variable-interval schedules in which components had equal rates of food reinforcement and appeared equally often on each of two keys. Interpolated between component changes on the final multiple schedule were 10-sec probes in which both schedule stimuli were present, one on each key. During multiple schedule training, variable-ratio response rates were greater than yoked-variable-interval rates; however, response rate differences in the components were not a function of the mean ratio value for the 40-to-320-ratio range studied. During the choice probes, subjects responded more to the stimulus associated with the interval schedule than to the one associated with the ratio schedule. It was concluded that pigeons prefer interval schedules over equal reinforcement rate ratio schedules, because the former generate fewer responses per reinforcement.     

A reevaluation of the CS− hypothesis for schedule-induced polydipsia under intermittent schedules of pellet delivery

Lashley and Rosellini (1980) have recently suggested that schedule-induced polydipsia (SIP) is determined by the occurrence of absolute periods within schedules of periodic food delivery which are associated with a low probability of food delivery, that is, CS? periods. To assess this hypothesis, SIP was examined in the present experiments under three schedules—fixed time, variable time, and random time (RT)—which differed in probability of occurrence and/or duration (Experiment 1), and under a range of RT schedules in which the CS? period was systematically varied by changing the interpellet interval (Experiment 2). In both experiments, the level and temporal distribution of SIP did not seem to be related to the absolute period associated with the absence of food. Instead, SIP was more systematically related to the average length of the interpellet interval and, therefore, to the average period associated with no food. It was suggested that drinking under intermittent schedules of pellet delivery, that is, SIP, is determined by an average CS? period and not by an absolute period associated with the unavailability of food.     

Positive behavioral contrast as a function of time-out duration when pigeons peck keys on a within-session procedure

Three pigeons pecked keys for food reinforcers delivered by multiple variable interval variable interval schedules in the first part of each session (baseline) and by multiple variable interval extinction schedules in the second part of each session (contrast). The variable interval schedules delivered reinforcers after an average of 4 min or 30 sec in different conditions. The duration of a time-out between the components varied in five steps from 5 to 120 sec. Positive contrast occurred for all time-out durations in both experiments. That is, the rate of responding emitted during the constant, variable interval component was greater during the contrast than during the baseline schedules. The size of contrast did not change systematically with changes in timeout duration. These results violate most theories of contrast. They are compatible with the idea that animals integrate reinforcers over intervals longer than 2 min.     

Response–food delay gradients for lever pressing and schedule-induced licking in rats

Eight food-deprived Wistar rats developed stable patterns of lever pressing and licking when exposed to a fixed-time 30-s schedule of food pellet presentation. The rats were trained to lever press by presenting the lever 10 s before the programmed food delivery, with the food pellet being delivered immediately upon a lever press. The operant contingency was then removed and the lever was inserted through the entire interfood interval, being withdrawn with food delivery and reinserted 2 s later. On successive phases of the study, a protective contingency postponed food delivery if responses (lever presses or licks) occurred within the last 1, 2, 5, 10, 20, or 25 s of the interfood interval. Lever pressing was reduced at much shorter response–food delays than those that reduced licking. These results demonstrate that reinforcement contributes to the maintenance of different response patterns on periodic schedules, and that different responses are differentially sensitive to delays.     

Factors affecting schedule-induced wood-chewing in rats: Percentage and rate of reinforcement,and operant requirement

In Experiment 1, rats were allowed to acquire either schedule-induced drinking or schedule-induced wood-chewing behavior under a fixed-interval (FI) 60-sec schedule of food reinforcement, following which food was omitted from 20% and then 50% of interreinforcement intervals. Omission of food severely disrupted induced drinking but had relatively little effect on induced wood-chewing. Experiment 2 investigated wood-chewing as a function of reinforcement rate, using a range of FI schedules from 5 to 180 sec in duration. Both the amount of chewing per session and the relative time spent chewing were bitonically related to reinforcement rate. In Experiment 3, schedule-induced chewing that had been acquired under a response-dependent schedule was found to persist under a response-independent schedule. Induced wood-chewing resembles other induced behaviors in important respects, but quantitative differences are also apparent.     

Reinforcer and response specificity in appetitive transfer of control

Three experiments examine transfer from appetitive Pavlovian conditioning to appetitive instrumental responding by varying the similarity between conditions of Pavlovian reinforcement and instrumental reward. After conditioning with rats confined in a restraining device, a CS for electrical stimulation of the brain (ESB) produced substantial facilitation of operant responding for ESB, while a CS for food facilitated operant responding for food. However, no effects on rate of responding for food were seen during a CS for ESB. In a fourth experiment, four groups of rats were trained to barpress for rewarding electrical stimulation of the brain (ESB) and then given discriminative Pavlovian conditioning with ESB. The groups differed in the degree of similarity between the stimulus-response sequences present during Pavlovian conditioning and those occurring in instrumental responding. As similarity increased, so did the degree of conditioned facilitation in subsequent transfer tests. These results indicate that conditioned incentive responses or reinforcer-derived expectancies are specific to the conditions under which they develop, rather than generalized emotional or motivational responses.     

Stimulus- and feeder-directed behavior in a long-box: Effect of fixed versus variable time schedules of food presentation

Two groups of 6 pigeons were exposed to either a fixed-time (FT) or a variable-time (VT) schedule of response-independent food presentation. The interval between two successive food presentations (food-food [FF] interval) was either 20 or 40 sec. The duration of stimulus presentation (stimulus-food [SF] interval) was varied relative to the FF interval. All subjects were exposed to different information ratios (IRs; IR = SF/FF; IR = 1.00, 0.50, 0.15). To study the relative contribution of sign- and goal-tracking behavior to keypecking observed in standard autoshaping procedures, pigeons were autoshaped in a long-box. In the long-box, the stimulus key and the feeder are located 60 cm apart. Stimuli were always presented at one end of the box, and food was presented at the other end. Locomotor behavior and keypecks were recorded. Pigeons engaged in sign-tracking behavior when IR = 0.15, but only when presentation of the food was unpredictable on the basis of other variables (e.g., the passage of time since the last food presentation, as in FT schedules). In the case of FT schedules, subjects engaged in feeder-directed activities. No effects of varying the FF interval were found. Keypecking was observed only when the SF interval was short (IR = 0.15) and food was presented on a VT schedule.     

Schedule-induced wood-chewing in rats and its dependence on body weight

In Experiment 1, 12 rats were exposed to an FT 60 schedule of food reinforcement, followed either by extinction or by a massed-food control condition, in the presence of a wood block. In 9 rats, wood-chewing behavior increased systematically during the FT 60 condition and declined again during extinction or massed food, while the other 3 rats showed virtually no chewing behavior at any stage of the experiment. In Experiment 2, frequency and bout duration of wood-chewing under an FT 60 schedule of food reinforcement declined as body weight increased, in 7 rats. We conclude that wood-chewing qualifies as a schedule-induced behavior, and that it resembles schedule-induced drinking in its dependence on body weight. Unlike drinking, however, induced chewing occupied the middle region of the 60-sec interreinforcement interval, declined markedly within the session, and showed considerable within- and between-subject variability.     

Contextual control of operant behavior: evidence for hierarchical associations in instrumental learning

Recent research has suggested that operant responses can be weakened when they are tested in new contexts. The present experiment was therefore designed to test whether animals can learn a context–(R–O) relation. Rats were given training sessions in context A, in which one response (R1; lever pressing or chain pulling) produced one outcome (O1) and another response (R2; chain pulling or lever pressing) produced another outcome (O2) on variable interval reinforcement schedules. These sessions were intermixed with training in context B, where R1 now produced O2 and R2 produced O1. Given the arrangement, it was possible for the animal to learn two distinct R–O associations in each specific context. To test for them, rats were then given aversion conditioning with O2 by pairing its presentation with lithium-chloride-induced illness. Following the aversion conditioning, the rats were given an extinction test with both R1 and R2 available in each context. During testing, rats showed a selective suppression in each context of the response that had been paired with the reinforcer subsequently associated with illness. Rats could not have performed this way without knowledge of the R–O associations in effect in each specific context, lending support to the hypothesis that rats learn context–(R–O) associations. However, despite a complete aversion to O2, responding was not completely suppressed, leaving the possibility open that rats form context–R associations in addition to context–(R–O) associations.     

Pavlovian contingencies and anticipatory contrast

Pigeons were trained on a four-component multiple schedule in which two target components with identical reinforcement schedules were followed by other components with either higher or lower reinforcement rates. The Pavlovian signal properties of the target-component stimuli were varied by changes in their duration relative to the following components, and by whether the two following components were cued by the same or different stimuli, When different stimuli occurred in the following components, response rates were higher in the target component preceding the following component with the lower reinforcement rate, and these contrast effects were larger with shorter relative durations. But with nondifferential stimuli in the following components, contrast consistently occurred only with the longer durations of the target components. Moreover, several subjects with the shorter duration target stimuli had higher response rates in the target followed by the richer schedule—that is, Pavlovian conditioning occurred to the target stimuli. This interaction suggests that the processes underlying anticipatory contrast and Pavlovian conditioning are in opposition, and that the Pavlovian effect can dominate ifthe signaling properties of the target components are sufficiently enhanced.