The influence of brief stimuli uncorrelated with reinforcement on choice between variable-ratio schedules

Experiment 1 investigated the behavior of rats trained to leverpress on a concurrent variable ratio (VR) 30 VR-30 schedule with a brief, 500-msec, light occurring at the midpoint of the ratio on one of the levers. Higher response rates were recorded on the lever associated with this stimulus, a finding that paralleled the effect produced by inserting primary reinforcement at the midpoint (i.e., by training on a concurrent VR-30 VR-15 schedule). Similar results were found in Experiment 2 using a concurrent VR-20 VR-20 schedule with a 2-sec visual stimulus presented midway through one of the components. In addition, a brief stimulus inserted midway through the VR-20 component of a concurrent VR-20 VR-10 schedule retarded the development of a difference in response rates between the components relative to a VR-20 VR-10 group lacking the signal. In Experiment 3, multiple VR VR schedules were used. Again, the response rate was higher in the component that had the added stimulus or, for a second group of subjects, on the component with the smaller response requirement. Probe-choice trials revealed a preference for the component that generated the higher rate in both groups. Presenting a stimulus partway through a ratio appears to reduce the effect on response rate and choice of a large ratio value.     

The influence of context-reinforcer associations on instrumental performance

In each of two experiments, rats were trained to press the lever in a Skinner box, food reinforcement being available on a variable-interval 60-sec schedule (VI 60). There followed an “exposure phase” for which the levers were removed from the boxes, and then a final test with the levers replaced to assess the effects of the intervening treatment on instrumental responding. Experiment 1 showed that simple exposure to the box reduced the vigor of instrumental performance in comparison with a condition in which food was made available during the exposure phase. Animals which received no exposure treatment also showed a relatively high rate of response. Experiment 2 demonstrated that an exposure treatment in which the occurrence of food is signaled by a light stimulus also leads to a decline in instrumental responding. These results are held to support the notion that associations between the context and the reinforcer serve to energize appetitive instrumental behavior.     

Applying Konorski’s model of classical conditioning to signal-centered behavior in the rat: Some functional similarities between hunger CRs and sign-tracking

Two experiments were conducted to investigate functional similarities between “hunger CRs” of Konorski’s (1967) model of appetitive classical conditioning and sign-tracking behavior in rats. Konorski’s model predicts that hunger CRs will be facilitated (1) when a nonrein-forced stimulus similar to the reinforced CS is introduced, and (2) when some CS presentations are unexpectedly nonreinforced. In Experiment 1, hungry rats acquired a leverpress response to a retractable lever that was paired with response-independent food. Following this training, a second lever was introduced whose presentation was not followed by food. The effect of the presence of this second lever was to facilitate responding to the original lever. In Experiment 2, single-lever autoshaping training was followed by a shift from 100% pairing of the lever with food to only 50% of the lever presentations being followed by food. The introduction of partial reinforcement produced an immediate and durable increase in leverpressing. The findings of both experiments are consistent with predictions from Konorski’s model of classical conditioning if sign-tracking is considered as a “hunger CR.”     

Negative correlation between tone (S−) and water increases target biting during S− in rats

Eighteen Long-Evans rats were exposed to a MULT VT 30-sec EXT schedule of water presentations. The EXT schedule was signaled by a 2500-Hz, 70-dB tone. Mean number of bites of a plastic target during atone (S?) increased across sessions, whereas the mean number of target bites during the VT component signaled byno tone (S+) decreased across sessions. Changing the multiple schedule to MULT EXT EXT produced a decrease in target bites during the tone. Reinstating the original MULT VT 30-sec EXT schedule produced an increase in number of target bites during the tone (S?). Changing the multiple schedule to VT 30-sec VT 30-sec produced a decrease in target bites during the tone. The results indicate that target biting in rats increases in the presence of the exteroceptive tone (S?) of the MULT VT 30-sec EXT schedule, and that target biting during the tone varies directly with the negative correlation between tone and water presentations.     

Effect of intertrial interval on variable-interval discrete-trial barpressing

In seven experiments, an effect of the intertriai interval (ITI) duration on barpressing by rats was studied. A stimulus signaled a 15-sec variable-interval trial. The first response after the interval elapsed turned the stimulus off and was rewarded with food. Trials were separated by long (about 300 sec) or short (about 10 sec) ITIs. A within-subjects design established that response rate on trials after long ITIs was lower than that after short ITIs (Experiments 1 and 3–7). The effect was not cumulative (the effect of one and five consecutive short ITIs was the same). Response rate after short and long ITIs was the same when a between-subjects design was used (Experiment 2). Response rate was higher after 160-sec ITIs than after 300-sec ITIs, suggesting that the ITI duration at which all longer ITIs are treated the same (i.e., the upper limit) is greater than 160 sec (Experiment 3). When food, the trial stimulus, a novel stimulus, or a familiar stimulus never paired with food, was presented 10 sec before the next trial during some of the long ITIs, response rate on the next trial was similar to that found after 10-sec ITIs (Experiments 4–6). This similarity suggested that these events could mark the start of the ITI. However, the familiar stimulus did so only when it reliably predicted that the next trial would occur after a short interval. The effect of ITI duration on responding was apparently attributable to response latency. Response latency was greater after long ITIs, but once responding began, it was similar after long and short ITIs (Experiment 7).     

Microanalysis of variable-interval performance during stimulus compounding

Additive summation is observed when more responses are emitted to the simultaneous presentation (tone-plus-light) of independently conditioned stimuli (tone and light) than to either stimulus presented alone. The current experiment sought to determine if this increased rate during tone-plus-light was a function of a new modal interresponse time (IRT) or a differential mixing of pauses with a modal IRT characteristic of the responding in tone and light alone. Three rats were trained on a three-component multiple schedule where tone and light were each associated with a variable-interval 30-sec schedule while a variable-interval 100-sec operated in the simultaneous absence of these stimuli, tone-off and light-out. Baseline response rates were 2–4 times as high in tone or light as in their absence. In testing, more responses were emitted to tone-plus-light than to tone or light by all animals, but the modal IRT was in the 0.2–0.4-sec IRT bin for all test conditions. Tone-plus-light controlled fewer long IRT values and more responses in the short modal category than tone or light alone. These results support the response mixing hypothesis of stimulus control; i.e., no “new” behavior was observed during the novel combination of stimulus elements, only a mixture of previously reinforced behavior patterns in different proportions.     

Acquisition of schedule-induced polydipsia by rats in proximity to upcoming food delivery

Food-deprived rats that receive intermittent delivery of small amounts of food develop excessive drinking--specifically, schedule-induced polydipsia (SIP). A main characteristic of SIP is its occurrence at the beginning of interfood intervals. The purpose of this study was to demonstrate that SIP can be developed toward the end of interfood intervals, in closer proximity to upcoming than to preceding food delivery. In Experiment 1, two groups were exposed to a fixed-time (FT) 30-sec food schedule with water available during the first or the last 15 sec of each interfood interval. Two additional groups, which had access to water throughout, were exposed to FT 30-sec or FT 15-sec schedules of food presentation. The FT 30-sec group with free access to water developed the highest level of intake; similar and intermediate levels were induced in all the remaining groups. In Experiment 2, three groups of rats were exposed to an FT 90-sec food schedule with water available during the first, the second, or the last 30 sec of each interfood interval. One additional group with access to water throughout was exposed to the FT 90-sec schedule of food presentation. The group with free access to water developed a higher level of consumption than did the other groups, but by the end of training none of the four groups showed statistical differences in polydipsic drinking. Results show that adjunctive drinking can be developed in proximity to upcoming food delivery even with long interfood intervals.     

Contextual effects in latent inhibition with an appetitive conditioning procedure

In three experiments, rats received presentations of a diffuse 30-sec stimulus (a light) and of food, and their tendency to enter the food tray was monitored. Experiment 1 showed that when the light was made to signal the delivery of food, the response of entering the food tray increased in frequency during the stimulus. The acquisition of this conditioned response to the light was retarded in subjects that had received preexposure to the stimulus. In Experiments 2 and 3, subjects received preexposure to the stimulus, some in the same context as that subsequently used for stimulus-food pairings and some in a different context. Those experiencing the change of context acquired the response more readily. It is argued that these results demonstrate a latent inhibition effect that is attenuated by contextual change.     

Spatiotemporal characteristics of serial CSs and their relation to search modes and response form

In four experiments, we examined how the spatiotemporal proximity to food of the two elements of a serial conditioned stimulus (CS) influenced the pattern of CS-directed versus food-site-directed behavior in rats. Experiment 1 showed that only temporal proximity affected responding when the serial CS consisted of two successive 4-sec presentations of either a spatially near or a spatially far lever (NN or FF). However, Experiment 2 showed that behavior depended markedly on whether rats received a near followed by a far lever (NF) or a far followed by a near lever (FN). Experiment 3 showed that the effects of Experiment 2 could be changed by increasing the duration of the second CS element, and Experiment 4 showed that these changes were not related to previous training. We concluded that behavior produced by the spatiotemporal qualities of the lever elements can be attributed to a mapping between the temporal qualities of the CS elements and an underlying sequence of search modes related to finding food.     

Signal-directed behavior in the rat: Interactions between the nature of the CS and the nature of the UCS

Two experiments are described, which involved the investigation of interactions between the nature of the conditioned stimulus (CS) and the nature of the unconditioned stimulus (UCS) in producing signal-centered behavior. In Experiment 1, rats received response-independent heat reinforcement in a cold environment. For some groups, this heat UCS was signaled by presentations of a standard aluminum retractable lever; for other groups, it was signaled by a retractable lever covered in acrylic fur (furry lever CS). Only the subjects that received the furry lever CS paired with heat exhibited differential CS-contact behavior, when compared with unpaired, aluminum lever, and warm control subjects. In Experiment 2, hungry rats received pairings of either an aluminum or a furry lever with food (UCS). When compared with unpaired controls, only the subjects that received the aluminum lever paired with food showed differential signal-directed behavior; the subjects receiving the furry lever CS did not show differential contact with the CS, but instead exhibited differential food tray entry behavior during CS presentation. In the two studies, the signal-directed behavior exhibited by subjects resembled either thermoregulatory or feeding behaviors characteristic of rats. The results suggest that signal-directed behavior is determined by a complex interaction between the ecological relevance of the CS and the nature of the UCS—an interaction that can best be described in terms of a behavior systems model of conditioned responding.     

Stimulus functions within a fixed-interval clock schedule: Reinforcement,punishment, and discriminative stimulus control

Pigeons received food for responding on a fixed-interval 32-sec schedule divided into three equal parts, each correlated with a distinctive, response-independent, visual stimulus. Response rate was very low during the first two thirds of the interval but high during the terminal third. When a response-dependent brief stimulus correlated with the terminal third was arranged for each response in the presence of the stimuli correlated with the first two thirds, response rate was enhanced, especially in the middle third. However, response rate was suppressed when each response in the presence of the stimulus correlated with the final third produced a brief stimulus correlated with the initial third. A similar suppressive effect occurred when each response produced a brief stimulus correlated with the middle third. Response suppression decreased over successive response-dependent brief-initial-stimulus manipulations. The results were interpreted in terms of reinforcement, punishment, and discriminative stimulus control by visual stimuli correlated with parts of a fixed-interval schedule.     

Behavioral momentum and relapse of extinguished operant responding

Previous experiments on behavioral momentum have shown that relative resistance to extinction of operant behavior in the presence of a stimulus depends on the rate of reinforcement associated with that stimulus, even if some of those reinforcers occur independently of the behavior. We present three experiments examining whether the rate of reinforcement in the presence of a stimulus similarly modulates the relative relapse of operant behavior produced by reinstatement, resurgence, and renewal paradigms. During baseline conditions, pigeons responded for food reinforcement on variable-interval 120-sec schedules in alternating periods of exposure to two stimuli arranged by a multiple schedule. Additional response-independent food presentations were also delivered in the presence of one of the multiple-schedule stimuli. Consistent with previous research, baseline response rates were lower in the presence of the stimulus with the added response-independent reinforcement, and relative resistance to extinction was greater in the presence of that stimulus. In addition, following extinction, the relative relapse of responding produced by reinstatement, resurgence, and renewal paradigms was greater in the presence of the stimulus associated with the higher rate of reinforcement. We suggest that a model of extinction from behavioral momentum theory may be useful for understanding these results.     

Resurgence of behavior during extinction depends on previous rate of response

In two experiments, food-deprived rat subjects leverpressed for food in three successive training phases. In the first phase of both experiments, rats were exposed to a multiple schedule, one component of which produced a high rate of response, and the other of which produced a lower rate of response (multiple random ratio [RR], random interval [RI] in Experiment 1, and multiple differential reinforcement of high rate, differential reinforcement of low rate in Experiment 2). Rats were then transferred to a multiple fixed interval (FI; 60-sec, 60-sec) schedule, until the effects of the first phase on response rate were no longer apparent and their response rates did not differ from those of rats responding on a multiple FI 60-sec, FI 60-sec schedule without previously experiencing a multiple RR, RI schedule. During the third stage oftraining, all rats were placed into extinction. During extinction, rates of responding were higher in the component previously associated with the high rate of responding in Phase 1, and they were lower in the component previously associated with low rates of responding in Phase 1. These results suggest that resurgence effects, like other history effects, are controlled by previous rates of responding.     

The role of trial tracking in rats’ working memory

The experiments reported in the present study tested whether decreasing intertrial intervals (ITIs) intensifies the disruptive effects of increasing retention intervals (RIs) in a delayed conditional discrimination by decreasing the animal’s trial tracking accuracy (Cohen & Armstrong, 1996; Cohen & Roberts, 1996). Rats responded on a fixed ratio (FR) 1 or fixed interval (FI) 10-sec reinforcement schedule at a second light or tone stimulus, S2, when the first light or tone stimulus, S1, had signaled an FI 10-sec or FR 1 schedule, respectively. RIs between S1 and S2 were increased from 3 to 24 sec and never exceeded ITIs that were reduced from 24 to 6 sec. For some rats, the trials were separated from each other by extending the lever at S1 and retracting it at the end of S2 (ITI lever-retracted group). For other, control rats, the lever remained extended throughout the session (lever-extended group, Experiment 1) or was extended and retracted with the onset and offset of each stimulus (RI/ITI lever-retracted group, Experiment 2). The rats under all trial conditions learned to delay leverpressing on the FI 10-sec schedule. Latency to begin leverpressing on the FI 10-sec schedule declined as RIs were increased, but this effect was attenuated in the ITI lever-retracted groups in both experiments, as would be predicted by thetrial tracking hypothesis. Decreasing ITIs from 24 to 6 sec intensified the disruptive effects of increasing RIs from 3 to 6 sec in the RI/ITI lever-retracted group (Experiment 2), as would be predicted by the trial tracking hypothesis.     

Generalization of fixed-interval response patterning in pigeons

Three pigeons were trained on a multiple schedule in which keypecking was reinforced on a fixed-interval (FI) schedule in the presence of a vertical line and on a variable-interval (VI) schedule in the presence of a horizontal line. Response rates in the former component were positively accelerated (the “FI scallop”), whereas response rates in the second component were relatively constant. In a subsequent generalization test on the line-tilt continuum, the index of curvature decreased systematically as the line was rotated from vertical to horizontal in 30-deg steps. Frequency distributions of indices of curvature suggested that intermediate average index values for intermediate generalization test stimuli were mainly the result of intermediate response patterns in the individual test stimulus presentation, rather than the averaging of VI-like patterns during some stimulus presentations with FI-like patterns during other stimulus presentations.     

Effects of changeover contingencies on auditory stimulus control of two responses

Rats were exposed to a procedure in which auditory stimuli signaled which of two levers was associated with a variable-interval 60-sec schedule of food presentation. Presses on the lever that was not associated with the variable-interval schedule (“errors”) postponed availability of reinforcement on the other lever by either a fixed number of responses or a fixed amount of time. Increasing the number of responses by which “errors” postponed food availability enhanced the level of stimulus control, and. alter a relatively high degree of control had been achieved, reduction of the requirement had no effect. Control experiments ruled out extended exposure to the discrimination procedure as a factor in the increase in stimulus control and suggested that the time of introduction of a changeover contingent is an important determinant of its effect.     

Latent habituation of the orienting response in the preweanling rat

The effect of inhibiting the orienting response on information processing was examined in four experiments. A nonsignal auditory stimulus was presented four times to preweanhng rats either 30 sec or 15 min after they had been placed in an unfamiliar environment (Experiments 1A and 2), shocked (Experiment 1B)5 or experienced a shift in environmental context (Experiment 1C). Both an autonomic (heart rate) and a behavioral componentx of the orienting response to the novel stimulus were recorded. In the 15-min condition, the auditory stimulus elicited a consistent orienting response on the first trial that habituated rapidly with successive trials. In contrast, the auditory stimulus did not elicit a detectable orienting response in the 30-sec condition on any of the four trials. However, when the auditory stimulus was re-presented after a brief retention interval, a comparable level of habituation was seen in both groups. These results demonstrate that animals in the 30-sec condition detected, attended to, and encoded the auditory stimulus even though they did not orient, either autonomically or behaviorally, to that stimulus when it was first presented. This process of response-independent habituation is best described as latent habituation. Like latent learning, latent habituation took place in the absence of any observable change in behavior. The implications of this effect for current theories of habituation and of the orienting response are discussed.     

Choice responding following multiple schedule training

Pigeons’ choice responding on 10-sec interpolated probes was studied after baseline training on multiple variable-interval variable-interval schedules of food reinforcement. Unreinforced choice following training with three different relative reinforcement rates (Experiment 1), with a 3-ply multiple schedule (Experiment 2), and with three different relative reinforcement durations (Experiment 3) was examined. Least squares lines were fit to choice relative response rate and schedule relative response rate as functions of training relative reinforcement rate; choice slope was significantly greater than schedule slope in all three experiments. This result is counter to the prediction of Herrnstein’s (1970) theory that these slopes should not differ. Luce’s (1959) theory also failed to account for the data. It was concluded that choice responding was controlled by both approach to the stimulus associated with the smaller mean interreinforcer interval or the longer duration, and avoidance of the other stimulus.     

The effect of redundant contextual stimuli on autoshaping the pigeon’s keypeck

Three experiments investigated the effect of contextual and trial stimulus lighting conditions on keypeck autoshaping in pigeons. White illumination of a response key before food presentation readily produced keypecking in a brightly lit chamber but failed to do so in a chamber without house illumination (Experiments I and III). Keypecking in a darkened cubicle progressively increased and the facilitatory effect of a houselight decreased as the keylight stimulus was varied from a color change (Experiment II) to a feature change (Experiment III). These findings support a “cue localization” hypothesis of autoshaping. according to which reinforcement signals select specific behaviors for expression and direct these behaviors toward the source of stimulation. This account was extended to superstitious and operant conditioning situations.     

Effects of stimulus spacing on steady state gradients of inhibitory stimulus control

When the response of pigeons is maintained to a number of stimulus wavelengths, but extinguished to one (S?), the birds peck more rapidly at stimuli near the S? than at more distant stimuli. The present study explores this “dimensional contrast” effect as a function of the number and spacing of test wavelengths. A fixed portion of the wavelength continuum was spanned by 5, 9, 13, or 49 stimuli, which appeared in random sequence behind a standard pecking key. At the end of each 20-sec trial, pecks to test stimuli produced a conditioned reinforcer (sometimes followed by food), while pecks to the S? stimulus produced only darkness. Dimensional contrast “shoulders” developed to test stimuli on either side of the S?; these shoulders were of approximately the same height and wavelength position for all but the 5-stimulus (widely spaced) condition, and were comparable to the original contrast results with 25 stimuli. The results strongly suggest that the extent and locus of contrast shoulders are largely independent of the number and spacing of test stimuli.