Reversibility of reinforcement between eating and running by schedule changes: A comparison of hypotheses and models

Rats increased eating that produced access to a running-wheel or increased running that produced access to food, depending on which response was potentially deprived, relative to baseline, by the scheduled ratio of responding. Under both schedules, instrumental responding significantly exceeded appropriate baselines of the noncontingent effects of the schedule. The results contradicted the hypothesis that reinforcement is produced by an overall or momentary probability differential between two responses; instead, they supported the condition of response deprivation as a key determinant of reinforcement. Of several recent quantitative models that predict reversibility of reinforcement by schedule changes, only the predictions of the relative response-deprivation model did not differ significantly from the data of either schedule.     

The effects of Pavlovian CSs on two food-reinforced baselines— with and without noncontingent shock

Two experiments explored the effects of Pavlovian (tone-shock) CS+, CS?, and truly random control (TRC) contingencies on two different food-reinforced instrumental baselines. One food-reinforced baseline contained noncontingent shock, while the other did not. In the first experiment, a TRC contingency was shown to produce suppression of food-reinforced responding, while a CS? contingency did not. When noncontingent shock was added to the baseline, however, the TRC stimuli failed to produce suppression, and the CS? contingency increased response rates over baseline level. In a second experiment, the effects of TRC and CS+ contingencies were compared on these same two baselines. While the CS+ produced suppression on both shock and no-shock baselines, the TRC contingency again produced suppression on only the no-shock baseline.     

The effect of noncontingent outcomes on extinction of the response-outcome association

The effect of noncontingent outcomes on an instrumental response-outcome (R-O) association was examined in four experiments using transfer tests. In each experiment, rats were first given instrumental discrimination training designed to establish different stimuli as signals (S+s) for different outcomes. Transfer responses were subjected to different treatments across the experiments and then tested with the S+s. In Experiments 1 and 2, two transfer responses were both initially trained with two contingent outcomes. Then, each transfer response was subjected either to the addition of noncontingent presentations of one of those outcomes (Experiment 1) or to the replacement of one of the contingent outcomes with noncontingent presentations of that outcome (Experiment 2). Transfer tests revealed no significant difference in the ability of an S+ to promote performance of a transfer response based on their shared association with either the contingent or the noncontingent outcome. These results suggest that a response reinforced with two outcomes remains equally well associated with both of those outcomes despite prolonged exposure to noncontingent presentations of one of those outcomes. In Experiments 3 and 4, the possibility that the noncontingent schedules of reinforcement used in Experiments 1 and 2 might be capable of establishing an association between a response and its noncontingent outcome was examined. Transfer responses were trained with one contingent outcome and a different noncontingent outcome. Performance of these transfer responses was augmented more by presentations of an S+ trained with the contingent outcome than with the noncontingent outcome. These results confirm previous reports that instrumental responses are sensitive to outcome contingencies in acquisition and that noncontingent outcome presentations do not weaken previously established R-O associations. Several explanations are considered for the failure of subsequent noncontingent presentations of an outcome to reduce the strength of its association with the instrumental response.     

Specificity of cue to consequence in aversion learning in the rat: Control for US-induced differential orientations

In four experiments, each using a single conditioning trial, rats avoided a light more than a saccharin solution after these stimuli had been paired with footshock, whereas saccharin was avoided more than the light after these stimuli had been paired with lithium injection. The use of a single conditioning trial precludes possible US-induced differential orientations from influencing which stimuli will be associated on the conditioning trial. This cue-consequence specificity effect was obtained even when subjects conditioned with lithium received a non-contingent footshock prior to the test session, and when subjects conditioned with footshock received a noncontingent lithium injection before testing (Experiments 2–4). Weak aversions to the light in rats given a light-lithium pairing and noncontingent footshock and to the saccharin in subjects that received a saccharin-footshock pairing and noncontingent lithium administration were obtained in Experiment 2. However, these weak aversions were not obtained when subjects were given three nonreinforced exposures to the test chamber before the test session (Experiments 3 and 4). These results indicate that US-induced differential orientations do not mediate the cue-consequence effect in aversion learning.     

Taste aversion learning and schedule-induced polydipsia in rats

Experiment 1 sought to determine whether schedule-induced drinking could be abolished by means of a taste aversion. Polydipsic rats were given access to a .4% saccharin solution while they were exposed to an intermittent food schedule. Immediately after the session, they received an intraperitoneal injection of either lithium chloride or sodium chloride. Following a recovery day with water in the experimental chamber, the animals were again exposed to the saccharin solution. The poisoned animals (lithium chloride) drank very little saccharin compared to the control animals (sodium chloride), indicating that they had learned a taste aversion in only one conditioning trial. Experiment 2 established that polydipsic rats can learn a taste aversion despite a long delay between schedule-induced saccharin consumption and poisoning, and that the delay gradient displayed by polydipsic rats is similar to that observed in thirst-motivated rats.     

Differential effects of delay of reinforcement on acquisition of affective and instrumental responses

Thirsty rats were allowed to choose between a striped arm of a T-maze that contained water and a black arm that contained a saccharin solution. After each exposure to the saccharin solution, half of the animals received a lithium chloride injection 1 min later and half received the same injection 30 min later. Comparable saline injections at these same delays were given after each exposure to the striped, water, arm. The results from saccharin aversion, spatial aversion, and instrumental learning tests generally supported the hypothesis that delayed reinforcement has a substantially greater negative impact on the acquisition of an instrumental response than it has on affective conditioning.     

Anticipatory contrast as a measure of time horizons in the rat: Some methodological determinants

In three experiments, the time horizon over which the rat evaluates alternative feeding sources was investigated. The time horizon was measured by the suppression of intake of one incentive (a 0.15% saccharin solution) when a preferred alternative incentive (a 32% sucrose solution) was available but delayed. In Experiment 1, we found a direct function between the amount of saccharin intake and the delay time before access to 32% sucrose. Compared with intake for a saccharin-only control, saccharin intake was suppressed before 4-min and 16-min sucrose delays, but not before a 32-min delay. Because previous work (Flaherty & Checke, 1982) had reported suppression before a delay of nearly 32 min, in the subsequent experiments we examined factors that might account for this difference. In Experiment 2, we found that saccharin intake was suppressed before a 32-min delay interval when saccharin and sucrose solutions were presented in a bright-novel test environment but not when the same solutions were presented in the home cage. In Experiment 3, we found that the time between testing and subsequent postsession feeding could also affect the suppression of saccharin intake. Saccharin intake was suppressed when access to 32% sucrose was delayed by 32 min and the test situation was followed by immediate postsession feeding, but not when postsession feeding was delayed by 90 min. These results thus extend estimates of the rat’s time horizon to at least 32 min, but indicate that the effective time horizon can vary, depending on the test situation.     

The relationship between momentary response probabilities and momentary reinforcement effects

Premack’s probability hypothesis provides a simple empirical rule for predicting reinforcement effects, but has always been applied to response probabilities estimated by averaging over entire sessions. If the rule is robust, it should also predict momentary (e.g., within-sessions) changes in reinforcement from parallel momentary probability changes. It seems to do so. Six rats received noncontingent water (base), then leverpressed for water (contingency), each for 15 sessions. All sessions were divided into six subsessions. Average leverpressing for individual rats was a simple monotonie, usually linear, function of the probability of drinking—estimated from that subsession ’s counterpart during base. Similar results were obtained from a second study even though different instrumental and contingent events were used. With some generality, then, it is possible to apply the probability hypothesis to momentary reinforcement effects.     

Rats’ responses to molar and local schedule constraints

Three experiments explored responses to molar and local schedule constraints. Thirsty rats pressed a lever for access to a water spout. In Experiment 1, response totals were unaffected by two local schedule characteristics—the variability of the instrumental requirement and the variability of the magnitude of contingent reward. Experiment 2 manipulated the correlation between the instrumental requirement and the magnitude of reward. This correlation did not affect the behavioral price ratio (presses per lick) at a molar level. At a local level, the positive correlation created a lower mean lick price than did the negative correlation. The rats licked more, and licked less efficiently, under the positive correlation than under the negative correlation. Experiment 3 compared two ways of manipulating the molar presses/lick ratio: The instrumental (contingent) series varied the instrumental (contingent) requirement, but held the other requirement constant. As the ratio increased, total leverpresses increased, and total licks decreased linearly; the two series did not differ significantly. At higher lick prices, the rats licked more efficiently and made more extra licks at the spout as it closed. The results help delimit the applicability of molar models of the organism’s response to schedule constraints.     

Contextual conditioning during free-operant extinction: Unsignaled,signaled, and backward-signaled noncontingent food

Three experiments were designed to study the effects of contextual conditioning on the extinction of instrumental leverpressing that had been reinforced on a random-interval schedule. In Experiment 1, noncontingent food retarded extinction, but signaling food delivery, a treatment that should reduce contextual conditioning, reduced the interference. Experiment 2 replicated the results of Experiment 1 and demonstrated that if the food preceded rather than followed the signal, the retardation of extinction was not reduced but was enhanced. In Experiment 3, non-contingent leverpressing was used to directly verify that the three treatments—forward signaling, noncontingent food, and backward signaling—differentially influenced contextual conditioning. Forward signaling produced the least, and backward signaling produced the most, contextual conditioning. This monotonic relationship between contextual conditioning and interference with extinction was used as evidence to support the argument that context-food associations are important in controlling instrumental responding.     

Saccharin aversions induced by lithium chloride toxicosis in a backward conditioning paradigm

Seven experimental groups of seven rats each were allowed to consume saccharin solution at different times relative to intubation of lithium chloride solution. Six backward conditioning (BWD) groups were intubed 0.5, 1, 2, 3, 4, and 8 h before saccharin consumption, and a forward conditioning (FWD) group was intubed 0.5 h after saccharin consumption. A no-lithium control group of 14 rats received no intubation. Only the 0.5-h FWD and the 0.5-h BWD groups showed an aversion to saccharin relative to the no-lithium controls. The aversion to saccharin in the 0.5-h FWD group was more pronounced than that in the 0.5-h BWD group. This shows that the aversive effects of lithium toxicosis dissipate far sooner than the aversive effects of X-irradiation.     

Anticipatory contrast as a function of access time and spatial location

Intake of a 0.15% saccharin solution was suppressed when it was followed by a 32% sucrose solution in brief daily pairings. With equal access durations to the two solutions, intervals of intermediate duration (2 or 3 min) produced a larger contrast than more extreme intervals (1 or 10 min). There was no evidence of inhibition of delay with the 10-min interval (Experiments 1A and 1B). When access times were asymmetrical, longer access time to the first solution reduced contrast, whereas longer access time to the second solution enhanced contrast (Experiment 2). Contrast was greater when the two solutions were presented at consistent and separate spatial locations than when location was changed randomly or when both solutions were presented in sequence at the same location. However, a degree of contrast occurred in all conditions (Experiment 3). Experiment 4, conducted with the solutions in opposite arms of a T-maze, showed that anticipatory approach to the location correlated with the 32% sucrose solution developed prior to lick suppression on the saccharin solution. However, within daily sessions, there was a reliable increase in contrast without correlated changes in anticipatory-approach behavior. Access-time effects were attributed to altered reward values, whereas spatial-separation effects suggest that goal-directed responses contribute to, but do not cause, anticipatory contrast.     

Stereotypy of spatial movements during noncontingent and contingent reinforcement

The degree of stereotypy in the movement patterns of 3 pigeons during noncontingent and contingent periodic food reinforcement was quantified by analyzing the distribution of turning angles, and by using information and Fourier analyses. The results indicated that (1) movement patterns were less stereotyped during noncontingent than during contingent reinforcement, (2) a reversal to noncontingent reinforcement resulted in a degree of stereotypy comparable to that during the first phase of noncontingent reinforcement, (3) movement patterns were maximally stereotyped immediately after food withdrawal and generally became less stereotyped as reinforcement approached, regardless of whether reinforcement was noncontingent or contingent, and (4) higher frequency movements generally accounted for more variance in the movements during contingent than during noncontingent reinforcement. Greater stereotypy in the movements during contingent reinforcement was likely due to a greater probability that similar movements were reinforced during contingent reinforcement. Momentary changes in the stereotypy of the movements within the interfood interval might reflect changes in the level of arousal.     

Reinforcement and the organization of behavior in golden hamsters: Sunflower seed and nest paper reinforcers

Undeprived hamsters received nest paper or sunflower seeds in their home cages, free or contingent on one of three activities. The pattern of effects of reinforcement was the same as that reported previously with food for hungry hamsters in an open field: Open rearing and scrabbling quickly increased in rate when reinforced with paper or seeds, but face washing showed little or no increase. Certain interrelations found previously between noncontingent effects of reinforcers, deprivation effects, and contingent effects were absent and therefore are not essential to the differences among these three behaviors during reinforcement. The hamsters could learn to interrupt fact washing to collect either paper or seeds, but even then they did not increase their time face washing to high levels. Raising the operant level of face washing by spraying the animals with water did not improve instrumental performance. Face washing could be classified as an involuntary activity, but this term does not provide a full account of its special properties, such as shortened bouts, during reinforcement and punishment.     

The role of amount consumed in flavor preexposure effects and neophobia

Rats received either a small or a large amount of a novel saccharin solution prior to a conditioning episode in which the saccharin flavor was paired with lithium-induced toxicosis. When the time between the preexposure and the conditioning episode was 3.5 h, both the small and large amounts of saccharin decremented conditioning. However, when the time between the preexposure and the conditioning episode was 23.6 h, only the consumption of a large amount of saccharin decremented conditioning. In a second experiment, rats received either a short or a long exposure to the novel saccharin solution and were then given a choice test between the saccharin and water. Rats given the choice test immediately following their preexposure to saccharin avoided consuming it. If they received the choice test 4 h later, they consumed more of the saccharin irrespective of the length of the preexposure. In contrast, when they were given a choice test 24 h after the preexposure, only rats that had received a long preexposure displayed a significant preference for the saccharin. The present results demonstrate that the flavor preexposure effect and the attenuation of neophobia are both determined by the time between preexposure to the novel flavor and the conditioning episode or choice test and the amount of preexposure to the novel flavor. These findings are discussed in relation to the information-processing model developed by Wagner (1976, 1978) and are used to provide an account, in terms of this model, of the delay-gradient seen in flavor-aversion learning.     

Generality of US preexposure effects: Transfer from food to shock or shock to food with and without the same response requirements

A series of four experiments, employing mice, investigated the generality of the learned helplessness phenomenon. The first two experiments used preexposure to aversive stimuli (shock), while the other two used preexposure to appetitive stimuli (food). In all of the studies, subjects were preexposed to contingent, noncontingent, or no stimuli (except for Experiment 2) in a Skinner box. During the test, animals preexposed to shock were tested with food, and those preexposed to food were tested with shock. The test was conducted in a similar situation, a Skinner box (Experiments 1, 3), or a different situation—a runway (Experiments 2, 4). Performance decrements were evident when subjects that were preexposed to a noncontingent stimulus were compared with subjects preexposed to contingent stimuli. The differences between the contingent and the noncontingent groups were significant, as were the differences between the contingent and the nonpreexposed groups (except for Experiment 1). The effects cut across the different types of stimuli, situations, and response requirements of the preexposure and test phases.     

Immunization to learned helplessness in appetitive noncontingent contexts

We examined immunization against learned helplessness in 36 dogs. The experiment consisted of five phases: (1) appetitive contingent training, (2) immunization training, (3) inescapable noise training, (4) recovery, and (5) an appetitive noncontingent test. There were six groups: (1) a group immunized by controllable and predictable noise, (2) a group immunized by controllable but unpredictable noise, (3) a group immunized by uncontrollable but predictable noise, (4) a group given uncontrollable and unpredictable noise during immunization training, (5) a group not submitted to any treatment during the immunization phase, followed by uncontrollable noise, and (6) a group not submitted to any treatment. The immunization effect was assessed by measuring the acquisition of an appetitive response when food was not contingent upon responding. Our results demonstrate that the immunization effect can be observed in a noncontingent appetitive context. The effects of escapable noises that ensure immunization against the motivational deficit and predictable noises that immunize against the associative deficit seem to be additive.     

The effects of conditioned taste aversions on schedule-induced polydipsia: An analysis of the initiation and postpellet temporal distribution of licking

Animals poisoned following the schedule-induced consumption of saccharin initially continued to drink following spaced pellet deliveries. Neither the initiation of postpellet drinking (i.e., bout, initiation) nor the size and duration of the bouts was effected by the conditioned aversion procedure. With repeated conditioning trials (i.e., repeated pairings of saccharin and LiCl), schedule induced drinking was eventually reduced. The specific components underlying schedule-induced consumption, however, were differentially affected by the aversion training. Specifically, the decrease in schedule-induced drinking was effected primarily by a decrease in licking occurring between 10 and 60 sec after pellet delivery. Bout initiation and licking immediately postpellet (i.e., within the first 10 sec following pellet delivery) were most resistant to suppression and appeared to be responsible for the relative insensitivity of schedule-induced drinking to conditioned taste aversions. The differential effects of taste aversion conditioning on individual components of elicited behavior are discussed.     

The effect of drug habituation before and after taste aversion learning in rats

In Experiment I, rats received eight habituation injections of either lithium chloride (LiCl) or sodium chloride (NaCl), then two aversion training trials in which access to saccharin solution was followed by LiCl injections, and finally eight extinction trials with saccharin but no injections. The rats habituated to LiCl showed less aversion to saccharin during training and extinction. In Experiment II, rats received two aversion training trials, then eight habituation trials to either LiCl or NaCl, then eight extinction trials, four more aversion training trials, and eight more extinction trials. The rats habituated to LiCl did not differ during the first extinction period from those habituated to NaCl, but showed less aversion to saccharin during the second training and extinction periods. Consequently, habituation to LiCl reduces the learning of an aversion to saccharin but does not reduce the performance of a previously learned aversion.