A comparison of resistance to satiation and resistance to extinction

In two experiments, resistance to satiation was compared with resistance to extinction. In Experiment 1, rats given initial trials in a straight-alley runway while satiated failed to show increased resistance to satiation in a later test phase. This negative finding contrasts with the increased resistance to extinction usually found following initial nonrewarded trials in a straight alley. In Experiment 2, rats were extinguished or were run while satiated following deprived acquisition, and then were either shifted to the other condition or maintained under the same condition. A greater response decrement was produced by extinction than by satiation, both when current performance was examined and when the persistent effect of satiation or extinction on later performance was examined. These results show that there are important dissimilarities in the effects of satiation and extinction, dissimilarities that suggest that extinction is more nonrewarding or aversive than satiation. It seems likely that extinction involves processes (such as frustration, arousal of aversive motivation, and conditioned inhibition) not involved in satiation, which account for the greater response decrement in extinction as compared with satiation.     

Observational extinction: Observation of nonreinforced responding reduces resistance to extinction in rats

Rats trained to push a joystick to the left or right for food reward were given two successive tests in which neither response was reinforced. Prior to Test 1, subjects were either confined in the apparatus with a passive conspecific (Group None), or allowed to observe a conspecific demonstrator making 50 nonreinforced responses in the direction that had beeirrewarded during observer training (Group Same) or in the opposite direction (Group Different). In Test 1, Group Same made fewer previously reinforced responses than did Group Different, which made fewer than Group None, and Groups Same and Different each made fewer previously nonreinforced responses than did Group None. In Test 2, Group Same made fewer previously reinforced responses than did Group None. These results indicate that observation of nonreinforced responding can reduce resistance to extinction (Test 1) and spontaneous recovery (Test 2) in rats.     

Classical conditioning of the rabbit nictitating membrane response: Effects of reinforcement schedule on response maintenance and resistance to extinction

Two experiments were conducted to assess the effects of the introduction of schedules of partial reinforcement (PRF), subsequent to continuous reinforcement training, on the maintenance and resistance to extinction of the rabbit’s nictitating membrane CR. Substantial response levels were maintained by schedules of reinforcement as lean as 15%, and the performance decrements, when observed to be reliable, could not be localized to the immediate effects of one, two, or three consecutive nonreinforced trials by the examination of conditional response probabilities. Moreover, reliable PRF extinction effects were obtained. The relevance of these findings to the purported empirical divergence of PRF effects on classical and instrumental conditioning were discussed.     

Differential reinforcement and resistance to change of divided-attention performance

Behavioral momentum theory provides a framework for understanding how conditions of reinforcement influence instrumental response strength under conditions of disruption (i.e., resistance to change). The present experiment examined resistance to change of divided-attention performance when different overall probabilities of reinforcement were arranged across two components of a multiple schedule. Pigeons responded in a delayed-matching-to-sample procedure with compound samples (color + line orientation) and element comparisons (two colors or two line orientations). Reinforcement ratios of 1:9, 1:1, and 9:1 for accurate matches on the two types of comparison trials were examined across conditions using reinforcement probabilities (color/lines) of .9/.1, .5/.5, and .1/.9 in the rich component and .18/.02, .1/.1, and .02/.18 in the lean component. Relative accuracy with color and line comparisons was an orderly function of relative reinforcement, but this relation did not depend on the overall rate of reinforcement between components. The resistance to change of divided-attention performance was greater for both trial types in the rich component with presession feeding and extinction, but not with decreases in sample duration. These findings suggest promise for the applicability of quantitative models of operant behavior to divided-attention performance, but they highlight the need to further explore conditions impacting the resistance to change of attending.     

Partial reinforcement effects on discrimination learning

Rats were trained on a series of reversals of a successive discrimination in which the percentage of S+ trials ending in food was varied. Changes in the discrimination index occurred more slowly with 50% reinforcement than with 100% reinforcement when the number of training trials was equated across conditions, but were approximately invariant when the conditions were equated with respect to the number of obtained reinforcements. Presentation of free reinforcement during the intertrial intervals reduced the overall rate of discrimination acquisition, but left this invariance unaffected. Invariance in reinforcements necessary to attain acquisition also occurred when different discriminations correlated with different percentages of reinforcement were intermixed within experimental sessions. The failure of the invariance effect to be disrupted by either manipulation suggests that previous accounts of the invariance effect in terms of “comparator” models of conditioning (e.g., Gibbon & Balsam, 1981) are inadequate.     

Effects of discrimination training on stimulus generalization in pigeons: Role of resistance to extinction and response-produced stimuli

In Experiment 1, two groups (n = 10) of pigeons received 17 sessions of TD (true discrimination) or ND (nondifferential) training with line angles. Seventeen sessions of SS (single stimulus) training with a wavelength preceded this training and two followed it. Subsequent wavelength generalization testing in extinction revealed a sharper TD than ND gradient. This slope difference was evident from the very first test stimulus presentation and remained stable throughout testing. As a consequence of substantial overtraining, there was no reduction of response strength and no sharpening of generalization during testing for either group. In Experiment 2, two groups (n = 16) of pigeons received 10 sessions of TD or PD (pseudodiscrimination) training with line angles, followed by four sessions of SS training with a single wavelength. During this training and in subsequent wavelength generalization testing in extinction, brief blackouts separated stimulus presentations. Again, the TD group yielded the sharper gradient. Although responding weakened and the gradients sharpened during the test, these effects were comparable in the two groups. Furthermore, gradients based on the percentage of trials with at least one response showed the same TD-PD slope difference. This finding indicates that differential control over responding by response-produced feedback is inadequate to account for the TD-PD difference in generalization slope. Both experiments indicate that a purported difference in resistance to extinction is also an inadequate explanation.     

Persistence early and late in extinction as a function of number of continuous reinforcements preceding partial reinforcement training

In two experiments rats were given straight-alley training in the following sequence: continuous reward (CR), partial reward (PR), extinction (EXT). Independent groups differed only in the amount of CR training. In both experiments,. early-EXT performance was directly related to amount of CR training and late-EXT performance was inversely related to amount of CR training. These data were related to a possible specific s_F intensity hypothesis, an extension of frustration theory.     

Effect of N-R transitions during partial reinforcement pretraining on subsequent resistance to discrimination

Three groups of 12 rats received 25 pretraining trials to each future discriminandum employed in a subsequent differential brightness conditioning problem. Groups NR and RN received partial reinforcement (PRF) pretraining either with or without, respectively, transitions from nonrewarded to rewarded trials (N-R transitions). Group CRF received consistent reinforcement during pretraining. A fourth group (n=12), Group NP, received no pretraining. During discrimination learning, one-half of the rats in each group received all their daily S+ trials preceding their daily S? trials (+? sequence); the remainder of the rats received an intermixed sequence of trials to S+ and S? (+?+ sequence). Discrimination learning was faster under the +? sequence than under the +?+ condition, and discrimination learning was retarded in Group NR relative to the other three groups, which did not differ from one another, under both the +? and +?+ discrimination sequence conditions. The results are discussed with Reference to previous experiments demonstrating N-R transition effects on discrimination learning, a theoretical extension of sequential theory to discrimination learning, and the effects of nondifferential reinforcement prior to discrimination learning on learned irrelevance.     

The influence of shock during response prevention upon resistance to extinction of an avoidance response

A recent study found that avoidance extinction is equally facilitated by response prevention (blocking) whether the latter involves CS-alone or CS-shock presentations. An experiment was performed to determine whether this result was due to the use of a lengthy shock (5 sec) during response prevention. Five groups of rats were extinguished: (1) without prior blocking, (2) after blocking with CS only, (3) after blocking with a lengthy (5 sec) CS-contingent shock, (4) after blocking with a brief (.5 sec) CS-contingent shock, or (5) after blocking with a brief (.5 sec) shock only. The group blocked with the brief CS-contingent shock was substantially more resistant to extinction than the other four groups. The unblocked group and the group blocked with brief shock only required more trials to extinguish than the groups blocked with CS only or with lengthy CS-contingent shock, but did not differ from each other. The groups blocked with CS only or with lengthy CS-contingent shock also failed to differ from one another. The data support a significant role for Pavlovian conditioning processes in the effect of response prevention upon avoidance extinction.     

The US-preexposure effect in honeybees

Signaled avoidance was studied in individual honeybees that visited the laboratory regularly to take sucrose solution from a target set on the sill of an open window. During feeding, substrate vibration or airstream was used to signal a brief shock that could be avoided by breaking off contact with the food for a few seconds. Aversive conditioning of the context was measured in terms of return time (the time between successive visits). In Experiment 1, experience with unsignaled shock was found to lengthen return time—which experience with signaled shock did not—and to impair performance in subsequent avoidance training with signaled shock (the US-preexposure effect). In Experiment 2, experience with unsignaled shock given after signaled avoidance training lengthened return time but had no effect on response to the signal in a subsequent extinction test. These results closely resemble the results obtained in analogous experiments with vertebrates.     

Resistance to discrimination and subsequent resistance to extinction as a function of the sequence of partial S+ reward in differential conditioning

The effects of transitions from nonrewarded (N) to rewarded (R) trials (N-R transitions) on discriminative behavior in differential conditioning and subsequent resistance to extinction were investigated in two experiments. In Experiment 1, groups given N-R transitions within S+ were more resistant to discrimination (ran fast in S?) and extinction than were groups given a partial reinforcement (PRF) schedule in S+ devoid of N-R transitions. Experiment 2 indicated that N-R transitions that occur when an N trial in S? is followed by an R trial in S+ are as effective in increasing resistance to discrimination, but not resistance to extinction, as are N-R transitions that occur within S+. The sequential effects obtained here were highly similar to those in conventional PRF and support the view that differential conditioning and PRF are highly interrelated phenomena. The results are discussed in terms of the extension of sequential theory to differential conditioning and the importance of internal reward-produced cues in discrimination learning.     

Blocking in landmark-based search in honeybees

Two experiments tested blocking in landmark-based search in honeybees. Honeybees in the experimental group were trained in Phase 1 with a single landmark in a constant spatial relation to the target (sugar water). In the compound training second phase, the landmark used in Phase 1 (blocking landmark) and a new landmark (blocked landmark) were presented at constant spatial relations to the target. The blocking and blocked landmarks differed from each other in color and position, and the blocking landmark retained the same spatial relationship to the target as in Phase 1. In Experiment 1, the control group experienced only Phase 2 training with two landmarks. In Experiment 2, the control group was trained with a different landmark in a different position in Phase 1. Blocking was found in both cases.     

Comparative analysis of learning in honeybees

The performance of free-flying and harnessed honeybees has been studied in a variety of experiments patterned after those in which learning in vertebrates has been studied—among them experiments on amount, quality, and probability of reward; on compound conditioning and discrimination; and on spatial learning and memory. Despite the remoteness of the evolutionary relationship and the vast differences in brain size and structure, the results for honeybees are strikingly similar to those for vertebrates in many respects and different in only a few. The extent to which phenomena of learning common to honeybees and vertebrates can be understood in terms of common functional principles and mechanisms remains to be determined. None of the differences in the results for honeybees and vertebrates points unmistakably to a difference in their learning.     

The effect of intensity and frequency of intermittent punishment in acquisition on resistance to extinction of an approach response in the rat

Rats trained to make an approach response with either nonreward or 150- or 175-V shock occurring on 2, 3, or 4 of 6 daily trials showed greater resistance to extinction than continuously reinforced-unpunished controls. Persistence during extinction was a function of both the type and the frequency of response-contingent event in acquisition. The significant interaction of these two factors was best interpreted as a result of the counterconditioning of the approach response (Amsel, 1972) to anticipatory conditions which varied in similarity to the frustrative nonreward of extinction.     

Recovery of extinction responding in rats following discontinuation of reinforcement of alternative behavior: A test of two explanations

One procedure which has been used to supplement extinction in order to produce faster and more complete response suppression is to provide reinforcement for some alternative response which is incompatible with the response undergoing extinction. When reinforcement for the alternative behavior is discontinued, however, substantial recovery of the original response has often been observed. The present set of experiments demonstrated that such recovery is best accounted for by a “response prevention” hypothesis rather than by a “discriminative cue” hypothesis. High-frequency reinforcement of alternative behavior during the first half of an extinction phase seems similar in effect to procedures which physically prevent rats from emitting the response programmed for extinction.     

Working memory for color in honeybees

Honeybees were tested in delayed conditional discrimination procedures (matching-to-sample and nonmatching-to-sample), using color stimuli presented on a video monitor. A small but reliable tendency to choose the color presented as the conditional cue was found, regardless of whether the contingencies reinforced or discouraged this tendency. The perseverative tendency occurred even with a delay of up to 1–2 min between the conditional cue and the choice. The tendency cannot be explained by changes in the associative value of the colors. Explanation of the results requires some form of working memory for color.     

Resistance to extinction: contingency termination and generalization decrement

We present an algebraic model of resistance to extinction that is consistent with research on resistance to change. The model assumes that response strength is a power function of reinforcer rate and that extinction involves two additive, decremental processes: (1) the termination of the reinforcement contingency and (2) generalization decrement resulting from reinforcer omission. The model was supported by three experiments. In Experiment 1, 4 pigeons were trained on two-component multiple variable-interval (VI) 60-sec, VI 240-sec schedules. In two conditions, resistance to change was tested by terminating the response-reinforcer contingency and presenting response-independent reinforcers at the same rate as in training. In two further conditions, resistance to change was tested by prefeeding and by extinction. In Experiment 2, 6 pigeons were trained on two-component multiple VI 150-sec schedules with 8-sec or 2-sec reinforcers, and resistance to change was tested by terminating the response-reinforcer contingency in three conditions. In two of those conditions, brief delays were interposed between responses and response-independent reinforcers. In both Experiments 1 and 2, response rate was more resistant to change in the richer component, except for extinction in Experiment 1. In Experiment 3, 8 pigeons were trained on multiple VI 30-sec, VI 120-sec schedules. During extinction, half of the presentations of each component were accompanied by a novel stimulus to produce generalization decrement. The extinction data of Experiments 1 and 3 were well described by our model. The value of the exponent relating response strength and reinforcement was similar in all three experiments.     

Control of performance by short-term memory in honeybees

Individual honeybees foraging at a laboratory window were trained with a correction method to choose between blue and yellow targets, one of which contained sucrose solution. There were two trials on each visit, with the locus of the sucrose predictable only on the second. Animals differentially rewarded on Trial 2 for choosing the rewarded color of Trial 1, for choosing the alternative color, or for choosing the target in the rewarded position of Trial 1 independently of its color, all showed a small but persistent preference for the rewarded color, with no significant preference for the rewarded position. When the positions of the colored targets were the same on Trial 2 as on Trial 1 (color and position confounded), there was a more substantial but equally persistent preference on Trial 2 for the rewarded color-position of Trial 1, whether the animals were differentially rewarded for perseveration or for alternation. The results provide further evidence of unlearned control of performance by short-term memory in honeybees but no indication of learned control.     

Distances and directions are computed separately by honeybees in landmark-based search

Honeybees were trained with two landmarks at some angle (e.g., 120°) apart from the target. On crucial unrewarded tests, only a single landmark was present. If distances and directions to landmarks are computed separately (independent averaging), the search distance to the landmark should equal the landmark-target distance found in training. If entire vectors are averaged, the search distance should be much shorter. Three experiments with short target-landmark distances showed results in between the predictions of the two hypotheses. A fourth experiment used longer target-landmark distances and isolated double peaks on single-landmark tests: one predicted by the independent averaging hypothesis, and one very close to the landmark. The near peak is interpreted as arising from approach and exploration of a landmark in a new location, and not from searching.