Transfer between nonreward and delay of reward following minimal acquisition training

Four groups of rats were given six acquisition trials under continuous reward, continuous delay of reward, partial reward, or partial delay of reward, following which all Ss received continuous delay. It was found that the partial reward and the partial delay of reward groups were significantly more persistent during the shift phase than the continuous reward group. No differences were found over trials between either the two partial groups or between each one and the continuous delay group.     

Partial reinforcement delayed extinction effect after regularly alternating reward training

When extinction is delayed very long, the superior resistance to extinction of the random schedule group relative to the alternating schedule group disappears (partial reinforcement delayed extinction effect, PRDE). Two experiments assessed the effects of reinforcement/nonreinforcement on Trial 1 on the PRDE. Following extended partial reinforcement acquisition training in a runway, rats received extinction training after a short (1-day) or long (23-day) retention interval. The schedules used in Experiment 1 were: a single-alternation (SA) schedule beginning each day with a rewarded (r) trial, for Group r-SA; an SA schedule beginning with a nonrewarded (n) trial, for Group n-SA; and a random (Rd) schedule, for Group Rd. The schedules and group names used in Experiment 2 were r-SA, Rd, and r-Rd. The results were that (1) rats given r-SA schedules yielded considerable resistance under delayed extinction, (2) those given Rd and r-Rd schedules showed a decline in resistance to extinction over a long retention interval, (3) those given the n-SA schedule showed relatively low resistance at both retention intervals, although retention deficit was not greater than in the case of the Rd schedule, and thus, (4) the PRDE was found in both experiments, although only weakly in Experiment 1. The results indicated that a regularly alternating reward pattern was a more important determinant than was type of reward on Trial 1 for the PRDE. The PRDE due to differential retention deficits among schedules is discussed on the basis of dual-process associative sequential mechanisms and cognitive rule-encoding mechanisms.     

Differential effects of delay of reinforcement on acquisition of affective and instrumental responses

Thirsty rats were allowed to choose between a striped arm of a T-maze that contained water and a black arm that contained a saccharin solution. After each exposure to the saccharin solution, half of the animals received a lithium chloride injection 1 min later and half received the same injection 30 min later. Comparable saline injections at these same delays were given after each exposure to the striped, water, arm. The results from saccharin aversion, spatial aversion, and instrumental learning tests generally supported the hypothesis that delayed reinforcement has a substantially greater negative impact on the acquisition of an instrumental response than it has on affective conditioning.     

Effects of schedule and delay of reinforcement on acquisition speed

The effects of schedule of reinforcement (partial vs. consistent) and delay of reward (0 to 20 sec) on running in rats were examined in two investigations. The effects of delay depended upon schedule of reinforcement; acquisition speed decreased as delay increased under consistent reinforcement, a common finding, while acquisition speed was independent of delay under partial reinforcement, a new finding. The partial-reinforcement acquisition effect or PRAE is defined as faster acquisition speed under partial than under consistent reinforcement. Because running speed was independent of delay under partial reinforcement, but decreased as delay increased under consistent reinforcement, the PRAE increased as delay of reinforcement increased.     

Replacement of event-generated memories of nonreinforcement with signal-generated memories of reinforcement during partial reinforcement training: Effect on resistance to extinction

Event-generated memory refers to the memory of a reinforcement (R) or nonreinforcement (N) event from an immediately preceding trial;signal-generated memory refers to the memory of a temporally remote R or N, retrieval of which is generated by presentation of a signal with which the memory is associated (Haggbloom, 1988). In each of three experiments, Group Signal-R received runway discrimination training in Phase 1 to establish a stimulus as a signal for R, and partial reinforcement training in Phase 2. An extinction test measured learning about the memory of nonreward (S^N)—learning that occurs when S^N is retrieved on R trials that follow N trials. In Group Signal-H, those R trials were accompanied by the signal for R, a treatment we hypothesized would generate retrieval of the memory of reinforcement (S^R) so that signal-generated S^R would replace event-generated S^N as the operative memory, thereby eliminating the increased resistance to extinction normally produced by PRF training. In each experiment, Group Signal-R was less resistant to extinction than was a control group conditioned to respond to-event-generated S^N. Extinction was as rapid in Group Signal-R as it was in a consistent reinforcement control group (Experiment 1) and in a group given intertrial reinforcements to interfere with learning about S^N (Experiment 3). Experiment 2 tested two alternative interpretations of the failure to learn about S^N in Group Signal-R. Those alternatives were found to be less viable than the hypothesis that the signal for R actively recruited retrieval of a competing memory.     

Persistence early and late in extinction as a function of number of continuous reinforcements preceding partial reinforcement training

In two experiments rats were given straight-alley training in the following sequence: continuous reward (CR), partial reward (PR), extinction (EXT). Independent groups differed only in the amount of CR training. In both experiments,. early-EXT performance was directly related to amount of CR training and late-EXT performance was inversely related to amount of CR training. These data were related to a possible specific s_F intensity hypothesis, an extension of frustration theory.     

Dissociation of patterned alternation learning and the partial reinforcement extinction effect in preweanling rats

Sprague-Dawley rat pups aged 14 or 18 days were trained on a patterned (single) alternation schedule with either an 8- or a 105-sec intertriai interval (ITI). At the 8-sec ITI, alternation learning was obtained at both ages, but the older age group learned more rapidly. There was no evidence of response alternation at the 105-sec ITI at either age. Continuously reinforced (CRF) and partially reinforced (PRF) groups trained and extinguished along with the patterned alternation (PA) group at the 105-sec ITI showed a robust partial reinforcement extinction effect (PREE) at both ages. Moreover, there was no difference in the rate of extinction of the PRF and PA groups at either age (i.e., no effect of N-length). A PREE can therefore be obtained in infant rats under conditions that apparently preclude the formation of sequential associations. The implications of this finding for the ontogeny of instrumental learning and extinction are discussed.     

Spontaneous recovery after reversal and partial reinforcement

Six experiments used magazine approach in rat subjects to explore changes with time in responding for stimuli brought to a common moderate level of performance through acquisition or extinction. They found no evidence for increases with time in behavior during stimuli given simple acquisition. However, stimuli brought to that same level by reversal learning, repeated reversal, or partial reinforcement all showed increases in responding with time. These results suggest that the decremental process established by nonreinforcement endures through subsequent reinforcement and is especially sensitive to the passage of time.     

Partial reinforcement effects on learning and extinction of place preferences in the water maze

In two experiments, two groups of rats were trained in a navigation task according to either a continuous or a partial schedule of reinforcement. In Experiment 1, animals that were given continuous reinforcement extinguished the spatial response of approaching the goal location more readily than animals given partial reinforcement—a partial reinforcement extinction effect. In Experiment 2, after partially or continuously reinforced training, animals were trained in a new task that made use of the same reinforcer according to a continuous reinforcement schedule. Animals initially given partial reinforcement performed better in the novel task than did rats initially given continuous reinforcement. These results replicate, in the spatial domain, well-known partial reinforcement phenomena typically observed in the context of Pavlovian and instrumental conditioning, suggesting that similar principles govern spatial and associative learning. The results reported support the notion that salience modulation processes play a key role in determining partial reinforcement effects.     

Eliminating behavior with omission and extinction after varying amounts of training

After rats were trained to leverpress for 1, 3, 9, or 27 days on a variable interval reinforcement schedule, omission training was compared with extinction in effectiveness of response elimination. Extinction produced faster response elimination than omission, although both procedures eventually led to equal response elimination. Resistance to response elimination increased with length of baseline training, although this effect did not interact with omission vs extinction. A test of the durability of elimination effects followed, using a response-independent variable time reinforcement schedule. After extinction, resumption of responding in the durability test increased with length of baseline training, but there was little response resumption following omission regardless of the length of the baseline training. These results amplify and extend previous findings which show omission to be an effective and durable response elimination method.     

The effects of reinforcement and strategy training on impulsive responding.

Impulsive second-grade children received strategy training or strategy training combined with primary or secondary reinforcement. All training groups showed significantly longer response latencies and significantly fewer errors on the Matching Familiar Figures test immediately following training. The reinforcement groups had significantly longer latencies than the training-only group, and there was some indication that primary reinforcement was more effective than secondary reinforcement in producing response delay. The 3 training groups did not differ significantly on error scores.     

Effects of the amount of acquisition and contextual generalization on the renewal of instrumental behavior after extinction

Four experiments with rat subjects examined the role of context during the extinction of instrumental (free-operant) behavior. In all experiments, leverpressing was first reinforced on a variable-interval 30-s schedule and then extinguished before being tested in the extinction and renewal contexts. The results identified three important variables affecting the renewal effect after instrumental extinction. First, ABA and ABC forms of renewal were strengthened by increasing the amount of acquisition training. This suggests that the strength of the association learned during acquisition, or the final level of performance, influences the degree of renewal after extinction. The effect of the amount of training was modulated by the second factor, the degrees of generalization from the acquisition and extinction contexts to the test context. The third variable was acquisition training in multiple contexts, which was shown to strengthen ABC renewal. Methodological, theoretical, and practical implications are discussed.     

Partial reinforcement and resistance to extinction in honeybees

Previous experiments have shown the partial reinforcement effect in honeybees under conditions which permit an interpretation in terms of sensory carryover. In the five experiments reported here, the effect was sought under conditions which would require an interpretation in terms of associative reinstatement. Since it is not feasible to train honeybees in widely spaced trials, several different interpolated-trials procedures were employed which had in common the feature that nonrewarded response to a stimulus never was followed by rewarded response to the same stimulus. Implications of the negative results for the interpretation of the overlearning-extinction effect and successive negative contrast in honeybees are considered.     

Acquisition and extinction in the presence of the same intertrial stimuli: The effects of partial reward

Two groups of rats (N = 11) were trained at two trials a day for 16 days in Phase I and 13 days in Phase II. Responses on Trial 1 were always rewarded in both phases. Percentage of reward (50% vs 100%) was varied on Trial 2 of each day of Phase I. Trial 2 on each day of Phase II was never rewarded. A partial reward effect (PRE) was observed on Trial 2 of Phase II. The implications of the results for intertrial explanations of the PRE were discussed.     

Recovery effects after extinction in the Morris swimming pool navigation task

In three experiments in which rats were used as subjects, we developed an extinction procedure using a Morris pool. The animals were trained to find a hidden platform located at a fixed position and were then given extinction trials in which the platform was removed from the pool. When training and extinction were carried out in the same context and time was allowed to elapse between extinction and test, spontaneous recovery of learning was observed. On the other hand, those rats that received extinction in a context different from the one used for training failed to show spontaneous recovery of learning when tested in the extinction context after an interval of 96 h. However, they did show renewal of spatial learning when tested in the training context. These results show that extinction in the spatial domain behaves like extinction in standard conditioning preparations.     

Extensive training,partial reinforcement,and temporal gaps do not affect S-S learning in second-order autoshaping

A second-order autoshaping procedure was used to examine the effects of three variables on the amount of information that could be learned about the stimulus properties of a reinforcer. All three experiments paired several keylight S2s with different keylight S1s and then carried out discriminative autoshaping with those S1s. Learning about the stimulus properties of S1 was inferred from changes in the response to its paired S2 when that S1 was changed in value. The sensitivity of S2 to changes in S1 was investigated as a function of number of S2-S1 pairings (Experiment 1), partial reinforcement (Experiment 2), and temporal distance between S2 and S1 (Experiment 3). Each experiment found evidence of a selective change in responding to an S2 as a function of the treatment of its S1. However, the amount of that change was not affected by any of the three variables studied. Those results imply that, within the ranges used here, none of these variables changes the degree of learning about the stimulus properties of a reinforcer.     

Choice, delay, probability, and conditioned reinforcement

The hyperbolic-decay model is a mathematical expression of the relation between delay and reinforcer value. The model has been used to predict choices in discrete-trial experiments on delay-amount tradeoffs, on preference for variable over fixed delays, and on probabilistic reinforcement. Experiments manipulating the presence or absence of conditioned reinforcers on trials that end without primary reinforcement have provided evidence that the hyperbolic-decay model actually predicts the strength of conditioned reinforcers rather than the strength of delayed primary reinforcers. The model states that the strength of a conditioned reinforcer is inversely related to the time spent in its presence before a primary reinforcer is delivered. A possible way to integrate the model with Grace’s (1994) contextual-choice model for concurrent-chain schedules is presented. Also discussed are unresolved difficulties in determining exactly when a stimulus will or will not serve as a conditioned reinforcer.