Replacement of event-generated memories of nonreinforcement with signal-generated memories of reinforcement during partial reinforcement training: Effect on resistance to extinction

Event-generated memory refers to the memory of a reinforcement (R) or nonreinforcement (N) event from an immediately preceding trial;signal-generated memory refers to the memory of a temporally remote R or N, retrieval of which is generated by presentation of a signal with which the memory is associated (Haggbloom, 1988). In each of three experiments, Group Signal-R received runway discrimination training in Phase 1 to establish a stimulus as a signal for R, and partial reinforcement training in Phase 2. An extinction test measured learning about the memory of nonreward (S^N)—learning that occurs when S^N is retrieved on R trials that follow N trials. In Group Signal-H, those R trials were accompanied by the signal for R, a treatment we hypothesized would generate retrieval of the memory of reinforcement (S^R) so that signal-generated S^R would replace event-generated S^N as the operative memory, thereby eliminating the increased resistance to extinction normally produced by PRF training. In each experiment, Group Signal-R was less resistant to extinction than was a control group conditioned to respond to-event-generated S^N. Extinction was as rapid in Group Signal-R as it was in a consistent reinforcement control group (Experiment 1) and in a group given intertrial reinforcements to interfere with learning about S^N (Experiment 3). Experiment 2 tested two alternative interpretations of the failure to learn about S^N in Group Signal-R. Those alternatives were found to be less viable than the hypothesis that the signal for R actively recruited retrieval of a competing memory.     

Latent discrimination learning under a regular schedule of partial reinforcement

In this investigation, which employed rats in a runway, discriminative responding consisted of faster running on the reinforced than on the nonreinforced trials of either the 4NR or R4N schedule, both schedules containing fixed, repeated sequences of nonreinforced and reinforced trials. Under the 4NR schedule, four nonreinforced trials preceded a reinforced trial each day, and under the R4N schedule, a reinforced trial was followed by four nonreinforced trials each day. The major finding obtained was that under the 4NR schedule, discriminative responding was improved very substantially by a shift to extinction. Rats maintained on the 4NR schedule did not show improved discriminative responding, nor did discriminative responding improve in extinction following training under either the R4N schedule or a schedule of consistent reinforcement. Latent discrimination learning was defined as discriminative responding which fails to reflect adequately the amount of discrimination learning accomplished. The present findings demonstrate latent discrimination learning for regular schedules of partial reinforcement, something already demonstrated for brightness differential conditioning and possibly DRL schedules, as well.     

Resistance to discrimination and subsequent resistance to extinction as a function of the sequence of partial S+ reward in differential conditioning

The effects of transitions from nonrewarded (N) to rewarded (R) trials (N-R transitions) on discriminative behavior in differential conditioning and subsequent resistance to extinction were investigated in two experiments. In Experiment 1, groups given N-R transitions within S+ were more resistant to discrimination (ran fast in S?) and extinction than were groups given a partial reinforcement (PRF) schedule in S+ devoid of N-R transitions. Experiment 2 indicated that N-R transitions that occur when an N trial in S? is followed by an R trial in S+ are as effective in increasing resistance to discrimination, but not resistance to extinction, as are N-R transitions that occur within S+. The sequential effects obtained here were highly similar to those in conventional PRF and support the view that differential conditioning and PRF are highly interrelated phenomena. The results are discussed in terms of the extension of sequential theory to differential conditioning and the importance of internal reward-produced cues in discrimination learning.     

Partial reinforcement effects on discrimination learning

Rats were trained on a series of reversals of a successive discrimination in which the percentage of S+ trials ending in food was varied. Changes in the discrimination index occurred more slowly with 50% reinforcement than with 100% reinforcement when the number of training trials was equated across conditions, but were approximately invariant when the conditions were equated with respect to the number of obtained reinforcements. Presentation of free reinforcement during the intertrial intervals reduced the overall rate of discrimination acquisition, but left this invariance unaffected. Invariance in reinforcements necessary to attain acquisition also occurred when different discriminations correlated with different percentages of reinforcement were intermixed within experimental sessions. The failure of the invariance effect to be disrupted by either manipulation suggests that previous accounts of the invariance effect in terms of “comparator” models of conditioning (e.g., Gibbon & Balsam, 1981) are inadequate.     

Effect on subsequent fixed-interval schedule performance of prior exposure to ratio and interval schedules of reinforcement

In three experiments, we examined the effect on the patterns of responding noted on fixed interval (FI) schedules of prior exposure to a range of interval and ratio schedules. Rats leverpressed for food reinforcement on random ratio (RR), random interval (RI), or variable interval (VI) schedules prior to transfer to FI schedules. In Experiment 1, prior exposure to an RR schedule retarded the development of typical FI patterns of responding. Exposure to a yoked RI schedule produced even greater retardation of typical FI performance. This effect was replicated in Experiment 2, using a within-subjects design. Rats responded on a multiple RR-RI schedule prior to a multiple FI-FI schedule. Typical FI performance emerged more slowly in the component previously associated with the RI than with that associated with the RR. In Experiment 3, exposure to an RR schedule retarded the development of FI performance to a greater extent than did exposure to a VR schedule. The latter schedule was programmed to allow the possibility that inhibitory control would develop after reinforcement. These results confirm that ratio schedules independently result in the disruption of FI responding. This effect was not long lasting and cannot be used plausibly to explain species differences in responding to FI schedules. However, it does suggest that temporal control—as manifested by the transfer of inhibitory control from one schedule to another—could facilitate movement between interval schedules.     

Durability of the partial reinforcement and partial delay of reinforcement extinction effects after minimal acquisition training

Four groups of 10 rats each were given six acquisition trials (Phase 1) under continuous reinforcement (CR), partial reinforcement (PR), constant delay (CD), or partial delay of reinforcement (PD) conditions. In Phase 2, all Ss were given 18 nonreinforced trials, followed by 12 continuously reinforced trials in Phase 3. In Phase 4, all Ss were given 12 more extinction trials. A constant 24-h ITI was observed throughout the experiment. A strong partial reinforcement extinction effect (PREE) was obtained in both Phases 2 and 4. Only a temporary partial delay of reinforcement effect (PDRE) was observed, which was restricted to the first nine trials of the first extinction phase. No constant delay of reinforcement effect (CDRE) was observed in either extinction phase. The results were discussed in terms of both frustration and sequential theories.     

Effect of prior nonreward on subsequent incentive growth during brief acquisition

Four experiments compared runway extinction or hurdle-jumping from nonreward performance following brief (10 trials) continuous or partial reinforcement acquisition. Some of the partial groups received all nonrewarded trials prior to any rewards. The major findings were that (l) rats receiving all nonrewarded experiences prior to rewarded ones were more persistent during extinction than continuously rewarded subjects; (2) rats receiving nonrewarded placements prior to rewarded ones in one compartment of a two-compartment box, failed to learn a hurdle-jumping response to escape nonreward, whereas rats not receiving the initial nonrewards did learn the escape response; (3) increasing the number of rewarded placements following initial nonrewarded ones offset the effect noted in (2). The results, which are discussed in the context of a frustration analysis of the small-trials partial reinforcement effect, suggest that incentive growth over rewarded trials is retarded when the rewards have been preceded by nonrewards. The similarity of these results to those investigating the phenomenon of latent inhibition is apparent, and possible mechanisms responsible for the present results are suggested in current theoretical accounts of latent inhibition.     

Partial reinforcement and resistance to extinction in honeybees

Previous experiments have shown the partial reinforcement effect in honeybees under conditions which permit an interpretation in terms of sensory carryover. In the five experiments reported here, the effect was sought under conditions which would require an interpretation in terms of associative reinstatement. Since it is not feasible to train honeybees in widely spaced trials, several different interpolated-trials procedures were employed which had in common the feature that nonrewarded response to a stimulus never was followed by rewarded response to the same stimulus. Implications of the negative results for the interpretation of the overlearning-extinction effect and successive negative contrast in honeybees are considered.     

Differential reinforcement and resistance to change of divided-attention performance

Behavioral momentum theory provides a framework for understanding how conditions of reinforcement influence instrumental response strength under conditions of disruption (i.e., resistance to change). The present experiment examined resistance to change of divided-attention performance when different overall probabilities of reinforcement were arranged across two components of a multiple schedule. Pigeons responded in a delayed-matching-to-sample procedure with compound samples (color + line orientation) and element comparisons (two colors or two line orientations). Reinforcement ratios of 1:9, 1:1, and 9:1 for accurate matches on the two types of comparison trials were examined across conditions using reinforcement probabilities (color/lines) of .9/.1, .5/.5, and .1/.9 in the rich component and .18/.02, .1/.1, and .02/.18 in the lean component. Relative accuracy with color and line comparisons was an orderly function of relative reinforcement, but this relation did not depend on the overall rate of reinforcement between components. The resistance to change of divided-attention performance was greater for both trial types in the rich component with presession feeding and extinction, but not with decreases in sample duration. These findings suggest promise for the applicability of quantitative models of operant behavior to divided-attention performance, but they highlight the need to further explore conditions impacting the resistance to change of attending.     

Effect of angle stimulation during development on adult discrimination ability in rats

Hooded rats were reared in very restricted environments with angular, curved, or white-field stimulation. The extent of transfer from this exposure was assessed in adulthood by testing subjects for (a) learning ability in form, brightness, and pattern discrimination tasks, and (b) exploration and activity differences in an open field. The angle-reared group performed better than the other two rearing groups in all three discrimination tasks. Moreover, the curved and white-field groups were equivalent. These learning differences did not appear to result from differences in (a) exploration or activity or (b) dependence on kinesthetic cues. The results demonstrated that one type of early visual stimulation produced transfer to discrimination tasks that varied in difficulty and in degree of relevancy to the rearing forms. This effect was related to the results and interpretations from previous form-rearing studies. Three interpretations were proposed for this transfer and for the advantage of angular vs. curved stimulation.     

Eye movements of monkeys during brightness discrimination and discrimination reversal

Eye movements of two stump-tailed monkeys were measured during performance on an easy and a difficult brightness discrimination problem with and without a relatively long fixation required at the beginning of each trial for purposes of calibration. The duration of an individual fixation of the discriminative stimuli was unrelated to most of the variables that were examined, including problem difficulty, response outcome, whether the S+ or the S? was fixated, and presence or absence of a long fixation at the beginning of a trial. Duration of fixation increased markedly, though temporarily, following reversal of the hard problem. The animals tended to do a minimal amount of scanning of the discriminative stimuli and to fixate most frequently on S+ before responding. In general, the results did not support an account of observing behavior in terms of conventional reinforcement.     

Enhancement of pigeons’ conditional discrimination performance by expectancies of reinforcement and nonreinforcement

Prior work has shown that when the separate correct responses of a conditional discrimination are followed by different reinforcing outcomes, performance is enhanced relative to that obtained under the conventional, single-reinforcer procedure. Four experiments with pigeons yielded the analogous finding when the different outcomes were reinforcement and explicit nonreinforcement. Controls indicated that the results could not be attributed to the effects of intermittent reinforcement, to possible differences in cue duration, or to a variety of potential sources of conditioned reinforcement. An interpretation in terms of expectancy learning is proposed.     

Additive summation by stimulus compounding irrespective of behavioral contrast during discrimination training: An investigation with positive reinforcement and avoidance schedules

The similarity in the discrimination training leading to behavioral contrast and that preceding tests producing response enhancement to combined discriminative stimuli suggested that the two phenomena might be related. This was investigated by determining if contrast indiscrimination training was necessary for this outcome of stimulus compounding. Responding to tone, light, and to the simultaneous absence of tone and light (T + L) was maintained during baseline training by food reinforcement in Experiment I and by shock avoidance in Experiment II. During subsequent discrimination training, responding was reduced in T + L by programming nonreinforcement in Experiment I and safety or response-punishment in Experiment II. In the first experiment, one rat exhibited positive behavioral contrast, i.e., tone and light rates increased while his T + L rate decreased. In Experiment II, rats punished in T + L showed contrast in tone and light, this being the first demonstration of punishment contrast on an avoidance baseline with rats. The discrimination acquisition data are discussed in the light of current explanations of contrast by Gamzu and Schwartz (1973) and Terrace (1972). During stimulus compounding tests, all subjects in both experiments emitted more responses to tone-plus-light than to tone or light (additive summation). An analysis of the terminal training baselines suggests that the factors producing these test results seem unrelated to whether or not contrast occurred during discrimination training. It was concluded that the stimulus compounding test reveals the operation of the terminal baseline response associations and reinforcement associations conditioned on these multicomponent free-operant schedules of reinforcement.

     

The effect of subadditive pretraining on blocking: limits on generalization

Recent evidence indicates that prior learning about a set of cues may determine how new cues are processed. If subjects are taught that two reliable predictors of an outcome do not summate when the cues are presented together (i.e., subadditive pretraining), the subjects will tend to show a less profound blocking effect when trained with different cues. Three experiments investigated the conditions necessary for subadditive pretraining to generalize to new cues. Experiment 1 demonstrated that subadditive pretraining is less effective in reducing blocking when it is experienced in a context other than that in which the blocking training is experienced. In Experiment 2, the effectiveness of subadditive pretraining waned with time. Experiment 3 showed that subadditive pretraining is more effective when the temporal characteristics of pretraining cues are similar to those of the cues used in blocking training. These results provide information concerning the conditions under which learning will generalize from one set of cues to another.     

Conditioned reinforcement and complex discrimination: Can conditioned reinforcing value be specific to a complex of three cues?

The present experiment is concerned with the nature of the cues that might acquire conditioned reinforcing value, and the ways in which such cues might interact with one another. Red and green colored keylights were differentially paired with food dependent upon the houselight context (A or B) and the trial type (training or choice/forced). The duration of the colored keylights was varied between groups in an attempt to manipulate the effectiveness of the short-term memory of trial-type cues at the trial’s end. The red and green stimuli were of 30 sec duration for Group 30 and of 3 sec duration for Group 3. The results indicated that the choices of the pigeons in Group 30 were influenced by the houselight context present and by the keylight color. The choices of the pigeons in Group 3 seemed to be influenced by the houselight context present, the keylight color, and the memory of trial-type cues. Memory cues for trial antecedents were not overshadowed by presumably more salient external houselight stimuli for the pigeons in Group 3. Two alternative explanations for the results are discussed, and determined to be unlikely based on the results of an earlier experiment. The present results are related to a model of the conditioned reinforcing value of momentary stimuli and of transmission of conditioned reinforcing value.