Remembering as discrimination in delayed matching to sample: Discriminability and bias

Task difficulty in delayed matching-to-sample tasks (DMTS) is increased by increasing the length of a retention interval. When tasks become more difficult, choice behavior becomes more susceptible to bias produced by unequal reinforcer ratios. Delaying reinforcement from choice behavior also increases both task difficulty and the biasing effect of unequal reinforcer probability. Six pigeons completed nine DMTS conditions with retention intervals of 0, 2, 4, 6, and 8 sec, in which reinforcer delays of 0, 2, and 4 sec were combined with ratios of reinforcer probabilities of .5/.5, .2/.8, and .8/.2 for correct red and green responses. Discriminability (logd) decreased with both increasing retention interval duration and increasing reinforcer delay. Sensitivity to reinforcement, the tendency for ratios of choice responses to follow unequal reinforcer probabilities, also increased as a function of both increasing retention interval and increasing reinforcer delay. The result is consistent with the view that remembering in DMTS tasks is a discriminated operant in which increasing task difficulty increases sensitivity to reinforcement.     

Differential reinforcement and resistance to change of divided-attention performance

Behavioral momentum theory provides a framework for understanding how conditions of reinforcement influence instrumental response strength under conditions of disruption (i.e., resistance to change). The present experiment examined resistance to change of divided-attention performance when different overall probabilities of reinforcement were arranged across two components of a multiple schedule. Pigeons responded in a delayed-matching-to-sample procedure with compound samples (color + line orientation) and element comparisons (two colors or two line orientations). Reinforcement ratios of 1:9, 1:1, and 9:1 for accurate matches on the two types of comparison trials were examined across conditions using reinforcement probabilities (color/lines) of .9/.1, .5/.5, and .1/.9 in the rich component and .18/.02, .1/.1, and .02/.18 in the lean component. Relative accuracy with color and line comparisons was an orderly function of relative reinforcement, but this relation did not depend on the overall rate of reinforcement between components. The resistance to change of divided-attention performance was greater for both trial types in the rich component with presession feeding and extinction, but not with decreases in sample duration. These findings suggest promise for the applicability of quantitative models of operant behavior to divided-attention performance, but they highlight the need to further explore conditions impacting the resistance to change of attending.     

On two effects of signaling the consequences for remembering

Five pigeons were trained to perform a delayed matching-to-sample task in which red- and green-colored keys were presented as sample and choice stimuli, and the duration of a delay interval varied across trials. Experiment 1 investigated the effects on delayed-matching accuracy of signaling different durations of food access for the two correct responses (the differential-outcomes effect), and of signaling nondifferential but larger durations for both responses (the signaled-magnitudes effect). In Condition 1, a vertical bar on either sample signaled different rewards (or different outcomes, DOs) for correct red and correct green responses (0.5 and 3.5 sec, respectively), and a horizontal bar signaled equal durations of food access (or same outcomes, SOs) for these responses (1.5 sec). In Condition 2, the horizontal bar signaled equally large rewards for the two correct responses (3.5 sec), and the vertical bar signaled equally small rewards (0.5 sec). Delayed-matching accuracies were higher on DO trials than on SO trials, and they were higher on large-reward trials than on small-reward trials. However, analyses of discriminability estimates as a function of delay-interval duration revealed differences between the forgetting functions reflecting these two effects. Signaling DOs increased the initial level of the function and reduced its slope relative to signaling SOs, whereas signaling larger rewards increased the initial level of the function but did not affect its slope relative to signaling smaller rewards. Experiment 2 investigated whether the difference between the initial levels of DO and SO functions in Condition 1 resulted from overall longer food access on the former trials. However, varying the food-access times on SO trials across three conditions (0.5, 3.5, and 1.5 sec) failed to produce systematic effects consistent with this hypothesis. The results are discussed with respect to the mechanisms that could be responsible for the two effects.     

Resistance to extinction: contingency termination and generalization decrement

We present an algebraic model of resistance to extinction that is consistent with research on resistance to change. The model assumes that response strength is a power function of reinforcer rate and that extinction involves two additive, decremental processes: (1) the termination of the reinforcement contingency and (2) generalization decrement resulting from reinforcer omission. The model was supported by three experiments. In Experiment 1, 4 pigeons were trained on two-component multiple variable-interval (VI) 60-sec, VI 240-sec schedules. In two conditions, resistance to change was tested by terminating the response-reinforcer contingency and presenting response-independent reinforcers at the same rate as in training. In two further conditions, resistance to change was tested by prefeeding and by extinction. In Experiment 2, 6 pigeons were trained on two-component multiple VI 150-sec schedules with 8-sec or 2-sec reinforcers, and resistance to change was tested by terminating the response-reinforcer contingency in three conditions. In two of those conditions, brief delays were interposed between responses and response-independent reinforcers. In both Experiments 1 and 2, response rate was more resistant to change in the richer component, except for extinction in Experiment 1. In Experiment 3, 8 pigeons were trained on multiple VI 30-sec, VI 120-sec schedules. During extinction, half of the presentations of each component were accompanied by a novel stimulus to produce generalization decrement. The extinction data of Experiments 1 and 3 were well described by our model. The value of the exponent relating response strength and reinforcement was similar in all three experiments.     

Directed forgetting in pigeons

Pigeons performed a version of delayed matching-to-sample in which different postsample cues signaled different trial outcomes. Cues to remember (R cues) signaled the usual comparison stimuli. Cues to forget (F cues) signaled either cancellation of comparison stimuli (comparison-omission) or presentation of a sample-independent discrimination (comparison-substitution). As assessed by occasional probe trials, F cues decreased matching accuracy during comparison-omission more than during comparison-substitution. The loss in accuracy of matching in F-cue probes was directly related to length of delays during comparison-omission but not during comparison-substitution. Because trials generally terminated in reward during comparison-substitution but not during comparison-omission, these findings were interpreted as suggesting the importance of end-of-trial reinforcement for the maintenance of short-term memory.     

Remembering: The role of extraneous reinforcement

In two experiments, pigeons' responding on an extraneous task was explicitly reinforced during delayed matching-to-sample trials. In Experiment 1, red or green sample stimuli were followed by retention intervals of 0.2, 1, 4, or 12 sec, during which pecks to a white center key were reinforced with 2.5-sec access to wheat according to extinction, variable-interval 30-sec, and variable-interval 15-sec schedules in different conditions. A proportion of .2, .5, .7, or .9 of subsequent red or green choice responses that matched the sample were reinforced with 3-sec access to wheat. The result was that increasing center key reinforcement, or reducing reinforcer probability, lowered overall accuracy. Initial discriminability fell, but with no change in the rate of forgetting. In Experiment 2, initial discriminability was affected by extraneous reinforcers that were contingent on center key pecking, but not by noncontingent reinforcers. A plausible conclusion is that initial discriminability decreases when reinforcers strengthen competing behaviors.     

Preference for rewards that follow greater effort and greater delay

Humans prefer (conditioned) rewards that follow greater effort (Aronson & Mills, 1959). This phenomenon can be interpreted as evidence for cognitive dissonance (or as justification of effort) but may also result from (1) the contrast between the relatively greater effort and the signal for reinforcement or (2) the delay reduction signaled by the conditioned reinforcer. In the present study, we examined the effect of prior force and prior time to produce stimuli associated with equal reinforcement. As expected, pressing with greater force or for a longer time was less preferred than pressing with less force or for a shorter time. However, participants preferred the conditioned reinforcer that followed greater force and more time. Furthermore, participants preferred a long duration with no force requirement over a shorter duration with a high force requirement and, consistent with the contrast account but not with the delay reduction account, they preferred the conditioned stimulus that followed the less preferred, shorter duration, high-force event. Within-trial contrast provides a more parsimonious account than justification of effort, and a more complete account than delay reduction.     

A failure to obtain magnetic discrimination in the pigeon

Six pigeons were trained on a multiple-concurrent schedule. During both multiple-schedule components, two response keys were available. In Component 1, responses to the left key were reinforced on a variable-interval 45-sec schedule, whereas responses to the right key were not reinforced. In Component 2, these contingencies were reversed. Following each reinforcer, the component next presented was chosen randomly with equal probability. The two components were differentially signaled by presentation of various stimuli during Component 1. Over the course of the experiment, these stimuli were various intensities of light, a noise, and various fluctuating and static magnetic fields. While subjects showed good discrimination of the light and noise stimuli, no stimulus control using the magnetic fields was obtained.     

A comparison of the short-term memory performances of pigeons and jackdaws

Two experiments employed a delayed conditional discrimination procedure in which half the trials began with the presentation of food and half with no food; following a retention interval, subjects were presented with a choice between red and green keys, a response to one of which was reinforced according to whether the trial had started with food or no food. In Experiment 1, after 38 training sessions during which the retention interval was gradually increased, pigeons performed at a moderate level with intervals of 5 to 7.5 sec. A final test produced a steep forgetting function for food trials, but not for no-food trials; performance was unaffected by the duration of the intertriai interval (10 or 40 sec). Experiment 2 used the delayed conditional discrimination procedure to compare short-term memory in jackdaws (Corvus monedulus) with that in pigeons. Although the performance of the jackdaws was below that of the pigeons at the start of training, they showed more rapid learning over long delays, and, in the final test, a shallower forgetting function for food trials than that shown by pigeons. The results suggested superior short-term memory in jackdaws, which may help to explain the better performance of corvids in general when compared with that of pigeons in certain complex learning tasks.     

The relation of multiple-schedule behavioral contrast to deprivation,time in session,and within-session changes in responding

Pigeons' keypecking was reinforced by food on baseline schedules of multiple variable interval (VI) x VI x and on contrast schedules of multiple VI x VI y. Deprivation of food was varied by maintaining subjects at 75%, 85%, and 95% (+/- 2%) of their free-feeding weights. Positive and negative behavioral contrast were observed. The size of the contrast was not systematically altered by changes in deprivation. Positive and negative contrast were both larger later in the session than they were earlier. Within-session decreases in responding were steeper for the baseline than for the contrast schedules for positive contrast. Within-session decreases were steeper for the contrast than for the baseline schedules for negative contrast. These results were predicted by the idea that different amounts of habituation to the reinforcer during the baseline and contrast schedules contribute to behavioral contrast. The results show that contrast occurs under conditions that reduce the effect of the following component. The results support the assumption that positive and negative contrast are produced by symmetrical theoretical variables.     

Real-time factors in the rabbit’s nictitating membrane response to pulsed and serial conditioned stimuli

Conditioning theories and recent real-time models commonly postulate that a reinforcer is signaled by a series of stimuli. In both Pavlovian and operant procedures, serial stimuli have been shown to control the likelihood and timing of responses over intervals of seconds and minutes. The present experiments were conducted to determine whether serial stimuli exercise similar effects over stimulus-reinforcer intervals in the order of hundreds of milliseconds. Such intervals typify those used in conditioning of the rabbit nictitating membrane response. A sequence of four tone pulses (50–100 msec) was used as the CS to assess the effectiveness of serial stimuli. After training with this CS, tests were conducted in which one or more of the pulses were removed. These perturbations of the sequence of stimuli over a 400-msec interval produced large deficits in CR likelihood and smaller alterations in CR timing. The results are discussed with respect to their implications for current real-time models of conditioning, and particularly with respect to their assumptions about the source of internal stimuli, rules for learning, and rules for generating CRs.     

Stimulus preference and the transitivity of preference

Four pigeons were exposed to multiple schedules with concurrent variable interval (VI) components and then tested for preference transfer. Half of the pigeons were trained on a multiple concurrent VI 20-sec, VI 40-sec/cuncurrent VI 4G-sec5 VI 80-sec schedule. The remaining pigeons were trained on a multiple concurrent VI 80-sec, VI 40-sec/concurrent VI 40-sec, VI 20-sec schedule-After stability criteria for time and response proportions were simultaneously met, four preference transfer tests were conducted with the stimuli associated with the VI 40-sec schedules. During the transfer tests, each pigeon allocated a greater proportion of responses (M=0,79) and time (M=0.82) to the stimulus associated with the VI 40-sec schedule that was paired with the VI 80-sec schedule than lo the VI 40-sec schedule stimulus paired with the VI 20-sec schedule. Absolute reinforcement rates on the two VI 40 sec schedules were approximately equal and unlikely to account for the observed preference. Nor was the preference consistent with the differences in local reinforcement rates associated with the two stimuli. Instead, the results were interpreted in terms of the differential value that stimuli acquire as a function of previous pairings with alternative schedules of reinforcement.     

Pretest cuing after forgetting of a food-motivated maze task in rats: Synergistic action of context and reinforcement

It is now well known in animal studies that spontaneous forgetting, as well as performance decrements resulting from other sources, can be alleviated by means of pretest treatments. Several previous experiments have shown that reliable forgetting is observed after a 25-day training-to-test interval on a relatively complex maze. This forgetting can be alleviated by pretest exposure to background stimuli in the experimental room. The effects of this treatment can be modulated by varying either the duration of the treatment or the length of the cuing-test interval. The purpose of the experiment presented here was twofold: (1) to demonstrate the multidimensionality of memories in our paradigm by comparing the effects of different reminder treatments, namely pretest access to the reinforcer and contextual cuing, and (2) to test the general hypothesis, formulated by Spear (1978, p. 418), that “the elicitation and retrieval of the target attribute of a memory depend on the arousal of a sufficientnumber or kind of the remaining attributes of this memory.” The data showed that pretest food reinforcement had no significant effect on retention performance; that pretest contextual cuing did, in some precise conditions, alleviate forgetting; and that some combinations of context and reinforcement had what amounted to a synergistic action on retention performance.     

Remembering in California sea lions: Using priming cues to facilitate language-like performance

Two sea lions(Zalophus californianus) were trained to respond to relational commands fey bringing the last designated object (transport item, or TI) to the earlier designated object (goal item, or GI). Most errors on this task involved forgetting of the GI due to interference from memory for the TI. We reasoned that instructing the animals to act directly on the same object several times in single-object commands immediately before a relational command that included the same object as the GI should make the GI associative episode less prone to interference by the TI associative episode. Results confirmed that prior cuing or prompting in this manner did enable the sea lions to perform significantly better on relational commands when no cuing was given.     

Effects of Headings on Text Summarization

A summarization task was used to study whether headings influence readers' representations of the topic structure of a text. College students (Experiments 1-3) and sixth- and eighth-graders (Experiment 3) summarized a multiple topic text that (a) included headings introducing every new subtopic, (b) included headings introducing half of the new subtopics, or (c) included no headings. In all experiments, topics were more likely to be included in a summary if they were signaled than if they were not signaled. This effect was magnified when the text was only half signaled: Signaled topics were more likely to appear in a summary if only half the text topics were signaled than if all of the topics were signaled; however, unsignaled topics were less likely to appear in a summary if half of the text topics were signaled than if none of the text topics were signaled. The findings demonstrate that readers rely heavily on headings in a task that emphasizes attention to a text's topic structure. It is suggested that previously observed signaling effects on text recall are mediated by effects on how readers represent a text's topic structure. Copyright 2001 Academic Press.     

Time dependent effects of double cuing in directed forgetting

Previous research in directed forgetting in pigeons has focused on the effect of single forget cues (F-cues) interpolated within the retention interval in delayed matching-to-sample (DMTS). The present series of experiments focuses on the ability of a remember cue (R-cue) to cancel the effects of a previously presented forget cue both when the forget cue occurs within the retention interval and when the forget cue precedes sample presentation. In the first experiment, an R-cue decreased the effect of an F-cue within the same retention interval in successive DMTS if the R-cue immediately followed the F-cue, but not if the second cue was delayed until the end of the retention interval. In Experiment 2, the double-cue effect was extended to choice DMTS. In addition, when a novel stimulus replaced the R-cue in the double-cue probe trials, matching performance was not restored, indicating that through its conditioning history the R-cue had gained control over memory processes in a direction opposite to that of the F-cue. Experiment 3 presents evidence that presample R- and F-cues can also effectively gain control over matching performance. Matching to R-cued samples was superior to matching to F-cued samples. When F-cued samples were followed immediately by R-cues, matching performance was not restored to R-cue levels, suggesting differential encoding of the R-cued and F-cued samples.     

The effect of crying on long-term memory in infancy

The influence of crying on infants' long-term memory for a learned response was investigated in 3 experiments. In each, infants were trained to move a crib mobile containing 10 identical objects by means of kicking and were then exposed to a reinforcer containing only 2 of these components. This shift in component numerosity produced crying in 53% of the infants. Infants who cried in response to the reward shift evidenced no retention of the contingency 1 week later (Experiment 1) but did have excellent retention at 1 day (Experiment 2). In Experiment 3, a brief reactivation treatment alleviated forgetting at 3 weeks regardless of the presence of crying in response to the change in mobiles. An unexpected recency effect characterized the efficacy of the reactivation treatment. The results indicate that crying in response to the violation of a reward-expectation habit functions as an amnesic agent to produce accelerated forgetting.     

Pigeon and human performance in a multi-armed bandit task in response to changes in variable interval schedules

The tension between exploitation of the best options and exploration of alternatives is a ubiquitous problem that all organisms face. To examine this trade-off across species, pigeons and people were trained on an eight-armed bandit task in which the options were rewarded on a variable interval (VI) schedule. At regular intervals, each option’s VI changed, thus encouraging dynamic increases in exploration in response to these anticipated changes. Both species showed sensitivity to the payoffs that was often well modeled by Luce’s (1963) decision rule. For pigeons, exploration of alternative options was driven by experienced changes in the payoff schedules, not the beginning of a new session, even though each session signaled a new schedule. In contrast, people quickly learned to explore in response to signaled changes in the payoffs.     

The rat’s simultaneous anticipation of remote events and current events can be sustained by event memories alone

After receiving events in a fixed order, A-B-C…, rats, like people, on being provided with A, may anticipate not only B, a current anticipation, but also C, a remote anticipation. In two experiments, we attempted to determine whether rats’ remote anticipations are mediated by item cues (C elicited by A) or by position cues (C directly elicited by Position 3 cues, which generalize to Position 2). In Experiment 1, rats in a runway received two series of three trials, XNY and ZNN, each in irregular order each day. N signified nonreinforcement; X, Y, and Z signified three qualitatively different food reinforcements. The rats manifested a remote anticipation by running faster on Trial 2 in the XNY series than in the ZNN series. Since the series were presented irregularly, Trial 2 performance cannot be explained on a positional basis alone. It can be explained on an item basis, by assuming that the memory of the Trial 1 reinforcer became associated not only with the Trial 2 event, but with the Trial 3 event as well. Thus on Trial 2 the memory of X signaled N and Y, whereas the memory of Z signaled N and N. Experiment 2 produced the same results, regardless of whether the XNY and ZNN series were presented in regular or irregular order. These results indicate that remote anticipations can be mediated by item associations. They offer no evidence that position associations can do the same, but they do not rule out that possibility.     

Dissociation of the effect of reinforcer type and response strength on the force of a conditioned response

A dissociation between the effect of reinforcer type and response strength on the force of the pigeon’s keypeck response was shown in three experiments. In Experiment 1, pigeons were trained to peck two conditioned stimuli, one paired with water and another paired with grain. The pigeons made more forceful pecks for grain than for water and also showed a tendency, albeit an unreliable one, to respond on a higher percentage of food trials than water trials. In Experiment 2, the pigeons from Experiment 1 were satiated with either food or water and were then presented with the two conditioned stimuli in an extinction test. It was found that, regardless of the drive state, the pigeons made more forceful pecks to the stimulus that predicted food than to the stimulus that predicted water. In the thirsty group, however, this difference in force was not accompanied by a difference in the percentage of trials with a response. In Experiment 3, pigeons trained with a single reinforcer pecked more often on instrumentally reinforced trials than on Pavlovian conditioning trials, but there was no difference in the force of the pecks. Taken together, these results imply that differences in response strength cannot account for the difference between the force of food- and water-reinforced pecks. Instead, stimulus-substitution theory may provide the best account of the topography of the two types of pecks.