Level of deprivation does not affect degree of discounting in pigeons

Two experiments tested the effects of food deprivation on discounting in pigeons. An adjusting-amount procedure was used to estimate the subjective value of food at delays ranging from 1 to 24 s. Experiment 1 compared pigeons’ discounting of delayed food reinforcers at 75 %–80 % and 90 %–95 % of free-feeding weight. Experiment 2 compared discounting under 1- and 23-h food deprivation. In both experiments at both deprivation levels, discounting was well described by the hyperboloid discounting function. No systematic effect of level of deprivation on degree of discounting was observed in either experiment. This finding is consistent with the view that pigeons’ choices are controlled by the relative, rather than the absolute, value of reinforcers.     

The generality of within-session patterns of responding: Rate of reinforcement and session length

Rats and pigeons responded for food delivered according to multiple schedules. The session length varied from 10 to 120 min, and the programmed rate of reinforcement varied from 15 to 240 reinforcers per hour. Response rates usually changed systematically within experimental sessions. For both rats and pigeons, responding reached a peak after an approximately constant amount of time since the beginning of the session, regardless of session length. When rats, but not pigeons, served as subjects, the peak rates of responding occurred later in the session and the within-session changes were smaller for lower than for higher rates of reinforcement. The similarities between the results for rats and for pigeons when session length varied suggest that at least one of the factors that produces the within-session changes in responding is shared by the present species, responses, and reinforcers. The differences in results when rate of reinforcement varied are more difficult to interpret.     

Positive behavioral contrast as a function of time-out duration when pigeons peck keys on a within-session procedure

Three pigeons pecked keys for food reinforcers delivered by multiple variable interval variable interval schedules in the first part of each session (baseline) and by multiple variable interval extinction schedules in the second part of each session (contrast). The variable interval schedules delivered reinforcers after an average of 4 min or 30 sec in different conditions. The duration of a time-out between the components varied in five steps from 5 to 120 sec. Positive contrast occurred for all time-out durations in both experiments. That is, the rate of responding emitted during the constant, variable interval component was greater during the contrast than during the baseline schedules. The size of contrast did not change systematically with changes in timeout duration. These results violate most theories of contrast. They are compatible with the idea that animals integrate reinforcers over intervals longer than 2 min.     

Potentiation of delayed matching with variable delays

In a delayed matching-to-sample procedure, pigeons chose a comparison stimulus that matched a sample stimulus presented earlier in the trial. The duration of the delay between sample-stimulus presentation and comparison-stimulus presentation was either varied over five values within each session or held constant within each session but varied over five blocks of sessions. Accuracy of matching to sample was higher overall with variable delays than with delays fixed within sessions. The result indicates that remembering depends on the temporal context provided by delay intervals.     

Development of preference and spontaneous recovery in choice behavior with concurrent variable-interval schedules

Pigeons pecked on two response keys that delivered reinforcers on a variable-interval schedule. The proportion of reinforcers delivered by one key was constant for a few sessions and then changed, and subjects’ choice responses were recorded during these periods of transition. In Experiment 1, response proportions approached a new asymptote slightly more slowly when the switch in reinforcement proportions was more extreme. In Experiment 2, slightly faster transitions were found with higher overall rates of reinforcement. The results from the first session, after a switch in the reinforcement proportions, were generally consistent with a mathematical model that assumes that the strength of each response is increased by reinforcement and decreased by nonreinforcement. However, neither this model nor other similar models predicted the “spontaneous recovery” observed in later sessions: At the start of these sessions, response proportions reverted toward their preswitch levels. Computer simulations could mimic the spontaneous recovery by assuming that subjects store separate representations of response strength for each session, which are averaged at the start of each new session.     

Resistance to extinction: contingency termination and generalization decrement

We present an algebraic model of resistance to extinction that is consistent with research on resistance to change. The model assumes that response strength is a power function of reinforcer rate and that extinction involves two additive, decremental processes: (1) the termination of the reinforcement contingency and (2) generalization decrement resulting from reinforcer omission. The model was supported by three experiments. In Experiment 1, 4 pigeons were trained on two-component multiple variable-interval (VI) 60-sec, VI 240-sec schedules. In two conditions, resistance to change was tested by terminating the response-reinforcer contingency and presenting response-independent reinforcers at the same rate as in training. In two further conditions, resistance to change was tested by prefeeding and by extinction. In Experiment 2, 6 pigeons were trained on two-component multiple VI 150-sec schedules with 8-sec or 2-sec reinforcers, and resistance to change was tested by terminating the response-reinforcer contingency in three conditions. In two of those conditions, brief delays were interposed between responses and response-independent reinforcers. In both Experiments 1 and 2, response rate was more resistant to change in the richer component, except for extinction in Experiment 1. In Experiment 3, 8 pigeons were trained on multiple VI 30-sec, VI 120-sec schedules. During extinction, half of the presentations of each component were accompanied by a novel stimulus to produce generalization decrement. The extinction data of Experiments 1 and 3 were well described by our model. The value of the exponent relating response strength and reinforcement was similar in all three experiments.     

Proactive interference in the retention of “attentional sets” in pigeons

In Experiment 1, six groups of pigeons (n=8) were tested for wavelength generalization either immediately or 24 h after learning a successive discrimination, with 550 nm reinforced and a black vertical line extinguished. The groups differed in the stimulus present during single stimulus pretraining, which was 550 nm (pretrain S+), the vertical Une (pretrain S?), or a neutral dim white light (pretrain Sn), respectively. The three immediate generalization gradients were steep and indistinguishable, reflecting only the immediately preceding discrimination training condition. The three delay gradients were flatter, with the flattening particularly marked in the pretrain S? group. This was interpreted as proactive interference (PI) resulting from the memory that both the 550-nm and the line stimuli had previously been reinforced. In Experiment 2, two (TD) groups of pigeons (n=16) were given single stimulus training with a 555-nm keylight followed by eight sessions of discrimination training with two line angles, then one session of non-differential (ND) training with the same two lines, and then a wavelength generalization test either immediately or after a 24-h delay. Two other (hold) groups (n=16) received similar training, except for the TD Une angle training sessions, in these hold groups, the wavelength gradient was flatter in a delayed test; in the TD groups it was steeper, indicating PI from the prior TD training. These two experiments suggest that the “attentional sets,” which purportedly result from TD and ND training, may fruitfully be viewed as target memories subject to the principles of interference theory.     

Past experience,recency, and spontaneous recovery in choice behavior

Pigeons’ responses on two keys were recorded before and after the percentage of reinforcers delivered by each key was changed. In each condition of Experiment 1, the reinforcement percentage for one key was 50% for several sessions, then either 70% or 90% for one, two, or three sessions, and then 50% for another few sessions. At the start of the second and third sessions after a change in reinforcement percentages, choice percentages often exhibited spontaneous recovery—a reversion to the response percentages of earlier sessions. The spontaneous recovery consisted of a shift toward a more extreme response percentage in some cases and toward a less extreme response percentage in other cases, depending on what reinforcement percentages were previously in effect. In Experiment 2, some conditions included a 3-day rest period before a change in reinforcement percentages, and other conditions included no such rest days. Slightly less spontaneous recovery was observed in conditions with the rest periods, suggesting that the influence of prior sessions diminished with the passage of time. The results are consistent with the view that choice behavior at the start of a new session is based on a weighted average of the events of the past several sessions.     

Home cage feeding time controls responding under multiple schedules

Eleven rats were exposed to a multiple variable-interval 1-min variable-interval 1-min schedule of reinforcement. All rats were initially fed a daily ration of food in the home cages immediately after the end of each session. In a later phase of the experiment, the same amount of food was fed 1 h after the end of each session. Later, five rats were again fed immediately after each session. Amount of food received and deprivation level in terms of percent of free feeding weight were constant across conditions. Response rates decreased within each session under immediate feeding. When feeding was delayed, rates in each component of the multiple schedule increased throughout the session and the decreasing trends were generally eliminated. The results suggest that home cage feeding time, apart from changes in deprivational level, is an important variable in the control of behavior in experimental sessions.     

Response sequence learning as a function of primary versus conditioned reinforcement

Two groups of pigeons were required to generate a fixed sequence of responses on three keys, for example, middle-left-right. One group received a small food reward (S^Food) following each correct response except the terminal one, which was followed by a large food reward. The second group received conditioned reinforcement from an overhead light (S^Light) for each correct response, with the terminal correct response followed by both S^Light and the large food reward. We manipulated length of sequence (3 or 7 responses) and duration of required interresponse interval (IRI; 1 to 9 sec). S^Light contingencies generated more accurate performances than did S^Food when sequence length was 3 responses but not when it was 7 responses. IRI duration influenced accuracy under the S^Light contingencies but not under S^Food. These results show that conditioned reinforcers sometimes generate more accurate sequence learning than do primary reinforcers, and that schedule contigencies influence which type of feedback will optimize performance. The results parallel those from the matching-to-sample and conditional discrimination literature.     

A comparison of the short-term memory performances of pigeons and jackdaws

Two experiments employed a delayed conditional discrimination procedure in which half the trials began with the presentation of food and half with no food; following a retention interval, subjects were presented with a choice between red and green keys, a response to one of which was reinforced according to whether the trial had started with food or no food. In Experiment 1, after 38 training sessions during which the retention interval was gradually increased, pigeons performed at a moderate level with intervals of 5 to 7.5 sec. A final test produced a steep forgetting function for food trials, but not for no-food trials; performance was unaffected by the duration of the intertriai interval (10 or 40 sec). Experiment 2 used the delayed conditional discrimination procedure to compare short-term memory in jackdaws (Corvus monedulus) with that in pigeons. Although the performance of the jackdaws was below that of the pigeons at the start of training, they showed more rapid learning over long delays, and, in the final test, a shallower forgetting function for food trials than that shown by pigeons. The results suggested superior short-term memory in jackdaws, which may help to explain the better performance of corvids in general when compared with that of pigeons in certain complex learning tasks.     

Rats' performance on an interval time-place task: Increasing sequence complexity

Rats were trained on an interval time-place learning (TPL) task in which the location of food availability depended on the time since the start of the session. Each of four levers (numbered 1, 2, 3, 4) provided food on an intermittent schedule for two nonconsecutive 3-min periods. The order in which the levers provided food was 1, 2, 4, 3, 2, 3, 1, 4. This order was consistent across sessions. Previous research conducted in our lab has shown that when only four “places” are used, rather than the eight in the present study, rats use a timing strategy to track the location of food. Pizzo and Crystal (2004) recently trained rats on an interval TPL in which each of eight arms of a radial arm maze provided food. They found evidence suggesting that rats used both spatial and temporal information. In the present study, in which a revisiting strategy was used (i.e., each lever provided food on more than one occasion), the rats tracked both the spatial and the temporal availability of food for the first half of the session. Interestingly, in the second half of the sessions, the rats appeared to be timing the availability of food even though they did not know where it would occur. That is, the rats knew the temporal, but not the spatial, contingencies for the second half of the session. It appears that the requirement of revisiting a previously reinforced lever resulted in rats' no longer being able to solve the spatial aspect of the task.     

On the determinants of induction in responding for sucrose when food pellet reinforcement is upcoming

Rats’ rates of leverpressing for low-concentration liquid-sucrose reinforcers in the first half of an experimental session increase when food pellet, rather than sucrose, reinforcers will be available in the second half. Experiment 1 determined that this induction effect was the outcome of food pellet reinforcement’s increasing response rates, not of continued sucrose reinforcement’s decreasing them. Experiments 2 and 3 showed that induction was primarily controlled by the conditions of reinforcement in the current session, not by those in the previous one. Experiment 4 showed little evidence that the induction was the outcome of Pavlovian processes. These results suggest that induction may occur because of processes operating at the level of the entire session. They also provide a link to a seemingly related area of study: contrast effects. Some of the results are consistent with what is known about contrast effects, but there are also several, yet unexplained differences.     

Within-session changes in responding during concurrent schedules that employ two different operanda

Five rats responded on several concurrent schedules in which pressing a key produced reinforcers in one component and pressing a lever produced reinforcers in the other component (Experiment 1). Four pigeons responded on several concurrent keypeck treadlepress schedules (Experiment 2). The programmed rates of reinforcement varied from 15 to 240 reinforcers per hour in different conditions. Rates of responding usually changed systematically within experimental sessions, and the changes were similar for the two components of a concurrent schedule. These results imply that within-session changes in responding may not confound the predictions of theories that describe the ratio of the rates of responding during the two components of concurrent schedules. Instead, within-session changes may be controlled by a mechanism that integrates the reinforcers obtained from the two components.     

Differences in food consumption under intermittent and continuous reinforcement schedules of water delivery: Some implications for schedule-induced behavior

Consumption of food pellets was examined in four water-deprived rats during 1-h sessions in which water was presented once every 30, 60, or 120 sec independently of the rats’ behavior according to three fixed-time (FT) schedules. Correlated with each FT condition was a continuous reinforcement (CRF) control condition in which the rats received, at the start of the session, the number of dipper presentations that were programmed to occur during the corresponding FT condition. During both the FT and CRF conditions, pellets per dipper presentation decreased and food intake rate increased with rate of water presentation, and there was a direct linear relation between log food intake and log water intake. For each of these three measures there was less eating under the FT condition than under the CRF condition, but the difference between the FT and CRF functions decreased at shorter FT values. These data are discussed in terms of the effects of amount of water on food consumption and the principle of temporal summation.     

On two effects of signaling the consequences for remembering

Five pigeons were trained to perform a delayed matching-to-sample task in which red- and green-colored keys were presented as sample and choice stimuli, and the duration of a delay interval varied across trials. Experiment 1 investigated the effects on delayed-matching accuracy of signaling different durations of food access for the two correct responses (the differential-outcomes effect), and of signaling nondifferential but larger durations for both responses (the signaled-magnitudes effect). In Condition 1, a vertical bar on either sample signaled different rewards (or different outcomes, DOs) for correct red and correct green responses (0.5 and 3.5 sec, respectively), and a horizontal bar signaled equal durations of food access (or same outcomes, SOs) for these responses (1.5 sec). In Condition 2, the horizontal bar signaled equally large rewards for the two correct responses (3.5 sec), and the vertical bar signaled equally small rewards (0.5 sec). Delayed-matching accuracies were higher on DO trials than on SO trials, and they were higher on large-reward trials than on small-reward trials. However, analyses of discriminability estimates as a function of delay-interval duration revealed differences between the forgetting functions reflecting these two effects. Signaling DOs increased the initial level of the function and reduced its slope relative to signaling SOs, whereas signaling larger rewards increased the initial level of the function but did not affect its slope relative to signaling smaller rewards. Experiment 2 investigated whether the difference between the initial levels of DO and SO functions in Condition 1 resulted from overall longer food access on the former trials. However, varying the food-access times on SO trials across three conditions (0.5, 3.5, and 1.5 sec) failed to produce systematic effects consistent with this hypothesis. The results are discussed with respect to the mechanisms that could be responsible for the two effects.     

Effects of identical context on visual pattern recognition by pigeons

The effects of identical context on pattern recognition by pigeons for outline drawings of faces were investigated by training pigeons to identify (Experiment 1) and categorize (Experiment 2) these stimuli according to the orientation of the mouth—an upright U shape representing a smiling mouth or an inverted U shape representing a sad mouth. These target stimuli were presented alone (Pair 1), with three dots in a triangular orientation to represent a nose and eyes (Pair 2), and with the face pattern surrounded by an oval (Pair 3). In Experiment 1, the pigeons trained with Pair 1 were most accurate, those trained with Pair 2 were less so, and those trained with Pair 3 failed to acquire the discrimination within eighty 100-trial sessions. The same ordering was found in Experiment 2 for pigeons trained on the three pairs simultaneously. The authors suggest that a contrasting finding in humans, the face superiority effect, might be due to a gain in discriminability resulting from recognition of the pattern as a face. An exemplar model of information processing that excludes linguistic coding accounts for the present results.     

Simultaneous discrimination reversal learning in pigeons and humans: anticipatory and perseverative errors

Pigeons were trained on a two-choice simultaneous discrimination (red vs. green) that reversed midway through each session. After considerable training, they consistently made both anticipatory errors prior to the reversal and perseverative errors after the reversal, suggesting that time (or number of trials) into the session served as a cue for reversal. In Experiment 2, to discourage the use of time as a cue, we varied the location of the reversal point within the session such that it occurred semirandomly after Trial 10, 25, 40, 55, or 70. Pigeons still tended both to anticipate and to perseverate. In Experiment 3, we required 20 pecks to a stimulus on each trial to facilitate memory for the preceding response and sensitivity to local reinforcement contingencies, but the results were similar to those of Experiment 2. We then tested humans on a similar task with a constant (Experiment 4) or variable (Experiment 5) reversal location. When the reversal occurred consistently at the midpoint of the session, humans, like pigeons, showed a tendency to anticipate the reversal; however, they did not show perseverative errors. When the reversal location varied between sessions, unlike pigeons, humans adopted a win–stay/lose–shift strategy, making only a single error on the first trial of the reversal.     

Rats’ choices between one and two delayed reinforcers

Rats chose between alternatives that differed in the number of reinforcers and in the delay to each reinforcer. A left leverpress led to two reinforcers, each delivered after a fixed delay. A right leverpress led to one reinforcer after an adjusting delay. The adjusting delay was increased or decreased many times in a session, depending on the rat’s choices, in order to estimate an indifference point& #x2014;a delay at which the two alternatives were chosen about equally often. Both the number of reinforcers and their individual delays affected the indifference points. The overall pattern of results was well described by the hyperbolic-decay model, which states that each additional reinforcer delivered by an alternative increases preference for that alternative but that a reinforcer’s effect is inversely related to its delay. Two other possible delay-discounting equations, an exponential equation and a reciprocal equation, did not produce satisfactory predictions for these data. Adding an additional free parameter to the hyperbolic equation as an exponent for delay did not appreciably improve the predictions, suggesting that raising delay to some power other than 1.0 was unnecessary. The results were qualitatively similar to those from a previous experiment with pigeons (Mazur, 1986), but quantitative differences suggested that the rates of delay discounting were several times slower for rats than for pigeons.