A time-series analysis of changeover performance on concurrent variable-interval schedules

Pigeons responded to concurrent variable-interval variable-interval schedules of food reinforcement. Interchangeover times and reinforcements were recorded in intact time series during acquisition following several schedule-density manipulations. Probability of a changeover, while constant as a function of time since the previous changeover, was found to be a nonmonotonically increasing function of time since reinforcement. Autocorrelation analyses of intact time series revealed the presence of statistically significant sequential dependencies in the time series during acquisition, but not during asymptotic performance. Lag transformations of actual interchangeover times revealed inhomogeneities in the time series not shown in the autocorrelations or in the overall distribution of interchangeover times.     

Preference and resistance to change in concurrent variable-interval schedules

Pigeons were trained on a multiple schedule in which separate concurrent schedules were presented in the two components of the schedule. During one component, concurrent variable-interval 40-sec variableinterval 80-sec schedules operated. In the second component, concurrent variable-interval 40-sec variableinterval 20-sec schedules operated. After stable baseline performance was obtained in both components, extinction probe choice tests were presented to assess preference between the variable-interval 40-sec schedules from the two components. The variable-interval 40-sec schedule paired with the variableinterval 80-sec schedule was preferred over the variable-interval 40-sec schedule paired with the variableinterval 20-sec schedule. The subjects were also exposed to several resistance-to-change manipulations: (1) prefeeding prior to the experimental session, (2) a free-food schedule added to timeout periods separating components, and (3) extinction. The results indicated that preference and resistance to change do not necessarily covary.     

Development of preference and spontaneous recovery in choice behavior with concurrent variable-interval schedules

Pigeons pecked on two response keys that delivered reinforcers on a variable-interval schedule. The proportion of reinforcers delivered by one key was constant for a few sessions and then changed, and subjects’ choice responses were recorded during these periods of transition. In Experiment 1, response proportions approached a new asymptote slightly more slowly when the switch in reinforcement proportions was more extreme. In Experiment 2, slightly faster transitions were found with higher overall rates of reinforcement. The results from the first session, after a switch in the reinforcement proportions, were generally consistent with a mathematical model that assumes that the strength of each response is increased by reinforcement and decreased by nonreinforcement. However, neither this model nor other similar models predicted the “spontaneous recovery” observed in later sessions: At the start of these sessions, response proportions reverted toward their preswitch levels. Computer simulations could mimic the spontaneous recovery by assuming that subjects store separate representations of response strength for each session, which are averaged at the start of each new session.     

Changeover contingencies and choice on concurrent schedules

Pigeons were trained on a multiple concurrent schedule with two components per session. In one component, changing schedules required the completion of a small fixed ratio on the switching key (a fixed-ratio changeover, or FRCO), and in the other component, changing schedules required only one switching response but engaged a changeover delay (COD) during which keypecks were not reinforced. Response ratios overmatched reinforcer ratios under the FRCO but undermatched under the COD. There was no difference in time allocation. In addition to these molar regularities in behavior, there were characteristic differences in performance at the molecular level. These local patterns of behavior, which can be explained within the context of contingencies created by the different changeover requirements, appear to underlie differences in performance at the molar level. Obtained molar differences in performance are not compatible with the assumption that there is a “general outcome” on concurrent schedules; and explaining these molar differences in performance in terms of the local contingencies of reinforcement is contrary to the assumption that behavior is allocated as a function of molar distributions of reinforcers.     

The effects of changeover delays of fixed or variable duration on concurrent variable-interval performance in pigeons

The effects of changeover delays of fixed or variable duration on concurrent variable-interval performance in pigeons were investigated in a series of three experiments. Experiment 1 compared the effects of a fixed, variable, or variable signaled changeover delay on interchangeover times and responding during and after the changeover delay. The duration of the changeover delays was systematically varied in Experiment 2, and the relative reinforcement frequencies were manipulated in Experiment 3. Interchangeover times were found to be shorter when changeover delays of variable duration were compared with those of fixed duration. Changeover delays of fixed duration produced higher response rates during the changeover delay than after the changeover delay had elapsed; changeover delays of variable duration produced such differences to a lesser extent. It was concluded that the changeover delay in concurrent variable-interval schedules of reinforcement functionally acts as a delay period to the next opportunity for reinforcement, possibly serving as a conditioned reinforcer for the behavior preceding it (the interchangeover time) and as a discriminative stimulus for the behavior in its presence (response rates during the delay).     

Preference between fixed-interval and variable-interval schedules of reinforcement: Separate roles of temporal scaling and predictability

Pigeons’ preference between fixed-interval and variable-interval schedules was examined using a concurrent-chains procedure. Responses to two concurrently available keys in the initial links of the concurrent chains occasionally produced terminal links where further responses were reinforced under either a fixed- or variable-interval schedule. In previous studies, preferences for the variable schedule with such a procedure have been interpreted as reflecting atemporal scaling process that heavily weights the shorter intervals in the variable schedule. The present experiment examined whetherpredictability, i.e., the presence of external stimuli correlated with the reinforcement interval, might also influence preference in such situations. When the two intervals in a variable schedule were made predictable by being associated with different key colors, preference for that schedule increased. This increase was reliable but small in magnitude and transient when initial-link responses only occasionally produced terminal links; it was large in magnitude when only one response in the initial link was required to produce the appropriate terminal-link schedule. The results suggest that preference between fixed and variable schedules may be influenced both by temporal scaling and to a lesser extent by predictability of the reinforcement intervals.     

The maintenance of matching behavior when contrast occurs in multiple concurrent concurrent schedules of reinforcement

Pigeons were trained on a multiple schedule of reinforcement in which each component was a concurrent schedule. The concurrent schedules were programmed by the changeover-key procedure. The primary purpose was to determine if the relative behavior allocated to two response alternatives is affected when absolute changes in these behaviors occur; i.e., to determine if matching is affected when positive behavioral contrast occurs. Results showed that (1) relative behavior in the unaltered component of the multiple schedule is not disrupted when positive contrast occurs in that component, (2) positive contrast occurred when the overall frequency of reinforcement in the reinforcement-correlated component(s) was high, but not when it was low, (3) changeover behavior was susceptible to positive contrast effects, and (4) changeover contrast and food-key contrast are independent phenomena.     

Sum of responding as a function of sum of reinforcement on two-key concurrent schedules

Four pigeons pecked for food reinforcers delivered by several two-key concurrent schedules. The sum of the rates of reinforcement provided by the component schedules varied from 25 to 300 reinforcers/h. The ratio of the rates of reinforcement remained constant at 1:4. The sum of the rates of responding generated by the component schedules increased with increases in the sum of the rates of reinforcement which the components provided. The increase in response rate was predicted by equations proposed by Catania (1963) and by Herrnstein (1970). But the data conformed more closely to Herrnstein’s equation.     

Herrnstein’s equation for the rates of responding during concurrent schedules

A modification of the nonlinear curve-fitting procedure proposed by Wetherington and Lucas (1980) was used to assess how well Herrnstein’s (1970) equation for the rates of responding during concurrent schedules described performance. The equation fitted some results very well, accounting for 80% or more of the variance in the data in studies that used moderate-duration changeover delays and provided the same positive reinforcers, operanda, and simple schedules in the two components. The equation fitted the data poorly in other studies. The k parameter changed with several variables; it was not as constant as Herrnstein (1974) suggested. R₀ did not fit Herrnstein’s interpretation as reinforcement from unprogrammed sources. Forty percent of all values of R₀ were negative, and another 23% were unreasonably large (greater than 50 reinforcers/h). The data suggest that Herrnstein’s equation is not a general theory of concurrent-schedule responding, and that Herrnstein’s interpretation of k and R₀ should be modified.     

Effects of signaled reward in instrumental conditioning: Enhanced learning on DRL and DRH schedules of reinforcement

The effects of signaled reward were examined using DRL and DRH schedules of reinforcement. In each case, one group of rats received a brief cue between the reinforced response and the reward, and a second group received brief cues at random times. With the DRL schedule (Experiment 1), signaled reward decreased response rate, increased response efficiency (number of responses per reinforcer), and increased resistence to satiation relative to the control group. With the DRH schedule (Experiment 2), signaled reward increased response rate, efficiency, and resistance to satiation. These results refute an overshadowing explanation of the effects of signaled reward and suggest that food-correlated cues enhance learning of the reinforcement contingencies.     

Within-session changes in responding during concurrent schedules that employ two different operanda

Five rats responded on several concurrent schedules in which pressing a key produced reinforcers in one component and pressing a lever produced reinforcers in the other component (Experiment 1). Four pigeons responded on several concurrent keypeck treadlepress schedules (Experiment 2). The programmed rates of reinforcement varied from 15 to 240 reinforcers per hour in different conditions. Rates of responding usually changed systematically within experimental sessions, and the changes were similar for the two components of a concurrent schedule. These results imply that within-session changes in responding may not confound the predictions of theories that describe the ratio of the rates of responding during the two components of concurrent schedules. Instead, within-session changes may be controlled by a mechanism that integrates the reinforcers obtained from the two components.     

The effects of signaled reward on sign tracking and response rate

If, while responding on a variable interval schedule, rats are given a brief cue prior to reward, their response rate is markedly lower than the rate for yoked partners who receive the cue randomly with respect to reward. This signaled-reward phenomenon has been explained in terms of sign tracking. Two experiments reported here replicated the phenomenon and examined sign tracking directly through visual inspection of the animals’ behavior. Although sign tracking did, indeed, occur more in the signaled reward condition, it did not fully account for the difference in response rates.     

Effect of intertrial interval on variable-interval discrete-trial barpressing

In seven experiments, an effect of the intertriai interval (ITI) duration on barpressing by rats was studied. A stimulus signaled a 15-sec variable-interval trial. The first response after the interval elapsed turned the stimulus off and was rewarded with food. Trials were separated by long (about 300 sec) or short (about 10 sec) ITIs. A within-subjects design established that response rate on trials after long ITIs was lower than that after short ITIs (Experiments 1 and 3–7). The effect was not cumulative (the effect of one and five consecutive short ITIs was the same). Response rate after short and long ITIs was the same when a between-subjects design was used (Experiment 2). Response rate was higher after 160-sec ITIs than after 300-sec ITIs, suggesting that the ITI duration at which all longer ITIs are treated the same (i.e., the upper limit) is greater than 160 sec (Experiment 3). When food, the trial stimulus, a novel stimulus, or a familiar stimulus never paired with food, was presented 10 sec before the next trial during some of the long ITIs, response rate on the next trial was similar to that found after 10-sec ITIs (Experiments 4–6). This similarity suggested that these events could mark the start of the ITI. However, the familiar stimulus did so only when it reliably predicted that the next trial would occur after a short interval. The effect of ITI duration on responding was apparently attributable to response latency. Response latency was greater after long ITIs, but once responding began, it was similar after long and short ITIs (Experiment 7).     

Microanalysis of variable-interval performance during stimulus compounding

Additive summation is observed when more responses are emitted to the simultaneous presentation (tone-plus-light) of independently conditioned stimuli (tone and light) than to either stimulus presented alone. The current experiment sought to determine if this increased rate during tone-plus-light was a function of a new modal interresponse time (IRT) or a differential mixing of pauses with a modal IRT characteristic of the responding in tone and light alone. Three rats were trained on a three-component multiple schedule where tone and light were each associated with a variable-interval 30-sec schedule while a variable-interval 100-sec operated in the simultaneous absence of these stimuli, tone-off and light-out. Baseline response rates were 2–4 times as high in tone or light as in their absence. In testing, more responses were emitted to tone-plus-light than to tone or light by all animals, but the modal IRT was in the 0.2–0.4-sec IRT bin for all test conditions. Tone-plus-light controlled fewer long IRT values and more responses in the short modal category than tone or light alone. These results support the response mixing hypothesis of stimulus control; i.e., no “new” behavior was observed during the novel combination of stimulus elements, only a mixture of previously reinforced behavior patterns in different proportions.     

Effects of reward amount and reward omission on short-term retention

A series of four experiments studied the retention of the response made on a just-preceding trial as a function of the presence, and amount, of food reward given on that trial. Rats were trained to alternate arm choices in a T-maze, and then were tested for alternation with 5- or 30-sec delays between runs. When the subjects had received prior experience with the reward amounts used in testing, larger rewards led to better retention than did small or no rewards. However, when reward omission first occurred during the test phase, it produced more alternation on the following trial than did reward presence. The results suggest that both reward amount and surprisingness determine short-term retention of responses paired with the rewards.     

Optimization of project payment schedules with Nash equilibrium model and genetic algorithm

To minimize the deviations of the net present values of project payment for both the owner and the client and optimize project payment schedules, a Nash equilibrium model based on game theory was set up and a genetic algorithm was developed to work out the Nash equilibrium solution with a two-stage backward inductive approach that requires the client responds to the owner’s payment schedule with an activity schedule so as to maximize the client’s net present value of cash flows. A case study demonstrated that a payment schedule at the Nash equilibrium position enables both the owner and the client to gain their desirable interests, thus is a win-win solution for both parties. Despite the computation time of the proposed algrithm in need of improving, combining Nash equilibrium and genetic algorithm into a complete-information dynamic-game model is a promising method for project management optimization.     

Differential punishment of other behavior: Effects of pause-dependent shock on variable-interval performance

Three albino rats were exposed to a differential-punishment-of-other-behavior shock schedule superimposed on a variable-interval schedule of food presentation. With this arrangement, failure to barpress for a specified interval since the previous response resulted in delivery of shock. This procedure reliably decreased the number of specified pauses and was accompanied by increases in barpress rates and shifts in the distribution of responses. Subsequent exposure to noncontingent shock produced similar, but quantitatively smaller, changes for two of the three subjects. Training with barpress-dependent shock and reexposure to noncontingent shock further diminished these effects. This study demonstrates that pauses are conditionable units of behavior insofar as they are sensitive to a punishment contingency. In terms of the targeted unit of behavior, the current findings are similar to those obtained with barpress-dependent or interresponse-time-dependent shock schedules and suggest a continuum of response specification.     

Memories of reward events and expectancies of reward events may work in tandem

A common assumption is that expectancies of reward events in instrumental tasks are established on the basis of Pavlovian conditioning. According to the tandem hypothesis, tested in the four runway investigations reported here employing rats, memories of reward events may serve as the conditioned stimuli eliciting expectancies. In Experiments 1–3, rats were trained under a schedule of partial reward (P), which did not produce increased resistance to extinction, and subsequently shifted to consistent reward (C). According to the tandem hypothesis, the shift to the C schedule should result in increased resistance to extinction if, as hypothesized, under the P schedule the memory of reward, S^R, came to elicit the expectancy of nonreward,_EN. This hypothesis was confirmed under a variety of conditions. It was shown that increased resistance to extinction could not be attributed to the P schedule alone, to the rats receiving two schedules, P and C, to stimuli other than S^R eliciting E_N, or to the rats forgetting reward-produced memories when expecting nonreward (Experiment 4). It was shown that the tandem hypothesis could explain the divergent findings obtained in prior studies employing a shift from P to C as well as in the present study.     

Duration of components and response rates on multiple fixed-ratio schedules

Pigeons were exposed to a two-component multiple fixed-ratio X fixed-ratio Y schedule of reinforcement in which X was always less than Y. Components were equal in duration and alternated at rates that varied between 2 sec and 23.5 h. Relative response rate in the FR X component: (1) increased as the duration of components increased between 2 sec and 15 min, (2) was at a maximum between 15 min and 6 h, and (3) decreased as the duration of components increased from 6 h to 23.5 h. The changes in relative response rate were attributable primarily to changes in absolute response rates during the FR Y schedule as absolute response rates during the FR X schedule were relatively invariant. These results pose complexities for several theoretical formulations.