PROBLEM SOLVING BEHAVIOR: AN EXPERIMENTAL EXAMPLE

M. Carmen LUCIANO

Departamento de Personalidad. Universidad de Granada

This article analyzes a type of Problem Solving Behavior in which (a) the responses were chained by direct contingencies, (b) a new response was included (one of the behavior chained was, then, new), and (c) all individual responses were maintained under the same motivational condition as that which controlled in the problem solving test. The first experiment systematically replicated Epstein's 1985 study by separately training a pigeon to push a box towards a green spot, after non-directional pushing, to climb and stand on the box, and to peck a facsimile of a banana in the bird's reach. Differences with regard to Epstein's study were: no extinction of flying or jumping with the banana out of the bird's reach, using a naive pigeon. In the test (with the banana out of reach and the box on the opposite side of the chamber) the pigeon stretched repeatedly towards the banana, pushed towards it, stopped under it, climbed, and pecked it several times.

Experiment 2 evaluated how the different responses contributed to problem solving. A test was conducted after the training of each behavior. Four pigeons participated; two of the four pigeons were trained to push a box towards a target and two were trained to push the box away from a target in and attempt to isolate the stimuli controlling directional pushing in the test. In each test, only responses acquired previously, either directly or by generalization, appeared during each test. Pigeons trained to push towards or away from the spot, pushed, respectively, towards or away from the banana in the test, except for one pigeon which never pushed in any direction. No flying or jumping appeared. The test conducted after all the responses had been trained (Final Test) showed sequences with all the possible responses, but the order in which they appeared formed unsuccessful or "almost" successful chains ("almost" because the pigeon stood on the box with its back to the banana, but pecking did not occur: the banana was not in the bird's visual field). Some factors likely to explain these results are discussed as a function of: (a) the comparative rates of reinforcement among the possible responses in the test, which probably accounts for the order of the chained responses (and hence, of the successful or unsuccessful chains), (b) the schedule of reinforcement for each behavior, which probably gave rise to successful connected chains (one response quickly followed by another) or successful but disconnected chains (unnecessary repetitions of the responses constituting the solution to the problem). Along with evidence from other studies these results led to a consideration of different types of successful problem solving.

Key Words: Chaining, novel behavior, variability, motivational conditions, functional stimulus control, problem solving behavior, contingency-shaped behavior, pigeons.

RESUMEN

Conducta de solución de problemas: Ejemplo experimental. Este artículo analiza un tipo de solución de problemas en el que (a), las respuestas se encadenan mediante contingencias directas, (b), se incluye una nueva, y (c), todas las respuestas individuales se mantienen bajo la misma condición motivacional del test de prueba. Los sujetos son pichones. En un primer experimento se replica el de Epstein (1985), encontrándose algunos diferentes resultados. Un segundo experimento evalúa la contribución de las diferentes respuestas. Se discuten los hallazgos, y se plantea la consideración de diferentes tipos de solución de problemas hasta ahora contemplados en la literatura.

Palabras Clave: Encadenamiento, conducta nueva, variabilidad, condiciones motivacionales, control de estímulo, conducta de solución de problemas, conducta moldeada por contingencias, pichones.

There have been many different definitions and explanations of Problem-Solving Behavior from different standpoints. From the perspective of behavior analysis a basic definition of problem solving is given, more or less explicitly, in Skinner (1969) and Bijou (1976) -an extended account is given in Luciano (in press). Hence, Problem Solving behavior can be defined as a new sequence of several links (behavioral segments in the subject's repertory) which are chained by direct contingencies and which culminate in an adaptive, functional consequence related to the current motivational condition defining the problem-solving situation. The double function of the stimulus (as a discriminative controlling factor and as a consequent event) is the key to the chaining of responses, according to the usual conceptualization of this process (e.g. Reynolds, 1968).

Accounts of problem solving usually often do not differentiate between problem solving involving direct contingencies and that which also involves rules, a distinction made by Skinner (1969). Most available studies emphasized the latter, which essentially requires research with human subjects, while studies involving direct contingencies might be a first step towards a parsimonious understanding of the phenomena. The most frequently cited study in problem solving (Kohler, 1925) reported chimpanzees solving problems by suddenly using sticks to reach food (insightful performances) or reaching the food by jumping (brute-force). Maier (1931) provided data about chaining different elements in a new sequence with rats; previous experience with the elements involved in the problem solving were reported by Birch (1945); Schiller (1952) found that general experiences were not sufficient for chimpanzees to solve the problem; the specificity of previous training for successful solutions was a finding addressed by Shurcliff, Brown, and Stollnitz (1971) with rhesus monkeys. Epstein , Kirshnit, Lanza, y Rubin (1984) and Epstein (1985) showed problem solving with pigeons through response chaining. Interpretations have been diverse. The present expands empirical evidence of problem solving by direct contingencies without rules. As the specific procedure of the present study is based on the last two studies, they will be described in more detail.

The "interconnection of two repertoires" (Epstein et al., 1984) was considered a replication, with pigeons, of Kohler's experiments with chimps (p. 61). Different histories were created with different pigeons; some were trained to push a box towards a green spot on the wall (directional pushing), and to climb onto the box and peck a facsimile of a banana suspended overhead, with the extinction of flying and jumping when the banana was out of the bird's reach; with other pigeons training involved only some of these responses. Each pigeon was then tested with the banana suspended out of reach and the box placed in the center of the chamber. It was concluded that all the repertoires described were necessary for successful problem-solving (pushing the box towards the banana, climbing onto the box and pecking the facsimile of banana, which was followed by access to food). The performances under such circumstances were called "insightful" (1984, p.132).

In "the interconnection of three repertoires" (Epstein, 1985) "a pigeon acquired three separate behaviors: (a) climbing, (b) pushing towards targets, and (c) pecking the banana. When the pigeon was confronted with the problem, a swift but erratic and not especially "insightful" solution emerged" (p. 131). Hence, the performances in both studies were not the same, as Epstein himself (1985, p. 140) indicated. Therefore, from our point of view they were not replications of Kohler's experiments (Luciano, in press).

Epstein (1985) assumed that some principles (reinforcement, extinction, resurgence and automatic chaining) "should predict different performances as a function of relevant parameters: the genes and history of the individual, current stimuli and the manner in which such stimuli are changed over time as a result of the organism’s behavior, and so on. If one knows the transformation functions and the relevant parameters, one should be able to predict where in this range the performance will fall, and to provide a detailed probability profile of the succession of behaviors that will appear" (p. 140).

In spite of the preceding data and assertions, this topic requires further experimental support for the following reasons. In the study by Epstein et al. (1984), two responses were chained in several pigeons, and the necessary components were usually evaluated between subjects. Epstein's (1985) study is of a different type, and the effect of all the responses in the problem solving test was evaluated in only one pigeon. Neither study said anything about the specific training criteria before the test, or about the functional role of the extinction of flying and jumping, which was emphasized as a component for successful problem solving. Neither paper provided training graphs. Without this information it is difficult to evaluate the specific function of each stimulus in order to analyze different types of problem solving behavior.

The goals of this study were four-fold: First, to replicate systematically Epstein's (1985) study. Second, to provide information about the performance in the problem solving test after the acquisition of each individual behavior. Third, to identify the current stimulus which would control directional pushing in the test (when the spot is not present), i. e. the most important behavioral segment for successful problem solving. Finally, to elucidate the functional roles of jumping and flying.

METHOD

Definition of the Problem-Solving Behavior and the Test Conditions.

The conditions for problem-solving testing involved a facsimile of a banana hung out of the bird's reach, a box on the opposite side of the chamber, and a healthy but food-deprived adult pigeon. Successful problem-solving was defined as a chain composed of pushing the box towards the banana, stopping the pushing when the box was under it, climbing onto the box and pecking the banana.

Subjects and Apparatus

Five experimentally naive adult male White Carneaux pigeons were individually housed with water available and maintained at 80-85 % of their free-feeding weight.

All sessions were conducted in a rectangular chamber (85 x 35 x 50 cm high) made of light grey wood for the floor, back and side walls, a transparent plexiglass front wall and wire mesh for the ceiling. At the base of the back left wall was an opening (6 x 5 cm) through which food was made available. During food presentations, a white 6 W bulb illuminated the hopper aperture. Extraneous sounds were masked by white noise and by sound-attenuating chamber in which the experimental space was located. Two small cardboard boxes, one cylindrical (8.5 cm in diameter and 7.2 cm high), and one almost rectangular (7.8 cm high and 9cm square); a facsimile of a banana measuring 6cm long by 1.5 cm wide, made of dry yellow Play-Doh material attached to a grey wire; and a round fluorescent green cardboard circle 3.5 cm in diameter, were all used. A video Camcorder mounted on a tripod recorded some training sessions and all test sessions.

Description of the experiments

Experiment 1 . The goal was to replicate Epstein's (1985) experiment "the interconnection of three repertoires", but in this case with a naive pigeon and without extinguishing flying and jumping when the banana was out of the bird's reach. Figure 1 shows the training sequence of each behavior.

After the Initial Test (see figure 1), directional pushing training began. This involved previous non-directional pushing training. Next pecking the banana was shaped, then climbing onto the box. Different training conditions consisted of daily session in which more than one different type of responses was trained. When the training criterion was achieved with all three types of responses, problem-solving was evaluated in the test conditions.

Experiment 2. The goals of this experiment were twofold. The first was to isolate the effect of each type of response upon the performance in the problem-solving test; and was evaluated by analyzing the responses produced under the test conditions after the acquisition of each type of responses (see Fig. 1). Four naive pigeons participated and the order of training of the first two types of responses was balanced. The second goal was to isolate the stimulus controlling directional pushing under the test conditions; this was evaluated by contrasting the directional pushing produced in the test, when two of the four pigeons were trained to push towards the spot, and two were trained to push away from the spot.

As Figure 1 summarizes, after the Initial Test, non-directional pushing was trained first in p. 22 and p. 55, until the training criterion was reached; this was followed by a second test (Test A). However, pecking the banana was the first type of responses trained previously to Test A in p. 44 and p. 33. Nondirectional pushing was then trained (p. 33 and p. 44) or maintained (p. 22 and p. 55), and pecking the banana maintained or trained, respectively in Different periods of each session; Test B followed. Pushing was trained towards (p. 22, p. 33) or away from (P55, p. 44) the spot; pecking the banana was maintained, and pushing the box in the absence of the spot was not reinforced. Test C followed. Finally, climbing was trained and pecking the banana as well as pushing towards or away were maintained. The Final Test followed in all the subjects except p. 55.

One additional test was carried out respectively with p. 22 and p. 44. A test was conducted before the Final Test with p. 22 to evaluate the effect of climbing and standing on the box with the head bowed (Pre-Final) and with the body upright (Final Test). The additional test (Test A-Ext) conducted with p. 44 preceded Test B; the goal was to evaluate the effects of the extinction of flying or jumping with the banana out of the bird's reach several sessions were conducted in which responses to the banana when it was out of the bird's reach were not reinforced, and pecking the banana when it was in the bird's reach was reinforced, Test A-Ext followed.

see Fig. 1. Training sequence of the different behavior and problem-solving tests.

Procedure

Sessions were carried out seven days a week. Each session was concluded when a specified number of reinforcers was consumed. This number was based on the pigeon's daily weight to maintain bird at 80-85% of its free feeding weight (range 45-55 reinforcers per session). The feeder operation was started by hand, with an automatic duration of there seconds. The pigeon was always put in the chamber with the light off. After twenty seconds the light was switched on and the session began. The light was off whenever the arrangement of the training material in the chamber had to be changed.

The adaptation period in the experimental chamber involved several sessions when the pigeon was in the chamber without any training material. This period concluded when the pigeon reached 80-85% of its free feeding weight, and when the sound produced by activating the hopper was a discriminative stimulus, judged by when the pigeon moved towards the feeder as soon as the sound was produced. The Initial Test followed.

Each test comprised one, two, or there consecutive trials, each involving different locations of both the banana and the box. Just before each test, except in the Initial one, there was a period in which the different, previously trained, types of responses were individually reinforced two or there times. A training session followed after each test.

Training Non-Directional Pushing . The pigeon was trained to push a box around die chamber with only the box present in the chamber. First pecking any part of the box was reinforced, but the reinforcement criterion was rapidly changed to pecking one of its upper comers in such a way as to rotate the box. Pecks in the upper comers that rotates the box were defined as correct pushes while any other pecks were considered incorrect, and were not reinforced. Correct pushes were maintained on variable ratio 5 (VR5) and calculated per session. Training ended when there consecutive sessions were conducted with 50% correct pushes each.

Training to peck the facsimile of banana. The banana was suspended from the ceiling of the chamber so that it was just overhead. Its position was changed after five or six reinforcements. Pecks to the facsimile were at first continuously reinforced; the criterion was quickly changed to intermittent contingencies (VR4). Criterion training was attained after there consecutive sessions on VR4. The number of pecks and reinforcements per session were recorded.

Training Pushing towards the Spot. Stimuli present were the box and the green spot (placed at different sites at the bottom of the walls so as to be visible even when the box was directly below). Correct directional pushing was defined as a sequence of responses beginning with pecking the spot or head facing the spot, followed by pushing the box towards the spot with responses to the spot in between (pecking it or head facing it), until the box was next to the spot, with the final component being "pecking the spot" or "head facing the spot". That is, directional pushing involved moving the box towards the spot regardless of the position of the box and the spot. Hence, a pushing sequence involved responses to the spot and the box without interruptions. Any other sequence was defmed as incorrect. Correct or incorrect sequences as well as responses to the box only, or responses to the spot only, were recorded per opportunity in each session.

An opportunity was defined under any of the there following conditions: (a) when the light was on at the beginning of this training period, (b) after a reinforcement occurred, that is, when the hopper was closed, and (c) after any behavior other than one involving the spot and the box, such as fluttering, scratching, moving around, or simply standing The percentage of opportunities in which correct directional pushing sequences were recorded and the percentage of opportunities in which only responses to the box were recorded were both used for observing shaping. The number of times the pigeon climbed onto the box during directional pushing training was also recorded.

Shaping ran as follows. After a few sessions in which pecking the spot was reinforced without the box present, the box was mounted on a wire which allowed movement of the box in a straight line; but the box was initially placed near the spot and a piece of elastic band around the wire prevented the box from being moved too far away from the spot. First, pecking the spot plus one response to the box regardless of the topography was reinforced; second, pecking or facing the spot plus one push in the direction of the spot was reinforced. The distance between the spot and the box was gradually lengthened, so that the number of responses to the spot and to the box increased. Third, a new component for reinforcement was incorporated when the pigeon pecked or faded the spot and moved the box from different locations towards the spot: pecking or head facing the spot when the box reached the box (the correct sequence as defined previously). As directional pushing was first shaped with the pigeon's head tilted to one side, the fourth step was shaping directional pushing with the head to the other side. In one of the pigeons (S33), the head topographies were shaped using an errorless discrimination procedure, which consisted of sticking a green spot on a side of the box which was gradually faded out (so that pecking occurred on the appropriate side of the box), while keeping the usual spot on the wall. Correct pushes were quickly produced with less variability than in the other subjects.

When correct directional pushing sequences (with both head topographies and the box still connected to the wire) reached 50%, an extinction period of pushes and pecks to the box in the absence of the spot was introduced alternating with periods of reinforcement of correct directional pushing sequences. When correct sequences reached 80% in one session, the wire was removed and the box was placed close to the spot and then gradually moved further away until the pigeon pushed the box towards the spot from the middle of the chamber, whatever the position of the spot in the wall. The training criterion was attained when, under the last conditions described, 80-100% correct directional pushing occurred in three consecutive sessions.

Training Directional Pushing away from the Spot. The main difference with respect to pushing towards the spot was that the pigeon pushed the box in the direction away from the spot. Hence, the correct sequence was defined as pecking or head facing the spot followed by two or three pushes from the spot, at which time the sequence was reinforced.

Training to Climb onto the Box. The box was fixed in position on the floor of the chamber and neither the spot nor the banana were present. ]le position of the box was changed during a session. Correct climbing was defined as climbing onto the box and standing with the body upright during three to five seconds. A fading stimulus discrimination procedure was used with respect to the box (the height of the box was gradually increased until it was the same as the box used in pushing training). Shaping proceeded as follows: approaching the box head first was initially reinforced; then, placing one foot onto it, placing both feet onto the box, keeping the body upright, and finally increasing the time spent until the pigeon remained on the box for 3-5 sec. An opportunity for climbing began when the pigeon touched the box with any part of the body and ended when it moved away from the box. The percentage of correct climbing was recorded taking into account the total opportunities in a session. Criterion was reached when correct climbing was 80-100 % during three consecutive sessions.

Reliability Assessment

Five independent trained observers recorded from videotapes each occurrence of the responses produced in all the tests. Agreement was 100% among all of them. All training sessions-were directly recorded by the experimenter and agreement between these direct data and the filmed sessions was intermittently checked by an independent observer. Agreement was calculated with regard to total correct occurrences of each type of responses in a session (dividing agreements by agreements plus disagreements); with a range of 91 % to 100% across sessions and pigeons.

RESULTS

Figure 2 (a) and 2 (b) show the data corresponding to each type of response throughout training as well as the moment (after achieving training criteria) in which test sessions were conducted.

Figure 2(a) and 2(b). Graphs illustrate the training of each behavior as well as the moment in which each test was conducted:

Non-directional pushing is indicated by the percentage of correct pushes per session during training, and responses to the box per minute during extinction.

Directional Pushing (towards or away from the spot) is indicated by the percentage of correct sequences ( ) and the percentage of sequences with only responses to the box ( ). Responses to the spot per session at the beginning of training are also indicated, and the removing of the wire is showed by an arrow.

Pecking the banana: Pecks to the banana suspended overhead (IN) and flying or jumping when the banana was out of the bird's reach (OUT) -in P44 only.

Climbing: percentage of correct climbing on the box and standing on it with the body upright.

see Figure 2(b).

see Figure 3. Summary of data from each of the trials ( ) in each Test. The different type of responses appearing for the first time and the last observed before ending each trial are indicated, as well as the duration of each trial. Repetitions of a given response are not indicated except the occurrences of complete directional pushing (indicated with a number).

Some of the data will be described; for example, the percentage of correct directional pushing increased when pushing the box (when the spot was not present) was not reinforced. The high percentage of correct directional pushing shows that the spot was controlling pushing. Therefore, the two types of correct pushing (towards or away) respectively in each pigeon illustrate the variables controlling the direction and termination of pushing. Other relevant data are: the double record of climbing with both head topographies in p. 22, and the recordings of climbing in the course of directional pushing training.

Figure 3 summarizes the order in which the different type of responses were produced in the trials of each test. This summary illustrates the order in which each response appeared for the first time, and last occurred upon conclusion of the trial; successive repetitions of a given response are considered, in this summary, as a single occurrence and are illustrated with a single pictogram (except the occurrences of directional pushing which were indicated). However, Figure 4 shows each type of response, occurrence-by-occurrence for two of the Final Test trials with p. 11 (the same type of recordings for all tests and pigeons are available upon request). This detailed figure shows the sequence of the different responses which were at times interpersed with periods during which irrelevant responses for the test were produced as well as the successive repetitions of a type of response. With regard to the first experiment, Figure 3 shows the Final Test conducted after the three types of responses had been trained. p. 11 stretched repeatedly towards the banana, pushed directionally towards it, stopped under it, climbed, and with the banana in its visual field, pecked it several times. No reinforcement followed this chain of responses (successful by definition but not functional for the subject in the absence of reinforcement). Two more trials were conducted to evaluate the point at which the chain could be disrupted in the absence of explicit reinforcement: the different responses appeared repeatedly. Two findings to be emphasized are, on the one hand, after complete directional pushing in the third trial, p. 11 climbed and stood on the box for a while without pecking the banana because it was not in its visual field (an almost successful problem solving). On the other hand, no flying or jumping were observed at any time.

Figure 3 shows the responses which appeared in each test in experiment 2. With regard to Tests A and B responses trained previous to each test appeared respectively; and p. 44, which was exposed to an additional test (Test Ext-A), did not fly or jump at any moment, but stretched towards the banana, as it did during Test A.

Variability was evident beginning in Test C (directional pushing and pecking the banana were the type of responses previously trained). In the first trial, p. 22 stretched towards the banana and pushed towards it, stopping when the box was under the banana (one complete directional push). In the second trial, incomplete directional pushing, stretching to the banana, climbing onto the box, stretching again, and a complete directional push, all occurred; and, in the third trial, p. 22 showed again complete directional pushing. p. 33 stretched towards the banana in all three trials, showed one complete directional push but only in the.. first trial, and incomplete directional pushing in the third trial, while non-directional pushing appeared in all three trials. p. 55 stretched towards the banana and pushed away from the banana five and two times respectively in each trial, followed by stretches towards the banana. Finally, p. 44 showed nondirectional pushing and stretching towards the banana, but at no time pushed the box towards or away from the banana. Hence, each subject, in test C, showed the type of responses possible from those previously trained.

see Figure 4. Occurrence-by-occurrence record of each response for the first and third trials in the Final Test of p. 11.

Pre-Final Test, conducted only with p. 22, shows all the responses trained up to that moment. The most relevant data from this test were that, in accordance with its previous training, the pigeon kept its head bowed once it had climbed onto the box, which prevented visual contact with, and hence pecking of, the banana in the second and third trial (an "almost" successful problem solving).

Before the Final Test all subjects except p. 55 (which did not received climbing training) had acquired three types of responses (pecking the banana, climbing and standing with the body upright, and pushing towards or away from the spot). p. 22 and p. 33 showed all three, but directional pushing was only complete in p. 33. P. 44 stretched towards the banana, climbed, and show non-directional pushing (in the second trial), but as in Test C, this subject's weak pushing was not under the control of the banana.

To synthesize, replication was obtained in the first trial with p. 11. Pigeons 22 and 33 showed a lot of variability in Test C and in Final Test. In Final Test, after all the responses necessary for correct problem solving had been trained in p. 22 and p. 33, none of them showed correct problem solving because of incomplete directional pushing. p. 44 and p. 55 never were supposed to show the correct problem solving because they were trained to push away from targets (to isolate the stimulus controlling pushing), and so p. 55 did in Test C while p. 44 never push away from the banana.

A final interaction was conducted with p. 22 after the Final Test, which in conjunction with the data of the Final Test with p. 11, provide information about the maintenance of the correct problem-solving (pushing towards the banana, climbing, and pecking it) under two conditions: when pecking the banana (the last response of the correct chain) was the only one reinforced (with p.22) and when it was not reinforced (as happened with p. 11). With p. 22, during two sessions after Final Test, only the correct chain of responses was reinforced, but incomplete directional pushing was very frequent and the occurrence of the correct chain was not very frequent (as also happened in Final Test with p. 11 during the second and third trial). Hence, a period was introduced in which only the last two behavioral segments of the correct sequence (climbing with the box under the banana and pecking it) were reinforced in order for the correct chain to appear easier. The two responses were produced quickly and were reinforced 23 times in a session. Testing followed in the next session, and 13 correct chains appeared which were reinforced, but again, mixed with incomplete directional pushing, repeated climbing and stretching. A relevant finding was that jumping and flying towards the banana appeared two times, by the first time, during this process when the box was placed far from the banana.

DISCUSSION

Results from the replication experiment show that the correct chain of responses defined as correct problem solving is also possible under the present experimental conditions (first trial), which were not the same as in Epstein's (1985) work because his study included a period in which jumping and flying were extinguished, the present was not; and the present study included a final component of directional pushing, e.g., an explicit response to the spot when the box reached the spot. This replication, in conjunction with data from Epstein et al. (1984), Epstein (1985), and data from the second experiment, shed some light on more explicit differences between all these problem solving performances discussed later. The type of successful problem-solving behavior used in this study is formed by several behavioral segments, two of which were explicitly trained and one of which was new ( because directional pushing was trained under the control of the spot, while under the test conditions the spot was absent and directional pushing was produced under the control of the banana). An additional characteristic of the type of problem solving studied is that all three behavioral segments were individually trained but under the same motivational condition (food deprivation), which is also the specific motivational factor involved in the problem-solving test. So, we first examine the data which evidence the appearance of the new behavioral segment; second, the variables responsible of this new behavioral segment; and third, the conditions under which the successful problem solving appear indicating those necessary conditions but hypothesizing the sufficient ones.

First, the type of directional pushing training helps to isolate the stimulus which controlled pushing in the test, since three pigeons were trained to push towards the spot and two pigeons were trained to push away from it. Hence, the topography of pushing with respect to the spot was different, and so were the topographies with reference to the banana in the test: p. 22, p. 33 and p. 11 pushed towards the banana, while p. 55 pushed away and p. 4 never pushed.

The variables responsible for stopping pushing under the banana (the complete directional pushing) were the same as those which led to stopping pushing with the spot, something which has been isolated by the different criteria for reinforcement corresponding to each type of directional pushing training: proximity of the spot to the box in pushing towards the spot, and distance between them in pushing away from it. Proximity was shaped requiring a final response to the spot (head facing it or pecking it) to activate the feeder, as a consequence of this training, when the box reached the spot, the pigeon pecked it and moved towards the feeder even before it was activated, which is similar to what happened under the test conditions when the pigeon continued to gently push the box within the area under the banana with stretches towards it in between pushes; moreover, when the pigeon moved the box far from the area under the banana, the direction of the pushes was changed to the area under the banana. On the other hand, p. 55 showed the same pushing topography in the test as did during training. Hence, the two topographies of pushing (towards and away) yield information on the stimulus which controlled pushing as well as for stopping pushing in the test.

Second, the new behavioral segment (pushing towards the banana) is necessary for correct problem solving. Epstein et al. (1984) and Epstein (1985) cited this phenomenon as "functional generalization". He wrote that "pecking the banana" and "directional pushing" were "apparently the right reasons for functional generalization" (p.62 and p.139, respectively). The present research shows that although these factors are indeed necessary, the sufficient conditions have yet to be identified, because one of the five pigeons in this study did not show functional generalization even when the two responses specified by Epstein had been trained. A parsimonious hypothetical argument follows regarding the sufficient conditions for this lack of generalization, based on the data obtained.

The main difference between p. 44 and the other pigeons lies in the five sessions conducted with p. 44 (before Test A-Ext) for the extinction of flying or jumping, although none of these responses appeared, hence what was really not reinforced during these sessions was stretching towards the banana. This variable can be ruled out to explain the lack of functional generalization on the basis on data from previous studies : functional generalization was reponed in the pigeons by Epstein et al, (1984) and in the pigeon in Epstein (1985), when all those subjects had extinction sessions like those conducted with p. 44. Hence, other conditions have to be responsible for functional generalization.

Common contingencies between training and test conditions were verified in this study : correct directional pushing (towards or away from the spot) began in all five pigeons with the head facing the spot (as the most frequent type of response to the spot in directional pushing training) which was also the most frequent type of final response of directional pushing in p. 33, while pecking the spot was the most frequent final response in p. 22 and p. 11. No final response to the spot was shaped in pigeons trained to pushed away from the spot. In the test, directional pushing began after several stretches to the banana. An equivalence between "stretching to the banana" and "head facing the spot", the first response of directional pushing could be involved. Both responses had been intermittently reinforced (while training pecking the banana and while training directional pushing, respectively), and this could be one of the variables for which the banana would have acquired the functional properties of the spot in the test. However, in spite of having this variable in common with the other pigeons, p. 44 did not push under the control of the banana. Hence, an additional common factor between the spot and the banana was verified: the number of direct responses to the spot (pecking it) reinforced, compared with the number of indirect responses to the spot (head facing it) reinforced while directional pushing was trained. The number of direct responses to the spot could be the most relevant factor here, because only direct responses to the banana had been reinforced during training.

Significantly, p. 44 differed from the others with respect to this factor: the percentages of directional pushing reinforced with the direct response to the spot were, respectively 78.3 (p. 11), 82.7 (p. 22), 47.9 (p. 33), 48.7 (p. 55) and 27.7 (p. 44). The type of topography reinforced with p. 44 with respect to the spot in most of the directional pushing sequences was "head facing the spot" rather than "pecking the spot". Moreover, occurrences of functional generalization in the tests was correlated with this variable in all the pigeons (the more directional pushing with direct responses to the spot were reinforced, the more directional pushing in the test). However this is only a hypothesis based on the observations, and specific further manipulation of those variables will be needed to account for this phenomenon.

Third, with respect to the conditions under which the successful problem solving chain appears results from tests in experiment 2 show that the pigeon's performance in each test depends on the previous responses acquired which provide the functional properties to the stimuli and the motivational condition, both present throughout the test conditions (see Tests A and B). However, variability was evident beginning in Test C and in the Final Test. The analysis of the conditions under which each response was maintained, and those of the Test conditions, allow us to interpret the data with respect to the order of appearance of each response in the test. Let us examine the different order that may arise as a function of the corresponding history of interactions.

When confronted with the test conditions, the first response to occur among all those possible in Test C and the Final Test might be:

(a) If visually confronted with the box without visual contact with the banana, then a few non-directional pushes in Test C. However in the Final Test climbing would appear because of the box would be the discriminative stimulus for it;

(1) Recorded as complete and incomplete directional pushing.

(b) If visually confronted with the banana without visual contact with the box, or if visually confronted with the banana and the box but with the variables responsible for functional generalization not operating, then stretching towards the banana in Test C and stretching or climbing in the Final Test;

(c) If visually confronted with the banana and the box, with the variables responsible for functional generalization operating, then directional pushing (which included stretches to the banana) towards or away from the banana in Test C. However, the Final Test, the test conditions would be functional for all three individual types of responses previously trained (pushing towards or away from the spot, climbing, and stretching towards the banana). Under such circumstances, the first response to appear would predictably be directional pushing (after some stretches to the banana) because this sequence of responses had the longest history of reinforcement (strength) with respect to the other responses (see Table 1, e.g., in p. 22, 2.183 directional pushing sequences were reinforced before Final test, compared to 374 responses of climbing). But, as previously mentioned, directional pushing in the test was new, and variables responsible for functional generalization must be invoked to account for complete and incomplete directional pushes.

In any case, when directional pushing occurred and was not reinforced, other responses would be possible (stretching to the banana or even non-directional pushing in Test C; climbing or stretching to the banana in the Final Test). When complete directional pushing appeared in the Final Test, different responses may follow: one possibility would be stretching or moving the head towards the banana; another possibility after complete directional pushing occurred would be climbing, for two reasons: firstly because the box was the discriminative stimulus for that response, and secondly because this response was also produced in the course of directional pushing training as indicated in Results². When climbing occurred, pecking the banana would be possible if body position was such that the banana was in the pigeon's visual field. However, when the banana was not in its visual field (an "almost" correct problem solving), then standing on the box and climbing could be partially extinguished, and another response, such as directional pushing, would once again be possible. Non-directional pushing would be another possible response, especially in pigeons trained to push towards the spot, because several consecutive responses to the box in directional pushing were necessary before the box reached the spot.

Results in Test C show that the first behavior was pushing towards the banana in pigeons trained to push towards the spot, and pushing away from the banana in p. 55, as expected from this subject's training (p. 44 did not show directional pushing). These nonreinforced responses were followed by the other possible responses, excluding climbing because it had not yet been trained (p. 22 climbed in this trial but this response was frequent during training2 and would hence be labeled accidental climbing because it was not under the experimental control of the box).

Data obtained in Pre-Final Test with p. 22 showed an "almost" correct problem solving according to training because standing on the box with the head bowed was the response shaped when climbing. Figure 3 shows that responses previously trained (directly or indirectly) appeared alternatively in the Final Test, showing variability from one pigeon to another, and from one trial to another. Hence, the order of appearance of the different responses, when more than one was possible, would be variable because of the training history.

(2) (See at the bottom of Directional Pushing graphs in Figures 2(a) and 2(b) where climbing responses are indicated). These data are of value in understanding why, in the Test C and Final Test, pushing towards the banana was intemupted by climbing.

The same happened during the Final Test with p. 11 which showed the sequence defined as correct problem solving, which was not reinforced, in the first trial. More variability appeared in the other trials of the Final Test: an almost correct problem solving occurred in the third trial ("almost correct problem solving because the last component of the chain was not produced since the banana was not in the bird's visual field). This variability could be a consequence of extinction during the tests, and of the way in which each stimulus became functional for any of the possible responses.

Extinction was operating during test conditions: when one response occurred and was not reinforced, other responses (among all these possible) were produced. Ibis point is relevant especially beginning in Test C, because there was more than one trial in each test. To say that extinction was operating is not to say that the pigeon had learned not to respond under such circumstances and to respond in a particular way under other circumstances; these conditions could be ruled out because systematic, sufficient differential contingencies did not exist between the test conditions and training conditions to discriminate the conditions under which reinforcement was available. The pigeons continued to respond with all the responses in the tests, alternating from one to another possibly as a function of the schedule of reinforcement and the conditions available becoming functional for any of them.

It follows from the preceding discussion that the necessary conditions for the sequences defined as correct problem solving are: the individual responses forming the problem solving behavior have to be in the subject's repertoire in order for each physical stimuli under the test conditions to become a functional stimulus for response chaining to occur. The specificity of the individual responses forming the correct problem solving has been demonstrated: p. 55, which pushed directionally away from the banana, would never be able to peck the banana because of the type of directional pushing training. As Skinner indicated, the key point in problem-solving is the construction of discriminative stimuli (1969), or according to Epstein (1985) "interconnection comes about moment-to-moment in time through a variety of processes.... ....One is automatic chaining: one behavior changes the environment or the orientation of the organism and hence produces stimuli that make other behaviors more or less likely" (p. 132).

However, the sufficient conditions have to be tracked down to explain the variability observed; these could well be the strength of one response compared to the strength of another among all these forming the correct problem-solving (comparative rate of reinforcement), and the schedule of reinforcement of each individual response forming the problem solving behavior. The first condition, the comparative rate of reinforcement, is the total number of one type of response reinforced, compared to the total number of another type of response reinforced. This variable helps to predict the most likely behavior. The second condition, the schedule of reinforcement of each behavior, is of value in indicating the frequency of each response under nonreinforcing conditions. In other words, the comparative rate would be probably responsible for the order of the chain, and the schedule of reinforcement would be probably responsible for the repetitions of particular responses which lead to the designation of the resulting chain as connected or disconnected.

Table 1 shows, on the one hand, that the schedules of reinforcement of each of the necessary responses for problem solving were not the same, however each was the same for all the pigeons, except the schedules maintaining the different pushes (towards or away from the banana).

Table 1. Number of responses of each type reinforced prior to each test (at the top); and schedules of reinforcement maintaining each type of response (at the bottom).

This variable may give rise to repetitions of a particular response under extinction, as happened in the Test. Moreover, the strength of each type of response was not the same in any test : the response with the greatest rate of reinforcement previous Test C or Final Test was directional pushing, and it was the most frequent behavior under Test conditions, except in p. 44. However, it was also the most susceptible to extinction because the last component of directional pushing (pecking the spot or head facing it when the box reached the spot) had always been reinforced. This seems likely to be the main reason for the quick appearance of incomplete directional pushing after complete directional pushing was not reinforced. The chain was broken at the point most susceptible to extinction.

To summarize up to this point, the variability observed in the problem solving test was probably a function of the compound effect of different variables when the responses were under extinction: (a) one of the componente forming the problem solving behavior was new, but the sufficient variable(s) which led to pushing towards the banana were not known; (b) the comparative rate of reinforcement, which probably accounts for the order of the chained responses; (c) the schedule of reinforcement of each response, which would probably explain why a given chain is connected or disconnected, and (d) the body position with respect to each stimulus.

Finally, some minor but significant data from the last interactions conducted with P22 after Final Test will be discussed because they offer additional evidence for different types of problem solving behavior. As mentioned, jumping or flying towards the banana (out of the bird's reach) appeared for the first time when pecking the banana, following directional pushing and climbing, was the only response reinforced. These data are interesting because they may shed light on the conditions under which one of the types of performance ("brute-force") in problem solving could emerge. An example of "brute-force" performances appeared in some of Kohler's chimps, and in some of the pigeons in Epstein et al.(1984) in the form of jumping or flying towards the banana hung out of the bird's reach. However, Epstein (1985) reported little jumping and no flying in his pigeon (p. 136), and, none of the five pigeons participating in this study showed any flying or jumping. Hence, brute-force performances may be a result of contingencies in which responses controlled by the banana were more likely than any other responses controlled by any other stimuli. This in fact was the case in the last interactions with p. 22, since the last component of the total correct chain was the only one directly reinforced with food, while the others were conditionally reinforced. The same was the case in the study of Epstein et al.(1984), in which climbing and pecking the banana formed one unique behavioral unit, and directional pushing another unit, both of which were reinforced with food, and both of which involved responses to the banana. Thus, successful but brute-force correct problem solving may be concerned with the motivational condition in the test which would be more compatible with the last behavioral segment of the correct problem solving than with the other responses.

CONCLUSION

This study has analyzed a type of problem-solving behavior which (a) was chained by direct contingencies, (b) contained a new behavioral segment, and in which (c) all the individual responses were maintained under the same motivational condition (food deprivation) as that controlling in the problem-solving test. Therefore, there was nothing in the problem-solving test to functionally indicate which response would be reinforced, because more than one response was possible under the test circumstances. This means that the correct problem solving appearing could justifiably be labeled "trial and error performance", and it is evident from these data that the specific behavioral segments comprising the problem-solving behavior have to exist in the subject's repertory (through specific training or through some type of generalization), although the extinction of jumping and flying were shown to be unnecessary for the correct performance evaluated here.

The variability observed, especially after all the necessary conditions were present for the type of problem solving studied here, made possible to tentatively identify relevant variables in order to explain it. Successful (p. 11); "almost" successful (p. 11 and p. 22), and unsuccessful performances appeared in the Final Test. Our analysis identifies two factors: (1) the comparative rate of reinforcement among all the possible responses in the test conditions, which could give rise to the most likely response. This variable would probably explain the order of the responses chained; (2) the schedule of reinforcement of each of these responses, which would explain the formation of connected (one response quickly followed by another different response) or disconnected (unneccessary repetitions of a response) chains. The combination of these two variables under extinction conditions may, or may not, set the stage for stimuli to be functional for any of the responses possible, thus permitting the appearance of different but successful problem solving, or unsuccessful chains.

These data have to be considered as exploratory because the availability of the different behavioral segments forming the problem solving are not sufficient for successful chaining of the responses (correct problem solving), as could be interpreted from the preceding studies in the literature cited

in the introduction. The sufficient conditions have to be experimentally isolated, but the present study, although demonstrative, provides plausible clues for identifying the variables responsible for correct problem solving, or for incorrect chains. Hence, the way in which the different individual responses have been trained are probably highly relevant to explain different performances in problem solving. For example, the schedule of reinforcement which maintains each type of response, and the strenght of one response with respect to another when both are possible. Furthermore, these data have also provided clues for identifying different types of problem solving traditionally classified as "trial and error", "insightful", "brute-force", and has given definitions based on the order of the different responses in the chain and the repetitions versus nonrepetitions of a given response, as well as on different motivational conditions controlling each response.

To summarize, the components involved in problem solving moment-to-moment are: (a) the organism's biological state and the general context (setting conditions); (b) the specific motivational conditions controlling each response (deprivation or aversive conditions correlated with the specific conditions in the novel situation when testing problem solving); (c) the functional stimulus conditions controlling each response, as a function of the previous reinforcement schedules; and the conditions for chaining, that is, (d) the consequences of each response, which may or may not be functional for the same or another response, as a function of the comparative rate of reinforcement and of the schedule of reinforcement of each behavior (taking into account that the body position acquires a relevant role, as it provides the contact media for the present stimuli).

Further research is needed to clarify and delimit the specific role of the variables described, but we emphasize problem solving experiments proceeding from examples which do not compris new behavioral segments unless the source of the novelty is explicitly identified. The applied importance of basic data in problem solving would enhance programs which provide with information to therapists and people, in general, to make possible successful chains of behaviors instead of unccessful ones or bloking effect in new situations. Therefore, this information could be applied to the appearance of new chains of behaviors other than problem solving, either normal or abnormal behaviors.

ACKNOWLEDGES

This research was completely supported by a Senior Fulbright Fellowship (*85-07422). This work was initiated while a Visiting Scholar in the Cambridge Center for Behavioral Studies, and at Boston University. The experiments were conducted entirely at the Department of Psychology, Boston University.

A summary of this paper was presented at the II European Meeting of Experimental Analysis of Behavior. Liege, Belgium, July 1988.

The author would like to thank to Henry Marcucella and Michael Harrison for all the facilities provided al Boston University which made this research possible, to Jesus G. Roales-Nieto, Carl Cheney and B. F. Skinner for continous support and helpful suggestions during the course of the experimental work; to Juana Gonzalo and Jesus G. Roales-Nieto for tabulating data and preparing figures; to students from the University of Granada for recording from videotapes; to Carl Cheney, Pere Julià and Murray Sidman for comments on an early draft, and to Inmaculada Gómez Becerra for comments on the final draft. Thanks are due to Karen Shashok for carefully and synthesizing the language.

Special thanks are due to the different anonymous reviewers for their appropriate and helpful suggestions.

REFERENCES

Bijou, S. W. (1976). The Basic Stage of Early Childhood. Englewood Cliffs, NJ: Prentice-Hall, Inc.

Birch, H. G. (1945). The relation of previous experience to insightful problem-solving. Journal of Comparative Psychology, 38, 367-383.

Epstein, R. (1985) The spontaneous interconnection of three repertoires. The Psychological Record, 35, 131-141.

Epstein, R., Kirshnit, C. E., Lanza, R. P., & Rubin, L. C. (1984). "Insight" in the pigeon: Antecedents and determinants of an intelligent performance. Nature, 308, 61.

Kohler, W. (1925). The mentality of apes. New York: Harcourt, Brace.

Luciano, M. C. (in press). Problem Solving Behavior: A conceptual account. Mexican Journal of Behavior Analysis.

Maier, N. F. (1931). Reasoning and learning. Psychological Review, 38, 332-342.

Reynolds, G.S. (1968). A primer of Operant Conditioning. Glenview, ILL: Scott, Foresman, & Company.

Schiller, P. H. (1952). Innate constituents of complex responses in primates. Psychological Review, 59, 177-191.

Shurcliff, A., Brown, D., & Stollnitz, F. (1971). Specificity of training required for solution of a stick problem by Rhesus monkeys. Learning and Motivation, 2, 255-270.

Skinner, B.F. (1969). Contingencies of reinforcement: A theoretical analysis. Englewood Cliffs, NJ : Prentice-Hall.