Psicothema was founded in Asturias (northern Spain) in 1989, and is published jointly by the Psychology Faculty of the University of Oviedo and the Psychological Association of the Principality of Asturias (Colegio Oficial de Psicólogos del Principado de Asturias).
We currently publish four issues per year, which accounts for some 100 articles annually. We admit work from both the basic and applied research fields, and from all areas of Psychology, all manuscripts being anonymously reviewed prior to publication.
Isabel Arend and Juan Botella
Universidad Autónoma de Madrid
It has been documented that high anxious subjects process emotional stimuli in a very special way that sometimes has been characterized as an attentional bias toward them. Our main purpose here is to test whether emotional and neutral words would produce different magnitudes of the effect known as attentional blink (AB) on high-trait anxious subjects, but not on low-trait subjects. Participants had to identify the only white word (target 1) in a stream of black words displayed using the Rapid Serial Visual Presentation (RSVP) technique, and detect if a probe word (target 2) is present in the series. The words used as target 1 could be emotional or neutral. The results show that the emotional information reduces the magnitude of the AB only for the high-trait anxiety group. The implications of these results for both theoretical and practical fields are discussed in terms of the subjects high level of automaticity to process emotional information.
Los estímulos emocionales reducen el parpadeo atencional (attentional blink) en sujetos con ansiedad subclínica. Se ha mostrado que los sujetos con ansiedad alta procesan los estímulos emocionales de una forma especial, que a veces se ha interpretado como un sesgo atencional hacia ellos. Nuestro principal objetivo ha sido comprobar si las palabras emocionales y neutrales producen magnitudes diferenciales en el efecto conocido como attentional blink (AB) en sujetos altos en ansiedad, pero no en sujetos bajos en ansiedad. Los participantes tenían que identificar la única palabra en blanco (target 1) que aparecía en una serie de palabras en negro presentadas mediante la técnica de Presentación Rápida de Series Visuales (PRSV), y detectar si estaba presente una palabra «prueba» (target 2) en la serie. Las palabras empleadas como target 1 podían ser de contenido emocional o neutral. Los resultados muestran que la información emocional reduce la magnitud del AB solo en el grupo de ansiedad alta. Se discuten las implicaciones de este resultado para los ámbitos teórico y aplicado, especialmente en términos de un alto nivel de automaticidad en el procesamiento de los estímulos emocionales por parte de los sujetos altos en ansiedad.
Over the past decade several studies have verified, through a variety of experimental paradigms, that high anxious subjects are sensitive to stimuli that represent their concerns. This can show up in at least two ways. First, a decreased ability to avoid that emotional distractors capture attention. Second, as a beneficial processing of emotional targets. The first way implies that a central aspect of this empirical evidence is that such worries produce biases in attention (Beck, Emery & Greenberg, 1985; Byrne & Eysenck, 1995). A variety of experimental paradigms have been used to demonstrate the association between attentional bias and emotion. Part of that body of research attempt to verify the selective aspect of attention toward emotional stimuli by using them as distractors in interference tasks. The degree in which performance on a central task is impaired by the different types of distractor stimuli can be taken as a measure of the degree to which these distractors selectively «capture» attentional resources. Studies using the Stroop paradigm (Dalgleish, 1995; Ehlers, Magraf, Davies & Roth, 1988; Fox, 1993; MacLeod & Rutherford, 1991; Sánchez & Serrano, 1997; Williams, MacLeod & Mathews, 1996) have demonstrated that high-trait anxious subjects show grater interference effects from emotional stimuli, indicating that they attend them selectively.
The second way is that this facilitation of processing for emotional information can enhance performance on tasks where the emotional stimuli play the role of targets. Among other, the use of dichotic listening (Mathews & MacLeod, 1986), has allow to show that there is a lower auditory threshold for stimuli related to the persons concerns (Burgess, Jones, Robertson, Ratcliff & Emerson, 1981; Foa & McNally, 1986) and also a lower visual threshold for such stimuli (Powell & Hemsley, 1984). These two alternatives to pursuit the relationships between emotion and cognition have been considered as two possible research strategies and both contribute on theoretical and practical aspects of the study of anxiety disorders.
Regarding theoretical issues, we summarize its implications on two aspects: the first involves the origin of anxiety states and the second the maintenance factors, where the cognitive characteristics are proposed by the cognitive models of anxiety. The practical importance of studying how anxious subjects process emotional information relies in that this measure can be used as an index to assess factors related with cognitive vulnerability and also the therapeutic progress.
Some explanations have been proposed to explain the adaptive role of attentional bias found on anxious subjects. Eysenck (1992, 1997) proposed that the bias to process emotional stimuli is related to one of the main functions of anxiety: the rapid detection of threatening stimuli. Some findings have supported this explanation (Byrne & Eysenck, 1995; Williams, Watts, MacLeod & Mathews, 1988), showing that pre-attentive and attentional processes associated with threatening stimuli are affected by anxiety.
Beck (1976; Beck & Clark, 1988) was one of the first authors to propose that the distorted content found in emotional disturbed patients arise from biases on the patients information processing system. He described the concept of schema as a representational structure that has been developed to accommodate information pertaining to a specific class of events. When a schema is activated it guides the course of information processing. Some types of early negative experiences result in the creation of idiosyncratic schema that later serves to guide attention selectively toward negative aspects of the environment. A main consequence is a high degree of automaticity for processing that type of information. These formulations have been used to explain the origin and maintenance of emotional disorders. This theory appears to be useful to explain why we can expect to find differences on tasks that are sensitive to the emotional valence of the stimuli processed. As we mentioned above, the activation of such schemas can bias attention on different ways. It can impairs subjects performance, delaying reaction time, or it can provoke a rapid detection of related stimuli, increasing the hit rate. That is, we can make different predictions about the subjects performance when the emotional information plays the role of target versus when it plays the role of distractor in attentional tasks.
It is important to note that the underlying mechanisms that facilitate or impair anxious subjects performance on such tasks are still unclear (Wells & Matthews, 1994). It seems worthwhile to use different paradigms to get additional information about this subjects cognitive functioning.
To explore the emotional and neutral information issue we have tested if the use of these two types of stimuli, when used as targets, would be processed on different ways by the low- and high-trait anxious groups. The experimental paradigms that are sensitive to the differential process load of the emotional stimuli on high and low anxious subjects share the idea that the emotional information has different strength on cognitive processing. One experimental paradigm potentially sensitive to such differential processing of emotional stimuli is the Attentional Blink effect (Raymond, Shapiro & Arnell, 1992).
What is the Attentional Blink?
The attentional blink effect can be summarized as a deficit on subjects performance to detect the second of two targets when they are presented in close temporal positions under conditions of Rapid Serial Visual Presentation (RSVP). Although this deficit was first reported by Broadbent and Broadbent (1987), it was systematically studied by Raymond, Shapiro and Arnell (1992) who employed for the first time the term attentional blink (AB). On their experiment 2 the subjects were presented with an RSVP stream of letters. One letter was white and the rest were black on the gray background of a computer monitor. The only white letter in the stream was designated as the target, and a black X that could appear in any position of the series after the target, was designated the probe (from here they are referred as target 1, or T1, and target 2, or T2). In the experimental trials the subjects were asked to identify the white letter and to detect if the black X appeared. On control trials the subjects were asked only to detect if T2. The black X was presented at different positions after T1, giving place to the different lags that constitute the main independent variable. The results showed that T2 detection was poor on the experimental condition when the lag between T1 and T2 was 2, 3, 4 and 5 positions. This deficit did not occur in the control condition. As the subjects had difficulties for reporting T2 only on experimental trials the authors concluded that the deficit has an attentional rather than only a sensory basis. An interesting point is that if the deficit for processing T2 is due to the processing of T1, then the deficit should be maximum when T2 is presented in the position closer to T1. It has been showed, however, that performance in that condition (lag 1) sometimes is better than when T2 is presented in positions 2-4, so that performance on T2 has a U-shape. The preservation of performance in that condition is known as lag 1 sparing, and the conditions in which it appear have an important role for the understanding of AB itself (Visser, Bischoff & DiLollo, 1999).
After the report by Raymond, Shapiro and Arnell (1992) many studies have been done to specify the conditions under which this effect is produced and to study the cognitive mechanisms that underlie it. On such studies many experimental manipulations have been used such as changing the target pattern and probes information by using geometrical patterns and letters (Chun & Potter, 1995; Joseph, Chun & Nakayama, 1997), personal names (Shapiro, Caldwell, & Sorensen, 1997), semantic related words (Shapiro, Driver, Ward, & Sorensen, 1997), etc. Using a prime paradigm, Maki, Friegen and Paulson (1997) verified that the information that is lost during the AB effect is in fact processed. The results are consistent with the idea that under some established conditions the AB can be observed whereas in others this effect can be minimized and, more important, that this effect is sensitive to T1 and T2 manipulations. For example, the AB can be minimized when a blank space is used as T2. The same happens when personal names are used as T1. They suggested that the pattern of information determines the resources needed to process T1 and T2 and, as a consequence, to maintain the level of performance.
Shapiro, Arnell and Raymond (1997) have identified five different theoretical explanations for the AB effect. Shapiro, Raymond and Arnell (1994) proposed that the AB stem from an interference in a short-term storage buffer; Duncan, Ward and Shapiro (1994) consider that the AB deficit represents the dwell time of attention; according to the two-stage model described by Chun & Potter (1995) T2 becomes degraded in a processing stage while T1 is processed in the other stage; another interesting contribution is the object-substitution account, in which T2 is said to be replaced in consciousness by the trailing mask (Giesbrecht & Di Lollo, 1998); the AB has been also viewed as an instance of a broader class of events known as the psychological refractory period (Jolicoeur, 1998).
Our interest was not to find empirical evidences regarding the basic processes implicated on this phenomenon. However, it is interesting to note that all these explanations share the point that the manipulation of factors that affect the processing load of T1 would have an effect on T2 performance. Our predictions were made considering this common point, as all models predict that a differential processing load would produce AB effects of differential sizes. Previous research with emotional stimuli in high-trait anxious subjects allowed us to make specific predictions about the size of the AB effect. As we mentioned above, we expect this effect to be sensitive to the attentional characteristics of high-trait anxiety subjects when the pattern of information is manipulated by using emotional and neutral words as stimuli (see the methodological discussion by Botella, 2000, about this point). As with other experimental paradigms, we expect that AB can detect differences on information processing of high-trait anxious subjects. Specifically, if emotional words are processed more automatically by high-trait anxiety subjects then the degree of interference on detection of T2 will decrease. On the contrary, the AB showed by low-trait subjects should be unaffected by this factor.
Forty-nine participants (25 high- and 24 low- trait anxiety subjects) were screened according to their scores in the Spielberger Trait Anxiety Inventory (STAI-T). We first applied the STAI-T (Spielberger, Gorsush & Lucshene, 1970) to about 400 students from the Universidad Autónoma de Madrid in order to obtain two extreme groups, one composed by low-trait anxiety subjects and the other composed by high-trait anxiety subjects. The 25 students with higher and lower scores were selected, and participated in a single individual session. A second anxiety scale (MAS; Taylor, 1953) was applied individually to those subjects after the experimental session in order to confirm their correct assignment to the groups. The results of one of the low-trait subjects were lost by technical problems. Table 1 shows a description of the scores of the participants on both scales.
Stimuli and materials
Stimuli were presented by means of an Inves monitor controlled by an Inves PC-compatible computer. The experimental programming was made using the MEL program (Schneider, 1988). 300 series of 14 words of 4-6 letters were constructed. One of the words was designated as target 1. It occupied, equiprobably, positions 6 to 8. In 200 of the series, the word theater (designated as target 2) was included, equiprobably, in one of the four positions after target 1 (conditions of lag 1-4). In the other 100 series target 2 was not included. All words appeared in black except the one designated as target 1, that was presented in white; the background of the screen remained gray throughout the experiment. The target 1 could be an emotional or a neutral word in content (e.g., THIEF/TREE). We selected 50 emotional words and 50 neutral words (figure 1), paired according to their length and frequency of use following a dictionary of frequency of Spanish words (Alameda & Cuetos, 1995). The words subtended 0.64º in height, while their width depended on the number of letters in each word. Each letter subtended 0.29º in terms of width.
Participants sat with their eyes about 40 cm from the screen. The environment was calm and half light. At the beginning of each trial a horizontal string of 6 plus symbols appeared on the center of the screen, in the position where the words would appear. The participant began each trial by pressing the space bar. After 500 ms the first word of the series appeared. The exposure time for each word was 83 ms and there was a blank interval of 33 ms between each two consecutive words, resulting an SOA of 117 ms. Trials of the eigth experimental conditions (4 lags by 2 conditions of emotionality) were presented randomly. On completion of the presentation of a series a response menu with six numbered options appeared. The menu contained the target 1 word and the two words presented before and after it, in random order, plus the option «I don’t know». In the conditions where the target 2 word was included in positions of lag 1 or 2, it was substituted in the menu by the word presented in lag 3 position (e.g., in the trial of figure 1 the response menu would include the words tree, pencil, fear, book and shirt). The participants chose his/her response by pressing the number of the corresponding alternative in order to identify the white word, or pressed the ‘6’ for the «I don’t know» option. Then the question «Did the word theater appeared on the series?» was asked. Numbers 1 and 2 of the keyboard were used for ‘yes’ and ‘no’ answers. Thirty of the experimental series were randomly selected for a practice block.
The data are presented in two parts. The first one includes the analysis of the mean percentage of first targets (T1) correctly reported. The percentages of correct on second target (T2) items were then conditionalized on correct first target report to study the AB. These data are discussed in turn.
First target data
The individual percentages of T1 correctly reported were submitted to a 3 factors ANOVA: the between-subjects factor group (high- versus low-trait anxiety subjects) and the within-subjects factors lag (4 levels) and emotionality of the words (emotional versus neutral). We did not find significant main effects of emotionality [F(1, 47)= .819, p=.370; MSE= 0.0121] and group [F(1, 47)=.091, p= .765; MSE= 0.337], but the main effect of lag was statistically significant [F(3, 141)= 8.787, p<.001; MSE= 0.0070]. The only interaction that showed significance is lag x group [F(3,14)= 2.70, p= .048; MSE= 0.0070]; all other first order interactions and the second order interaction were not significant [emotionality x group, F(1,47)= .413, p= .524, MSE= 0.0121; emotionality x lag, F(3,141)= 1.61, p= .19, MSE= 0.0061; emotionality x lag x group, F(3,141)= .182, p=.908, MSE= 0.0061]. The nature of the main effect of lag and of the interaction lag x group is showed in figure 2. High-trait anxiety subjects perform better with short lags (1 and 2) than with long lags (3 and 4), whereas the low-trait anxiety group show no change along the four lag conditions.
Second target data
The percentage of hits on T2 detection conditionalized to a hit on T1 identification were submitted to a similar ANOVA. We found that there was no significant main effect of group [F(1,47)= 0.57, p= .813; MSE= 0.334], and emotionality [F(1,47)= 3.264, p= .077; MSE= 0.0190], but the main effect of lag was significant [F(3,141)= 24.860, p<.001; MSE= 0.0385]. This main effect is what constitutes the AB. We did not find significant effects of the first order interactions: group x lag [F(3, 141)= .640, p= .590; MSE= 0.0385], emotionality x group [F(1, 47)= 3.383, p= .072; MSE= 0.0190] and emotionality x lag [F(3, 141)= 1.822, p= .146; MSE= 0.0166].
However, as we predicted, there is a significant second order interaction of group x emotionality x lag [F(3, 141)= 2.692, p= .049; MSE= 0.0166]. This interaction is due to the fact that whereas in low-trait anxiety subjects the size of the AB is similar for emotional and neutral stimuli, the high-trait anxiety subjects show a reduced AB when emotional stimuli are employed. The high-trait anxiety subjects showed a minimized AB effect only when emotional words were used, but the low anxious group showed the same magnitude on the AB for both emotional and neutral words (figure 3 and table 2).
Some false alarms («yes» responses on trials where T2 was absent) were also found. The percentages of false alarms were submitted to an ANOVA with the within-subjects factor emotionality and the between-subjects factor group. We found no significant main effects of emotionality and group [F(1,47)= .786, p= .38, MSE= 0.0026; and F(1,47)= .132, p= .718, MSE= 0.0174, respectively] neither of the interaction between them [F(1,47)= .232, p= .632; MSE= 0.0026]. The mean averages of false alarms for the high- and low-trait anxiety groups were 68 and 64, respectively, in the condition with emotional words, and exactly the same in the condition with neutral words.
We have found a robust AB using words as stimuli, with the shape usually found on previous studies (Raymond, Shapiro & Arnell, 1992). As predicted, the magnitude of the effect was not affected by emotionality in low-trait anxious subjects, but it was for the high-trait anxious group. Specifically, the emotional words produced a smaller impairment on T2 detection. Assuming that anxious subjects process more automatically emotional stimuli, this result match with the theoretical explanations of AB that relate it with the processing load imposed by T1 (Chun & Potter, 1995). The AB is reduced in this type of subjects when emotional words are used. This prediction is supported by the second order interaction. It seams reasonable to conclude that the bias in attention found on high-trait anxiety subjects is stable among the use of different experimental paradigms, since these results goes on the same direction of other empirical evidences previously reported. As we mentioned in the introduction section, anxiety can impair or facilitate performance on experimental tasks that involve emotional stimuli, depending on the role they play. It can be interpreted in terms of the theoretical formulations of how emotional stimuli are processed on the cognitive system in terms of the strength of representation. The two-stage model proposed by Chun and Potter (1995) postulates that the AB is a consequence of the delay in the processing of the first target on the two stage model. If we relate our findings with this idea we can say that when T1 is an emotional word the anxious subjects process it more rapidly (more automatically) and, as a consequence, the interference on T2 processing is reduced. In the same way, the interaction of lag and group in the levels of performance on T1 reflects the larger automaticity of high-trait anxiety subjects when they process emotional stimuli. These findings are also compatible with the formulations made by Eysenck (1997) and Broadbent and Broadbent (1988), in which the anxious subjects are sensible to the content of the stimuli used in different experimental paradigms. We have verified that this paradigm is sensitive to study the high level of automaticity when sub-clinical anxious are used as subjects. This result should be replicated with clinically anxious patients and also with other clinical populations. It is reasonable to consider that important questions regarding the underling mechanism of how these subjects process information remain unanswered.
Alameda, J.R. and Cuetos, F. (1995). Diccionario de frecuencias de las unidades lingüísticas del castellano [Dictionary of frequencies of castilian linguistic units]. Oviedo, Spain: Servicio de Publicaciones de la Universidad de Oviedo.
Beck, A.T. (1976). Cognitive therapy and the emotional disorders. New York: International Universities Press.
Beck, A.T. and Clark, D. A. (1988). Anxiety and depression: An information processing perspective. Anxiety Research, 1, 23-36.
Beck, A.T., Emery, G. and Greenberg, R.T.(1985). Anxiety disorders and phobias: A cognitive perspective. New York: Basic Books.
Botella, J. (2000). Algunos problemas metodológicos en el estudio de la atención selectiva. [Some methodological problems in the study of selective attention]. Psicothema, 2, supl.2, 91-94.
Broadbent, D.E. and Broadbent, M.H. (1987). From detection to identification: response to multiple targets in rapid serial visual presentation. Perception & Psychophysics, 42, 105-113.
Broadbent, D.E. and Broadbent, M. H. (1988). Anxiety and attentional bias: State and trait, Cognition and Emotion, 2, 165-183.
Burgess, I.S., Jones, L.M., Robertson, S.A., Ratcliff, W.N. and Emerson, E. (1981). The degree of control exerted by phobic on non-phobic verbal stimuli over the recognition behavior of phobic and non-phobic subjects. Behavior Research and Therapy, 19, 233-243.
Byrne, A. and Eysenck, M.W. (1995). Trait anxiety, anxious mood, and threat detection. Cognition and Emotion, 9, 549-562.
Chun, M.M. and Potter, M.C. (1995). A two-stage model for multiple target in rapid serial visual presentation. Journal of Experimental Psychology: Human Perception and Performance, 21, 109-127.
Dalgleish, T. (1995). Performance on the emotional Stroop task in groups of anxious, experts and controls participants: A comparison of computer and card presentation formats. Cognition and Emotion, 9, 341-362.
Duncan, J., Ward, R. and Shapiro, K. L. (1994). Direct measurement of attentional dwell time in human vision. Nature, 369, 313-315.
Ehlers, A., Margraf, J., Davies, S. and Roth, W. (1988). Selective processing of threat cues in subjects with panic attacks. Cognition and Emotion, 2, 201-219.
Eysenck, M.W. (1992). Anxiety: The cognitive perspective. Hove, UK: Lawrence Erlbaum Associates Ltd.
Eysenck, M.W. (1997). Anxiety and cognition: A unified theory. Psychology Press: UK.
Foa, E.B. and McNally, R. J. C. (1986). Sensitivity to feared stimuli in obsessive-compulsives: A dichotic listening analysis. Cognitive Therapy and Research, 10, 477-486.
Fox, E. (1993). Attentional bias in anxiety: selective or not? Behavior Research and Therapy, 31, 487-493.
Giesbrecht, B.L. and Di Lollo, V. (1998). Beyond the attentional blink: visual masking by object substitution. Journal of Experimental Psychology: Human Perception and Performance, 24, 1.454-1.466.
Jolicoeur, P. (1998). Modulation of the attentional blink by on-line response selection: Evidence from speeded on unspeeded task decisions. Memory and Cognition, 26, 1.014-1.032.
Joseph, J.S., Chun, M.M. and Nakayama, K. (1997, June 19). Attentional requirements in a preattentive feature search task. Nature, 387, 805-808.
Mathews, A. and MacLeod, C. (1986). Discrimination of threat cues in anxiety states. Journal of Abnormal Psychology, 95, 131-138.
Maki, W.S., Frigen, K. and Paulson, K. (1997). Associative priming by targets and distractors during rapid serial visual presentation: Does word meaning survive the attentional blink? Journal of Experimental Psychology: Human Perception and Performance, 23, 1.014-1.034.
MacLeod, C. and Rutherford, E. (1991). Anxiety and the selective processing of emotional information: mediating roles of awareness, trait and state variables, and personal relevance of stimulus material. Behavior Research and Therapy, 30, 479-491.
Powell, M. and Hemsley, D.R.G. (1984). Depression: A breakdown of perceptual defense? British journal of Psychiatry, 143, 358-362.
Raymond, J.E., Shapiro, K.L. and Arnell, K. M. (1992). Temporary suppression of visual processing in an PRSV task: An attentional blink? Journal of Experimental Psychology: Human Perception and Performance, 18, 849-860.
Sánchez, M.S. and Serrano, J.M. (1997). Influencia del nivel de alexitimia en el procesamiento de estímulos emocionales en una tarea Stroop. [Effect of the alexithimia level in the processing of emotional stimuli in a Stroop task]. Psicothema, 9(3), 519-527.
Schneider, W. (1988). Micro experimental laboratory: An integrated system for IBM PC compatibles. Behavior, Research, Methods, Instruments, & Computers, 20, 206-217.
Shapiro, K., Arnell, K.M. and Raymond, J. E. (1997). The attentional blink. Trends in Cognitive Science, 1, 291-291.
Shapiro, K., Caldwell, J. and Sorensen, R. (1997). Personal names and the attentional blink: A visual «Cocktail Party» effect. Journal of Experimental Psychology: Human Perception and Performance, 23, 504-514.
Shapiro, K., Driver, J., Ward, R. and Sorensen, R. (1997). Priming from the attentional blink: A failure to extract visual tokens but not visual types. Psychological Science, 8, 95-100.
Shapiro, K., Raymond, J.E. and Arnell, K. (1994). Attention to visual pattern information produces the attentional blink in RSVP. Journal of Experimental Psychology: Human Perception and Performance, 20, 357-371.
Spielberger, C.D., Gorsush, R.L. and Lucshene, R. E. (1970). Manual for the State Trait Anxiety Inventory, Palo Alto, CA: Consulting Psychologists Press. Spanish version in: C.D., Spielberger, R.L., Gorsush, R.E. and Lucshene. (1988). Cuestionario de Ansiedad Estado-Rasgo, Madrid.TEA.]
Taylor, J. (1953). A personality scale of manifest anxiety. Journal of Abnormal and Social Psychology, 48, 285-290.
Visser, T.A., Bischof, W.F. and DiLollo, V. (1999). Attentional switching in spatial and nonspatial domains: evidence from attentional blink. Psychological Bulletin, 125, 458-469.
Wells, A. and Matthews, G. (1994). Attention and emotion: A clinical perspective. Lawrence Erlbaum Associates: UK.
Williams, J.M., Watts, F.N., MacLeod, C. and Mathews, A. (1988). Cognitive psychology and emotional disorders. Chichester: John Wiley.
Williams, J.M., MacLeod, C. and Mathews, A. (1996). The emotional Stroop task and psychopathology. Psychological Bulletin, 120, 3-24.