Since David Green and John Swets (1966) published their classic
book entitled Signal detection theory and psychophysics, psychologists
have made a wide use of this approach as a way to explain human performance
on detection tasks conducted in a noise background. The number, however, of
problems to which the theory have been applied is in these days much larger.
At the beginning, the common user of signal detection (SDT) was a psychologist
studying sensory processes. Today, psychologists interested in perception, memory
and cognition. In general, as well as other social and medical scientists are
also potential users of the SDT analysis.
The type of problems to which SDT can be applied are numerous,
a few of which are:
a) Determining the ability of a person to discriminate whether
a familiar object or a photograph has been previously seen or corresponds to
b) Assessing the capacity of a medical student in distinguishing
X-rays showing a normal organ from an organ with a tumour.
c) Measuring the ability of a blind person to discriminate
between a closed and an open shape haptically explored.
These are only just a few examples to which detection theory
can be applied. Detection theory is today a widely used approach to study human
performance in a large number of experimental as well as clinical settings.
The signal detection approach has many important advantages
over classical psychophysics methods in which sensitivity is confounded with
response bias (see Jáñez, 1992; Muñiz, 1991). The main
advantage of SDT and Choice Theory (Luce, 1963) is that both provide sensitivity
indexes that are unaffected by bias toward a particular response (see MacMillan
& Creelman, 1991).
SDT and Luce theory are useful tools in situations in which
performance is not perfect and accordingly errors appear. In other situations
in which «noise» does not affect performance and a near perfect correspondence
between stimuli and response is obtained, these approaches are not useful. However,
in situations in which errors arise, SDT as well as Luce' Choice Theory, are
useful approaches to compute and interpret subject's sensitivity, as well as
bias from different experimental designs.
Computations, however, are sometimes rather complex and very
time consuming. For this reason, Psychologists working in several areas, medical
as well as social scientists using detection theory in their work may find useful
a comprehensive software that speeds up calculations.
A first version of the SDT_SP was presented at the 23rd Annual
Meeting of the Society for Computers in Psychology that gathered at Washington,
DC, United States, in the fall of 1993. A paper has just appeared in the journal
Behaviour Research Methods, Instruments and Computers (Reales & Ballesteros,
In this paper we will present very briefly the main features
of the program. We will also show its accuracy comparing SDT_SP results to outputs
from a small software program written by Boice and Gardner (1988) and to results
from a well-known textbook on detection theory. Finally, we will present the
most recent extension of the program prepared to calculate the parameters corresponding
to «same-different» designs.
SDT_SP main features
SDT_SP is a computer program written in Pascal 7.0 which runs
on IBMcompatible personal computers. The main features of this program are:
accuracy, quickness, easiness of use, and comprehensiveness. The program is
menudriven and easy to use, SDT_SP computes descriptive statistics from different
experimental designs. It also computes inferential statistical tests that make
possible to know the goodness of an estimate and whether the parameter values
differ significantly from 0 or from some other value.
Furthermore, the program allows to plot receiver operating
characteristic (ROC) curves for individual as well as for group's d',
SDT_SP was prepared for practical purposes related to our ongoing research program on haptic and visual perception and implicit and explicit memory for objects (Ballesteros, 1993; Ballesteros, Manga & Reales, 1994a.b,c; Ballesteros & Reales, 1992). In these studies we need to compute fast and accurately a number of descriptive as well as inferential SDT statistics. The starting point of SDT_SP was the Appendix 6 of MacMillan and Creelman's (1991) manual for computing d', c, and β. From here, a comprehensive program developed in an intent to provide the user with the necessary statistical tools to compute parameters from a large number of experimental designs.
A question that naturally arises in relation to any new statistical package is how accurate the computations performed by the program are?
SDT_SP is very accurate in its calculations. Table 1 (top) presents results comparing performance from an «one interval experiment» on haptic perception of symmetry using 3-D unfamiliar wooden objects conducted in our laboratory (Ballesteros et al. 1994b).
The leftmost column presents the results obtained with the
SDT_SP. The central column contains the results obtained with an independent
statistical program (Boice & Gardner, 1988). These data correspond to performance
of one of the subjects who participate in our experiment (subject number 2).
The second part of Table 1 (bottom) presents the parameters from a «same-different»
experiment design computed with SDT_SP (see below). These results are compared
to examples provided by MacMillan and Creelman's (1991, p. 142) textbook, Signal
detection theory: A user's guide (rightmost column). As can be observed
from these comparisons, the program's computations are very accurate. A larger
number of comparisons are provided by Reales and Ballesteros (1994. p. 152)
including results from significance tests for two as well as for more signal
detection parameters (Marascuilo, 1970). All these tests have shown that the
program's accuracy is very high.
How to use the program
Figure 1 (top) shows a printout of the SDT_SP principal menu, showing the available options. The program requires to enter the option number. The user may choose among the available options according to the experimental design and the theoretical assumptions. The available options are: (1) Yes/No Experiments (or one interval discriminations); (2) Nonparametric Analysis; (3) The Rating experiments; (4) Forced-Choice experiments; and (5) «Same-Different» experiments; (6) Setup: and (7) Exit. Figure 1 (middle) shows a printout of the menu with the available options. This menu appears at the screen when option number 1 corresponding to «Yes/No experiments» is selected.
Option number 1 of STD_SP, allows the computation of parameters from one interval experiments. Consider, for example, a situation in which unfamiliar objects are presented under blind conditions and subjects have to judge the objects as «symmetric» or «asymmetric».
Any of the four joint events presented in Table 2 can occur
in each trial. The number responses under any event is presented in the corresponding
box of the stimulus-matrix (in parenthesis). The correct recognition of a symmetric
object is called a hit. The failure to recognize it is called a miss.
Recognizing an asymmetric object as «symmetric» is called a false-alarm while
correctly responding «asymmetric» to an asymmetric object is called a correct
The best way to determine the observers sensitivity is to compute
a measure of the discrepancy between the hit and the false alarm rate. The following
step is to introduce the program the number of hits, false-alarms, misses and
correct rejections from the stimulus-response matrix for each subject. Then,
the program faster and accurately generates the indexes presented in Table 3:
a) The most important SDT sensitivity index is d'. This
index is the distance between the noise and the signal+noise distributions defined
in terms of z scores.
b) d' standard deviation and its confidence interval.
c) α y log α from the choice theory .
d) c, or criterion, is the basic SDT bias index. This
is the best index of subject bias because it is statistically independent of
d' (see MacMillan & Creelman, 1991). The program also provides c's
standard deviation and the likelihood ratio, another bias measure.
e) SDT_SP provides the corresponding bias indexes for the choice
Reales and Ballesteros (1994) provided the formulae used by
the program to calculate these indexes. SDT_SP allows three options at this
point: The first option creates a file in which store this information, the
second option presents a plotting of the ROC curve in the computer screen, finally,
the third option allows to exit from this menu.
Figure 2 presents the ROC curve corresponding to the detection of symmetry data showed in Table 2. This graph represents the functional relationship between the proportion of hits (in our example, to say «symmetric» when the object is symmetric) and falsealarms (to say «symmetric» when the object is asymmetric). If a print of the graph is wanted, before to enter the program a graphic manager, for example, the GRAPHYCS.COM from the MS-DOS, should be loaded. The type of printer to be used must also be specified. The program allows to obtain a graph in z-coordinates as shown in Figure 3.
Figures 2 and 3 show that the subject's sensitivity in detecting
objects' symmetry or asymmetry, is quite accurate (d'= 2.1).
The third option of the menu allows to escape the program.
Detection Theory computation for group data is also available
from option number 3. SDT_SP used the formula provided by MacMillan and Kaplan
(1985.,Appendix A) which allows to compare the variances of two statistics than
estimate d' (sensitivity) or c (the bias index).
SDT_SP allows also to compute the significance test for hypothesis testing from option 4 (see Reales & Ballesteros, 1994).
The rating experiments
To illustrate the rating experiments we are using an example
from the Handbook of perception and human performance (Vol. 1) provided
by Falmagne (1986) in the chapter entitled «Psychophysical measurement and theory»
(p. 1-40). See Table 4.
Rating experiments are those in which subjects are presented
with two stimuli or events and have to indicate their level of confidence that
one or the other event is present using a rating scale with several levels.
When this option is selected the program asks for the number of possible responses.
In our example we enter 6 as this number. The data must be entered from more
certainty to less certainty, first the signal trials and then the noise trials.
The program, then produced the parameters shown in Table 5.
Pressing just a computer key a receiver operating characteristic (ROC) curve can be presented at the computer screen and a printout can also be obtained in linear coordinates or in z-coordinates. See Figures 4 and 5.
To estimate the parameters of line fitting in rating experiments
we have followed the procedure proposed by Churchhause (1981) in the Handbook
o f applicable mathematics (Vol III). Instead of minimizing the vertical
distance between the points and the estimated curve. The reason is that both,
the z(F)i and z(H)i are subjected to experimental error. The main advantage
of using this procedure is that it takes into account that experimental errors
can be obtained in both coordinates.
The «same-different design»
The most recent development in the SDT_SP includes the possibility to analyze data from «same- different» designs. In a «same- different» experiment the observer is presented in each trial with two stimuli <S1S2>; <S1S1>; <S2S1>; or <S2S2> which may be «same» or «different» and he/she has to classify the pair as «same» or «different».
The results can be presented in a 2 x 2 table as show in Table
6. These data correspond to an example provided by MacMillan and Creelman (1991,
p.142, see Table 1, bottom). Suppose we want to compare the ability of children
to discriminate syllables such as /ba/ and /pa/. We create artificial syllables
in the speech perception laboratory varying gradually «voice onset time» (the
point in which the sound starts to vibrate) from, for example, 0 msec (/ba/)
to 60 msec (/pa/) in 20 msec steps (i.e., 0,20,40,60). A way to investigate
the listener's sensitivity to discriminate between pairs of sounds is using
a «same/different» design. If the problem is to discriminate between two sounds
(i.e., 0 and 20), the experimenter creates the four different combinations of
the sounds (<S1S1>, <S2S2>, <S2S1>, and <S2S1>. The
perceiver task is to decide whether each pair of sounds is «same» or «different».
This experimental procedure can be very useful as it does not require an extensive amount of training and it requires very simple judgments. This main characteristics make the task very adequate to be used with young children. As in other options, the program asks for data input. Following the program's prompts, a Table is obtained (see Table 7). This table shows results for fixed as well as for roving experimental designs. In a fixed experiment only two stimuli are presented in the whole block of trials. In this example, 0 and 20 msec «voice onset time». In a roving experiment, however, the stimuli vary within the block in a continuum scale. In the example on the /da/, /ba/ speech discrimination experiment, four different stimuli must be artificially constructed (S1, S2, S3, and S4), corresponding to 0, 20, 40, and 60 msec «voice onset asynchrony».
We are now interested in the assessment of subjects' sensitivity
and bias corresponding to any adjacent pair of sounds. A roving design can use
an only block of trials, instead of the three different blocks required by a
fixed experimental design.
SDT_SP provides the basic sensitivity and bias indexes corresponding to fixed and roving data (see Table 7). The user can also obtain a printout of the «same/different» ROC curve in linear coordinates following the same procedure described in the «yes-no» experiments section as shown in Figure 6.
Other experimental designs
Other experimental designs that can be analyzed using this statistical program are the 2AFC and the mAFC designs. The mAFC designs are general cases of 2AFC experiments in which two stimulus classes are presented in each trial but the number of response alternatives is m instead of just 2.
Summary and Conclusion
The SDT_SP User's Manual and the corresponding software
is being published by the Editorial Universitas.
SDT_SP is a program fast and accurate, written in Pascal 6.0,
that runs in IBM-compatible computers. The program calculates indexes from SDT
and Luce's Choice Theory and allows to plot ROC curves in linear as well as
in z-coordinates. It is menu driven, accurate, easy to use and fast.