Since David Green and John Swets (1966) published their classic
book entitled Signal detection theory and psychophysics, psychologists have
made a wide use of this approach as a way to explain human performance on detection
tasks conducted in a noise background. The number, however, of problems to which
the theory have been applied is in these days much larger. At the beginning,
the common user of signal detection (SDT) was a psychologist studying sensory
processes. Today, psychologists interested in perception, memory and cognition,
in general, as well as other social and medical scientists are also potential
users of the SDT analysis.
The type of problems to which SDT can be applied are numerous,
a few of which are:
a) Determining the ability of a person to discriminate whether
a familiar object or a photograph has been previously seen or corresponds to
b) Assessing the capacity of a medical student in distinguishing
X-rays showing a normal organ from an organ with a tumour.
c) Measuring the ability of a blind person to discriminate
between a closed and an open shape haptically explored.
These are only just a few examples to which detection theory
can be applied. Detection theory is today a widely used approach to study human
performance in a large number of experimental as well as clinical settings.
The signal detection approach has many important advantages
over classical psychophysics methods in which sensitivity is confounded with
response bias (see Jáñez, 1992; Muñiz, 1991). The main
advantage of SDT and Choice Theory (Luce, 1963) is that both provide sensitivity
indexes that are unaffected by bias toward a particular response (see MacMillan
& Creelman, 1991).
SDT and Luce theory are useful tools in situations in which
performance is not perfect and accordingly errors appear. In other situations
in which "noise" does not affect performance and a near perfect correspondence
between stimuli and response is obtained, these approaches are not useful. However,
in situations in which errors arise, SDT as well as Luceí Choice Theory, are
useful approaches to compute and interpret subjectís sensitivity, as well as
bias from different experimental designs.
Computations, however, are sometimes rather complex and very
time consuming. For this reason, psychologists working in several areas, medical
as well as social scientists using detection theory in their work may find useful
a comprehensive software that speeds up calculations.
A first version of the SDT-SP was presented at the 23rd Annual
Meeting of the Society for Computers in Psychology that gathered at Washington,
DC, United States, in the fall of 1993. A paper has just appeared in the journal
Behaviour Research Methods, Instruments and Computers (Reales & Ballesteros,
In this paper we will present very briefly the main features
of the program. We will also show its accuracy comparing SDT-SP results to outputs
from a small software program written by Boice and Gardner (1988) and to results
from a well-known textbook on detection theory. Finally, we will present the
most recent extension of the program prepared to calculate the parameters corresponding
to "same-different" designs.
SDT-SP main features
SDT-SP is a computer program written in Pascal 6.0 which runs
on IBM-compatible personal computers. The main features of this program are:
accuracy, quickness, easiness of use, and comprehensiveness. The program is
menu-driven and easy to use. SDT-SP computes descriptive statistics from different
experimental designs. It also computes inferential statistical tests that make
possible to know about the goodness of an estimate and whether the parameter
values differ significantly from 0 or from some other value.
Furthermore, the program allows to plot receiver operating
characteristic (ROC) curves for individual as well as for groupís dí.
SDT-SP was prepared for practical purposes related to our ongoing
research program on haptic and visual perception and implicit and explicit memory
for objects (Ballesteros, 1993; Ballesteros, Manga, & Reales, 1994a,b,c;
Ballesteros & Reales, 1992). In these studies we need to com-pute fast and
accurately a number of descriptive as well as inferential SDT statistics. The
starting point of SDT_SP was the Appendix 6 of MacMillan and Creelmanís (1991)
manual for computing dí, c, and ß. From here, a comprehensive program
developed in an intent to provide the user with the necessary statistical tools
to compute parameters from a large number of experimental designs.
A question that naturally arises in relation to any new statistical
package is how accurate the computations performed by the program are?
SDT-SP is very accurate in its calculations. Table 1 (top)
presents results comparing performance from an"one interval experiment"
on haptic perception of symmetry using 3-D unfamiliar wooden objects conducted
in our laboratory (Ballesteros et al. 1994b). The leftmost column presents the
results obtained with the SDT-SP. The central column contains the results obtained
with an independent statistical program (Boice & Gardner, 1988). These data
correspond to performance of one of the subjects who participate in our experiment
(subject number 2). The second part of Table 1 (bottom) presents the parameters
from a "same-different" experiment design computed with SDT-SP (see
below). These results are compared to examples provided by MacMillan and Creelmanís
(1991, p. 142) textbook, Signal detection theory: A userís guide (rightmost
column). As can be observed from these comparisons, the programís computations
are very accurate. A larger number of comparisons are provided by Reales and
Ballesteros (1994. p. 152) including results from significance tests for two
as well as for more signal detection parameters (Marascuilo, 1970). All these
tests have shown that the programís accuracy is very high.
How to use the program
Figure 1 (top) shows a printout of the SDT-SP principal menu,
showing the available options. The program requires to enter the option number.
The user may choose among the available options according to the experimental
design and the theoretical assumptions. The available options are: (1) Yes/No
Experiments (or one interval discriminations); (2) Nonparametric Analysis; (3)
The Rating experiments; (4) Forced-Choice experiments; and (5) "Same-Different"
experiments; (6) Setup; and (7) Exit. Fi-gure 1 (middle) shows a printout of
the menu with the available options. This menu appears at the screen when option
number 1 corresponding to "Yes/No experiments" is selected.
Option number 1 of STD-SP, allows the computation of parameters
from one interval experiments. Consider, for example, a situation in which unfamiliar
objects are presented under blind conditions and subjects have to judge the
objects as "symmetric" or "asymmetric". Any of the four
joint events presented in Table 2 can occur in each trial.
The number responses under any event is presented in the corresponding
box of the stimulus-matrix (in parenthesis). The correct recognition of a symmetric
object is called a hit. The failure to recognize it is called a miss. Recognizing
an asymmetric object as "symmetric" is called a false-alarm while
correctly responding "asymmetric" to an asymmetric object is called
a correct rejection.
The best way to determine the observers sensitivity is to compute
a measure of the discrepancy between the hit and the false alarm rate. The following
step is to introduce the program the number of hits, false-alarms, misses and
correct rejections from the stimulus-response matrix for each subject. Then,
the program faster and accurately generates the indexes presented in Table 3:
a) The most important SDT sensitivity index is dí. This index
is the distance between the noise and the signal+noise distributions defined
in terms of z scores.
b) dí standard deviation and its confidence interval.
c) α y logα from the choice theory.
d) c, or criterion, is the basic SDT bias index. This is the
best index of subject bias because it is statistically independent of dí (see
MacMillan & Creelman, 1991). The program also provides cís standard deviation
and the likelihood ratio, another bias measure.
e) SDT-SP provides the corresponding bias indexes for the choice
Reales and Ballesteros (1994) provided the formulae used by
the program to calculate these indexes. SDT-SP allows three options at this
point: The first option creates a file in which store this information, the
second option presents a plotting of the ROC curve in the computer screen, finally,
the third option allows to exit from this menu.
Figure 2 presents the ROC curve corresponding to the detection
of symmetry data showed in Table 2. This graph represents the functional relationship
between the proportion of hits (in our example, to say "symmetric"
when the object is symmetric) and false-alarms (to say "symmetric"
when the object is asymmetric). If a print of the graph is wanted, before to
enter the program a graphic manager, for example, the GRAPHYCS.COM from the
MS-DOS, should be loaded. The type of printer to be used must also be specified.
The program allows to obtain a graph in z-coordinates as shown in Figure 3.
Figures 2 and 3 show that the subjectís sensitivity in detecting
objectsí symmetry or asymmetry, is quite accurate (dí= 2.1).
The third option of the menu allows to escape the program.
Detection Theory computation for group data is also available
from option number 3. SDT-SP used the formula provided by MacMillan and Kaplan
(1985, Appendix A) which allows to com-pare the variances of two statistics
than estimate dí (sensitivity) or c (the bias index).
SDT-SP allows also to compute the significance test for hypothesis
testing from option 4 (see Reales & Ballesteros, 1994).
The rating experiments
To illustrate the rating experiments we are using an example
from the Handbook of perception and human performance (Vol. 1) provided by Falmagne
(1986) in the chapter entitled "Psychophysical measurement and theory"
(p. 1-40). See Table 4.
Rating experiments are those in which subjects are presented
with two stimuli or events and have to indicate their level of confidence that
one or the other event is present using a rating scale with several levels.
When this option is selected the program asks for the number of possible responses.
In our example we enter 6 as this number. The data must be entered from more
certainty to less certainty, first the signal trials and then the noise trials.
The program, then produced the parameters shown in Table 5.
Pressing just a computer key a receiver operating characteristic
(ROC) curve can be presented at the computer screen and a printout can also
be obtained in linear coordinates or in z-coordinates. See Figures 4 and 5.
To estimate the parameters of line fitting in rating experiments
we have followed the procedure proposed by Churchhause (1981) in the Handbook
of applicable mathematics (Vol III). Instead of minimizing the vertical distance
between the points and the estimated curve, the distance of the points was minimized
from the nearest point of the curve. The reason is that both, the z(F)i and
z(H)i are subjected to experimental error. The main advantage of using this
procedure is that it takes into account that experimental errors can be obtained
in both coordinates.
The "same-different" design
The most recent development in the SDT-SP includes the possibility
to analyze data from "same-different" designs.
In a "same-different" experiment the observer is
presented in each trial with two stimuli <S1S2>; <S1S1>; <S2S1>;
or <S2S2> which may be "same" or "different" and he/she
has to classify the pair as "same" or "different". The results
can be presented in a 2 x 2 table as shown in Table 6. These data correspond
to an example provided by MacMillan and Creelman (1991, p. 142, see Table 1,
bottom). Suppose we want to compare the ability of children to discriminate
syllables such a /ba/ and /pa/. We create artificial syllables in the speech
perception laboratory varying gradually "voice onset time" (the point
in which the sound starts to vibrate) from, for example, 0 msec (/ba/) to 60
msec (/pa/) in 20 msec steps (i.e., 0, 20, 40, 60). A way to investigate the
listenerís sensitivity to discriminate between pairs of sounds is using a "same/different"
If the problem is to discriminate between two sounds (i.e.,
0 and 20), the experimenter creates the four different combinations of the sounds
(<S1S1>, <S2S2>, <S1S2>, and <S2S1>). The perceiver
task is to decide whether each pair of sounds is "same" or "different".
This experimental procedure can be very useful as it does not
require an extensive amount of training and it requires very simple judgments.
This main characteristics make the task very adequate to be used with young
children. As in other options, the program asks for data input. Following the
programís prompts, a Table is obtained (see Table 7). This table shows results
for fixed as well as for roving expe-rimental designs. In a fixed experiment
only two stimuli are presented in the whole block of trials. In this example,
0 and 20 msec "voice onset time". In a roving experiment, however,
the stimuli vary within the block in a continuum scale. In the example on the
/da/, /ba/ speech discrimination experiment, four different stimuli must be
artificially constructed (S1, S2, S3, and S4), corresponding to 0, 20, 40, and
60 msec "voice onset asynchrony". We are now interested in the assessment
of subjectsí sensitivity and bias corresponding to any adjacent pair of sounds.
A roving design can use an only block of trials, instead of the three different
blocks required by a fixed experimental design.
SDT-SP provides the basic sensitivity and bias indexes corresponding
to fixed and roving data (see Table 7). The user can also obtain a printout
of the "same/different" ROC curve in linear coordinates following
the same procedure described in the "yes-no" experiments section as
shown in Figure 6.
Other experimental designs
Other experimental designs that can be analyzed using this
statistical program are the 2AFC and the mAFC designs. The mAFC designs are
general cases of 2AFC experiments in which two stimulus classes are presented
in each trial but the number of response alternatives is m instead of just 2.
Summary and Conclusion
The SDT-SP Userís Manual and the corresponding software is
being published by the Universidad Nacional de Educación a Distancia.
SDT-SP is a program fast and accurate, written in Pascal 6.0,
that runs in IBM-compatible computers. The program calculates indexes from SDT
and Luceís Choice Theory and allows to plot ROC curves in linear as well as
in z-coordinates. It is menu driven, accurate, easy to use and fast.