Raven Standard Progressive Matrices
Abbreviation: SPM
Background
In the mid-1930s, John C. Raven, a student of Charles Spearman, constructed the “Progressive Matrices” as part of his Master's Thesis, which we know today as the Standard Progressive Matrices (SPM)1). The aim of the test construction was to cover the entire range of intellectual development from childhood to old age. At the same time, the test was to be as independent as possible of language, education, nationality, or health status.
The items consist of incomplete geometric figures or patterns, which must be completed from six or eight response alternatives (multiple choice), respectively, so that a recognized construction pattern is logically continued. Figure 1 shows an example of an item that corresponds in its construction principle to SPM items.
Figure 1: Example of a matrix item. The matching piece must be selected.
The SPM consist of 5 series (labeled A to E) of 12 such items each, with increasing difficulty. The solution of the first item of each series is usually immediately apparent, especially in the first four series A to D. In the later items of each series, more difficult problems are then presented. They always build on what was required in the preceding items.
The SPM have remained practically unchanged since the first edition in 1938. The still current Manual for Raven's Progressive Matrices and Vocabulary Scales2) indicates that there were only two small changes to the items. In 1947, a correction was made to the eighth item of Series B, and for the 1956 edition, both the items themselves and the response options were rearranged, but not the position of the correct answer.
In the upper intelligence range, the SPM no longer measure very precisely. In the course of the cross-cultural rise in intelligence test scores3), a clear ceiling effect was observed for the SPM in adolescents and younger adults. For this reason, the SPM-Plus was developed at the same time as a parallel version of the SPM corresponding to the classic 1956 form, which again allows improved differentiation in the upper intelligence range. For the usual clinical purposes, differentiation in the normal and subnormal range is more important than in the upper intelligence range. In this respect, the differentiating power of the SPM is usually sufficient for clinical purposes.
The SPM are used with both children and adults. Within TDB2Online, only the application in adults is covered.
Frank Gierschmann wrote a quite detailed review of the Progressive Matrices in 2003.4) The book is still available.
History of the test and the publishers
For a long time, the three versions of the Progressive Matrices were self-published. After J.C. Raven's death, his sons initially continued the publishing house (J. C. Raven Ltd.) under the leadership of the eldest son, John Raven. Under John Raven's supervision, the most extensive norming studies in adults to that date took place in Scotland and the USA in the late 1980s. In the mid to late 1990s, a comprehensive documentation of Raven's tests appeared in 7 volumes. Of these, Part 1 (general overview)5), Part 3 (Standard Progressive Matrices)6), and Part 7 (research data and references)7) are relevant for the SPM. At the beginning of the century, the Psychological Corporation in Texas, which by this time already belonged to Harcourt, took over the family publishing house including all rights to the tests. The test division of Harcourt was later taken over by Pearson Assessment, which currently (October 2012) publishes the English manuals without substantial changes.
In Germany as well, the SPM have changed test publisher several times. A first edition with a German standardization for children by Kratzmeier and Horn appeared in 1979 from Beltz Test, and a (slightly) expanded second edition in 1988.8) A further German edition with new norms (but likewise predominantly for children and adolescents) appeared in 1998 under the editorship of Heller, Kratzmeier, and Lengfelder.9)
Almost simultaneously (since 1998), Swets Test Service in Frankfurt began to publish the complete English manuals and the three matrices tests in German. The SPM appeared there in the adaptation by Bulheller and Häcker10) based on the British edition of 1998.11) The German norms published in this edition were collected with the SPM Plus, not with the classic SPM. However, there is a conversion table with the help of which SPM-Plus raw scores can be converted into SPM raw scores (and vice versa). Mainly adolescents between 14 and 18 years of age were studied, only a few students, and no non-student adult sample.
Currently, the German versions that first appeared at Swets Test Service, or their new editions, are published by the legal successor, Pearson Assessment & Information GmbH, in Frankfurt. They can be obtained directly from there or through the Testzentrale (test distributor) in Germany or Switzerland.
Test material
To administer the classic SPM, one definitely needs the (classic) SPM test booklet, which contains the five test series (A, B, C, D, E) of 12 items each, and the (classic) SPM answer sheets. In the current version, these are self-duplicating and thus facilitate counting the raw scores. At least a cursory familiarity with the content of the German SPM manual12) is necessary, even if it is somewhat laborious to read. In addition, the test authors recommend reading the basic manual for Raven's matrices tests13).
Copyright
J. C. Raven's rights to his test were passed on from publisher to publisher and today lie internationally with Pearson Assessment, represented in Germany by Pearson Assessment & Information GmbH in Frankfurt.
Test administration
The current manuals for Raven's matrices tests14) 15) recommend administering the SPM as an untimed power test. The test can be administered both individually and as a group test (although the group size, especially in clinical applications, should be limited to a few persons). Administration takes about 45 minutes on average.
Administering the SPM with a binding time limit, which does not allow some examinees to finish the test, would lead to an uneven and invalid score distribution. The reason is that under such test conditions, some examinees spend a lot of time solving the difficult items from the first series, while others skip these items and considerably increase their scores by solving the easier items from the higher series.
Test raw scores
For all versions of Raven's Progressive Matrices, there is only one raw score, the number of correct solutions. For the SPM, this number ranges from 0 to 60.
Distributional properties of the raw scores
The SPM can measure across a very wide range of ability. After all, the test consists of 60 items (which is a lot compared with other ability tests), and the items differ considerably in difficulty. Actually, if the difficulty coefficients of the items are well chosen, this should result in a normal distribution of the raw scores. Nevertheless, restrictions at both ends of the distribution must be expected: in very young children, and certainly also in very old persons, there will be floor effects. It is known from the literature that ceiling effects have occurred in young adults in recent decades (see below). Both factors should lead to truncated distributions in the respective subpopulations.
The literature contains no exact SPM distribution data from random samples of adults. However, rough distribution shapes can be reconstructed because most SPM normative data are available as percentile values. One of the larger adult samples (N=952) was collected by Deltour in Belgium and published in 1993. These data cover a wide age range and are available - unfortunately rather coarsely graded - as percentile values for the 5th, 10th, 25th, 50th, 75th, 90th and 95th percentiles. This information makes it possible to depict the distribution shape of the raw scores roughly. Figure 2 shows these back-calculated distributions for three age groups.
Figure 2: Back-calculated distribution shape of the SPM raw scores in the Belgian norming of 1993 for three selected age groups
Similar data are available for the British norming; they are shown in Figure 3.
Figure 3: Back-calculated distribution shape of the SPM raw scores in the British norming of 1992 for three selected age groups
Both figures show that there are indeed ceiling effects, more in the young than in the old and more in the British data than in the Belgian data (peculiarities of and differences between the normings are discussed later in the section Norming studies). The descent of the distribution curve is steeper than its ascent, which means poorer differentiating power in the high ability range. A floor effect, by contrast, cannot be seen.
It is striking that the vast majority of examinees - even in the 70-79 age group - solve more than 20 items correctly. The items of series A and B are so easy that over 95 percent of all adults master them. John C. Raven, the test author, and his eldest son John Raven (Jr.) always emphasized that this is intentional: the initial series always also served to teach general solution strategies for this type of item. They are thus not superfluous but are to be seen as a kind of preparation for the test, which is intended to even out differences in speed of comprehension.
Even if there are probably no empirical tests of this concept, it sounds plausible to a clinician. Especially in clinical settings, performance anxiety and inhibitions at the start of a new task are particularly disruptive, and it is helpful to have a larger number of easy items at the beginning of a test so as not to give rise to unnecessary anxiety.
Linearization of the measurement scale
From the beginning, norms for the Raven tests were given as percentile ranks. This takes into account the fact that the relationship between raw scores and IQ scores is not linear. In the lower performance ranges, solving one additional pattern is associated with a smaller IQ increase than in the upper performance ranges. On the basis of the available percentile rank norms, conversion into linear performance scores via the relationship between percentile ranks and units of the normal distribution is formally simple.
Norming studies
Since none of the three German SPM manuals reported satisfactory German norms for adults (details see below), all international norming studies that meet at least a certain standard of representativeness of the sampling have been compiled here. This work was facilitated by the English SPM manual16), in which all relevant studies up to about 1999 are printed in tabular form.
Germany
The three German editions of the SPM collected normative data predominantly17)18) or exclusively19) for children and adolescents. For the adult range, Heller et al.20) examined 324 students. 295 were between 20 and 27 years old, the remaining 29 between 30 and 44. Bulheller & Häcker21) examined 167 persons over 19 years of age in 1998 and 1999, 99 of them between 20 and 29, the rest designated undifferentiated as “30+”. Little is reported about the composition of the examinees, and nothing about the intended source population. Neither study is usable for norming purposes, but their results are reported descriptively below.
One must therefore conclude that, unfortunately, there are no German SPM norms for adults. As with many other neuropsychological tests, we will have to make do for the time being with a meta-analytic summary of norms from other countries.
Great Britain 1992
The first British norms for adults were collected by J. C. Raven during the Second World War and later supplemented by individual samples. A representative sample survey (with the SPM and APM) did not take place until 1992 in the town of Dumfries in Scotland. It is presented in detail in the English SPM manual22). At that time Dumfries had about 25,000 inhabitants. With regard to demographic characteristics, Dumfries corresponded to the overall picture of Great Britain. A random sample (every 29th person entered in the electoral register) was notified in writing of a visit by a research assistant. At the home visit, the aim of the study was then explained and Part I of the APM was administered. If the examinee reached a minimum score (the cutoff was initially set at 8 points and later raised to 10 points), Part II of the APM was administered. If the examinee could not reach the minimum score, the SPM were administered. 80% of the persons contacted agreed to participate in the study. In total, 645 persons completed the APM I and either the SPM or the APM II. For the standardization, a conversion of SPM scores into APM scores (and vice versa) previously established by Andrich & Dawes was used; details of the procedure are missing from the manual, but the conversion table is printed. The norms are available only as relatively coarse percentile norms for the percentile ranks 5, 10, 25, 50, 75, 90 and 95.
USA 1993
One year later, the SPM (likewise together with the APM) were standardized on an American sample in Des Moines, Iowa. This study, too, is reported in the SPM manual23). Analogous to Dumfries, the demographic characteristics of the residents of Des Moines corresponded approximately to the overall picture of the US population. The design of the study essentially corresponded to the conditions of the study in Dumfries. However, assembling a random sample proved more difficult in the USA than in Great Britain, since no lists of names and addresses were available in the USA. For this reason, census data were used to locate the different residential areas through which a representative picture of the population to be examined with regard to age, ethnic background and socioeconomic status could be obtained. From 30 such residential areas, a group of 1000 examinees representative of the total population of Des Moines was selected. In practice, 972 addresses could then be identified with this procedure. The response rate of the persons contacted was 70%. The study yielded a total of 625 usable data sets. 8% of the examinees were Black, 87% White and the rest of Asian origin or Hispanic. Test administration and analysis corresponded to those in Dumfries. The norms were created for each age group by fitting the curves to smoothed overall norms.
The American norms of 1993 are printed in great detail in the English SPM manual. For each raw score, the percentile values for each age group are tabulated there in five-year intervals. The table ends at percentile values of 3, and for the higher age groups also at 4 or 5. McKinzey & Raven supplemented these data in 2005 in a paper in the online journal WebPsychEmpiricist24) with more precise values in the lower ability range.
Belgium 1993
The Belgian psychologist Deltour undertook a norming of the SPM in Belgium between 1984 and 1990.25). Psychology students examined 10 examinees each with the SPM over several years, with a fixed specification of how many examinees from each of four school education levels were to be examined. The results of the norming are reproduced in tabular form in the German26) and English27) SPM manuals, albeit somewhat coarsely for the percentiles 5, 10, 25, 50, 75, 90 and 95. The sample size of 952 persons is large, and the age range extends from 25 to 84 years with large cell sizes in all age groups.
Special samples
The English SPM manual prints the results of two further groups of adults which - like the small German samples - can help to assess the validity of the larger samples. For both groups, rough percentile rank norms are available.
The first group consists of French workers who, as part of a mandatory safety screening for risk-related jobs (e.g., bus and locomotive drivers), had to complete the SPM, among other tests. The study was first reported in 1987 and was compiled and smoothed by J. Raven for printing in the English SPM manual.
The second group is similarly composed. It consists of job seekers who were tested by the UK Employment Service. It is not reported in which year the data originate.
Evaluation of the norming studies
Of the available norming studies, only three at all aimed to draw representative samples of an adult population. Raven's studies in Scotland and Iowa assume that the chosen regions are representative of the UK and the USA, respectively. In the Belgian study such information on regional representativeness is missing; moreover, the sampling was stratified exclusively by school education, which is at least probably the most important sociodemographic parameter for this type of data.
In all three studies the examinees were able to work without a time limit, which corresponds to the recommendations in the test manual. In the British and US studies, however, there were deviations from the usual standard procedure in two respects. The interviewers, who made the first contact with the examinees, first presented Part I of the APM and scored it. If the examinees reached 8 (later raised to 10) or more points in Part I of the APM, the interviewers left Part II of the APM with the examinees to complete; if the score was lower, they left the complete SPM. The results were collected later, often only after a week. Thus, on the one hand, the standard procedure for completing the SPM was in fact never given. Either there was the additional practice period with Part I of the APM, or the SPM was not administered at all but rather the APM, from which the presumed SPM raw score was calculated with the help of previously collected equivalence data. On the other hand, it was also no longer guaranteed that the examinees worked alone, and it is conceivable that the norms are biased upward by additional solution attempts by other persons. This particular procedural detail has also been discussed controversially in the literature.28) 29) 30) The arguments with which Raven counters Gudjonsson's view in his reply are indirect and not very expedient.
From the percentile values of the two studies (for the British ones Table SPM8 in the manual, for the US ones Table SPM13, detailed by Kenzie & Raven's later supplement31)) the raw score distributions can be approximately reconstructed. For the British data this was already shown above in Figure 3; the following Figure 4 shows the back-calculated distribution for the US data, which can be depicted in much finer detail. If one compares both with the data of the Belgian study already shown above in Figure 2, the markedly higher medians stand out first, but also a different distribution shape.
Figure 4: Back-calculated distribution shape of the SPM raw scores in the US norming of 1993
For the evaluation it is useful to bring these three normings together with the other data sets presented above, namely the two small German adult samples and the two studies with employment office data from France and Great Britain, and to look at the age trajectories of the SPM medians. Figure 5 shows these data in synopsis.
Figure 5: Mean SPM raw scores across age for all SPM studies discussed in the text
The British and the US data are in fact very close to each other. Especially the US data are obviously also very heavily smoothed. It would otherwise be extremely unlikely that the medians per age group from 18 to 47 years are exactly the same. Only after that does the expected age-associated decline in performance appear. In the UK data the age-related decline begins at around 40 years. The Belgian norms are lower by about 5 raw score points. The age trajectories are similar. In the Belgian data the decline begins somewhat earlier, which is also plausible for a problem-solving test.
Where, then, are the medians and means of the other four studies? The German student sample lies where one must expect it: above the three large normative samples. However, it is quite astonishing that the UK examinees in two age groups come close to the German student sample. That could well have to do with the long and uncontrolled completion time having led to elevated performance. The second German (convenience, not further specified) sample lies somewhere between the three large normings. That is not implausible and would probably be about what one would expect. The two employment office samples differ enormously. While the British Employment Service clients are close to the Belgian means (with a dip in the first two age groups), the French data are far below. The difference is considerable and cannot be explained by the 15-year difference in the time of data collection. It certainly plays a role here that in France applicants for simpler (albeit safety-relevant) jobs were examined. Unfortunately, nothing is reported about the administration details. The age-associated decline in performance in particular would need an explanation.
Meta-analytic summary of the norms
Only the UK, US and Belgian samples come into consideration for a meta-analytic summary, because only these make a certain claim to representativeness. All three samples cover a relatively wide age range, which makes them interesting as a normative basis. A shortcoming of the British study is the very coarse percentile information, which gives raw scores only for the percentile ranks 5, 10, 25, 50, 75, 90 and 95. The US norms are considerably finer; they give the percentile value for each raw score, with McKinzey & Raven's supplement down to a percentile value of 1. A problem with both is the long and unsupervised completion time, which could have allowed artificially elevated norms. In order not to give this possible shortcoming too much weight, only the US and the Belgian studies were used for the tdb2 normative basis. Together they comprise 1558 persons in the age range between 20 and 84 years.
The Belgian norms were available in 10-year intervals (e.g., 50-59), the US norms in 5-year intervals, but offset from them (e.g., 48-52, 53-57, 58-62). Since the US norms are smoothed across the age groups anyway, only those age groups that fell exactly within a (Belgian) 10-year group were considered for averaging (i.e., 50-59 of the Belgian norms was combined with 53-57 of the US norms). The percentiles were converted into IQ-normed standard scores (100;15) and averaged at this level.
With distributions percentile-to-normal-transformed in this way, unattractive effects arise at the edges of the distribution, because empirically obtained percentile rank distributions are always censored at the ends with usual sample sizes. This problem has been discussed several times - above all in connection with the assessment of intellectual disability (Lit) - and the US norms were in fact already once extended downward with the supplement by McKinzey & Raven. However, even this extension was insufficient for representing the performance scores of low-performing older persons. The relationships between raw scores and performance scores were therefore corrected manually in the lower ranges, the pragmatic solution principle being to keep the difference between the IQ scores of examinees of absolutely equal performance but different ages constant in the sense of an interval scale.
Figure 6 shows - like Figure 5 - the SPM raw scores for all SPM studies discussed in the text across age. In addition, the medians of the meta-analytic integration (dark blue and bold) and their 16th and 84th percentiles (light blue and bold) were superimposed. Only for this purpose were they interpolated into 5-year groups.
Figure 6: Mean SPM raw scores across age for all SPM studies discussed in the text and for the meta-analytic summary
The figure shows that the mean performance of the (young) German students lies roughly in the range of IQ 115 (at PR84 of the meta-analytic integration, i.e., one standard deviation above its median), a value that is not implausible.
Overview of performance norms and age norms
Figure 7 shows the ability and age norming of Raven's SPM in a graphical overview. In this figure, the vertical bars mark the performance scores from 40 to 145; the raw scores (items solved) are entered on the black horizontal line. The performance scores were created on the basis of the 20-29-year-olds. The lines drawn alternately in red and blue contain the norm limits for all age groups in overview. The five markers on each line stand for the percentile ranks 2.5, 16, 50, 84 and 97.5. One such line, namely the one corresponding to the examinee's age group, is drawn (in a different form) in the TDB2Online profile sheet to help with the individual interpretation of the test results.
Figure 7: Performance scores of the SPM with raw scores and age norm limits (see text)