Raven Advanced Progressive Matrices
Abbreviation: APM
Background
The Advanced Progressive Matrices (APM) by John C. Raven are a more demanding alternative to the Standard Progressive Matrices (SPM). The starting point for their development was the low discriminatory power of the SPM in the upper quartile of the test score distribution, especially in above average gifted adolescents and adults. The first version of the APM dates from 1943 and was developed for personnel selection in the military. Between 1947 and 1962, the test was revised and analyzed several times. Since 1962, it has been available in the version that is customary today1).
The APM are used with children as well as with adults. Within TDB2Online, only the application in adults is covered.
The test description of the Standard Progressive Matrices is somewhat more detailed than the present one of the APM; an example of the type of tasks can also be found there.
Test material
The APM consist of two parts (Set I and Set II), whose tasks are published in two separate test booklets. For both there is a separate answer sheet. Part I consists of 12 rather easy tasks that are used as learning and warm-up material for Part II. As with the SPM, the tasks consist of incomplete geometric figures or patterns that are to be completed from eight response alternatives each (multiple choice). The 36 tasks of Part II are comparable to the first part in presentation and basic structure but substantially more complex. Normally, only the sum of correct solutions in Part II is used as the test score.
The currently valid version of the original test was published in 19982). After a change of publisher and a merger, it is currently produced and distributed in unchanged form by Pearson Assessment in San Antonio, TX.
A first German version by Kratzmeier and Horn was published in 1980 by Beltz Test3). It contained German norms only for 15- and 16-year-olds, and, deviating from the normal scoring, both test parts were included in the sum score.
As part of the complete edition of the Raven tests by Swets Test Service (later Harcourt Test Service, now Pearson Assessment & Information GmbH, all in Frankfurt/Main), the APM were published in 1998 in the adaptation by Bulheller and Häcker4). This is the currently valid German version. It contains German norms from 1997, which, however, essentially cover school children aged 13 years and older and university students. The age groups over 30 are comparatively thinly populated, those over 40 very thinly. The sample does not aim at representativeness and is therefore not usable for general norming purposes in adults.
The German version first published by Swets Test Service is currently published by Pearson Assessment & Information in Frankfurt. It can be obtained directly from them or via the test distributors in Germany or Switzerland. To administer the APM, one definitely needs the two test booklets (Set I and Set II) and the corresponding answer sheets. At least a cursory familiarity with the content of the German APM manual5) is necessary, even though it contains much theory and practically no directly usable norms for adults. In addition, the test authors recommend reading the basic manual on the Raven matrices tests6).
Copyright
J. C. Raven's rights to his test were passed on from publisher to publisher and today are held internationally by Pearson Assessment, represented in Germany by Pearson Assessment & Information GmbH in Frankfurt.
Test administration
The current manuals recommend administering the APM as a power test without a time limit. Part I serves to explain the task principle of the test in detail (and nonverbally!). The test can be administered both individually and as a group test (although the group size, especially in clinical applications, should be limited to a few persons). Administration takes under 10 minutes for Part I and about 40 minutes for Part II, but in individual cases can take much longer.
In contrast to the SPM, for which a time-limited administration is always discouraged because of the inhomogeneous arrangement of task difficulty, the APM can in principle also be administered as a time-limited test. However, the norms presented below refer to the unlimited administration.
Test raw scores
The test raw score is the number of correct solutions in Part II of the APM. The range of values goes from 0 to 36.
Distributional properties of the raw scores
There are no precise APM distribution data from random samples of adults in the literature. However, distribution shapes can be reconstructed. The data of the 1992 APM norming in Great Britain are tabulated in the APM manual as relatively fine percentile values. With this information, the distribution shape of the raw scores can be displayed; Figure 1 shows it separately for three age groups.
Figure 1: Back-calculated distribution shape of the APM raw scores in the 1992 British norming for three selected age groups
In all three age groups, the raw score distributions are symmetric but truncated at both ends. In practice, this means that the test measures relatively finely only in the middle range, not at the extremes.
Linearization of the measurement scale
From the beginning, norms for the Raven tests were given as percentile ranks. This takes into account the fact that the relationship between raw scores and IQ scores is not linear. On the basis of the available percentile rank norms, the conversion into linear performance scores via the relationship between percentile ranks and units of the normal distribution is formally simple.
Norming studies
The two German APM manuals did not report satisfactory German norms for adults (for details see below). For this reason, as already with the SPM, all international norming studies that meet at least a certain standard of representativeness of the sampling were compiled for the results output of TDB2Online. This work was made easier by the APM manual7), in which the relevant studies up to about 1997 are printed in tabular form.
Germany
In the first German edition of the APM from 19808), only children were studied. The 1998 version9) does contain normative data, but without any claim to representativeness. Of the 1,142 persons studied, 466 were between 13 and 18 years old. 442 were in the age range between 20 and 29 years. However, they consisted almost exclusively of persons with a vocational high school or university entrance qualification (357) or a university of applied sciences or university degree (38). Only 21 had a lower secondary school certificate. In the upper age ranges, the sample thins out. In the age range between 40 and 49 years there were 51 persons, between 50 and 75 only 27 persons. Nothing is reported about the intended source population. This study is not usable for norming purposes; its results are, however, reported descriptively below.
One must therefore conclude that, unfortunately, there are no German SPM norms for adults. As with many other neuropsychological tests, we will have to make do for the time being with a meta-analytic summary of norms from other countries.
Great Britain 1992
The first early norming studies with the APM were rather unsystematic and are outdated from today's perspective. A representative sample survey (with SPM and APM) did not take place until 1992 in the town of Dumfries in Scotland. It is also presented in detail in the German APM manual10). At that time, Dumfries had about 25,000 inhabitants. With regard to demographic characteristics, Dumfries corresponded to the overall picture of Great Britain. A random sample (every 29th person entered in the electoral register) was notified in writing of a visit by a research assistant. During the home visit, the aim of the study was then explained and Part I of the APM was administered. If the test taker reached a minimum score (the cutoff score was initially set at 8 points and later raised to 10 points), Part II of the APM was administered. If the test taker could not reach the minimum score, the SPM were administered. 80% of the persons contacted agreed to take part in the study. In total, 645 persons completed the APM I and either the SPM or the APM II. For standardization, a conversion of SPM scores into APM scores (and vice versa) previously established by Andrich & Dawes was used; details of the procedure are missing from the manual, but the conversion table is printed. The norms are available in the form of percentile values for raw scores between 4 and 35 and are thus fairly complete for the younger examinees. What is missing are the low raw scores of the older examinees. A raw score of 4 already corresponds to a percentile rank of 10 in the group over 67 years.
USA 1993
One year later, the APM (likewise together with the SPM) were standardized on an American sample in Des Moines, Iowa. This study, too, is reported in the German APM manual11). Analogously to Dumfries, the demographic characteristics of the inhabitants of Des Moines corresponded approximately to the overall picture of the US population. The design of the study essentially corresponded to the conditions of the Dumfries study. However, assembling a random sample proved more difficult in the USA than in Great Britain, since no lists of names and addresses were available in the USA. For this reason, census data were used to identify the different residential areas through which a representative picture of the population to be studied with regard to age, ethnic background, and socioeconomic status could be obtained. From 30 such residential areas, a group of 1,000 test takers representative of the total population of Des Moines was selected. In practice, 972 addresses could then be identified with this procedure. The response rate of the persons contacted was 70%. The study yielded a total of 625 usable data sets. 8% of the test takers were Black, 87% White, and the remainder of Asian origin or Hispanic. Test administration and analysis corresponded to those of Dumfries. The norms were created for each age group by fitting the curves to smoothed overall norms.
The American norms of 1993 are printed in somewhat less detail; there, the table for each age group begins only at a percentile rank of 3, which is why the lower raw scores are censored.
Linking of SPM and APM
The manuals for the SPM and APM, including the German versions, each contain tables with the help of which one raw score can be converted into the other. The tables go back to work by Andrich and Dawes, which is described only in the Research Manual12) and even there only very briefly. Evidently, this is an equipercentile equating (see e.g. Kolen & Brennan13) ) that was carried out on a relatively broad sample and covers the usable range of values of both tests.
The literature contains one further study on the equipercentile equating of APM and SPM14). This study was conducted with 261 undergraduate college students. Here, too, corresponding conversion tables were provided.
In Figure 2, the data of both studies are shown together graphically.
Figure 2: Equipercentile equating of SPM and APM in two studies
It is immediately apparent that the equating curve from the Raven manual covers a broader range of values and at the same time either included more persons or is more smoothed than the curve based on the data of Jensen et al. Because of the restriction to college students, the intellectually less capable persons are missing from the latter study, and the curve breaks off at an SPM score of around 40. It is unclear why the curves lie relatively far apart in the upper range. Jensen's students scored 2 or 3 raw score points lower on the SPM at comparable APM scores. Perhaps it has to do with the time limit in this study: both tests had to be administered within 40 minutes, which leaves comparatively less time for the 60 tasks of the SPM than for the 36 tasks of the APM, even if the latter are on average more difficult.
For the practical purposes of equating, the data from the Raven manual in any case appear to be more comprehensive and reliable than those of Jensen and colleagues.
Evaluation of the norming studies
Of the available norming studies, only two even aimed to draw representative samples of an adult population. Raven's studies in Scotland and Iowa assume that the regions chosen are representative of the UK and the USA, respectively. In both studies, the examinees were able to work without a time limit, which corresponds to the recommendations in the test manual.
In the evaluation of these two norming studies for the SPM (these are, after all, the same studies!), concerns were raised that have to do with the deviation from the usual procedure. There were indications that the norms, at least in the upper half of the distribution, turned out rather hard. If one applies these arguments to the APM as well, it does not seem sensible to make only these two studies the normative basis.
The available data on the equipercentile equating of SPM and APM could also be used to convert the Belgian SPM norms into APM scores. This approach offered the advantage that both tests are based on the same normative basis.
This resulted in a total of four data sets, namely the UK and US studies, the Belgian study on the SPM, and the German norming of the APM. Figure 3 shows the age trajectories of the mean APM raw scores for these four studies.
Figure 3: Mean APM raw scores across age for four studies
As expected, the British and US American data lie as close together as their SPM raw scores. The Belgian norms lie - likewise similar to the SPM - by around 5 raw score points lower. The age trajectories are similar. The German sample is still three to four raw score points above the Anglo-American values. Given the predominantly university entrance or even university-level education of this sample, this is plausible.
Meta-analytic summary of the norms
By analogy with the procedure for the SPM, only the US American study and (via the conversion from SPM to APM) the Belgian study were used for the tdb2 normative basis. Together, they comprise 1,558 persons in the age range between 20 and 84 years.
The Belgian norms were available in 10-year intervals (e.g., 50-59), the US American ones in 5-year intervals, but offset from these (e.g., 48-52, 53-57, 58-62). Since the US norms are smoothed across age groups anyway, only those age groups that fell exactly into one (Belgian) 10-year group were considered for averaging (that is, 50-59 of the Belgian norms was combined with 53-57 of the US American norms). The percentiles were converted into IQ-normed standard scores (100;15) and averaged at this level. The oldest group was not presented because, at low SPM scores, the conversion into APM scores is no longer very accurate. This is not a real limitation: only very rarely will the APM be the test of choice in persons over 70.
With distributions percentile-to-normal-transformed in this way, unsightly effects arise at the margins of the distribution, because empirically obtained percentile rank distributions are always censored at the ends for usual sample sizes. This problem has been discussed several times before - above all in connection with the assessment of intellectual disability (Lit). The relationships between raw scores and performance scores were therefore corrected manually in the lower ranges, the pragmatic principle of the solution being to keep the difference between the IQ scores of examinees of absolutely equal ability but different ages constant in the sense of an interval scale.
Figure 4 shows - like Figure 3 - the mean APM raw scores for the four studies across age. In addition, the medians of the meta-analytic integration (dark blue and bold) and their 16th and 84th percentiles (light blue and bold) were superimposed. Only for this purpose were they interpolated into 5-year groups.
Figure 4: Mean APM raw scores across age for the APM studies discussed and for the meta-analytic summary
As with the SPM, the mean performance of the German sample, with predominantly secondary school (Gymnasium) or university education, lies approximately in the range of IQ 115 (at PR 84 of the meta-analytic integration, that is, one standard deviation above its mean), a value that is not implausible.
Overview of performance norms and age norms
Figure 5 shows the ability and age norming of Raven's APM in a graphical overview. In this figure, the vertical bars mark the performance scores from 40 to 145; the raw scores (tasks solved) are plotted on the black horizontal line. The performance scores were constructed on the basis of the 20-29-year-olds. The lines drawn alternately in red and blue contain the norm limits for all age groups in overview. The five marks on each line stand for the percentile ranks 2.5, 16, 50, 84, and 97.5. One such line, namely the one that corresponds to the age group of the examinee, is drawn (in a different form) in the TDB2Online profile sheet to help with the individual interpretation of the test results.
Figure 5: Performance scores of the APM with raw scores and age norm limits (see text)
Discussion
Fields of application of SPM, APM, and CPM
Quite clear recommendations can be derived from the available data as to when which version of the Progressive Matrices is best applied. Figure 6 shows the raw scores of the SPM, APM, and CPM on the performance score scale. The conversion of raw scores into performance scores for the SPM and APM is based on the meta-analytically derived norms in TDB2Online. For the CPM, the CPM raw scores were first converted into SPM raw scores, using a conversion table by Andrich and Dawes given in most Raven manuals (for example, in Table 11, page 62, of the German SPM manual15)). The SPM raw scores obtained in this way were then converted into performance scores like normal SPM raw scores.
Figure 6: Raw score points of the SPM, APM, and CPM on the performance score scale
Each point in the curves corresponds to a raw score sum. Considering first only the maximum range of values, which goes from 0 to 36 for the APM and CPM and from 0 to 60 for the SPM, the SPM can measure performance scores from 57 to 131, the APM those from 64 to 135, and the CPM those from 57 to 118. However, one must also keep the discriminatory power of the tests in view. The narrower the horizontal distance between the points at a given location, the more finely the test can measure at that location (determined simply by raw score density).
Comparing SPM and APM in this respect, an advantage for the APM can be seen from a PS of about 105 upward, with an increasing tendency at the high performance scores. Between 90 and 105 there is no difference in discriminatory power; below 90 the SPM measure more finely. In the clinical setting, use of the APM would therefore be recommended if one has reliable prior information that a person will achieve at least average performance (in absolute terms, that is, performance scores) on such tasks. In the usual clinical examination situations, the Standard Matrices are therefore probably mostly preferable, especially since finer differentiation in the upper performance range is rarely likely to be among the clinical questions.
Comparing the SPM with the CPM, one must conclude that in adults there is actually no point at which the CPM have really better discriminatory power, but they have markedly worse power above performance scores of about 65. The CPM are therefore only very rarely appropriate in adults.