The detailed documentation consists of two parts:

• This page describes the background, the concepts, and the general procedures of TDB2Online
• A description of the individual tests, in particular of the methods applied in processing them, can be found under the navigation item App and Test Documentation and is accessible only to registered professional users. A brief description of the tests is also accessible without login via the home page.

Two short versions are available for quick information:

• A one-page description of the properties of the TDB2 profile is intended to be handed out together with the printed test report.
• A nine-page introduction explains the concepts and procedures of TDB2Online in brief form and gives a number of notes for practical report writing.

TDB2 and TDB2Online are programs for visualizing neuropsychological test data. They were developed by Rolf R. Engel (psychometrics) and Ralf Mayr (programming). Scoring algorithms and presentation of results are identical in TDB2 and TDB2Online. TDB2Online collects raw test scores via a simple online form and outputs a graphically processed test profile as a pdf file. TDB2, in contrast, runs within a protected data network. In addition, it maintains a database of patient and test data, supports report writing, and manages an archive of the digitized original documents. TDB2 has been in use since 2009 in the Psychological section of the Department of Psychiatry and Psychotherapy, University Hospital, LMU Munich, and a predecessor program (TDB, “Testdatenbank”) since 1995.

This documentation describes the general procedure for the psychometric processing and presentation of test results. First, the fundamental psychometric concepts on which the processing of the data is based are presented. In applied assessment, however, theoretical assumptions quickly reach their limits, because there are considerable insufficiencies in the practical feasibility of theoretical demands. The documentation summarizes this under the heading Problem areas in practice. The main part of the documentation then presents, as a compromise of sorts between theoretical desiderata and the needs of practice, the general procedures used in the programs.

TDB2Online is intended and able to score and present tests that are constructed in very different ways. Different measures are therefore applied for each individual test. This documentation describes only the general principles of processing and presentation, regardless of whether or not they are (or can be) applied to an individual test. In the protected area of the TDB2Online wiki, the actual measures that were necessary to include the test in TDB2Online are described for each individual test.

The documentation is not intended for laypersons in psychological assessment. It assumes that readers have a background in psychological assessment such as is generally acquired through a university degree in psychology. Relevant textbooks include, for example, Cohen & Swerdlik (2018)¹⁾ for the overall field of psychological diagnostics, and Furr (2022)²⁾ for the background in test theory and statistics.

Raw scores are the elementary values by which performance on a test is evaluated. They depend on the type of task. Examples are:

• Number of correct solutions in a knowledge test (example: Information subtest in the Wechsler test batteries)
• Number of errors in a categorization task (example: Halstead Category Test)
• Time to complete a visuoconstructional task, in seconds (example: Trail Making Tests)
• Number of correctly reproduced words in a memory test (example: Auditory Verbal Learning Test)
• Difference between the number of correctly recognized words minus the number of falsely “recognized” words in a recognition task (example: Auditory Verbal Learning Test)
• Points in a complex problem-solving task that is scored according to correctness and speed of performance (example: Block Design subtest in the Wechsler test batteries)

Raw scores are test-specific. The points in Block Design, for example, have numerically nothing to do with the times in the Trail Making Test; they are not even keyed in the same direction. With raw scores, performance cannot be compared between tests (and thus between different cognitive domains).

For comparisons between persons, on the other hand, raw scores are quite suitable. Simple raw scores are also often used when comparing groups (especially if they are comparable with regard to other parameters).

Standardized test scores leave the numerical level of raw scores. They are based on a comparison: The performance of the person currently being tested is compared with the performance of a normative sample that has already completed the test. Ideally, the data come from norming studies on random samples in which several hundred persons per age group are tested. Standardized test scores thus indicate the relative standing of an examinee within a meaningful population-based comparison group. In general, scales derived from the normal distribution are used for standardization. In TDB2Online, the IQ scale with a mean of 100 and a standard deviation of 15 is used uniformly for all tests.

Depending on the choice of the reference group for the comparison, several types of test scores can be distinguished, representing different aspects of application.

Performance scores (PS, “Leistungswerte” (LW) in German) are, in general usage, measured values that describe a particular performance quantitatively. In medicine, there are performance measures for cardiac function or breathing under particular stress conditions. In psychology and education, performance scores are used in the PISA study to measure cognitive performance across subjects and countries.

An essential peculiarity of performance scores is that all objects or persons to be measured are assessed with the same scale. While this is the case for physical measurements anyway, the results of psychological tests (e.g., IQ scores) are almost always reported on relative scales that apply only to particular groups. Here, performance is reported in comparison to persons of similar age or similar education. In the process, the absolute performance dimension is lost, and a comparison of performance across age groups (e.g., “Sixty-year-olds are 20 IQ points worse on this test than twenty-year-olds”) is no longer possible at the level of scale scores. In the usual scoring of tests, only the raw score level remains for such a comparison, but then the comparison across subjects or domains between different tests is no longer possible.

In neuropsychological assessment, the measurement and comparative evaluation of deficits in different performance domains is the focus. For this task, an absolute unit of measurement is a significant advantage, because it allows direct comparisons over time and/or across domains. Performance scores make performances comparable across different tests and types of raw scores. They provide a transferable yardstick for evaluating individual cognitive performances in standardized form. The relation to age, which is likewise necessary for interpretation, can be established, as in most other areas of clinical assessment, by means of age norms.

Performance scores are ideally standardized on a sample of young adults roughly between 20 and 30 years of age. The rationale for choosing this age group is that this is when the individual peak of cognitive performance on standardized tests is generally reached. (The only exception is knowledge tests. Here the performance maximum occurs later, but the difference from persons in their twenties or thirties is small.) In persons younger than 20 or older than 30, performance declines. There are large differences in the rate of decline between cognitive domains. In some areas (for example, almost all verbal abilities), the rate of decline is very low (example: Information subtest in the WAIS-IV). In tests that make demands on visuomotor speed, it is very high (example: Digit Symbol Coding subtest in the WAIS-IV). Because of the differing decline of abilities, age groups other than young adulthood are not suitable for standardization if one wants to compare performances across different abilities. Even with a standardization across all age ranges, the differing rates of decline would distort the assessment between performance domains.

David Wechsler applied this principle prominently with his classic “scaled scores”. This enabled him to compare performance between the subtests of his intelligence batteries (something that is not possible with raw scores). This form of absolute performance measurement was intended for the profile presentation of performance in the individual subtests up to the WAIS-R (the test documentation for the WAIS-III gives details on this). It was abandoned in the WAIS-III and, currently, also in the WAIS-IV in favor of an age-dependent presentation of subtest performance, whereby the absolute performance yardstick was lost.

For many tests used in dementia assessment, there are no norms for young adults; a prominent example is the CERAD Test Battery. For these tests, the norms mostly come from examinees in the higher age range. For pragmatic reasons, there is no alternative but to relate the performance scores to these persons. Details are given further below in the section on the calculation of performance scores.

Age-normed scores were introduced to facilitate the interpretation of an individual patient's performance relative to his or her age cohort. They, too, are standard scores, just like performance scores. The difference is that the reference basis for the standardization comes from the part of the normative sample that is comparable to the patient in age. These scores are hardly usable for an absolute assessment of performance across different tests, because different abilities change differently over the course of age. They are, however, an important interpretive aid in assessing the performance of an individual patient relative to his or her age cohort. The clinical assessment in the verbal test report is based essentially on the age-normed scores. For most tests, manual scoring usually yields, besides the raw scores, only these age-normed scores, not age-independently standardized performance scores.

There is a considerable difference in substance between performance scores and age-normed scores. Performance scores are actual “measured values”. They are merely transformed into standard scores in the same way for all examinees (with the same formula for everyone). The age-normed score, in contrast, incorporates information that is not the same for all persons. A different conversion path from raw score to standard score applies to each age group. With these scores, one leaves the level of measurement and enters the level of interpretation. More on this later.

Scales derived from the normal distribution are generally used as the yardstick for cognitive performance. The simplest variant would be to use the abscissa values of the normal distribution, the well-known z scores. These have a mean of zero and a standard deviation of 1. The range of the characteristic relevant for clinical purposes would thus lie roughly between -4 and +3. However, z scores are not particularly suitable for communication, because one would have to deal with decimal fractions and negative numbers. For this reason, only scales derived from them have been used for decades. The most important of these are shown in Figure 1.

The IQ scale is mostly used in the measurement of cognitive abilities, the T scale more often in personality tests. The standard score scale with a mean of 100 and a standard deviation of 10, which is quite popular for intelligence tests in Germany, is hardly used internationally.

The particular numerical properties of the IQ scale can be seen from Figure 1. Most important here is the comparison between the values of the IQ scale and the values of the percentile rank scale. One can see that an IQ score of 130 corresponds to a percentile rank of 97.7, which means nothing other than that 97.7 percent of the comparison sample have a lower IQ than 130 and the remaining 2.3 percent a higher one. For every IQ score there is a corresponding percentile rank value, which can be calculated or taken from more detailed tables.

Figure 1: Normal distribution and some of the standard scales derived from it

The IQ scale has the pleasant property that easily remembered and interpretively meaningful verbal descriptions suggest themselves for particular sections of the scale. Between the IQ scores 90 and 110 lie about 50 percent of the normative sample. According to David Wechsler's recommendations, somewhat simplified and smoothed here, this range can be described verbally as “average”, occasionally of course with additional qualifying remarks (“lower average range”, “at the upper end of average”, or the like). The IQ range of 80-90 comprises about 16 percent and is described as “low” or similar; below it, at 70-80, lies the “very low” range (about 7%), and still below that (under 70) the “extremely low” range. Statistically (viewed from the normal distribution), this accounts for about 2.3% (in reality it is larger by 1 to 2 percentage points, because this range contains the disease-related intellectual disabilities (genetic/prenatal/perinatal), which are responsible for the fact that cognitive abilities do not show completely symmetrical distributions). The upper intelligence range is divided accordingly; Table 1 gives the overview.

Table 1: Interpretation ranges of the IQ scale

A percentile rank scale can be formed for any ordinal scale. This requires no normal distribution of the characteristic, not even a unimodal distribution. The percentile rank scale gives the relative standing of a person within the entire normative sample as a percentile rank (PR). If an examinee has a percentile rank of 90, then 90 percent of the normative sample achieve a lower performance and 10 percent a better one than he or she does.

Using the percentile rank scale in the measurement of cognitive abilities has great pitfalls. Normally one can assume that cognitive abilities show a unimodal distribution, often also a fairly symmetrical one. For such distributions resembling the normal distribution, the percentile rank scale has the unpleasant property that its scale values are not equidistant. On the scale underlying such a distribution, the percentile ranks 40 and 60 lie closer together than the percentile ranks 60 and 80. We are not used to dealing with scales whose values have a nonlinear relation to the underlying measured quantity. The percentile rank scale would be equidistant only if the underlying characteristic had the form of a rectangular distribution.

The percentile rank scale is therefore ultimately suitable only for the coarse formation of classes, for example in the sense that it separates persons below a PR of 16 from those above this percentile rank.

The scale of z scores underlying the normal distribution fulfills the characteristics of an interval scale, because all differences between adjacent scale values are equal. Without this characteristic, the shape of the distribution would not be defined.

For many raw score scales with which we evaluate cognitive performance, this does not necessarily hold. Raw scores based on the number of correct answers usually yield approximately symmetrical distributions, at least when there are no ceiling effects. Raw scores based on error frequencies or on time measures, in contrast, often yield strongly asymmetrical, positively skewed distributions that are due to a floor effect: Downward (toward zero) the scale has a limit that cannot be undercut, whereas upward the time or the error frequency is hardly limited. These distributions additionally have the unpleasant property that, in subgroups, the standard deviations are correlated with the means: Groups of persons who need more time (for example, older persons) also have higher standard deviations. Such scales no longer meet the prerequisites of an interval scale, because equal differences at different points of the scale have different meanings. In the Trail Making Test A, for example, an improvement in performance from 20 seconds to 15 seconds is an enormous step. In this range, the distribution curve is very steep. An improvement from 70 to 65 seconds, on the other hand, is insignificant in terms of content. Both values characterize a very poor performance. Here the distribution curve is very flat. Equal raw score differences are thus of different substantive importance here.

We have good reasons to assume that cognitive performances are normally distributed. Non-normal raw score distributions are a consequence of the chosen raw score scale not being suitable for measuring cognitive performance, at least not without transformation. Only when a normal distribution has been produced by a transformation of the raw scores does the (new) scale correspond to an interval scale suitable for the measurement of cognitive performance.

In test construction and/or the creation of norms, this problem is handled in different ways. Many test authors ignore the problem; some try to find a solution. Ultimately, only a linearization of the scale by adapting the raw score distribution to a normal distribution is able to remove the disadvantages of a nonlinear distribution. There are various methods for this, which are discussed in the following.

Arithmetic transformations have the advantage that they can be carried out repeatably in both directions. However, a formula that normalizes the raw score distribution cannot always be found. Sometimes there are empirical rules of thumb. With a square root transformation or an arcsine transformation, one or another raw score distribution can be normalized. There are also general families of equations with which unimodal distributions can often be normalized. Some cases remain, however, for which no formula for a satisfactory linearization can be found.

If the cumulative percentile ranks of an empirical distribution are transformed into z scores of a normal distribution, any distribution can be normalized. Here the transformation takes place not at the level of the scales but at the level of the distribution function. The non-normal distribution function of the original scale is converted into that of a normal distribution by assigning to the empirical percentiles of the cumulative original distribution the abscissa value that corresponds to this percentile in a cumulative normal distribution. These transformations can be called, briefly and succinctly, percentile-to-normal transformations.

The disadvantage of this procedure is that no function can be given for the transformation. The transition function is determined purely empirically and defined only by the support points supplied by a table of raw and standard scores. Most raw scores, however, are graded finely enough to supply a sufficient number of support points. It becomes difficult when the raw scores actually occurring are limited in number. This is the case, for example, with some memory tests, when only raw scores between 0 and 10 occur. It also becomes difficult when higher or lower raw scores occur in application samples than in the norming population. Without extrapolation, such values are not defined.

Psychological assessors measure or estimate psychological characteristics and derive from them statements of current or future practical relevance. To guide this judgment into comprehensible channels, test authors generally provide test users with aids. An important aid for interpretation is the norming of test scores. By norming, all numerical processes are meant here that a test author prescribes between the collection of the raw score and the determination of the test score. Looking at the tests used in the clinical setting, one finds very large differences in the elaborateness of the norms. The range extends from the use of raw scores with simple cutoff scores on the one hand to very complex test scores that take the influence of several variables into account. Many users are not always sufficiently aware of the difference between the various techniques.

For some tests, test scores are reported only as raw scores. Examples are the Mini Mental State Examination (MMSE) or the Frontal Assessment Battery (FAB).

This by no means has only disadvantages. Laboratory medicine as a whole works practically only with raw values. In any case, it first of all facilitates communication if there is only one kind of measured value for a given variable. However, one has to know a test very well if one wants to do something with the raw scores directly.

In laboratory medicine, a cutoff score is frequently used to separate the pathologically abnormal range of values from a normal range. Sometimes the normal range is also bounded on both sides by cutoff scores. The abnormal range, too, can be divided into different zones or stages by several cutoff scores.

This can be illustrated with the example of the Mini Mental State Examination (MMSE). There, the normal range consists only of the scores 28 to 30; everything below 28 is already abnormal. The abnormal range is also subdivided. From 27 to 24, one generally speaks only of “mild cognitive impairment”. Below 24, the dementia range begins.

As a clinician, one can certainly live with such a coarse classification. One knows and takes into account that the MMSE is only one of many indicators of dementia, that the various indications can well be contradictory, and that each one must therefore be weighted clinically. As a diagnostician, too, one can live with it, at least if further observations and tests enter into the clinical report to be written.

For some tests, norming by forming standard scores does not make sense simply because the characteristic measured does not show a normal distribution. The MMSE in particular spans such a wide range as a test that a norming based on the normal distribution is possible at most in the vicinity of the normal range (more in the description of the processing of the MMSE normative data).

Most psychological tests today use standard scores. In the simplest variant, the mean and standard deviation of the complete normative sample are used to form standard scores. In many personality tests this is done. Among ability tests, in contrast, only the few tests for which the age effect on performance is small are normed in this way. Examples are the Vocabulary Test (WST) or the Multiple-Choice Vocabulary Intelligence Test (MWT-B)³⁾. In both, the raw scores are converted into standard scores using a table that is the same for all test takers.

The direct conversion of raw scores into standard scores using the mean and standard deviation of the raw score distribution is justified only if the raw score distribution roughly corresponds to a normal distribution and the effect of age on test performance is small.

For many tests with asymmetrical distributions, the conversion into standard scores also includes a linearization. Mostly it is implicitly built into tables, as for example in the conversion of raw scores into scaled scores for the subtests of all Wechsler tests.

For many cognitive tests (for children as well as for adults), raw scores are transformed into age-dependent standard scores. The cognitive performance of children and adolescents increases with increasing age, that of adults decreases (in the higher age range). Since these processes proceed very differently for the cognitive domains, age-normed scores can facilitate interpretation considerably. One should not forget, however, that – as we have seen above – this leaves the level of measurement.

The increasing use of computers in test scoring makes it easier for test authors to include further characteristics in the calculation of standard scores. Indicators of educational history, for example the number of years of schooling, and often also characteristics such as sex or race, are popular predictors, above all in the USA but increasingly also in Europe. Mostly the standard scores are then calculated via regression equations. We view this practice critically, because the diagnostician can no longer trace how a standard score comes about. To give an example: In an examinee of advanced age and high education, his or her age leads to an increase in the standard score (compared with younger examinees), and his or her education to a decrease (compared with less educated persons). To what extent both factors have become effective for the standard score calculated in this way can no longer be seen without analyzing the formula of the regression equation.

Taking into account characteristics of the person, effects of the illness, or circumstances of the examination situation is important for the interpretation of findings in all clinical disciplines. This applies in test-based assessment just as in any other clinical assessment. Normally, however, biographical characteristics are taken into account only secondarily. The measured value itself remains unaffected by them. A usual psychological assessment consists of several tests and subtests from different cognitive domains and mostly also of tests constructed in different ways. It makes little sense if some of these tests and subtests now pack biographical characteristics (often also different ones from test to test) into a single standard score by arithmetic operations. Such a standard score is merely a construct that combines portions of the measured value and portions of the included biographical background information in a way that is practically no longer traceable for the end user.

As with many other intelligence test batteries, it is customary in the Wechsler tests to form an index score for some subtests (e.g., Verbal IQ, Performance IQ) or for all subtests (Full Scale IQ). The practice goes back to the beginning of intelligence test assessment, when the aim was above all to capture global intellectual ability and less its structure. In current neuropsychological assessment, by contrast, the focus is on the differentiated capture of individual performances and less on global ability parameters.

The simultaneous capture of individual performances and global scores has certain psychometric pitfalls. This can be explained quickly with an example. Suppose that two individual performances (general knowledge and arithmetic) are to be both captured and presented individually on an IQ scale with a mean of 100 and a standard deviation of 15 and also combined into a global performance in the sense of a “Verbal IQ”. A psychometrically naive observer will probably think that this is simple and that the Verbal IQ is calculated as the mean of the two individual performances. This is by no means the case, because in conventional IQ tests the individual performances and the total scores are standardized separately. The double standardization means that the correlation between the two subtests determines whether the same or a different metric is used for the combination. The lower the correlation between the subtests (and the greater the number of subtests included), the more the new metric deviates from the old one.

Table 2 shows the consequences in our simple example. If both subtest scores are 100 and thus lie exactly in the middle of the distribution, the level of the correlation plays no role. In this case, the combined “Verbal IQ” is always 100. It is different when the individual scores deviate from 100. If both individual scores are exactly 85 IQ points, the “Verbal IQ” would be 85 only if the two individual performances correlate with each other at 1. In practice, the correlations between subtests that are combined into a total score are mostly somewhere between .30 and .70. With the already relatively high correlation of .70, the “Verbal IQ” drops to 84 instead of 85, with the rather low one of .30 already to 81 instead of 85. The more extremely the individual scores depart from the mean of the distribution, the greater the differences become. If the individual test scores are at 70 IQ points, the “Verbal IQ” falls to 66 points at a correlation of .70, and to 63 points at a correlation of .30. The effect is symmetrical: With high test scores, the “Verbal IQ” increases accordingly. The spread factor for the metric becomes greater the lower the correlation between the subtests and the more subtests are combined.

Table 2: Effects of separate standardization on the metric

These psychometric relationships between subtests and global measures have long been known. They follow from the equation for the variance of a sum of random variables, in which, besides the variance of the individual variables, the covariance between the variables also enters (formula: Var(X + Y) = Var(X) + Var(Y) +2Cov(X,Y)). This is usually also taught in statistics courses. The different metric of subtests and IQ scores, however, is rarely really noticed in clinical assessment, for example with the Wechsler intelligence tests. Contributing to this is that scaled scores are standardized with (10;3) and IQs with (100;15). As long as one does not convert the two types of standardization and plot them on the same profile sheet, the discrepant metric is hardly noticeable.

A brief trip down memory lane: In German-speaking countries, the HAWIE-R (the German version of the WAIS-R) replaced the HAWIE (the German version of the Wechsler-Bellevue I) for adults in 1991. In the mid-1990s, many practitioners were surprised to find that, for individuals with low ability, the IQ scores calculated using the new HAWIE-R were much lower than those previously determined using the HAWIE. This could be related, on the one hand, to the phenomenon of the population's IQ gain over time (Flynn effect). The long-term trend of rising intelligence in the second half of the 20th century, first described by Flynn, means that newer tests, normed later, measure lower IQs than older ones. However, in the change from WAIS to WAIS-R this explained only part of the difference. The other part came from the fact that the mean intercorrelation of the subtests in the standardization sample of the WAIS-R (for whatever reason) was only .40, and in the age groups 20-34 years used for the IQ calculation even only .32, both clearly lower than in the old WAIS, where the mean subtest intercorrelation was about .60. The lower intercorrelation of the subtests in the standardization sample led, in the WAIS-R, to a greater spread of the metric of the IQs relative to the metric of the subtests, an effect that is especially pronounced at the extremes of the distribution and was accordingly noticed above all in the assessment of intellectual disability.

Figure 2 shows this in an example case of a 31-year-old examinee with low ability who achieved merely four scaled scores in all subtests of the WAIS-R. The classic, non-age-dependent scaled scores of the WAIS-R have a mean of 10 and a standard deviation of 3. Four scaled scores therefore correspond to a value of 70 on the IQ scale. Because of the low intercorrelation of the subtests, however, the partial IQs actually determined are, instead of the naively expected 70, only 57 for the verbal part and 57 for the performance part. Because the effect on the test metric becomes greater the more subtests are included, the Full Scale IQ is only 48. On the basis of the subtest results, one would have expected one of 70, and on the basis of the partial IQs one of 57. Figure 2 was generated with an old program that displayed the scaled scores and IQs calculated according to the manual instructions and merely added the age-specific expected norm range.

Figure 2: Example of the different metric of subtests and IQs in the WAIS-R

Nobody can deal with such different metrics in practice. The idea that different levels of aggregation of tests and subtests also entail different metrics is psychometrically consistent because of the incomplete covariance of the individual tests, but in practice it is unexpected and disturbing. That many “low” individual performances lead to a “very low” or even “extremely low” overall result is unfamiliar to most diagnosticians. For the recipients of psychological clinical reports, generally psychometric laypersons, a variable metric is likewise not comprehensible. In everyday practice, overall grades are averages of individual grades. In school, four individual grades of “sufficient” lead to an overall grade of “sufficient” and not to one of “poor”. If one wanted to actually retain the aggregation that is psychometrically correct because of the incomplete covariance of the subtests, one would have to adapt the verbal descriptions of the test results to the test levels. This is difficult, however, because the conditions differ from test to test and from norming to norming.

Figure 3: Mean performance scores as a substitute for intelligence quotients, using the WAIS-IV as an example

In a neuropsychological context, the issue is the differentiated capture of different abilities and their comparative evaluation. For this reason, only the metric of the individual tests is used in the TDB2 profile. Each individual performance is presented in a performance score that is comparable across different tests. Global scores are not presented in TDB2 in a separate standardization, because this would lead to a different metric. Instead, the performance scores of the individual tests are averaged into global scores. Figure 3 shows, in an example case, that the same test performances as in Figure 2 are aggregated in the TDB2 profile into mean performance scores that lie at the same metric level as the individual performances. As usual, the age norm limits are additionally displayed and an age-normed score (ANS) is calculated for the aggregate score and presented in the right column of the profile. These age-normed scores correspond in their composition to the usual intelligence quotients. They are age-standardized and indicate the average ability across the subtests included. In contrast to the conventional intelligence quotients, however, they are expressed in the same metric as the subtests. Linguistically, we distinguish them from the intelligence quotients of the WAIS-R, WAIS-III, and WAIS-IV, or from the index scores of the WAIS-III and WAIS-IV, by speaking of mean age-normed scores (100;15). The red dots added by hand in Figure 3 show, for comparison, the values of the intelligence quotients when calculated according to the manual's instructions.

In the practical use of psychological assessment instruments, hardly any single methodological feature is likely to weigh more heavily than the adequacy of the norms. Theoretically it is clear that adequate norms can be produced only by examining representative random samples. Unfortunately, such samples are almost never found for existing tests, in particular for cognitive ability tests. Even for established tests published by test publishers, the descriptions of the sampling remain terse. Evidently no standard has yet been established here. For the user, it is often extremely difficult to judge the adequacy of a normative sample from the descriptions in the test manuals. This also holds when the size of the normative sample would be quite satisfactory: Against systematic distortions in drawing the sample, however, even large numbers do not help.

The norm sources are particularly problematic for some classic standard neuropsychological tests that are no longer looked after by any test author or test publisher. Prominent examples are the Trail Making Tests, the Halstead Category Test, and almost all verbal memory tests. For these tests there is mostly no standard manual. Here most users rely on cursory, ad hoc compiled norm tables in neuropsychological reference works^{4) 5) 6)}. Since there are, however, several of them, drawing on different material, it can hardly be avoided that in practice highly different norm sources are sometimes used, which in an individual case arrive at highly divergent results.

In developing norms for old and very old examinees, problems arise that do not occur, at least not to this extent, in adults of working age. The primacy of population representativeness of the sample can begin to waver when, with increasing age, more and more external causes of cognitive underperformance occur. The question then arises to what extent persons should be excluded from the normative sample who, for clearly identifiable reasons, in particular because of illness, are currently (or permanently) not in full possession of their cognitive abilities. Many investigators hold on to random samples here as well; others aim for ideal samples consisting exclusively of healthy examinees. In the test documentation for the Mini Mental State Examination, this problem is pursued further in view of very divergent norming results.

Among experts in psychological assessment, it is probably undisputed that tests should be revised and renormed from time to time. Opinions differ, however, about the length of the tolerable intervals between two editions. At least the tests with larger numbers of editions that have become well established in practice are in fact revised every 5 to 20 years and occasionally renormed in the process. Unfortunately, test users' persistence is high in most cases: The situation arises that younger diagnosticians use the newer versions of a test, while the older ones mostly stick with the old ones. It is probably largely unknown that massive norm shifts can sometimes occur from one edition to the next. The texts or prefaces of the manuals are unfortunately not a good source if one is looking for information on this. There are almost never data on the equivalence of the new norms with the old ones; as a user one has to analyze the tables oneself. For the d2, a test that has been revised regularly, such massive differences were found, for example, between the 7th and 8th editions in individual age groups that the deviations could amount to 15 standard score points. Unfortunately, stating the edition of a test used in a test report is more the exception than the rule.

Tests of which there are several variants, whether in the test material or in the administration instructions, are similarly problematic. For the two Trail Making Tests A and B, different templates were in use for a time. One of the templates, the one with serifs in the typeface, was much harder to recognize for persons with poor eyesight than the other. For the Mini Mental State Examination, there are in practice various administration variants that are mostly not stated explicitly. In the working memory task of this test, sometimes a word (different ones are in use) is spelled backward, and sometimes counting backward is used instead. In the TAP, there are again variants in many subtests. In too many test reports, it is not mentioned which of these variants was used. The list could easily be extended, for example for the various memory tests. For the possible scoring variants as well, it is rarely specified exactly which norms were used (for example, general or school-specific norms).

The tests included in TDB2Online are processed according to a uniform scheme, although not all steps apply to every test. This section presents the general procedures and techniques in the processing of the tests and, where necessary, justifies them. Which of them apply to an individual test is described in the test-specific texts.

TDB2 is a project that arose from clinical psychological assessment practice. The tests included are those that are (or were) used so frequently by the psychologists working in assessment at the university psychiatric hospital in the University Hospital of LMU Munich that inclusion is (was) worthwhile. The appearance of outdated tests on the list is not an encouragement to use them in practice. The old tests remain in the database only because reports were created with them in the past.

The list is not closed; new tests may be added. Suggestions are welcome.

All test-specific texts begin with a description of the test, specifically a description of exactly the version to which the norms of TDB2Online refer. For published tests whose manuals are readily available, the description is rather brief. If there are several editions of a test, exactly the edition for which the norms apply is described. For “orphaned” tests that no test publisher looks after (any longer) or never had one, the test material is described in more detail, because there are often various variants. Where possible, sources of supply are also mentioned. The description of the copyright situation, which is given to the best of our knowledge, has a special status.

Some of the tests included in TDB2Online consist of very many subtests, of which not all are used equally often in practice. Examples are the Test of Attentional Performance (TAP)⁷⁾ or the Regensburg Word Fluency Test (RWT)⁸⁾. For the selection of subtests in TDB2Online, the same applies as for the selection of tests: Only those subtests are included that were administered so frequently in the Psychiatric Hospital of LMU that inclusion was worthwhile. The size and quality of the normative basis (for example, for the TAP) also played a major role. If a special administration variant was chosen for a test or subtest, this is described.

The properties of the raw scores of a test are described in a separate paragraph. Normally these are not long sections, because the coding rules for the raw scores can mostly be explained quickly. Sometimes, however, there are also situations that require explanation. We state explicitly, for example, which number serves as the raw score when the measurement range has ended (for instance, a time limit in a test whose raw score is the processing time).

The distributional properties of the raw scores are important. Often the distributional properties can already be inferred from the type of raw score. Whenever there are empirical data on the raw score distributions, they are presented and discussed here. The distribution of the raw scores has a great influence on the discriminative power of a test.

Complex ability characteristics, such as those measured by cognitive tests, are generally normally distributed in the population. If a distribution of raw scores in a random sample from the population deviates markedly from a normal distribution, this usually has to do with the chosen raw scores not being particularly suitable for measuring the characteristic. They then depict the characteristic not linearly but in a way that is distorted in terms of measurement. Nonlinear measurement scales are not well suited as starting material for tests. For one thing, the assumptions of classical test theory then do not hold (which would be tolerable); for another, we can no longer interpret nonlinear measurement scales intuitively. We are accustomed to equal-sized intervals at different points of a scale, for the units of measurement we deal with daily, also representing an equal-sized difference in the underlying characteristic (length, weight, but also IQ, memory competence, whatever). This works only if the scale depicts the underlying characteristic linearly and accordingly leads to a normal distribution in random samples.

For all tests included in TDB2Online, linear test scores are therefore aimed for. For many tests this is simple, because the raw scores are already linear. For some tests, the test author has already given thought to a linearization and built it into the scoring, e.g., David Wechsler at the time through the transformation of raw points into scaled scores. Unfortunately, however, a whole series of tests remains whose raw scores are nonlinear and which must be linearized retrospectively in TDB2Online. The retrospective solutions are complicated and generally unpleasant. Only rarely are the raw scores of a sample of young adults available, which would be the ideal case for TDB2. Sometimes at least a percentile distribution from a larger sample of healthy young adults is available. On the basis of such an empirical distribution, a heuristic transformation table that linearizes the scale can also be created. Samples with a wide age range are hardly suitable, because additional age-related effects that are relevant to the distribution then come in. For similar reasons, patient samples are rather unsuitable. Occasionally, approximate methods had to be used in the linearization. This is unsatisfactory, but still better than forgoing linearization although the raw score distribution is clearly skewed.

The exact procedure is documented for the individual tests; an overview of the available methods was already given in the section Methods of linearization.

In view of the lacking adequacy of the norms of many tests, an attempt was made within TDB2Online to improve the normative basis wherever possible. This was possible above all for the standard neuropsychological tests for which there were many norm sources in the literature. For each test, TDB2Online uses all practically available norm sources that meet the following criteria:

1. An unselected (or at least not recognizably selected) sample of healthy examinees was studied.
2. The sample could be assigned to one or more not too broad age ranges.
3. The test was administered in a standardized way, and administration and choice of the raw score corresponded to those recommended in TDB2Online.
4. As a minimum, the mean and standard deviation per age group were available.

The sources are documented in the texts on the individual tests. The norms were combined using (simple, descriptive) meta-analytic methods. The aim was to provide a norming that is as representative as possible of the entire adult population. No stratification by characteristics other than age was performed. Norm sources that proved to be statistical outliers were not used. The values were smoothed across the age groups.

Occasionally, the adequacy of the normative data of a test was examined by testing its equivalence with another, similar test. Results can be found in the documentation texts on the individual tests in the Discussion section. The aim of such work was to obtain indications for estimating the plausibility of the existing normative data.

For the presentation of all test scores, TDB2Online uses the IQ scaling with a mean of 100 and a standard deviation of 15. Two arguments influenced the decision for the (100;15) system:

1. The IQ scaling is probably the one most frequently used for ability tests. The T score scaling (50;10) is very common for personality tests, but not for intelligence tests and less so for other ability tests. The “SW” scaling (100;10) is quite common for German intelligence tests but is hardly found internationally.
2. For the verbal interpretation of test reports, the IQ scale has the advantage that the middle 50 percent of the distribution, a range that in statistics is often regarded as the average range (interquartile range), lies between the salient values 90 and 110. A further average range (plus/minus one standard deviation) is also still well marked by the values 85 and 115, even though, for continuously distributed variables, one needs good reasons to call almost 70 percent of a distribution “average”. The IQ scale thus makes it relatively easy to adopt Wechsler's verbal characterization of the level of cognitive performance (see Table 1 in the section "Scales based on the normal distribution"). With seven levels from “extremely low” through “very low”, “low”, “average”, “high”, and “very high” to “extremely high”, a performance can mostly be characterized verbally well enough, especially since nuances such as “at the lower end of an average performance” can qualify the statement further. Such a characterization does greater justice to the measurement precision of cognitive tests than the often-read “below average”, “average”, “above average”.

TDB2Online pursues the goal of a pure performance measurement on a scale that is comparable for all measuring instruments used. The interpretation of a measured value with regard to the patient's age is facilitated by aids, but age is not used to alter the actual measured value. Other biographical parameters (education, sex, whatever) are not taken into account. What significance they have must be interpreted verbally in the test report; this is not the task of performance measurement.

For performance measurement, a scaling is used where possible that makes the test performance of healthy adults at the lifetime peak of their cognitive ability the yardstick of presentation. The procedure is by no means new. In principle it corresponds to Wechsler's “scaled scores”, specifically in the classic (“non-age-corrected”) form that was introduced with the WAIS-R (see also the section Scaled score transformation in the test description of the WAIS-III). In Wechsler's adult tests, the conversion of raw scores into scaled scores involved two things: first, a correction of skewed and distorted distribution shapes of the raw scores toward a normal distribution of the scaled scores (which in TDB2Online is mostly carried out in a separate step), and second, the standardization on a random sample of young healthy adults, mostly in the age range between about 20 and about 30). TDB2Online draws on this concept, though with a change in scaling technique. Wechsler had scaled his scaled scores to a mean of 10 and a standard deviation of 3. Because of the greater familiarity and the better ability to differentiate, we relate these performance measures to a mean of 100 and a standard deviation of 15 and call them performance scores (PS). Performance scores are always keyed positively: Higher numbers mean better performance.

For many tests used in dementia assessment, there are no norms for young adults. A prominent example is the CERAD Test Battery. Norms for healthy examinees exist here only in higher age ranges, from 50 upward at the earliest, often not particularly well specified. For such tests, the performance scores relate not to young adults but to older persons. In the heading for each test, the age range for the calculation of the performance norms is always stated.

For the clinical assessment of a test score, a comparison with the performance of the corresponding age group is normally necessary. Not for nothing, for most published tests the raw scores are converted directly into age-dependent standard scores. In TDB2Online as well, in addition to the performance scores, age-dependent standard scores (ANS, with a mean of 100 and a standard deviation of 15) are calculated and given numerically in the test profile. The scores are calculated with the normative data pooled in TDB2Online. As presented above, these are based on an empirical basis that is as broad as possible and a psychometric processing that is as efficient as possible. They therefore do not necessarily correspond to the normative data found in other publications. Even for published tests for which there are no further norm sources besides the published normative data, deviations are likely, because, for example, smoothing between raw score levels and between adjacent age groups was always carried out for the TDB2Online norms.

All test scores are presented in a uniform graphical frame of reference that is oriented to the performance scores as a vertical grid. For each test, selected raw scores are also depicted in the performance score grid as small numbers. In addition to the test score of the examinee, the distribution parameters of the age reference group are shown in the graphic. In this way one obtains, on the one hand, the relation to absolute performance, measured against healthy young adults (performance score grid), and, on the other hand, the relation to the values of the respective age group.

Figure 4: Example of a performance score profile

Figure 4 provides a sample of this type of visualization of test results. Shown is a performance score profile (range 40 to 145) with two individual test scores in the TMT A and B for a 58-year-old, shown as black circles. In the example, the raw scores are 25 and 79; as performance scores (abscissa values), approximately 96 and 84 can be read off. The age reference values are superimposed. The white square marks the mean of the age group (here 55-59 years), or the age-normed score (ANS, also referred to briefly as age-normed score) of 100 for this age. The gray area covers the ANS of 85 to 115, corresponding to a percentile rank of 16 to 84; the end bars extend from ANS 70 to ANS 130, corresponding to the percentile ranks of 2.5 to 97.5. Within these age norms, the examinee's test score lies in Part A at the upper edge of the gray bar and in Part B just below the age-appropriate mean. The corresponding age-normed scores (113 in Part A and 98 in Part B) are entered in the column at the right edge. (Click to enlarge!)

The uniform graphical processing of the test results in TDB2Online has a number of functional features that support clinical report writing.

In TDB2Online, each test score is plotted as a black dot in a performance score grid that remains the same from test to test and on which performance can be compared across different tests. The scale with the small label “performance score” underlying the grid runs from 40 to 145 for every test, and the values are always in the same position. The small numbers in the rectangle that represents an individual subtest indicate the position of the possible raw scores of the subtest. In Figure 5 one can see that a raw score of 30 in the WST corresponds approximately to the performance score 100, and one of 20 approximately to the performance score 85. The performance scores themselves are comparable from test to test. This makes it possible to state that in this 24-year-old patient the memory performance in recalling a word list (AVLT, recall after interference, raw score 12, performance score about 104) is better by about 20 IQ points (more than one standard deviation) than his or her performance in Part B of the TMT (raw score 79, performance score about 84). Without the common measurement level of the performance scores, comparative interpretations of performance in different domains would not be possible.

The smallest and the largest raw score that a test can yield is always plotted in the profile if it lies within the displayable performance score range between 40 and 145. In Figure 5, using the example of the Vocabulary Test WST, one can thus trace that the smallest possible raw score is 1 and the largest possible raw score is 41. This corresponds to a relatively large measurement range, because the corresponding performance scores range from about 60 to about 140. For the subtest delayed recognition in the AVLT (last row in Figure 5), this is different. This subtest reaches its test ceiling (the largest possible raw score of 15) already at a performance score of about 113. Better performances can thus no longer be differentiated by this subtest. It is not possible to distinguish a high from a very high performance. On the other hand, the measurement range extends very far downward. It would even reach much further than is captured here. Raw scores below 9 already lie outside the display range. They occur extremely rarely and are therefore no longer normed.

Figure 5: Example profile for a 24-year-old patient

A basic estimate of measurement precision is obtained by comparing the resolution of the raw score scale with the resolution of the performance score scale. For the Vocabulary Test WST (see Figure 5), the raw scores between 10 and 20 correspond approximately to performance scores between 75 and 85. At this point of the scale, an additional correct word thus raises the measured IQ by one point. This is equivalent to a relatively high measurement precision. In the high performance range, an additional correct word (e.g., from raw score 40 to 41) raises the measured IQ by 6 or 7 points. If a single additional correct word that is known (or even guessed) can change the measured cognitive ability that much, the measurement precision at this point of the scale is rather low.

This example shows very practically that the measurement precision of a test is by no means the same at all points (as reliability theory and the concept of the standard error of measurement derived from it suggest). Most tests measure more precisely in the middle range than at the ends. With item response theory, a scaling method for test items, this can also be demonstrated regularly.

A look at the Auditory Verbal Learning Test AVLT (see Figure 5) suffices to recognize that all four subtests measure only relatively coarsely: Even one additional word learned or recalled raises the performance score by many IQ points.

Once more as a reminder: The performance score itself is not output numerically. It can be read off only from the position of the black dot in the test profile (abscissa value).

For the clinical assessment of a test score, a comparison with the performance of the corresponding age group is necessary, especially in older patients. Since the values in the TDB2 test profile appear graphically as age-independent performance scores, the comparison ranges of the age group must be plotted in addition. In the TDB2 profile, comparison ranges are plotted that were determined via percentile ranks (PR). The white square marks the median ( = PR 50) of the patient's age group, the gray rectangle the range from PR 16 to PR 84, and the two flags at the ends of the gray rectangle the percentile ranks 2.5 and 97.5. If the performance scores are normally distributed (which is mostly achieved by transformations in TDB2Online), the percentile ranks mentioned correspond exactly to the mean and the one- and two-sigma limits.

One can of course read off the individual standing of a patient within the limits of his or her age reference group from the graphic, but this is somewhat laborious and imprecise. For this reason, the age-normed scores are calculated by the program and displayed on the right side of the profile in the ANS column. These age-normed scores (ANS) are also displayed IQ-scaled, that is, with a mean of 100 and a standard deviation of 15.

With an example one can show how the ANS is calculated: In the Part B row of the Trail Making Test in Figure 5, one can see that the patient's individual test score lies about one third of a standard deviation below the gray area. Had it been on the white square, that would be the age-normed score 100. Had it been at the left end of the gray area, that would be the age-normed score 85. Since it actually lies about 1/3 standard deviation below the gray area, this corresponds to the age-normed score 80. This value is calculated by TDB2Online and entered in the ANS column on the right side.

In young adults, the age-related standard scores hardly deviate from the performance scores. The white squares thus mostly lie in the vicinity of the performance score 100. The older a patient is and the more performance in a given test declines with increasing age, the greater the difference between performance scores and age-related standard scores. One sees this clearly in Figure 6, which shows the same test scores as Figure 5, but this time for a patient aged 74 instead of 24 years. The position of the black dots in the performance score grid remains the same. Nothing changes in the absolute performance. What changes are the age reference ranges and thus the age-related standard scores in the ANS column. The change in age has a very strong effect in the visuoconstructional Trail Making Test (TMT) and in the memory test AVLT. Here the age reference ranges shift far to the left. It has no effect in the vocabulary test WST, because there are no age-graded norms for it and the age of 78 years still lies within the age range of the normative sample. In the Vocabulary subtest of the WAIS-IV, one can see somewhat more precisely that at the age of 74 years there are also slight age-related losses in vocabulary in the normative sample, which help our examinee, with unchanged performance, to a higher ANS of 106. One can quickly see, however, that the age effects in the TMT and AVLT are much larger.

Figure 6: Test performance as in Figure 5 for a 74-year-old patient

Not all tests span a large or even a symmetrical performance range. Exemplary is the measurement range in the Vocabulary subtest of the WAIS-IV, which extends from the performance scores 45 (raw score 0) to 143 (raw score 57) (see Figure 6). With the subtests of the AVLT, in contrast, very high performances can no longer be measured adequately. In all subtests, 15 is the highest attainable raw score. In the subtest delayed recognition (last row in Figure 6), a raw score of 15 corresponds only to a performance score of about 112. Higher performances can no longer be measured.

This occurs particularly often in tests that are not constructed at all to quantify abilities across a broad normative range but serve primarily to quantify deficits. This applies in particular to the tests for quantifying cognitive impairment in dementia patients. Figure 7 shows a test profile obtained with the CERAD.

Figure 7: CERAD test scores in a 70-year-old

We have already pointed out that for many subtests of the CERAD there are no norms for young adults. The title line of the CERAD battery in Figure 7 also points this out explicitly. Nevertheless, in some subtests of the CERAD, the test ceiling is reached even with average performance. This is most conspicuous in the subtest Word List Recognition. Here almost half of the normative sample of 60–64-year-olds achieve the full score of 10 correct. Consequently, a raw score of 10 also leads only to a performance score of just over 100. In such cases, we give in the ANS column, instead of a fixed value, only a range, in the example of Figure 7 >105. In the example, only ranges are given for two further subtests as well.

TDB2 and TDB2Online arose from diagnostic practice in a university psychiatric hospital. The starting point was dissatisfaction with the heterogeneity of the existing tests, not of their content, but of their psychometric processing. It was hardly possible to teach psychology students, in the course of a six-week internship, to write a sensible and consistent test report on the basis of the manual of each individual test.

With the introduction of TDB2 in the hospital (2009), this became significantly easier. Once an intern had grasped the principles of the TDB2 profile presentation (this did not happen immediately, but was mostly completed in the second week), discussing the findings with the supervisors became much easier. In this respect we owe to our interns (about 10 per year) the external stimulus that was necessary to advance TDB2.

TDB2 and TDB2Online are not finished programs. Changes will arise in the tests as new norms become available. New tests may be added. We are grateful for suggestions at any time.