Planning Test

Authors: Joachim Kohler & Ulrich Beck

The Planning Test by Kohler and Beck is a computerized variant of the Tower of London by Shallice¹⁾. This action-based test consists of a series of problem-solving tasks in which the actions leading to the solution must be planned a few steps in advance. The test thus assesses a subcomponent of executive thinking and acting.

The Planning Test is a computer program that runs on Windows PCs from Windows 95 onward. Whenever Windows operating systems changed, corresponding program updates were produced. In the past, the test was distributed through the authors' own website; for the past few years, it has been available through the publisher Normdaten.

On the screen, the patient sees three pegs of different lengths that can hold 1, 2, or 3 balls. In addition, there are three balls of different colors. At the beginning of each task, the three balls are placed on the pegs in different arrangements. At the same time, the patient sees the desired target configuration above on the screen. The task is to produce the target configuration with as few moves as possible (using a mouse or trackpad). After two practice tasks, 6 tasks are presented that can be solved in 3 to 8 moves with optimal planning. The test is available in two parallel versions.

The results are output in a table. The essential data are the total number of moves, the number of tasks solved without errors, the number of errors, and the processing times.

The Planning Test is a copyrighted program.

Many examinees complete the test in 5 to 8 minutes, but it can also take longer. There is no time limit.

The program comes with an instruction text that can also be shown to the examinees. It is better, however, to explain the test task to them using the two examples. Experienced computer users like to work with the mouse right away; with less experienced computer users, it has proved useful if they point out the planned ball movements on the screen and the test user makes the corresponding mouse movement. In terms of content, one should adhere to the instruction text in the manual, even if the tasks are explained freely. The instruction that the task should be planned mentally as far as possible before beginning the moves is particularly important.

The results table of the computer scoring provides many raw scores, which, however, are highly intercorrelated. In TDB2Online, three of them are evaluated:

1. Total number of moves: This is the most useful indicator of overall performance on the test. The range of values starts at 33 (the minimum number of moves across all 6 tasks). Raw scores above 50 are rarely achieved. The range of values thus comprises about 20 values.
2. Number of tasks solved correctly (that is, with the minimum number of moves): Raw scores between 0 and 6 are possible here; in practice, almost only scores between 2 and 6 occur.
3. Total number of errors: Theoretically, this scale is open-ended at the upper end; realistic values are between 0 and 3.

Only the total number of moves is suitable for distribution analyses; the other two raw scores are occasionally important for the qualitative interpretation of test performance, but because of their small range of values they cannot be used for more elaborate scaling.

The first version of the Planning Test had only preliminary norms based on a few examinees. Version 2.0.0.3 from around 2003 included data from 241 healthy persons aged between 16 and 65 years. Only these data are used in TDB2Online.

Since version 3.0 of the Planning Test, new norms are integrated into the scoring. They are based on 419 persons collected in German-speaking Switzerland as part of the norming of the MNND (Materials and Norms for Neuropsychological Assessment). The advantage for the Planning Test is that the norms are available separately by age group (in ten-year age groups from 20 to 60 years). The serious disadvantage is that the norms are not documented in the manual of the Planning Test but are (somehow) integrated into the automatic scoring. Instead of the previous raw scores for “tasks solved correctly” and “quality of the solution” (a transformation of the number of moves required for all tasks), “age- and education-corrected raw scores” (sic) are now output. These are then converted into z scores and percentile ranks using a conversion table that is likewise not documented further. Owing to the compulsory education correction alone, these norms are unsuitable for presentation in TDB2Online.

There remains a small consolation: The enlargement of the sample from 241 to 419 persons is unlikely to be so substantial that major changes can be expected (they are not documented anyway), and in the relatively narrow age range of the new norming, the age effects are probably not as pronounced as they would be in higher age groups.

The parameters evaluated in TDB2Online can also be obtained from the new results sheets of version 3. What is needed are the total number of moves, the number of tasks solved with the lowest possible number of moves, and the number of errors. All three can be derived from the results in the middle section of the scoring sheet.

see detailed documentation

see detailed documentation

see detailed documentation

As with many test procedures that measure executive thinking, the qualitative assessment of test behavior is also very important for the Planning Test. The measurable performance parameters can be sensibly interpreted only within the context of the observed test behavior.

Of the three parameters presented here, two (correct tasks and errors) measure only extremely coarsely. Nevertheless, the number of tasks solved with the minimum number of moves is an important parameter for the quality of the solution on the whole test. For errors, this is not necessarily the case. The Planning Test registers mouse clicks very precisely. If, for example, one grasps the lower of two balls and tries to move it, this is scored as an error. For people inexperienced with the mouse or trackpad (they still exist), this can quickly lead to several errors. In this respect, I do not share the opinion of the test authors in the manual that the number of rule violations is the most sensitive parameter for identifying executive disorders in this test. Even if, as mentioned above in the section Administration, the test user guides the mouse in inexperienced persons, unintended errors occur.

Regarding psychometric properties, the total number of moves is certainly the best parameter, at least for describing deficient planning behavior. The usable raw score range is so large that one raw score corresponds to about 4.5 performance scores, that is, about one third of a standard deviation. In the lower performance range, this allows sufficiently fine measurement; in the upper range, the test ceiling is reached at a performance score of 115.

For the total number of moves as well, careful observation of test behavior is decisive for the interpretation of the test scores.