The old SAT had an extraordinarily high ceiling. Only about three out of a million test takers scored a perfect 1600 on the old SAT, but since only a third of American 17 year-olds took the test, and virtually 100% of the high IQ ones took the test, 1600 was a one in a million performance. An SAT score of 1600 equaled an IQ of roughly 170.

But did it really?

The smartest person I have ever known once claimed on an internet forum that he knew the SAT scores of many, many people, and that the SAT (old or new) did a good job measuring intelligence up to about 1400, but beyond 1400, he could never see any kind of pattern. That is, there were obvious differences in brain power between people who scored 1400 and 1300 and between people who scored 1300 and 1200, but no cognitive differences between people who scored 1500 and 1400, or between people who scored 1600 and 1500. 1500+ people were more academically successful than 1400+ people, but they did not seem any more capable of solving truly complex and novel problems. For example if you handed a Rubik’s Cube to a bunch of SAT takers who had never seen it before, the 1500+ might be no better at it than the 1400+, but the 1400+ would be much better than the 1300+ who would be much better than the 1200+.

Further, if you scanned the brains of SAT takers, you might find that 1500+ people have the same brain size as 1400+ people, but 1400+ people have bigger brains than 1300+ people who have bigger brains than 1200+ people. Both types of evidence would show that the SAT is just not measuring g (general intelligence) above 1400.

But why not? Imagine a test that consisted of the following 10 items (to be answered without use of pen or paper):

1) 1 + 2 = ?

2) 22 + 33 = ?

3) 444 + 555 = ?

4) 5555 + 6666 = ?

5) 66666 + 77777 = ?

6) 88888 + 99999 = ?

7) 11111 + 22222 = ?

8) 22222 + 33333 = ?

9) 33333 + 44444 = ?

10) 44444 + 55555 = ?

On such a test, there would be huge differences in intelligence between people who scored 1 out of 10 and people who scored 5 out of 10, but very little difference in intelligence between people who scored 5 out of 10 and people who scored 10 out of 10. This is because items 1 through 5 increase in difficulty in a very linear way, while items 5 through 10 are all roughly equal in difficulty, so the only difference between someone who scores 5 out of 10 and 10 out of 10, is the ability to avoid dumb mistakes.

However intelligence is not just about the number of problems you can solve (intellectual breadth), it’s also about the hardest problem you can solve (intellectual depth), and the latter is probably much more g loaded than the former. But when test items stop increasing in difficulty, the differences between a high score and a super high score becomes entirely a function of just avoiding dumb or careless mistakes, rather than doing anything brilliant.

I suspect this is what happens with the SAT. Most of the people who write the test items probably have IQs around 135 so they are good at creating problems that discriminate well between a top 10% mind and a top 1% mind, but they have trouble imagining and identifying problems that discriminate between the top 1% and the top 0.1%. As a result, you get a pile up of items at the top 1% level, hence the SAT fails to measure g beyond 1400, but it still predicts academic success since the latter is as much about avoiding dumb mistakes and studying as it is about g.

So even though only one in a million Americans scored perfect on the old SAT, their intelligence might not have been higher than those who scored around one in 200 level.

Advertisements