A respected blogger named Emil has responded to my recent post about Harvard students regressing precipitously to the mean when they take an official IQ test. Although some studies have found the SAT only correlates 0.4 with official IQ tests like the WAIS, Emil writes:

The lower values are due to restriction of range, e.g. Frey and Detterman (2004). When corrected, the value goes up to .7-.8 range. Also .54 using ICAR60 (Condon and Revelle, 2014) without correction for reliability or restriction.

While it’s certainly true that the SAT’s correlation with official IQ tests goes way up when you correct for range restriction, I’m not sure how appropriate the correction is here. The point of such corrections is that if a sample has a restricted range of general intelligence (g), but an unrestricted range of non-g variance, then almost by definition, variance in g will have less predictive power than non-g variance, since the latter variance exceeds the former.

However people who take the SAT, particularly at the same high school, are not just restricted in g, but are also restricted in academic background and test preparation which likely correlates with SAT scores independently of g, thus studies that correct for range restriction in g, while ignoring range restriction in non-g variance, may grossly overestimate the SAT’s correlation with IQ in a random sample of all American 17-year olds.

Emil also notes the average IQs of Harvard students in the study I cited might be deflated by an oversampling of social science students who are less intelligent than STEM students. I definitely agree that STEM students are more intelligent than social science students, however I’m not sure this would have a significant effect because most Harvard students are not in STEM, so the non-STEM students would probably be a lot more representative of the average Harvard undergrad than STEM students are. However this needs to be explored in more depth.

Emil then writes:

SAT has an approx. mean of ~500 per subtest, ceiling of 800 and SD of approx. 100. So a 1500 total score is 750+750=(500+250)+(500+250)=(500+2.5sd)+(500+2.5sd), or about 2.5 SD above the mean.

I realize Emil is just doing a rough estimate, but it’s important to note that verbal and math sections of the SAT are said to only correlate about 0.67, so someone who scored +2.5 SD on each subscale should be about +2.8 SD on the composite (relative to the SAT population, who are already above the U.S. population mean). At least in theory..

Official stats from the year 2000 (around when the Harvard students in the cited study were tested) showed that the national mean verbal SAT was 505 (SD = 111) and the mean math SAT was 514 (SD = 113) and the composite score had a mean of 1019 (SD = 208). Assuming Harvard students have a mean SAT of 1490, they would have scored 2.26 SD higher than the average SAT taker. Roughly the top one in 85 SAT takers, and probably the top one in 255 level for all American 17 year olds (+2.66 SD).

Emil then applies the 0.86 test-retest correlation to estimate how SAT takers will score on the WAIS, however this correlation might be way too high because it is based on people taking the same test twice and the SAT and WAIS are not the same test. One’s true score on the SAT will not correlate perfectly with one’s true score on the SAT.

People who score +2.26 SD above the SAT population on the SAT will average 0.86(2.26 SD)= 1.94 SD when they take the SAT again, which is the top 2.6% with respect to SAT takers, and the top 0.88% of all 17 year olds, equivalent to an IQ of 136 (U.S. norms) or IQ 134 (U.S. white norms) or IQ 132 (U.S. normal white norms). By contrast on the WAIS Harvard students average IQ 122 (U.S. normal white norms; corrected for test abbreviation).

In short, the unreliability of the SAT does not seem to explain the severe regression to the mean Harvard students experience when tested on the WAIS.

Advertisements