There’s a fair amount of social science and anecdata that tall males are more reproductively fit. More precisely, males one to two standard deviations above the norm in height seem to be at the “sweet spot” as an idealized partner (e.g., leading males). And, short men often have fewer children. Short women will pair up with tall men. Tall women will generally not pair up with shorter men. The question then has to be asked: why isn’t natural selection producing a situation where we’re all tall?
As it is, height is a highly heritable trait where there’s a lot of genetic variation present in the population. One hypothesis might be that short(er) people are simply individuals with a higher mutational load. In other words, there’s going to be variation in the load of deleterious alleles from person to person, and one’s value on quantitative traits (intelligence, height) is a reflection of one’s genetic fitness. There are problems with this model, starting with the fact that one you need to tease apart inter-population variation. Also, within families there doesn’t seem to be a correlation between height and intelligence, which you would expect to …