This post is also available on LessWrong. Thank you to Justis Mills for proofreading and giving feedback. Most scientific personality models are, directly or indirectly, based on the lexical hypothesis, which roughly speaking states that there is a correspondence between important personality traits and abstract behavior-descriptive adjectives. For example, the
Did you consider asking people a lot of frequency type questions, a kind of semi-objective measure of personality? E.g. how many times the last 2 weeks did you: cry, drink alcohol with drinks, play a board game with drinks, play an electronic game on computer/console/phone, browser Twitter, eat candy, watch TV in bed/sofa.
Thank you very much for posting this where we could read it! It's great to see this kind of thing on substack, and seeing this post convinced me to put up some of my own research, which I'd previously decided against - you can see it up here.
My personal take on your own work here (which you probably won't agree with) is that you've done a good job showing that the usual lexical pathway psychology has been using is actually on the correct track. When you say things like "Creativity (appears to conflate Creative problem-solving and Artistic creativity)," this is a feature, not a bug, of dimensional models. No one says that things like A) creative problem solving, and B) artistic creativity are literally the same; there's a large body of research on the difference between scientific and artistic success out there.* Yet there's obviously going to be some similarity between these two things that allows them to be positioned nearby in a space of personality traits.
More useful to you would probably be this bit of advice: The impact of personality on behavior is strongest when options are open.
Twelve different people wandering in the desert for three days are all going to be fighting over a bottle of water they come upon lying in the crevace between a rock. "If you were thirsty in the desert, would you want water?" is more a question designed to flush Lizardmen than an attempt to measure personality.
In other words, when you design concrete questions, a good strategy would be to focus on choices over duties. Having a stressful job, spending a lot of effort on parenting, and many other concrete details often depend on other situations or people around you much less than the effects of innate personality. I know that just over the past six months, my answers to those two questions would have changed dramatically.
* For example, I liked: Kaufman, S. B., Quilty, L. C., Grazioplene, R. G., Hirsh, J. B., Gray, J. R., Peterson, J. B., & DeYoung, C. G. (2016). Openness to experience and intellect differentially predict creative achievement in the arts and sciences. Journal of personality, 84(2), 248-258.
One problem with concrete items is that they mean different things in different places. Take engaging in politics, for example. I have lived in Utah, Boston, and Mexico. It is very common to engage in politics in Boston, less so in Utah, and even less so in Mexico. So if you ask about politics to infer if someone is easy-going, Boston will come out looking extraordinarily high strung. Concrete items may be better because they require less interpretation, but even then people are quite bad at remembering even basic things. For example, you can ask large samples of heterosexual men and women their number of partners and the math just doesn't add up; men over-report (comparatively). Not to say we shouldn't use concrete items, just not obvious to me that they are better.
In my view there is a personality space which, thankfully, we can reduce to low dimensions. That's where any method starts. We can find that space via factor analysis of survey items or adjectives. I don't get so hung up on specific factors, per say. Any direction in personality space is real. Though perhaps a word like "creative" is interpreted more contextually than "agreeable". In personality space, I would interpret this as it referring to several directions, kind of a diffuse factor. I wouldn't exactly call that real, it's the descriptor's fault, not a lack of veracity of individual (and intra-individual) variation along those axes. One of my problems with the Big Five is that the interstitial space then gets short shrift; instruments are designed only to measure personality at each pole. But any other rotation in that space is just as real!
Much of this extends for other data reduction techniques as well such as network analysis, though the interpretation is slightly different than "factor".
It seems that to explore personality space we need data generated from humans, usually self report (though I have also used word vectors). With adjectives these are filtered through millions of people who evolved to gossip, ie find important individual differences and articulate them. I put a lot of stock in that filter, but one can get pretty similar results from items (Revelle and Condon found the Big Five as well as the little 27: https://www.sciencedirect.com/science/article/pii/S0191886920300945).
Did you consider asking people a lot of frequency type questions, a kind of semi-objective measure of personality? E.g. how many times the last 2 weeks did you: cry, drink alcohol with drinks, play a board game with drinks, play an electronic game on computer/console/phone, browser Twitter, eat candy, watch TV in bed/sofa.
Thank you very much for posting this where we could read it! It's great to see this kind of thing on substack, and seeing this post convinced me to put up some of my own research, which I'd previously decided against - you can see it up here.
My personal take on your own work here (which you probably won't agree with) is that you've done a good job showing that the usual lexical pathway psychology has been using is actually on the correct track. When you say things like "Creativity (appears to conflate Creative problem-solving and Artistic creativity)," this is a feature, not a bug, of dimensional models. No one says that things like A) creative problem solving, and B) artistic creativity are literally the same; there's a large body of research on the difference between scientific and artistic success out there.* Yet there's obviously going to be some similarity between these two things that allows them to be positioned nearby in a space of personality traits.
More useful to you would probably be this bit of advice: The impact of personality on behavior is strongest when options are open.
Twelve different people wandering in the desert for three days are all going to be fighting over a bottle of water they come upon lying in the crevace between a rock. "If you were thirsty in the desert, would you want water?" is more a question designed to flush Lizardmen than an attempt to measure personality.
In other words, when you design concrete questions, a good strategy would be to focus on choices over duties. Having a stressful job, spending a lot of effort on parenting, and many other concrete details often depend on other situations or people around you much less than the effects of innate personality. I know that just over the past six months, my answers to those two questions would have changed dramatically.
* For example, I liked: Kaufman, S. B., Quilty, L. C., Grazioplene, R. G., Hirsh, J. B., Gray, J. R., Peterson, J. B., & DeYoung, C. G. (2016). Openness to experience and intellect differentially predict creative achievement in the arts and sciences. Journal of personality, 84(2), 248-258.
Consider adding TTS to your stack 🙃
One problem with concrete items is that they mean different things in different places. Take engaging in politics, for example. I have lived in Utah, Boston, and Mexico. It is very common to engage in politics in Boston, less so in Utah, and even less so in Mexico. So if you ask about politics to infer if someone is easy-going, Boston will come out looking extraordinarily high strung. Concrete items may be better because they require less interpretation, but even then people are quite bad at remembering even basic things. For example, you can ask large samples of heterosexual men and women their number of partners and the math just doesn't add up; men over-report (comparatively). Not to say we shouldn't use concrete items, just not obvious to me that they are better.
In my view there is a personality space which, thankfully, we can reduce to low dimensions. That's where any method starts. We can find that space via factor analysis of survey items or adjectives. I don't get so hung up on specific factors, per say. Any direction in personality space is real. Though perhaps a word like "creative" is interpreted more contextually than "agreeable". In personality space, I would interpret this as it referring to several directions, kind of a diffuse factor. I wouldn't exactly call that real, it's the descriptor's fault, not a lack of veracity of individual (and intra-individual) variation along those axes. One of my problems with the Big Five is that the interstitial space then gets short shrift; instruments are designed only to measure personality at each pole. But any other rotation in that space is just as real!
Much of this extends for other data reduction techniques as well such as network analysis, though the interpretation is slightly different than "factor".
It seems that to explore personality space we need data generated from humans, usually self report (though I have also used word vectors). With adjectives these are filtered through millions of people who evolved to gossip, ie find important individual differences and articulate them. I put a lot of stock in that filter, but one can get pretty similar results from items (Revelle and Condon found the Big Five as well as the little 27: https://www.sciencedirect.com/science/article/pii/S0191886920300945).