Global Strategy - blog

In the TV Series How I Met Your Mother, one of the characters, Marshall, a young lawyer, leaves his friends at a bar one night by saying:

In recent times, there has been a lot of discussion about the questionable practices of scholars, universities and publishers. In a related fashion, the economist Ronald Coase once said that "if you torture the data long enough, Nature will confess." He referred to a common research practice of flexibly changing one's model, collecting more data and using different measures until you find interesting and publishable results; it is likely that at some point you will find statistically significant results purely by chance. For a qualitative researcher, flexibility is a good thing. In fact, going back and forth between data and theory is the primary mode of doing qualitative research. However, in quantitative research, flexibility is somewhat counter-intuitively considered a bad thing. Let me explain.