President’s update: Waiting for Science Comms to be rocked to its underpants…

Waiting for Science Comms to be rocked to its underpants…

So some really significant research was released in the last few years – and I have been patiently waiting for science communications globally to be rocked to its underpants.

But nothing has happened.

Let me explain. The first research was conducted by a collaborative effort of over 270 psychology researchers who got together to try and replicate the findings of 100 key psychological studies.

What did they find? – They could only replicate about a third of them.

The implications of this are pretty profound (to quote Back to the Future III), as it has potential impacts across lots of social science research – including science communication research – that is rarely replicated.

And why is that?

It is key tenant of good science that an experiment be replicated to ensure it is valid. But in the social sciences, not only are there no rewards for replicating research, but you can actually be subtly punished for it – most often through not achieving publication because your work was deemed not new.

And this means that research that is conducted at a particular time with a particular audience is held up as the gold standard to how all audiences at all times and in all places will undoubtedly react or behave.

But what if that is not the case? What if the gold standards of Cultural Cognition and Values and Biases and Framing and so on are not very replicable, or are very dependent on particular situations? Can you hear the collective Uh-oh?

And that brings us to the second study that I referred to.

The key researcher, Joe Henrich, had been doing work amongst people in South America and Africa and noted that social experiments conducted there obtained very different results from the ones that were conducted in North America. And where are the majority of social science experiments conducted? 70 per cent are conducted in the USA, and a huge number of those are with undergraduates. And I would argue that that is not the most typical of audiences to extrapolate data from.

With his colleagues, Heine and Norenzayan, they started applying studies more widely across different cultures and they found that over and over there was one group of people who were particularly unusual when compared with the broad population of the globe. They even called their research paper ‘The Weirdest People in the World’

And you have probably guessed by now that the weirdest people were North Americans! And yet they are the main core for global social science experiments.

They stated, “American participants are exceptional even within the unusual population of Westerners—outliers among outliers.”

They concluded that social scientists could not possibly have picked a worse population from which to draw broad generalizations about how we might all behave.

Granted, we in Australia can sometimes be more like North Americans than we’d like to admit, but we do have some distinct differences. And of course for researchers working in Asia, Africa, the Middle East, South America and the Pacific and so on, the differences will be much, much greater.

Individually, the findings of these two research projects are quite startling, but when you mix them together, they are like the Mentos in the Coke bottle that all science communicators have tried at one time or another.

For when it comes to science communication research gold standards (or even the silver, bronze and other less Olympic metal standards), we really don’t know how much science communication wisdom might not be replicable, nor how much is not relevant in other cultures than where it was undertaken. I don’t think I’m going out a limb here to say – probably quite a bit.

And just maybe that is exactly why nobody wants to talk about it!

If you want to read more on the studies, fasten your undies from a rocking, and check them out here:

http://www.psmag.com/books-and-culture/joe-henrich-weird-ultimatum-game-shaking-up-psychology-economics-53135

https://theconversation.com/we-found-only-one-third-of-published-psychology-research-is-reliable-now-what-46596

http://www.sciencemag.org/content/349/6251/aac4716

Dr Craig Cormick

President

Australian Science Communicators

Leave a Reply