laurahobbs
Hi Chris, thanks for coming to talk to us. It usually takes a few minutes to get going, so in the meantime would you mind telling us a little bit about you and what you do please? (I should warn you, there is a character limit on what will display in these boxes, but there’s no indication of what it is – so lots of shorter messages are better than one long one).
Chrispeters
I am the scientific affairs manager at Sense About Science. Which means I coordinate a lot of the responsive work we do when science doesn’t go quite right in the media, or in policy announcements. I also run the Ask for Evidence campaign and our expert panels on plant science and energy. This year I am also running the John Maddox Prize for standing up for science. In a previous life I was a plant scientist and did my PhD at the University of Sheffield where I also completed my undergraduate masters in biological sciences. I got a little bored with freezing plants to death in liquid nitrogen so I took a step away from the lab bench and worked at the British Ecological Society and then at the climate science and energy policy blog Carbon Brief. Mostly I enjoy making a fuss about things that are wrong. So long as it is a constructive fuss.I see what you mean about the character limits! Good thing I type quick too 😉
Grace-lee
can you give an example of when it hasn’t gone right and you have had to get involved
Chrispeters
So this was a couple of years ago. An article in the Daily Express reckoned alternative treatments for cancer were effective.
Grace-lee
Oh dear. Do you feel much damage was done?
Chrispeters
I’m not going to link to it – but it was talking about natural remedies, coffee enemas, vitamin jabs and eating lots of avocados… We contacted their health editor to see if they could amend the article and introduce some actual science. Who flatly refused So we pulled together a group of experts including Cancer Research UK, the Institute of Cancer Research, oncologists, Professor Edzard Ernst and Professor Paul Pharoah. And wrote our own version of the article which we called ‘Better Than The Daily Express’.
http://www.senseaboutscience.org/data/files/News/Better_than_the_Daily_Express.pdf
Which was downloaded about 20,000 times in less than 48 hours. So hopefully that might have corrected some of the damage done by the original article
Laurahobbs
You might lose us all while we read that…
Chrispeters
I’ll try and keep the wall ‘o text to a minimum from now on!
Grace-lee
You can bet you that people who want to hear what the original article said will be using it to justify their position
Chrispeters
You’re right. It was a really poor bit of journalism. The worst I’ve seen for a while.
Laurahobbs
““Homeopathy is disregarded because it does not treat the whole person or any part of the person.” Might keep that on standby
Chrispeters
If you want homeopathy debunking check out http://www.senseaboutscience.org/data/files/resources/54/Homeopathy.pdf
And also the Good Thinking Society.
Laurahobbs
We have some questions that were asked in the Facebook group by people who can’t make it tonight. I’m going to start pasting them over now (so get ready for a load of posts in rapid succession!)
I have tried looking up studies behind various claims before. There are some that aren’t hidden behind pay walls, which is great, but the reading can be dull, dense and difficult to understand. Which is probably why I rely so much on science journalism to explain things for me. Which has its own risks. Are there any pro tips for proactive parents who do try to “do their research ” and wade through nih and pub med? Is there a “plain English” movement in science, the way there has been in legislation and contract writing?
Chrispeters
Wading through pub med can be tough! Trish Greenhalgh has written a great book – “How to read a paper” And the BMJ has a series by her too.
http://www.bmj.com/about-bmj/resources-readers/publications/how-read-paper
Best simple tips are to look for review papers otherwise you can get caught out unconsciously cherry picking findings from individual papers.
Laurahobbs
And are there any reliable science journalists out there? Ones who consistently get it right, and yet can effectively explain someone’s crazy looking graph to a parent who just wants to know if she really should avoid parabens in cosmetics, or aluminium in deodorant or a specific vaccine for her kids (as examples…)?
Chrispeters
On the question about science journalists who are consistently good. I’d say Ian Sample, Tom Chivers, Tom Whipple, Claire Coleman, David Derbyshire are usually spot on.
Laurahobbs
Ah, that ties into the next question I think: let’s say we ask a company for evidence into a claim, and they post something on their website or social media, which looks like some charts and analysis that a lab put together. How do we know if they’ve posted a good, or poor, example of evidence? What standards should we be looking for?
Chrispeters
It’s really tough to know if you’re looking at good evidence or not. http://askforevidence.org/help/evidence
There was one example where a chap was sent a science paper to back up some claims a product was making about weight loss. He wrote to us asking that very same question – was that good evidence or not? He thought it probably was – it was a science paper after all. But we have a database of over 8000 scientists who can help in that situation. We put it to one of them and we found it wasn’t good evidence. In fact, the paper was about an ingredient the product didn’t even contain…. Here’s the full story
http://askforevidence.org/help/irrelevant-evidence
So if you are confronted by evidence that you’re not sure about – then get in touch!
Laurahobbs
What are the most common science topics you get asked about?
Chrispeters
Usually things to do with health. Cancer scare stories are a big one, and anything saying a risk or something or other. Diets are also high on the list – there are so many claims out there. New fad diets are in the papers most weeks and it seems like dietary advice is coming more and more frequently from celebs as opposed to people with a nutrition background. Which I find very frustrating. It all adds up to making it very difficult to know what to believe. Other topics that are common are plant science and energy related – because we openly advertise online expert panels for people to get answers to their questions. So lots of questions about farming and GM and fracking and nuclear radiation.
Laurahobbs
Who do the questions tend to come from? Journalists, general public, teachers, scientists in other disciplines?
Chrispeters
We get lots of questions from the general public. We also get journalists calling fairly regularly – usually wanting to speak to an expert asap about an article they’re planning to write.
Laurahobbs
Well at least they ask! I can remember staring at a TV in disbelief in Iceland during the Eyjafjallajokull eruption because Sky News had put up a map showing the volcano in the wrong place. Like, really wrong. And thinking well if you can’t get something that basic right…
Chrispeters
You’d think they’d be able to check Google maps for Eyjafjallajokull , There can’t be many places by that name…
Laurahobbs
The best bit was that it looked very much like a screenshot of Google Maps. I haven’t watched Sky News since. I just can’t trust anything they say!
Laurahobbs
I’m aware of time so I’m just going to copy over the other questions quickly now… And how does one “do your research” anyway, without a university’s access to science libraries and up to date journals? How can a layman look into someone’s claims when we’re stuck with Google?
Chrispeters
We’ve found that actually asking someone for the evidence behind their claims is a great way of showing that firstly you care about evidence, and secondly that you’re prepared to hold someone to account for the claims they make. That’s a powerful incentive for change. If you’re after a specific paper though, you can usually contact the author who’ll send you a copy.
Laurahobbs
(This one was actually me, following on from the last one): I’m interested in how the growing availability of open access papers might affect engagement of the general public with research/evidence. Will people be more likely to go and seek it out? Is it common knowledge that this is happening, are there easy ways to ‘promote’ the fact that more and more articles aren’t pay-walled these days?
Chrispeters
Even with more and more papers becoming open access – it still takes someone pretty dedicated to go to the original science papers.
But it would mean that people writing articles have no excuse not to link to their sources and I’d like to see much much more of that.
Laurahobbs
I also don’t know much about which sources are more reliable than others. Like, I’ve heard of nature. But how do you know if the open source you’ve come across is a good one?
And then the last one was simply how do you know what is a reliable source? (So very similar to the previous question).
Chrispeters
There are more and more ‘predatory journals’ out there. That take money to publish something without any scrutiny. The Beall’s list of predatory journals highlights a few of them. And it’s always worth checking to see if a paper has gone through peer review – which is a good benchmark to say whether it’s robust or not
Laurahobbs
The original papers can be pretty hard to read in some cases anyway though can’t they. I can remember trying to read some medical papers when I was pregnant, because I could access them, but some of them I just couldn’t decipher because it’s not my background (in the same way I’m assuming a medic wouldn’t get very far with a volcanology paper).
Chrispeters
There is a great resource that we’ve been working on called Understanding Health Research. It’s going to be launched later this summer but it’s an interactive online tool to help you go through an individual science paper and make sense of it.
Kate-ohara
It sounds like your job is Mission Very Important And Near Impossible.
I read something the other day which was peer reviewed and also funded by a group of companies. It was comparing their product with others. Can it still be good evidence if it was funded by someone with a vested interest like that? So I thought in that case that the peer reviewed aspect probably trumped the guided funding aspect.
Chrispeters
Company funding is a tricky one because it doesn’t necessarily mean that the science is bad. There’s good science done that is corporately funded, and there’s bad science done that isn’t. Our director wrote about this quite recently and we’re looking at helping develop a code of practice to make sure subconscious bias is addressed and that research can get the funding it needs.
http://www.theguardian.com/commentisfree/2016/may/14/research-corporations-funding-science
Grace-lee
How do you know what Peers are used? What if they are all inherently biased? Is there a way to check?
Chrispeters
Different journals do peer review differently. Sometimes it’s blinded so you don’t know who has written the paper you’re reviewing. Journals will have different ways of choosing the people who do their reviewing – but there isn’t a really a great deal of support for early career researchers who are just staring down the road of peer review.
Laurahobbs
I don’t think there’s a way to check who’s reviewed a paper is there? If they’re anonymous then even the authors don’t know who’s done it?
Chrispeters
Mostly it’s anonymous – although it can be quite simple to work it out as some fields of research are quite niche. It’s not always anonymous and I think there is a bit of a movement to make it more transparent. But on the other side if you know you’re reviewing a very senior academic and they’re going to see your comments – that can be pretty scary for someone who is more junior.
Laurahobbs
Haha yes I have worked it out/narrowed it down on stuff I’ve written (small specific area). But officially, if they’re anonymous, the journal wouldn’t reveal it would they? I can imagine that causing problems – if you’re more junior and you think criticising someone’s work might affect your future career, that could affect the review you give. Even subconsciously, maybe.
Chrispeters
@Laura – exactly
We run workshops to debate the challenges in the system. It’s not a perfect system so we need to know what the flaws are if we’re to address it.
http://www.senseaboutscience.org/pages/peer-review-workshops.html
Kate-ohara
@Laura that’s interesting. I had always thought that the peer review system relied on people putting their names to having reviewed things, so you wouldn’t risk your reputation by saying a rubbish thing was good.
Laurahobbs
No, people are often anonymous (and reviews can be really quite harsh, especially if people aren’t saying who they are). The journal will know who they are though, so they’re still putting their name to it in that respect. Not publicly though.
Kate-ohara
I wondered if anyone studies the whole psychology and sociology around discussing these types of things on the internet. Our group is unusual in that we question and look for evidence, but when we do that other users sometimes perceive conflict or even abuse.
Chrispeters
I bet someone has looked at that. If they haven’t it would be an interesting topic! But it can be difficult when you’re asking for evidence and genuinely interested in what that evidence will be. People can often get defensive.
Kate-ohara
I think I find that people are more open to requests for evidence that alternative things work, than they are to requests for evidence that conventional things don’t.
Chrispeters
The fact that it’s asking for evidence as opposed to demanding it helps! We’re very keen to show off people who are using good evidence too – we don’t just want to hit people with a stick. And if you’re not using evidence then it’s about how you can change after being asked. Vision Express and Ann Summers both responded really well when asked for evidence – and changed the way their training was done as a result
Laurahobbs
Do I want to know what Ann Summers were trying to claim without evidence??
Kate-ohara
I do.
Chrispeters
Vision Express –
http://askforevidence.org/articles/vision-express-scientific-claims-launches-investigation
Ann Summers had given some training to staff about recommending a particular product for cleaning their toys and made a pretty strong claim that it would ‘prevent infections’. Turns out that there wasn’t any evidence, Ann Summers changed staff training to make sure that advice wasn’t given to customers again.
http://askforevidence.org/articles/buzz-fresh-wipes
Laurahobbs
Ahhhhh, ok. There’s a few potential lawsuits in that if they get it wrong!
Kate-ohara
That’s quite relevant to the things we discuss, actually!
Chrispeters
So being asked for evidence is actually an opportunity to show that you’re evidence based. Or that you’re willing to improve and appreciate the importance of evidence, and the more people that ask for it – the more companies, individuals, politicians, journalists etc. will expect to be asked in the future. So I hope the Nappy Science Gang keep on asking for evidence – just like the non-bio detergent NHS advice.
Laurahobbs
Have you ever had any questions about alternative laundry products like Ecoegg? We asked them for evidence but they never gave it.
Chrispeters
You can look on http://www.askforevidence.org to see if something has been asked about. So far nothing on the Ecoegg, but if you’ve asked then let us know and we can help chasing that request.
Laurahobbs
We might just take you up on that
Grace-lee
We are desperate for a response from Ecoegg I fear we may have complain to ASA
Chrispeters
Not responding doesn’t look good on behalf of the company and it might be that making a fuss about that can prompt a response.
Laurahobbs
Thank you, and thank you so much for this. We should let you go and do teething duty
Chrispeters
So far so good, but I’m sure as soon as my head hits the pillow she’ll wake up! Email us about the Ecoegg ‘ask’ and we’ll see if we can help. (askforevidence@senseaboutscience.org)
Thanks for having me!
Laurahobbs
Thanks everyone. May you all have a peaceful unbroken night!
