{"id":59031,"date":"2015-06-01T12:00:25","date_gmt":"2015-06-01T11:00:25","guid":{"rendered":"https:\/\/www.transcend.org\/tms\/?p=59031"},"modified":"2015-06-01T09:56:12","modified_gmt":"2015-06-01T08:56:12","slug":"rotten-science-how-we-got-duped-by-fake-chocolate-science","status":"publish","type":"post","link":"https:\/\/www.transcend.org\/tms\/2015\/06\/rotten-science-how-we-got-duped-by-fake-chocolate-science\/","title":{"rendered":"Rotten Science: How We Got Duped by Fake Chocolate Science"},"content":{"rendered":"

28 May 2015 – <\/em>It\u2019s been a bad month for science. First, a highly-cited study published in Science<\/em> that found that voters\u2019 opinions on same-sex marriage could be swayed by conversations with openly gay canvassers was retracted<\/a> after UC Berkeley grad students pointed out irregularities <\/a>and one of the study\u2019s authors failed to produce the raw data. That author may also have lied<\/a> about who funded the study. Then, yesterday, science journalist John Bohannon unveiled in io9<\/a> how he tricked the world into thinking that eating a daily dose of chocolate could help you shed pounds by publishing a bogus study<\/a> in the journal International Archives of Medicine<\/em>.<\/p>\n

On Retraction Watch<\/a>, a site dedicated to tracking scientific papers that are retracted, newly debunked studies are piling up. The reason we keep getting duped is that science isn\u2019t just science anymore. It\u2019s Big Business<\/em>. And it\u2019s time we start thinking about it that way because, as in any big industry, there are some disturbing things going on that most people outside of scientific circles don\u2019t know about \u2014 but should. After all, these missteps affect our lives.<\/p>\n

Some poor sweet-toothed soul probably started eating chocolate in the hopes that it would tip the scales. That sounds silly enough, but there are other science frauds that have far more serious ramifications. Anti-vaxxers are the direct result of a now widely refuted study in a high-caliber journal linking autism to childhood vaccines.<\/p>\n

People are outraged that science is broken<\/a>, but part of it is on us. There are ways to avoid being duped by the charlatans. We should always have our BS radar on when reading about science, be it in a mainstream publication or in an academic journal. Here are things to remember to be a better, more skeptical reader of science news.<\/p>\n

Journals are money-making operations. <\/strong><\/p>\n

Not every journal is created equal. There\u2019s a very clear hierarchy based on something called the impact factor<\/a>, a measure of how often a particular journal is cited in a given year. Top-tier journals, like Science,<\/em> Nature <\/em>and Cell<\/em>, have impact factors in the 30s. The higher the impact factor of the journal, the more validity your paper gets by association. That\u2019s why most people didn\u2019t initially question the canvassing study, which came with Science<\/em>\u2018s high-caliber seal of approval. But that\u2019s like saying that every person who goes to an Ivy League school is a prodigy. It\u2019s simply not true.<\/p>\n

On the other end of the impact spectrum, there\u2019s been a huge proliferation of crappy \u2018journals,\u2019 if you can even call them that. In 2013, the journalist behind the chocolate study stunt wrote another bogus paper about a cancer-fighting chemical, which he then submitted to a bunch of journals. It was part of an investigation<\/a> into open-access publishing, testing out if journals were conducting rigorous peer review\u2014the process by which other scientists judge the merit of a given study. His shitty study was accepted by more than half the journals to which he submitted it. So what\u2019s going on? According to Bohannon, these journals are nothing more than money-making machines:<\/p>\n

Open-access scientific journals have mushroomed into a global industry, driven by author publication fees rather than traditional subscriptions. Most of the players are murky. The identity and location of the journals\u2019 editors, as well as the financial workings of their publishers, are often purposefully obscured\u2026Internet Protocol (IP) address traces within the raw headers of e-mails sent by journal editors betray their locations. Invoices for publication fees reveal a network of bank accounts based mostly in the developing world.<\/p>\n

If scientists actually read these studies, they\u2019d be able to tell they weren\u2019t reliable and the results wouldn\u2019t go anywhere because they\u2019d know not to cite them. (In science, the number of citations you get is a measure of how important your research has been.) But we now live in the age of the Internet, where any person can access scientific studies, whole or as abstracts, and blog about them. That can be a recipe for pseudoscience gone viral<\/a>. And that\u2019s exactly what happened with Bohannon\u2019s chocolate study. Several media organizations including Shape, Cosmopolitan<\/em>\u2019s<\/a> German website, and the Daily Star, covered it. Ooops.<\/p>\n

Peer review is flawed.<\/strong><\/p>\n

The public, probably thanks to how the media covers science, often regards the findings in peer-reviewed papers as irrefutable fact. But peer review is highly imperfect. As Bohannon points out in io9<\/a>, his study was published just two weeks after he and his co-authors paid the journal\u2019s publication fee. The journal claimed it has a rigorous peer review but that\u2019s doubtful. \u201cNot a single word was changed,\u201d he wrote<\/a>.<\/p>\n

Having peer-reviewed papers myself, that\u2019s not the norm. At least it shouldn\u2019t be. If you\u2019re submitting to a respected journal like Nature<\/em>, Cell<\/em>, Neuron<\/em>, PNAS<\/em>, Science<\/em>, or the Journal of the American Medical Association<\/em>, a paper will have to pass, first through a staff editor, and then through a handful of reviewers, other scientists who are specialists in the area of research the paper deals with. (Yes, a tiny group of people get to decide what is good science and what isn\u2019t.) Usually, good reviewers tease a paper apart, looking for holes in the data. Often, they\u2019ll ask for extra experiments to round out a study, or suggest ways in which an experiment could be done better. This is supposed to serve as a filter, a quality-control to weed out bad studies. Of course, that doesn\u2019t always work<\/a>: the latest Science<\/em> retraction is testament to that. But the problem is bigger. A recent study, for instance, found that 100 of psychology\u2019s most seminal papers were not reproducible<\/a>.<\/p>\n

For now, peer review is the best we\u2019ve got, which is to say we need to do better. Some journal startups, like PeerJ, are trying out a more crowdsourced approach to peer review, but the verdict is still out on whether this model works.<\/p>\n

Part of the problem is that reviewers aren\u2019t paid. It\u2019s a volunteer gig. You get an email from a journal asking you to review a paper. Can we really expect overworked scientists to dedicate the necessary amount of time to dissect each paper they\u2019re asked to review? Not always.<\/p>\n

Also, what passes peer review can reflect people\u2019s biases rather than a valid assessment of the research. Take, for instance, deep learning\u2014the AI technique that\u2019s en vogue right now at all of the biggest tech companies. In the early aughts, top researchers in this field, like Facebook\u2019s Yann LeCun and Google\u2019s Geoff Hinton, had a hard time getting their papers published if they as much as mentioned deep learning because most computer-vision experts thought the technique was subpar, despite some promising results and commercial applications, like reading digits on checks<\/a>. Now, things have shifted because the science politics have shifted. Hinton and his grad students, for instance, blew competitors away in a computer vision competition some years ago, and the rest of the field had no choice but to take notice. Now deep learning summits are standing-room only and journals like Nature<\/em> are publishing articles on the topic.<\/p>\n

Science journals can fall prey to trends, and even chase Internet memes<\/a>. What we see in journals isn\u2019t absolute truth or reflective of the entirety of what\u2019s happening in the scientific world. The gatekeepers\u2019 biases play an important part in what gets published where. Keep that in mind.<\/p>\n

Enter rat race to the top.<\/strong><\/p>\n

Because we place such a high premium on high-impact, peer reviewed papers, scientists are judged almost exclusively by their publication record. As Adam Marcus and Ivan Oransky wrote in their editorial about scientific fraud in The New York Times<\/em><\/a>, \u201cscientists view high-profile journals as the pinnacle of success \u2014 and they\u2019ll cut corners, or worse, for a shot at glory.\u201d<\/p>\n

A paper in Science<\/em> or Nature<\/em> as a grad student can set you on the road to academic success. If you can keep that up through your post-doc years, you might even get a shot at a tenure track position. And getting one of those is increasingly difficult. Graduate programs are pumping out more and more grad students, but very few actually manage to climb to the top of the ivory tower. That has bred a highly competitive culture in which \u201csexy\u201d results are rewarded more readily than meticulous research.<\/p>\n

I think part of the problem is funding and job security. Federal funding for science has been drying up in recent years. Pay for young scientists is just atrocious. The baseline salary for a recently minted PhD, as set by the National Institutes of Health, is a measly $42,000<\/a> \u2014 and that\u2019s for working 80-100 hours a week<\/a>. Post-docs and grad students are basically cheap, high-quality labor. So the way to work your way up is to come up with interesting data because that can translate into grants, a raise, and perhaps down the line a coveted tenure-track professorship.<\/p>\n

Show me the money.<\/strong><\/p>\n

And with that comes the promise of even more money. Many researchers have funding from industry. They get paid to give talks or to do consulting, which can lead to conflicts of interest. When publishing studies, corporate-funded researchers are supposed to disclose this, especially if the way they interpret the data could be biased by company interests, but that doesn\u2019t always happen. Again, something to keep in mind.<\/p>\n

We need better ways to make science and its shortcomings more transparent. But this month\u2019s news isn\u2019t all bad: we found out things went awry and how<\/em>, thanks to those Berkeley students and Bohannon. The next step is doing something about it because these things won\u2019t just fix themselves. Scientists should work together to devise a better way to screen research and the public should be more critical. If it sounds too good to be true, wait for a second study before filling your larders with chocolate.<\/p>\n

_________________________________<\/p>\n

Daniela Hernandez is a senior writer at <\/em>Fusion. She likes science, robots, pugs, and coffee.<\/em><\/p>\n

Go to Original \u2013 fusion.net<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

On Retraction Watch, a site dedicated to tracking scientific papers that are retracted, newly debunked studies are piling up. The reason we keep getting duped is that science isn\u2019t just science anymore. It\u2019s Big Business. And it\u2019s time we start thinking about it that way because, as in any big industry, there are some disturbing things going on that most people outside of scientific circles don\u2019t know about \u2014 but should. After all, these missteps affect our lives.<\/p>\n","protected":false},"author":4,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[200],"tags":[],"class_list":["post-59031","post","type-post","status-publish","format-standard","hentry","category-academia-knowledge-scholarship"],"_links":{"self":[{"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/posts\/59031","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/comments?post=59031"}],"version-history":[{"count":0,"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/posts\/59031\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/media?parent=59031"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/categories?post=59031"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/tags?post=59031"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}