Engaging With Bad Science Episode 2 – What Is A Study And Why We Care

I have recently been challenging myself by looking at the science, or rather pseudoscience, behind the anti-vaccination claims that are making such a horrible difference on children’s lives these days. So far, I have looked at how the anti-vax movement deals with how real science works and on how it often cites low quality studies. After that piece, I have been asking for my readers to give me suggestions for more “anti-vaccination” studies – and I got some really interesting suggestions and I hope I get many more in the comments below this post. One of the interesting things I noticed by looking through the comments I received was that while I was asking for scientific studies, a lot of what I received was links to Youtube videos and books on sale on Amazon.

This made me realize that not everybody in the world necessarily understands what a scientific study is. In conversational English, a “study” is just the time that one or more people have put into investigating a specific topic. On the other hand, in the world of science, a “study” is a very precise type of publication. So I thought it would be helpful if I went through what a scientific study is and why this matters. If you are reading this because you are trying to make up your mind about vaccinations or are just interested in learning more about what the scientific community has to say, please stay tuned. It might change the way you look at people’s claims online. If you are keen to read more, please check out my article on Scienceseeker.com for some practical tips on how to read about science when you are not a scientist.

A scientific study is a very specific type of document that is designed to allow scientists from around the world to communicate their work to each other and to the public. After a group of scientists is finished performing experiments testing their hypothesis, they write them up in the form of a paper. The paper contains not only an explanation of the experiments that have been performed and an interpretation of what the results mean, but also a detailed description of how these experiments were performed. What’s more, the paper includes the actual data the scientists performed. After the scientists are done writing the article, they send it off to a scientific journal. There are thousands of scientific journals, covering every field of science in existence. Some scientific journals are highly respectable and only publish the best quality of work, whereas others can be scam operations pumping out low-quality work for a profit. Of course, as a non-scientist it can be quite difficult to work out which journals are reputable and which are predatory. A simple way to determine if a journal is reputable or otherwise is to cross check it in the Beall’s List of predatory journals.

If the article is sent off to a good-quality journal, it is assessed by the editor. If the editor thinks it is interesting and it looks to be of good quality, it is sent off to three or four other scientists who are highly specialized in the field. These other scientists are kept anonymous and are known as the “peer-reviewers” or the “referees”. They go through the manuscript and they pick on it for scientific accuracy, errors, overdrawn conclusions and all other manners of bad or incomplete scientific practice. Once they are done with their review, it is sent back to the authors, who then have a few months to make any changes required by the referees.Most of the time, this requires them to perform more experiments to satisfy the referees that their conclusions are correct. They might have to show more control-experiments to make sure that their results are accurate. This process of peer-review can continue for a number of rounds, until the reviewers are satisfied with the quality of the work. Of course, at any point they also have the option to flat out reject the manuscript, because they don’t feel it is of high enough quality, or because they are afraid that it overstates its conclusions or simply because they are not convinced that the underlying scientific evidence is strong enough to substantiate the claims the authors make.

At the end of the process of peer-review, a scientific paper is published and it becomes part of the scientific literature – the body of work of all scientists from all over the world trying to solve big problems together from all different angles. Scientific literature has value because thanks to the process of peer review it is officially the best that the scientific community can do at any given moment in time. Individual scientists’ talks, books, Youtube videos, blogposts etc all have the abilitynto convey a message, but are absolutely not scientific studies. They do not bear the same credibility, because the have not gone through the same process of peer review.

If you feel as though you want to point people towards a Youtube video as a way to show them a particular scientific theory or idea, take the time to look for the real scientific study that supports that idea. If you cannot find a peer-reviewed scientific study that will fit the bill, you might want to take a minute to figure out if you want to believe something that the scientific literature does not support.

6 thoughts on “Engaging With Bad Science Episode 2 – What Is A Study And Why We Care”

  1. It is worth noting that, typically, the reviewers are required to stipulate that there is no conflict of interest, either financially or scientifically, between themselves and the authors of the submitted manuscript.

  2. What. Is needed is a PR push to publish true stories of the horrible consequences of measles and smallpox
    We need true investigative reporters to dramatize those complications to the public.
    Forget publishing recommendations to vaccinate. Rather publish in media the deafness and deformities

  3. When people truly understand the consequences, their behavior changes
    When my 16 year old son wanted a motorcycle, I said sure, but First visit the motorcycle ward at our hospital. This went on every year. He never did get a motorcycle

  4. I am one of those who posted links to several YouTube videos. I chose ones that included links (PMID’s) to their peer reviewed studies and were collected and organized by an MD with decades of experience. I thought this would be a good way to give you a collection of links organized by subject with comments by a well educated scientist. I also provided a laundry list of concerns that do not appear to have been adequately addressed by the ProVax groups and asked if you could point to any research, as opposed to expert opinions, on these topics.

    For example take our modern knowledge of disease and the immune system: how difficult it is to avoid contamination in a small lab as opposed to a production facility, how many diseases are carried by the animals used to incubate diseases for vaccines, and our very limited ability to remove diseases from solutions without killing the desired live viruses. How many diseases, over the last 200 years were and are spread around the world because of vaccination programs? Also what is the expected result of the massive evolutionary pressure, imposed by world wide vaccination programs, on the biome (strain replacement, germ replacement, unencapsulated bacteria, …)? What happened when we did this to a smaller extent with other drugs?

    I like the topic you’ve chosen but I’m hoping you can explore the bad science on both sides and what’s been done to correct it. Perhaps your next installment could address what you see as bad science on the ProVax side. What things need to be addressed? Such as conflict of interest in the major medical magazines, publication restrictions on research funded by pharmaceutical companies, limited monitoring for non-specific effects, inability to detect longer term effects, and lack of inactive controls (actual placebos) in almost all vaccine studies.

  5. Unfortunately the concerns noted in the original Gardasil trials, that the vaccine raised the cancer rates in girls already exposed to HPV prior to vaccination, may be being observed in Sweden.
    Increased incidence of cervical cancer in Sweden: Possible link with HPV vaccination


    A side note: my first comments were posted as Bill Smith. Then my hand tremor started acting up and I started using auto-complete to fill in some fields. Sorry for any confusion.

  6. Oh. My. God.

    “Bad science on both sides.”

    “Youtube videos.”

    “Cervical cancer in Sweden.” (This has since been retracted, which you’ll see if you follow the posted link. The cited comment was evidently a hoax perpetrated by another anti-vaxxer.)

    It’s like we needed a live example of the anti-vax insanity.

Comments are closed.

Share This