An Alternative to Alternative: Assessing the Validity of Information or Knowledge
News Flash! The wedding between Big Foot and The Loch Ness Monster to be the event of the century! It’s going to be huuuuge! A-list celebrities plan on attending…
Now, is that a true news item or a false one? Before you hit the buzzer and say it is false, consider for a second that it just might be true. People can and do give themselves or their children unique monikers (Big Foot Smith is not outside of belief). The point is, we do not know many things for certain, but we can deduce validity — at least in part — from the source of the info. However, it can be a difficult task even for professional seekers of truth, e.g., scientists.
Traditionally, scientists turn to scholarly journals for valid information. In the olden days, researchers would dive into the bowels of the library to find the journal issue carrying a sought-after article, or send an undergrad to fetch it. Not anymore. Today, a few internet clicks and POOF! Millions of hits in mere seconds. Though it has become easier to access information, it can sometimes be more difficult to discern the legitimacy of the information or the source.
A great example of discerning legitimacy comes in the form of so-called “predatory journals.” What is a predatory journal? It is a journal that exists solely for profit making rather than disseminating knowledge. For a fee, the predatory journal publishes pretty much anything thrown at them, without thorough review of the findings. It can be difficult to spot these predators for they go by titles that sound legit, and can (and do) fool even the most senior of scientists (Cobey, 2017).
How can scientists (and the rest of us) evade these noxious predators? Confirming that a journal is listed in the PubMed database, which has banned some predatory journals, is one strategy (Deprez and Chen, 2017). Other databases such as Journal Citation Reports or Directory of Open Access Journals might be useful (Moher et al., 2017). Looking at impact factors, questioning librarians, or seeing if the journal has any characteristics that have been attributed to a predatory journal (Moher et al., 2017) may also help get to the valid data/knowledge.
However, predatory journals are not the only sources of suspect information. Digital apps and other software can crunch away data and offer something “insightful.” Yet, these data and the resulting findings are not always useable or accurate. For instance, the software that converts experimental data into a DNA sequence has about a 50% reproducibility rate (Keshavan, 2017). Think about it. Your genetic test results might be different if the samples were re-analyzed. If these results were used to decide medical treatment, the treatment might be inappropriate 50% of the time! To rein in the variability seen in DNA sequencing, the FDA is taking action. Though still short of implementing regulations, the FDA beseeches the sequencing companies to scrutinize their software and improve it (Keshavan, 2017).
So, how are consumers using results from genetic tests? Recently, a survey showed how much consumers actually used the knowledge acquired from commercial tests (Barton, 2017). Consumers can be told of their risk for cancer based on genetic tests that look for single point mutations. These tests did not greatly sway health-related behaviors in either a negative direction or positive direction for many of the participants (except in the case for prostate cancer tests) (Barton, 2017). On the bright side, it would appear that the news is not causing everyone (except for those with worrisome test scores for prostate cancer) to rush and get potentially unnecessary diagnostic tests done, which can carry their own set of problems.
Where do we go from here? With the sheer crush of data available it seems like it is almost impossible avoid predators and wring credible and reproducible information/knowledge from the internet or companies. Yet, it is not. We need to just take the time to carefully scrutinize the information/ knowledge, investigate the publishing practices of journals, query how a company validates their results, put the claims in the context of other knowledge, etc. This is not always easy because we do not always have the time to do this well, and still read entertaining stories on the internet, such as the Big Foot and Loch Ness Monster nuptials.
Barton, M. K. (2017). Health behaviors not significantly changed by direct-to-consumer genetic testing. CA Cancer J Clin, 67(3), 175-176. doi:10.3322/caac.21368
Cobey, K. (2017). Illegitimate journals scam even senior scientists. Nature, 549(7670), 7. doi:10.1038/549007a
Deprez E. and Chen C. (2017, August 29). Medical journals have a fake news problem. Bloomberg. Retrieved from https://www.bloomberg.com/news/features/2017-08-29/medical-journals-have-a-fake-news-problem.
Keshavan, M. (2017, August 1). FDA pushes to bring order to the chaotic world of DNA sequencing. Statnews. Retrieved from https://www.statnews.com/2017/08/01/fda-dna-sequencing/
Moher, D., Shamseer, L., Cobey, K. D., Lalu, M. M., Galipeau, J., Avey, M. T., . . . Ziai, H. (2017). Stop this waste of people, animals and money. Nature, 549(7670), 23-25. doi:10.1038/549023a