A Research Paper On Things That Are Not, In Fact, Research Papers
Breaking: science journalism exists and is influential!
Hi there! This is TITLE-ABS-KEY(“science journalism“), a newsletter about science journalism research. In the previous issue, I tested some curious scicomm tools, in part because my inner magpie likes shiny new things but also due to a local anomaly caused by way too much science journalism in the issue before that one.
This time, I’m back to reading studies and wrapping up 2022 for this section as the whole newsletter is going on a (hopefully) short break. But let me tell you, we’re going out with a bang because today’s preprint was destined to be reviewed here. Onwards!
Today’s paper: Ioannidis, John. "Prolific non-research authors in high impact scientific journals: meta-research study" (bioRxiv, 2022). DOI: 10.1101/2022.11.19.517227.
Why this paper: I can’t even — just read the abstract.
Abstract: Journalistic papers published in high impact journals can be very influential, especially in hot fields. This meta-research analysis aimed to evaluate the publication profiles, impact, and disclosures of conflicts of interest of non-research authors who had published >200 Scopus-indexed papers in Nature, Science, PNAS, Cell, BMJ, Lancet, JAMA or New England Journal of Medicine. 154 prolific authors were identified, 148 of whom had published 67,825 papers in their main affiliated journal in a non-researcher capacity. Of 25 massively prolific authors with over 700 publications in one of these journals, only 3 had a PhD degree in any subject matter. Only 2 of the 25 disclosed potential conflicts with some specificity. The practice of assigning so much power to non-researchers in shaping scientific discourse should be further debated and disclosures of potential conflicts of interest should be emphasized.
Wow. As I am grasping for words to respond to this, I’m going to post a tweet I saw in the discussions of the preprint that I think summarizes what’s happening here:
Okay, but let’s be real and professional and respond to this stunning exercise in… I’m not even sure what this is. Journalistic papers? Non-research authors? Do you maybe mean news and feature articles and journalists?
You’ve got to admire the sheer in-your-faceness of gatekeeping in the opening part of the preprint. Publications in high impact peer-reviewed scientific journals are coveted by scientists, and they are struggling with single-digit acceptance rates – but just look at these editorial staff and science writers publishing hundreds and even thousands of pieces of writing expediently (sometimes even within hours of submission) and without formal peer review, and with acceptance rates that are probably very high!
(I had to step away from the blinding light reflected by all the pearls being clutched here – and used this opportunity to check for emails from multiple editors currently ghosting me, one of them since about 2019.)
Okay, I’m back (no emails), but the AUDACITY of these NON-RESEARCH AUTHORS!
Again, these are called journalists doing journalism and publishing writing that is clearly marked as such, usually with unambiguous labels like news. There is, however, one wrinkle in this argument which, astonishingly, is not brought up explicitly in the abstract or introduction – in fact, it’s one of my favorite scientometric jokes. Despite having published zero research papers, I somehow have an h-index of 2 because Scopus refuses to treat my news and feature stories properly and thinks of them as papers. Ioannidis knows about this oddity as it is in fact used later in the paper (but that is never his specific gripe).
It’s not a problem of my or my editors’ doing — rather, it’s a known bug in the academic metrics. We are strictly following journalistic practice in presenting this writing as what it is (science journalism, not science), and I only mention my h-index in countless self-deprecating jokes because, as a journalist, I get absolutely nothing from it. We’re not part of the h-worshipping congregation, you know.
But anyway, since this preprint is supposed to be an account of an actual study and not a “journalistic paper,“ let’s have a look at the study. Ioannidis takes eight journals – Nature, Science, PNAS, Cell, NEJM, Lancet, JAMA, and BMJ – and looks at authors who published more than 200 “papers“ in at least one of them. The threshold is arbitrary as ostensibly too extreme a number of papers to be published by an academic researcher. He then takes a separate more in-depth look into what he calls the most massively prolific authors, with more than 700 publications in a single high impact journal.
I have worked for Nature, Science, and BMJ, and as far as I know the other journals do not do a lot of news and features, so I was a bit surprised to see them on the list. But then, alas, Cell came up with no prolific authors at all, and PNAS had three (and all were in fact researchers). The other journals, in total, had 146 “prolific authors” – which included editors-in-chief, senior editors, and even the whole Lancet editorial board. So it’s not just the journalists who are uppity!
By the way, still no mention of the fact that this entire analysis is sponsored by made possible by a clear and well-known error in Scopus. Conversely, the author kind of doubles down when he mentions that Scopus automatically categorizes most of these publications as “articles“ – an absurd decision which, for that matter, I cannot even override as their author! So I have no way of telling Scopus to stop treating my writing as something other than journalism, but maybe the actual argument here is that there simply should be no journalism in Science and Nature? We’ll see.
After a remarkable discovery that the most impactful bits of this JOURNALISM since 2020 were done on hot (and often controversial), rapidly emerging areas of COVID-19 – stop the presses, journalists are writing about current events! — we go on to the 25 individuals with more than 700 "papers” in their main affiliated journal. (Quotation marks are mine here because I can’t bear to refer to news stories as papers in earnest.)
Subsequent analysis shows that these are mostly staff journalists doing nothing but journalism on a wide range of topics including energy/climate change, addiction/behavior, gene therapy, global arsenic toxicity, junk DNA, diverse infectious diseases, 3-dimensional cultures, translational research, and opioid prescription abuse. So basically science journalism, although I’m not sure what 3-dimensional cultures are. And I have my new productivity target now that I’ve learned that one BMJ reporter filed 232 stories in 2020.
I’m now going to quote one paragraph from the preprint in its entirety:
Based on Google searches, 3 of the 25 massively prolific authors had obtained a PhD degree (in oceanography, pharmacology, and organic chemistry); however, the topics that they covered journalistically were typically not related to their PhD degree. At least two had an MD degree and another 7 has a Master’s degree in journalism or a related field. The other 13 seemed to have neither any doctoral degree (in any subject) nor a Master’s degree in journalism or in a related field. However, one cannot exclude that some education credentials were not disclosed/retrievable.
I can almost hear the hyperventilating about the lack of proper credentials, can’t you?
Okay, in short, we’ve established several things in this preprint:
Scopus still hasn’t read the memo on Science, Nature, BMJ et al having news desks with news journalists and news editors;
These journalists dare to only do journalism, as evidenced by the lack of proper scientific papers;
Most of these journalists dare to not have a graduate degree in the subjects they are covering;
Proper scientists dare to cite journalistic work in their papers, thus boosting citation metrics for these PhD-less journalists;
(the one thing I’m not going to be sarcastic about because it’s a 100% valid point) These journals are not very consistent in making journalists disclose conflicts of interest in their reporting.
First, I’m going to address the valid point: part of it is that these reportable conflicts of interest usually preclude journalists from writing about something in the first place – unlike researchers describing the results of their patented work that will make them rich, most of us are very replaceable, and the story will normally just be assigned to someone else without a COI. But also I’ve disclosed conflicts of interest in my bylines, and I think it’s worth being more transparent about this with the readers (who likely don’t know much about the editorial back office processes).
Now, the rest: so it turns out that the argument isn’t that there should be no science journalism in Science and Nature. And it’s not, if you can believe it, that Scopus should read the damn memo and stop mislabelling journalism as science. It’s that journalists are not smart and trained enough to understand complex scientific fields – and these journalists, not smart and trained enough as they are, influence policy.
Proposed solutions? Co-authoring these pieces with knowledgeable subject-matter specialists; offering more space to knowledgeable field-specific specialists rather than science writers and editors; training of science writers in rigorous meta-research methods (I kid you not, this was a self-citation in the original preprint) and/or pairing with meta-researchers in co-authorship.
Yeah, no. Just no. We’re not going to give up science journalism, nope, and we’re certainly not going to let you “optimize this science writing corpus.” Nor, for that matter, are we going to let you exploit the incredibly obvious Scopus loophole for your citation metrics through co-authorship of journalistic writing! I saw what you did there, yeah.
Ultimately, this preprint is almost 30 years too late to the “should science journalists have PhDs“ party. No, and no we shouldn’t have scientist coauthors. We are doing a different job. Unless you also ask to co-bake your bread or co-cut your abdomen for surgery, please stay away from news journalism (you’re more than welcome to write your own op-eds, though.)
I’m just going to leave you with quotes from a Nieman Reports story published in 1993, when I was blissfully unaware of science journalism at age 5:
And it's true that the modern science writer probably was trained not as a reporter but as a scientist or a doctor, something news organizations imagine lends credibility to their coverage of these topics but which also accounts for a remarkable willingness to take things on faith. Journalistic instincts are developed by daily contact with blowhards, poseurs, rogues and knaves, by covering the cops and the city council or working night rewrite. "If your mother says she loves you, check it out," isn't taught in graduate school. <…> Why should journalists, and particularly newspapers, be the one to ask these questions? Because it's our readers whose lives are most affected by the answers, and because there isn't anyone else. Congress keeps the executive honest. The executive keeps big business honest. The courts keep journalists honest. Among the many fields of human endeavor, only science is virtually exempt from external oversight.
Please read the rest of the Nieman Reports story. It is soul crushing but in a good way, one that is appropriate for New Year Resolutions season.
That’s it! If you enjoyed this issue, let me know. If you also have opinions or would like to suggest a paper for me to read in one of the next issues, you can leave a comment or just respond to the email.
Cheers! 👩🔬🎄