In 1963, Martin Luther King famously said:
I have a dream that my four little children
will one day live in a nation
where they will not be judged by the color of their skin
but by the content of their character.
In 2015, I would like to say:
I have a dream that my research papers
will one day exist in a world
where they will not be judged by their journal of publication
but by the content of their equations.
Everyday life is messy and complicated. As human beings, we do not have the capacity to deal with the world in its full-fledged complexity. Instead, we must settle for methods of simplification and coarse-graining in order to cope with the thousands of events and interactions that make up our busy lives. Consider for example the case of a company evaluating the job application of a prospective employee. Like the rest of us, the applicant is a complex individual with dreams, ideas and fears. She cannot be reduced to a few pages of curriculum vitae. And yet, the interviewers are often forced to make this simplification, for they have neither the time nor the resources to do otherwise.
As scientists, we are faced with a similar problem when we attempt to evaluate the quality of a research paper. Ideally, we would read the paper thoroughly and make a careful assessment of its merits. Can we do this every single time? No, we already have our hands full trying to write papers in the first place! What do most of us do instead? We look at the number of citations and the journal of publication. For young articles – which have not had a chance to be cited significantly – it is common to focus only on the journal of publication.
Now let me be perfectly clear about this: there is nothing wrong with making a quick assessment of quality based on the journal of publication. Indeed, these two properties are correlated and as I have explained before, we are often forced to make these types of simplifications. The real problem arises when we forget that it is a crude and frequently incorrect method of assessing quality. Here are two facts to illustrate this point.
Fact 1: Terrible articles have been published in prestigious journals.
My favourite example is this Nature paper: Information transmission under conditions of sensory shielding. It is a comically non-rigorous experiment trying to argue for the possibility of extra-sensory perception. It was quickly debunked. Schroedinger's rat provides some more examples, but I am sure most of the readers can think of some of their own.
Fact 2: Ground-breaking results have been published in obscure journals.
My favourite example is John Bell's original paper introducing his now famous theorem. Another example is Wiesner’s quantum money scheme: a paper which, like many others, was way ahead of its time. After being rejected by several journals, it had to wait 13 years to be published. In fact, during QCMC 2014 – where there was a big debate concerning scientific publishing – the award recipients Nicolas Gisin and Reinhard Werner both publicly stated that none of the articles they consider to be their best were published in prestigious journals.
From these facts we can logically conclude that if we judge a paper by its journal of publication, we will often conclude that some lousy articles are great, and that some excellent papers are mediocre. It is a faulty method which we should only use when forced to, and whose limitations we should always keep in mind. So what is the problem?
The problem is that the scientific community has fooled itself into believing that the quality of a research paper is equivalent to the prestige of its journal of publication.
I am not talking about funding agencies or university administrators who may not have the technical knowledge to judge a paper properly – I am talking about us. I am talking about group leaders assessing applications of postdocs and graduate students. I am talking about students thinking about the merits of their own results. I am talking about postdocs referring to the work of their colleagues. Most of you who are reading these words have probably been guilty of this mistake, myself included, and it is indeed a corrosive error that has become engraved in the way we speak and think about science. Like all forms of prejudice, this flaw in judgement is unfair, discriminatory, and dangerous. It is a serious problem and we have to make it stop.
Consider the following statements which are typical of conversations between researchers. All of them are examples of things I have actually heard.
Actual Statement #1: "I am just happy to have my name appearing in an article in Science".
Actual Statement #2: "This professor in my home university is amazing. She got an article published in Nature!"
Actual Statement #3: "This student has published three PRLs during her PhD. She must be really good."
Actual Statement #4: "You become famous by publishing in Nature and Science."
Actual Statement #5: "That paper is only a PRA."
Actual Statement #6: (To the editors of Nature): "Who are you to decide what the best papers in the world are?"
Actual Statement #7: “But even if you make all those great improvements to the experiment, you will only get in published in IEEE, not in Nature Photonics”.
Actual Statement #8: “I guess my paper is not as good as I thought – it wasn’t accepted in PRL."
Do you see a problem with those statements? I sure do. Do they sound familiar? Of course they do.
As a community, we need to stop thinking along these lines and refrain from using this type of language. We must remind ourselves of a prejudice we have taken too far. We are the ones to blame. Think about how much better it would be to hear the following statements as opposed to the ones listed above:
Better Statement #1: "I am happy to have been part of such a fantastic paper."
Better Statement #2: "This professor in my home university is amazing. She solved a long-standing problem with a very creative new technique!"
Better Statement #3: "This student has published three PRLs during her PhD."
Better Statement #4: "You become famous when you are responsible for ground-breaking results that are widely recognized by your peers."
Better Statement #5: "That paper is a PRA."
Better Statement #6: (To the editors of Nature): "Who are you to decide what papers get published in Nature?"
Better Statement #7: “But even if you make all those great improvements to the experiment, aren’t you afraid people will not see it as innovative?”
Actual Statement #8: “My paper is as good as it always has been, even if it wasn’t accepted in PRL."
I am definitely not the only one speaking out about this issue. Carlton Caves has written a masterful column about what he calls the ‘High-impact-factor Syndrome’. Valerio Scarani has also blogged about this before. So why do I say the same things again but in different words? Because I believe that the more of us that speak out, the higher the chance we have of actually making a difference. Much like the problem of judging people based on their skin colour or sexual orientation, this is an issue that arises from a flaw in our mentality, and it is hard to change mentalities with isolated arguments. We need several people to repeat the argument many times.
If you think I am exaggerating the importance of this issue, allow me to persuade you otherwise. The stakes are very high - we are talking about people's careers and the integrity of science! Do we want to steer innovative and creative young scientists away from a research career because their results are not of enough general interest to appear in Science? Do we want to incentivize scientists to be dishonest in order to boost their chances of getting their papers accepted in PRL? Do we want an environment where so much energy is spent working on over-hyped topics instead of pushing science forward? Do we want to be part of a community that judges the accomplishments of its members based on a form of prejudice?
I would like to invite you all to raise your consciousness on this issue and to recognize the role that you can play. Please think twice before you convince yourself that a paper can’t be any good unless it appears in a top journal. Please be careful when you speak about a scientist’s worth based only on how many of her articles appear in these journals. Please find some time to read the papers of the people who apply to your group. Please remember that the quality of your work is unchanged whether it gets accepted or rejected by a particular journal. Please help me make our community an even more enjoyable place to do science.