Second, perhaps we should spare a thought for the brilliant individual who is stuck, through some early stroke of fate, in a tiny group in Nowheresville, Antarctica. Without a journal system, the ideas of that person might have no chance at all. Although I have sometimes despaired of the ability of academic journals to accept papers that are truly novel, the journal system does work to a limited degree. Some important stuff makes it past the grey-minded reviewers. If there were no peer-reviewed journals, I bet researchers would pay even more attention than they currently do to status symbols such as a university’s name and nation.
The simple truth is that the scholar who is tired of referees is tired of life. Journal reviewers are going to keep on being incoherent, making fundamental mistakes, being childlike and gratuitous, forcing us to cite their irrelevant articles, thinking that after 20 minutes they understand our work better than we do after 20 months, leaving out the crucial references they claim have already shown the finding, and all the rest.
Between you and me, the only good referee reports I ever saw were written by me and by – oh, wait, no: there isn’t anyone else. This is how we all feel, and in some ways it is surprising how well a voluntary system of refereeing, based on a form of honour system, actually works.
Peer review is a good way of seeing what the next generation is doing, but I am becoming increasingly selective in which invitations I accept
The worst comment I received on anything I have ever written was in the form of a question: “What is this muck?” It was an essay for a journal that claimed to be pioneering new research in cultural and media studies. Doubtless writing about a feminist rock star was just too much for someone!
In principle, it seems like a fine idea for work submitted to a journal, publisher or funding body to be assessed anonymously by independent experts. Surely it is more equitable and transparent than individual editors and research managers taking subjective decisions, as was the norm in my early career.
But, as with all the ventures by academics into quantifiable systems of assessment, it has grown into a monster. Academics – and those academics transformed into “managers” – have learned how to play around with peer reviewing and turn it to their advantage in determining who wins and loses in the local promotion stakes. And their use of peer review is used by open access publishers to excuse the venality of asking for money up front from authors (something that used to be derided as vanity publishing).
That everyone now needs to confirm that they have been peer-reviewed, even if the piece in question was published by the smallest online journal, means that the task of reviewing is now gigantic. In the first six months of this year, I was asked to peer-review more than 20 articles, half a dozen book proposals, and so many bids for funding that my memory fails me. Not all come through anonymously, and if you work in a specialist area, you can often identify the writer.
Of course I don’t agree to peer-review everything I am asked to, but I am told by editors and publishers that such is the demand to publish that they are becoming desperate to find people willing to do the assessments. Because it isn’t just a matter of reading 20-odd pages of typescript and passing an opinion. No, you have to log on to complicated websites and answer a host of questions. This can take ages, especially if you are asked to provide information that can be relayed back to the writer as distinct from a box for non-disclosable comments. The complexity of some of the processes may explain the logjams and long delays in getting feedback to writers, some of whom are kept waiting for months.
Attempts have been made to raise the status of peer-reviewing to something that can be put on a CV. An example of this is the fanfared creation a few years ago of the Arts and Humanities Research Council’s “Peer Review College”: a publicly identified body of about 1,500 individuals who referee applications for AHRC funding and from whom the research council’s assessment panels are largely drawn.
I went along to one of their training days, during which we were divided into groups and asked to assess previously submitted applications. In my group we failed two and agreed that two were excellent. It then turned out that our failures had already received some £750,000, while our excellents had failed to win anything. So much for expert opinion, I thought. After receiving a string of half-baked proposals and struggling with the problems of the website, I resigned and wrote to the AHRC explaining why. I could also have mentioned that some other funding bodies pay their peer reviewers – which is some small compensation for their time.