Thursday, June 29, 2017

How to Cure a Cancer: Thoughts on Improving Academic Journals

So, I currently have a paper that has been under review since last December, coming up on 7 months. A friend of mine recently waited something like 16-17 months at the JME. This is ridiculous. It doesn't seem like it would be that difficult to design a system in which this doesn't happen, or happens only very rarely. It's led me to think about all the ways the academic economics publishing system could be improved. Particularly since economists study incentives, you'd think we could design a good system.  All economists know and believe there are obvious problems in the academic publishing system, and yet, somehow, the profession soldiers on like a drunken sailor.  Thus, here are a few proposals, most of which are obvious and probably not new:

1. A recent positive trend is that many journals pay referees for finishing a report within a certain number of days. I approve. However, my paper is currently at such a journal, which has limit of 4-6 weeks (range given to protect the anonymity of the journal). After that, a referee gets nothing. What I wonder, then, is if a referee misses the deadline, they no longer have any incentive to write a report sooner rather than later. I also wonder if this doesn't make them more likely to delay. If the deadline was 30 days, and you wake up on day 31, and realize you've missed a payday, it seems you might be less likely to submit the report on day 31, feeling like a chump. A phase out of the payment over several months would likely be more effective. Here's another idea, why I'm throwing them out: referees should be given an option to have a larger payoff with a short duration of a referee report -- say, two weeks -- while also agreeing to pay if they don't finish a report within two months, with a fee increasing incrementally each month. And would need to provide their credit card information in advance. The fee for backing out could be set to be even worse than submitting a report at each date, to make this incentive compatible.

2. Another problem is that much research is funded by the government, or by universities paid for by tuition dollars, and yet academic research is not a public good. Academic journals make bank. First, academics write papers and submit them to journals, and not only do they do this free of charge, but they actually pay for the right to have the fruits of their free labor published, for the benefit of the journal owners. We must do this, of course, as our academic careers depend on publishing in fancy journals. Our careers are also helped by prestigious editing positions at journals, which we are also happy to do for free or low wages. Referees then, also, typically work for free or at least below-market wages. Next, the journals charge lots of money to the same universities which pay their professors for access to the same research the universities pay professors to produce. It's a great business to be in, if you can get it. The issue is that the most prestigious journals have near monopolies -- the tradeoff is that one Top 5 publication is seen as being worth 4-5 publications in Top Fields (at least for your first one) -- a crazy ratio, but the profession is obsessed with rank and status. The solution here is some form of collective action. For example, the government should tax academic journal profits, and could use the proceeds to help fund education, or it could regulate the fees that journals charge for libraries or the public to access the research. If the Ivy League, or the UC system were to spell out that only research published in open access journals or which obey some other criterion would result in tenure, this could be a start. The problems and solutions here are so obvious I'm hesitant to write it down. A dream of mine is also to devise a journal ranking system which penalizes journals for bad behavior (e.g., for high fees to libraries, or high submission fees), so that departments could change the ranking system which they use to award tenure and promotion decisions, in order to better incentivize journals. 

3. A third problem is that referees, and even journal editors, have limited incentives to try and do a good job, much less a timely job. One thing I propose here is for journals to ask authors for feedback on how the referee and editorial process went.  Clearly, when someone gets a negative referee report, they will give someone a bad ranking, but overall referee scores could be computed controlling for the recommendation of the report. Partly, I think this would be good for purely therapeutic reasons, even if the journals didn't use this for anything. When you're pissed off after getting yet another clueless report, you now have an outlet -- you can kill them in your referee Eval! However, the journals could band together to create a referee ranking. Editors and other referees could also rate referees. There are certainly cases where referees say crazy things, and this is acknowledged by not only the authors, but the editor and other referees. And yet, the said referee pays no penalty at all. On the contrary, they may receive the benefit of not being asked to referee again for some time. If a system were designed to give feedback to departments: "your professor X has a history of providing unprofessional reports", or simply: "here is the rank of your APs in terms of their prowess writing referee reports", this could change incentives for the better. 
4. Another problem is that journals have no incentives to publish comment papers, which tend to reflect poorly on the journal, and which also tend to be poorly cited. Thus, in the ranking system I devised above, I would penalize journals that don't accept comment papers on papers published in its own journal. 







3 comments:

  1. Long ago, when papers were read at society meetings, the editors recruited discussants, whose comments and a response from the author were printed together with the paper. These made for interesting reading. Since the whole thing was public, everybody got credit or blame. Even the editor's choice of discussants was on display, so that anyone in the know could tell whether he (and it would have been a he in those days) was being fair. I don't think some modern version of that would speed things up, but having the reviews public would certainly improve their quality.

    ReplyDelete
    Replies
    1. This is an interesting idea. I support it. We spend so much time writing referee reports, and put so much time and effort responding to referee reports, that what is the point of keeping all this information secret? One issue is that if the referee is known, then the referee might have incentive to recommend acceptance if it is a big, powerful guy or a journal editor. In that case, some anonymity is useful.

      In fact, I also have experimented with making the referee reports I receive public. I think, in one case this worked out well. A later referee recommended accept, and said he found my responses to previous referees helpful and persuasive. In the other case, I'm not so sure. Referees might have looked at it and thought -- many previous referees didn't like it, it must be bad work. Sure, I didn't think the previous referees had raised any serious issues, in fact, I thought most the points raised were either wrong or minor, but others could well come to a different view. Another worry, which I may be in danger of here, is insulting my past referees, even though they are anonymous. So let me also say that I've also received many brilliant referee reports! I also made public a referee report in which I recommended "reject" on a paper topic closely related to one of my papers public and put it on my webpage. I put it up since I stand by my comments, and also because I don't want to be an academic gatekeeper. I won't do this when I recommend accept, however, as that could be self-serving.

      Delete
  2. My Cancer that was cured by a great herbal medicine man. Since last 7 months i have been suffering from cancer. last 5 weeks I was in a great pain so I told one of my friends about it she told me that there is a great herbal medicine man that can heal cancer and so many diseases and he also help people in getting pregnant and bringing back EX. I asked her if she had the man contact so i could give it a try, she gave me the email, I emailed him he talked to me, and he send me the herbal medicine and herbal soap via DHL service to my country, told me how i will be using the herbal soap and drinking the herbs and after few weeks I should go for a test. Which I did, when the doctor told me that am now cancer free I couldn't believe myself I went to see another doctor the result was still the same i couldn't keep this to myself so i decided to let it out on this page that if anyone here need help in curing his/her cancer or trying to conceive should also contact him once again you can email him DRTOKUBOTEMPLE@GMAIL.COM or WHATSAPP him via his mobile number +2349037990322 wish you good-luck

    ReplyDelete