THE REVIEW PROCESS OF THE BRITISH JOURNAL OF SPORTS MEDICINE

When a manuscript is submitted to the Journal, an editorial management decision is first made as to whether it is an appropriate paper for this journal. For example, a paper on sports astrology would be considered inappropriate whereas a randomised control trial on the treatment of groin pain would be eminently suitable. It is worth authors noting that, as the years pass and the Journal receives increasing numbers of high quality papers, then papers that may have been published in years gone by are now more likely to be rejected. Attempting to pass off a personal case series as an original research paper is a common example of this.

The number of papers published in the Journal is limited by the total page numbers, which are fixed annually by the Board of Management. With six issues per year and a maximum of 10 research papers per issue, the equation is simple. Manuscript submissions have steadily risen to about 350 a year. This means (hopefully) that the quality of the Journal will rise as more papers are inevitably rejected. It also means that more reviewers are required each year.

Once the manuscript has been editorially assessed, a decision is made on reviewers. Generally three reviewers are used for each paper or review for reasons that are outlined below. The reviewers are selected from various sources: authors' suggestions, the journal's reviewer database, Medline searches of similar recent articles, assistant editor advice, or known experts. When the paper is sent to the reviewer, it is usually a “blind” copy with no author names/institutional information provided. This is an attempt to make the process as fair as possible. A paper should be rightly judged on its merits not on who wrote it! There are exceptional occasions when a paper needs author identification in order to be accurately assessed. A reviewer would have to make a fairly persuasive argument for the identity of the author to be revealed.

Reviewers are then solicited by email to ascertain their availability for reviewing a particular paper. With our new manuscript tracking software, a prospective reviewer will receive the request along with the manuscript abstract to make a decision about their availability. Once they agree, the full paper is forwarded to them electronically.

We generally aim to turn around reviewer comments in a six week time frame. As can be readily appreciated, this is not always possible given the demands on clinicians' time. This may be a particular problem in highly specialised areas of research where relatively few “experts” exist. Nevertheless, email reminders are initiated at standard times to attempt to achieve this deadline.

Once the reviewers' comments are known, the authors are notified. It is rare in this day and age for a paper to be accepted without revisions. Those who publish regularly will realise that manuscript rejection is a normal and healthy part of the publishing process. Most of the papers require an extensive revision and resubmission, which requires the authors to revise the paper as suggested by the reviewers, and then the resubmitted manuscript goes back to the original reviewers for a further assessment. In some cases, three or four major revisions are required to get a paper into a publishable shape. When this happens, the process may become protracted over many months. If the reviewers' suggestions are “minor”—for example, typographical errors—then we can notify the authors that their paper can be “accepted, pending revisions”. Clearly the more timely the authors' responses to suggested revisions, the faster the publishing process.

The advent of electronic paper submission, electronic manuscript tracking, and, from February 2002, online reviews should help to minimise processing delays that occur during communication between the Journal, reviewers, and authors. It is hoped that this will translate into faster decisions for authors.

If major debate occurs between the reviewers and the authors, we often use an impartial reviewer or “ombudsman” to determine the outcome of the paper. Fortunately this is rarely required, and most authors see the review as enhancing the final paper rather than a negative process.

BEING ASKED TO BE A REVIEWER

As I mentioned earlier, for the first time reviewer, the initial feeling is often the same as receiving a letter from the tax office. One of the common worries is the feeling that “it is not my area”. It is worth noting that the journal seeks many types of reviews: some are purely statistical, some are expert comments about methodology or techniques, and some are a scientific critique or a “readability” assessment from a person who may have a more generalist knowledge of the area under review.

The concern of many prospective reviewers is time. Generally we are all busy folks with a clinical practice or research programme to run, and the additional burden of journal reviews simply adds to the stress levels. The old maxim “if you want something done, ask a busy person” is never more true than here. Journal editors are extremely grateful for the time and expertise of all reviewers. One comment exemplifying this concept came from a reviewer asked to review a case report: “I shall give it a try. My mother-in-law is being buried tomorrow and my daughter has just gone into labor so no problems”. My hat goes off to that reviewer.

THE PERFECT REVIEW

From the perspective of both the editor and author, a perfect review is rapid, impartial, and constructive. It should be an educative process for the author and result in an unambiguous recommendation for the editor.

The reviewer as “gatekeeper”

Inexperienced reviewers often see themselves as a “gatekeeper” trying to hold back the process by which authors seek to be published. Such “hawks” often simply produce a list of negative comments. In many cases, although it provides the editor with a firm opinion, the review offers nothing to an author who may seek to improve his or her research or scientific writing.

For example, a comment from a recent review went as follows: “I reviewed the submitted paper and started my list of deficiencies. After two pages I began to realize that there was no part of the manuscript that meets reasonable standards in terms of science, logic and even English expression”. Such a damning response leaves the author little to go on. Another similar comment went as follows: “I find the paper totally non contributory to any biomechanical territory and not worthy of publication”. In some cases, the entire review is a single derogatory sentence. To the journal (as well as the author), such a review is not worth the paper it is written on. Generally, such reviewers do not get asked to review further manuscripts.

In part, I suspect that this problem relates to our lack of training in this area. To be asked to review a paper for the first time is a little unnerving and our role as a reviewer is often unclear. We feel the need to be excessively critical to try to justify our selection by demonstrating our academic teeth. Nothing could be further from the truth. If a paper is worth damning, then it should be rejected. It is the manner or style in which this is done that becomes the key element. Scientific problems can be identified along with suggestions on how this may be overcome in the future. Some of the best reviews I have read express the reviewer's difficulties in assessing the paper. In one superb review, the reviewer stated that he “agonised” over the manuscript and then attempted to annotate and rewrite much of the paper in order to show the authors how he thought it should be written. Although the paper was rejected, I am confident that the authors came away from that process empowered to improve their paper in a positive light.

We need to remember that the review process is part of a wider education of an author. When all of us begin a research career, scientific writing is often the most difficult skill to develop. A good research supervisor or mentor can assist this process, but the process of publication helps us to refine these skills further, and good quality reviews are the key.

The Peter Principle

One of the common phenomena seen by editors of journals world wide is a variation of the Peter Principle. If two reviews are sought, then it is likely that these will produce polarised outcomes—that is, one will “reject” the manuscript and the other will “accept without changes”. This is often despite such comments as outlined above. This is the primary reason why we seek three reviews as a general principle, in order to arbitrate this process more fairly. It never ceases to amaze me as an editor how two experts can read the same manuscript and come up with entirely different viewpoints.

Hopefully with our new software system, reviewers will have access to other reviews and any correspondence. This may assist in their personal development as a reviewer by seeing what other experts say.

A good reviewer generally tries to see what message the authors are attempting to communicate and in what fashion. Methodological concerns typically arise, but criticism can be constructive or destructive. It is far more useful to make suggestions on how to improve the paper to enable the authors to understand the problems than to savage the paper in an uncompromising fashion.

One of the most difficult problems is language. Although each of us has a writing style that is unique, the Journal is global in its outlook. This means that many of the papers are sent from countries where English is not the primary language. Difficulties in spelling, syntax, verb construction, and so forth often limit the readability of the article. In general, we would recommend to such authors that they seek an opinion on the manuscript from someone who speaks English as a first language. In this situation, a reviewer has an even more difficult job. The guiding principle should be to see whether there is scientific merit in the work that may be hidden by the grammatical difficulties. Remember grammar can be improved but the science often cannot.

An editor's dream

One of the concerns in journal publishing is the fear of duplicate or redundant publications. A reviewer who is familiar with the topic under scrutiny is often familiar with similar publications that may need closer inspection. In the last 12 months, the Journal has had two cases, of which we are aware, of previously published data being submitted as original research. In both cases, the reviewers picked this up because of their expertise in the area.

It is a good habit for a diligent reviewer to carry out a Medline search of the topic or the authors' other publications to assist in this process. In many good reviews, the comments are not only constructive but they also point out recent research that may have been missed by the authors. With the online reviews in our new manuscript tracking system, this process will be considerably simplified and should become almost an automatic process for the reviewer.

Authors need to bear these issues in mind. With computerised manuscript tracking and the ever increasing coordination of journal software, it is foreseeable that a reviewer will not only have access to Medline but also to similar manuscripts submitted to other journals, which makes the likelihood of detection much greater. Any author who deliberately attempts this type of academic deception will be blacklisted from future publication. Hopefully as journals increasingly collaborate, such a “red card” system for fraudulent publication will be more widely adopted.

The truly obsessive reviewer not only carries out the Medline search but also reanalyses the authors' data and comments on the appropriateness of the conclusions drawn from this information. Unfortunately this is an extraordinarily rare and somewhat frightening phenomenon.

CONCLUSION

To be a good journal reviewer is an educative process in many ways similar to that of the development of an author. A widely published author generally has experience of good and bad review comments and should be able to provide a fair and appropriate manuscript review. Nevertheless, reviewers need to hone their skills and perhaps their contributions need to be formally assessed to enable them to improve their future contributions. All journals attempt to make this process as fair and impartial as possible, but the vagaries of individual reviewers often surprise even experienced editors.

The perfect reviewer provides the journal with rapid review turnaround, detailed analysis, helpful comments, an assessment of the current literature in this area, and an unambiguous recommendation. For the author, the reviewer should provide a constructive analysis of the paper, with a Medline review of any recent work omitted, and clear recommendations on how the paper may be improved. Although guidelines may be suggested, finding perfect reviewers is a bit like hunting the proverbial Snark.