Month: August 2014

Dear Colleagues,

You’ve made it to the big leagues – the (figurative) stack of papers sit in front of you, ready to be ‘peer-reviewed.’  You’ve done it, right?  You’re a peer?

Peer reviewing papers is one of the most important processes in the academic world.  These peers are the gatekeepers to what everyone else will consider adequate for citation.  Before any Wikipedian or college student can refer to the piece in front of you, it has to be approved… by you!

The power that is in your hand is not to be wielded lightly… the world’s collective knowledge depends on your judgment.  Accept?  Reject? Encourage to resubmit?  The fate of the piece in front of you is in your hands.

Perhaps the hyperbole above is a bit too dramatic.  Let’s put this in context.

You, with adequate funding and permission from your boss and/or parents, will be at the Transportation Research Board’s Annual Meeting.  And therefore, you’re my peer.  Always remember that even the ‘experts’ are your peers as well.

Remember that, as peers of this esteemed institution, you are qualified as the ‘peer’ in peer-reviewed.  And your opinions are important.

During my first peer review session with ASCE, I questioned myself.  Am I really qualified to judge these papers? (Yes.)  Can I really provide comment even if I’m not the expert of that material? (Yes.)  What do I tell them in the comments? (The Truth.)

First off, you’ll most likely be qualified to read something if you’ve self-selected yourself on to committees are within your realm of study.  Sometimes we get caught up in our own world, forgetting that the population at large is not the transportation folks that we interact with on a daily basis.  If you only hang out with transportation people with knowledge that sits two standard deviations and higher, like coworkers, researchers, professors, and the train nut in me, then your world is a bit… skewed from normal.

But still, check out that first paper in your stack and ask yourself, is this something that you understand?  Remember that the expert on that paper will be the author, so perfect understanding will only be as good as the words on the (metaphorical) paper.  After your initial scan, perhaps the abstract is starting to make some sense.  You’re probably qualified.

Though, if you’re into policy and cities, and you get something examining the effects of precipitation draining on rural asphalt roads, then maybe reconsider.

Given that you’ve self-qualified yourself to review this paper, perhaps you may think that you cannot provide comment, even though you’re a peer-expert.  Remember that a good paper is constructed well – composition is important.  The author is responsible for clearly communicating their contribution to the web of knowledge, and you’re not responsible for guessing what the author means.  If the author isn’t clear, then you should note that in your comments.

Does the work match the thesis they provide?  (Is there a thesis?)  Do they adequately cite other pieces, and contribute something new?  Is it relevant to the committee?  Then those are good signs that it is a good paper.

Check out Andy’s guide on TRB reviews, check out what TRB has to say about its own reviews.  Be honest, kind, and constructive in your comments.

Best,

Raymond Chan

As August rolls in, the Transportation Research Board’s paper review season begins. Over the next few months TRB staff and committees will pull into high gear to review over three thousand papers to determine suitability for presentation and publishing. For many of us involved with TRB, paper reviews can seem like a time-consuming chore, however reviews are central to TRB’s mission. Here’s some tips to guide you as a paper reviewer:

The 8 Cardinal Rule of Paper Review

Reviewing TRB manuscripts begins with eight simple rules. These rules help ensure that the review process upholds TRB’s ethical and research standards:

1.  Determine whether you are qualified to review a paper. Only review a paper if you know enough on a particular topic to successfully review the paper. Don’t be intimidated by the title; give the paper a quick scan to determine if it’s within your area of expertise.
2.  Objectively review a paper for merit and don’t make the review personal. The paper reviewer should be focused on reviewing a paper for originality, accuracy, and interest to TRB and its committees. Reviews should keep their comments professional and constructive.
3. Avoid conflicts of interest. Notify your paper review coordinator if there is any conflict of interest in reviewing a particular paper. As TRB uses a single-blind review process, authors are not anonymous. If you feel that you would have any personal bias, recuse yourself from the review process.
4.  Keep paper manuscripts confidential. Manuscripts are confidential documents and as such you should avoid discussing the manuscripts or its findings with others.
5.   Support your judgment. Review managers and the paper authors should be able to understand your basis for criticism.
6. Call out similarities between this document and other published or unpublished work. TRB strives to publish and present original research. Note any instances where the manuscript has been published elsewhere or submitted to a concurrent journal
7.  Don’t use or disclose any unpublished information without the consent of the author(s).  Receive permission from the author(s) and properly attribute the paper if you intend to disclose any information, findings, or interpretations from the manuscript.
8.  Call out any instances of plagiarism. Notify the review manager if you have evidence of plagiarism or falsification of results.

The Paper Review Process:

The paper review process gets going on August 1^st, the due date all TRB manuscripts. Committee paper review coordinators take these papers and assign them to at least three reviewers. Once comments are returned, the committee leadership, paper review coordination and TRB staff work together to determine which papers should be accepted for presentation and publication. As the paper reviewer, you will be asked to make separate publication and presentation recommendations.

How to Recognize a Good Paper

One of the great things about TRB is the sheer diversity of research submitted every year, ranging from public policy synthesis to scientific research and analysis. Because papers are so different, there is no one universally accepted benchmark for determining whether a manuscript meets TRB standards. TRB recommends that reviewers focus on four areas:

·         Composition: Is the paper well written and easy to follow?

·         Accuracy: Are the conclusions of the paper valid, is the methodology sound, and are the findings of the paper well defended?

·         Originality: Is the paper adding something to our field’s body of research or state of practice?

·         Interest to the TRB Community: Is the paper relevant to the committee’s mission and TRB’s mission overall?

Other Things to Note

At our July meeting, a few additional pointers for reviewers were discussed. First, don’t wait until the last minute to review papers! While the sun and sand might beckon, waiting until after Labor Day to review your paper will surely stress our hard-working paper review coordinators. For those more senior paper reviewers, if you distribute a paper to your staff to read and then consolidate the review comments, please make note of who made what comments so you can go back with questions/clarifications. When reviewing papers, don’t wordsmith the paper; focus instead on the four factors discussed above: composition, accuracy, originality, and relevance to the TRB community.

Finally, enjoy the paper review process! Being a reviewer allows you to learn about cutting edge research in our field; I have been consistently impressed by the quality and insight of the research submitted to TRB.

Want to Know More?

TRB has a number of great resources for us paper reviewers. TRB’s more detailed reviewer instructions can be found here: http://onlinepubs.trb.org/onlinepubs/AM/ReviewerInstructions.pdf

TRB also has a number of tutorials on reviewing papers, including two webinars on August 4^th and August 21^st. For more information visit:

http://www.trb.org/abouttrb/AnnualMeetingTutorials.aspx