Sci Snippet – Who reviews papers?

ReviewingThis blog is called “Things I Tell My Mom” for a reason. These are all things that I really do talk to my Mom about.  In fact, after posting about the trials and tribulations of publishing a paper, my Mom asked who these reviewers are anyway.

Reviewers are usually (hopefully) in the same field as the researcher so they have a background knowledge that will allow them to evaluate the research carefully and thoughtfully based on what is already known in the field. The editors select several (usually 3) reviewers who each review the manuscript independently. The good news is that in an ideal world, these well-informed reviewers will be in the best position to provide the journal with insightful feedback.  The bad news is that they may like what you’re working on so much that they provide suggestions for lots of additional experiments, steal the ideas in the paper and then quickly publish them before you get a chance. Is this “scooping” ethical? Nope. Does it happen? Yup.  Often? Probably not that often.

You also have to keep in mind that the reviewers know who the authors of the paper are, but the reviewers comments are anonymous. So if you get a poor review, you don’t always know if it’s because the manuscript is terrible or if the reviewer is someone who you are competitive with professionally or don’t get along with. As my Dad aptly said, “That system sucks.” This is in part why some journal are starting to offer double blind review (described in more detail here).

Beside this apparent conflict of interest, reviewers are also active researchers and therefore super busy people. If the reviewer doesn’t take the responsibility of reviewing seriously, this can mean one of two things – it will take a lot of time for them to get to reviewing the paper (dragging out the waiting) or they will look through it quickly and provide a crappy review. Crappy reviews can reject great papers or accept terrible papers. It’s an imperfect system. Some of this imperfection is highlighted on this hilarious and depressing website S**t My Reviewers Say Tumblr.

Also, keep in mind that reviewers don’t get compensated in any way for reviewing – it’s part of a scientist’s service to the scientific community. I have been a reviewer many times, and I take the job very seriously and try my best to provide a fair, complete review in a timely manner – and I expect to receive the same when I submit manuscripts as well. This is the ideal, but not always the reality.

For more Sci Snippets, click here.

How do scientific papers get published?

I had a busy week last week.  Besides doing experiments, I was also working on a manuscript about sustainability in biobanking. I’ve talked in other posts about what a scientific paper generally looks like (see here), but not what it takes to get from experiments in the lab to publication.  This is what I’m going to talk about today.

First, you have to know enough of the field to have an understanding of what is known and not known.  Second, you have to identify a hole in that knowledge that you could fill by doing experiments to test hypotheses. Third, you needed to do well-controlled experiments that will hold up to careful analysis.  These will be the basis of your manuscript’s results section.

nytpuzzle

From the NYT article

Let’s talk about this point a bit more, but first, go to the New York Times and take this little puzzle (it’ll only take a minute or so). This puzzle boils down the essence of doing a good experiment.  You start with a hypothesis (e.g., the numbers are all even). You test that hypothesis and see the result. If you get a negative result, you come up with a new hypothesis and test that.  If it’s a positive result, you can do a few things.  You can decide that your hypothesis is correct and try to guess the answer or, in the case of the laboratory, publish a paper about your new amazing result.  However, it won’t get published because you could be wrong. You will have to find other ways to confirm your hypothesis using different or complimentary experiments.  You should also do your best to disprove your hypothesis. Even in the NYT puzzle, you should try to find negative results. This will help you better understand the limitations of your hypothesis and the results you obtain.  In the lab, it will likely also lead you down roads of abject failure that will never see the light of day, but that’s the reality of science.

Now that you have your experiments completed, your forth step is to actually write the paper.  My undergraduate research adviser always had a draft outline of the paper in process as experiments were being done. That way, he could fill in results as he went along. Most researchers aren’t that organized and they get to writing the paper when their adviser (or the head of the lab) looks at them essentially tells them to “get their butt in gear and write the damn paper, already!” Why is starting a manuscript so difficult? Mostly because it involves synthesizing information. Results on their own are beautiful pieces of success that could go anywhere and do anything.  Synthesizing these into a “story” and discussing the implications of these results on the broader field is hard work. And the wrong analysis can affect the chances of the paper being published or your reputation in the field.

The other challenge when starting to write is figuring out what journal to submit the manuscript to – and keep in mind, there are thousands of possibilities.  Depending on the scientific significance, novelty of the results, and general scientific interest, you may submit your paper to one of the top journals (like Nature, Science or Cell) or you may submit to one that isn’t.  Maybe you’re wondering why this matters?  Each journal has an impact factor, which indicates how often papers published in that journal get cited by other authors.  The more prestigious the journal, the more citations your paper gets, indicating generally, that your research is more influential.  This affects how likely it is for you to (fill in the blank): get a job, get tenure, get invited to give talks at meetings around the world, get a raise, get more grants, get more students, be more successful overall etc. Therefore, the higher the impact factor, the more likely it is for you to become a wildly famous (in the ideal) or successful (in reality) scientist.

paper

The first page of a publication I wrote about the DNASU plasmid repository published in the journal Nucleic Acids Research

Once the journal is decided upon, and the manuscript is reformatted to fit the requirements of that journal (which is a whole other pain in the butt sometimes requiring you to cut the manuscript length by 60%) you submit your manuscript and wait. And you may wait a long time.  I’m not a journal editor so I can’t provide all the insider details, but once at the journal there are  levels of review.  Internal reviews by the journal editors sending the manuscript out for review by other scientists in the field, additional editorial review, etc.  At each stage the manuscript can be rejected and each step takes time.

The manuscript reviewers take on an essential role in the scientific process. They help to ensure that the research is high quality and that the claims made in the paper are valid and supported by evidence. Any scientist can be a reviewers (I’ve reviewed a half a dozen manuscripts), but they should ideally have knowledge of the scientific topic being addressed in the manuscript. Typically each manuscript that is sent out to review to three scientists, and they each anonymously provide their comments back to you and to the editors.  This is where the process can get messy.  The reviewers could just flat out suggest your paper should be rejected or accepted.  Then again (and more likely), they may ask for edits – these can be as simple as fixing typos and adding a few clarifying sentences to doing years worth of additional experiments. And the reviewers may not agree – one reviewer could accept the manuscript, one could reject it and one could ask for edits.  It’s then up to the journal editor to decide what to do. For more about reviewers, check out this Sci Snippet about who these reviewers actually are and the crazy things they sometimes do and say.

When you get the reviews back, if your paper hasn’t been immediately accepted, you can choose to make edits, do the suggested experiments and resubmit to the same journal.  If this is your choice, just by re-submission, you are not guaranteed to have your paper accepted.  You may spend months doing experiments, just to have your manuscript rejected at the end of the day.  If the reviewer comments are too negative or the paper was rejected, you may realize that you set your sights too high and you should submit to a different journal that may have a higher likelihood of acceptance.  At that point, you start the review process all over again.

If this sounds frustrating to you, it is.  I have friends who have had papers in the review, edit, review process for over a year.  I have friends who have submitted papers over and over and haven’t been able to get them published. I also have friends that submit to a high profile journal and get accepted right out of the gate.  However, with the stakes being so high and affecting you scientific career so directly, the publishing process is a necessary frustration and if done correctly, can make your science and your research better.