Sci Snippet – Who reviews papers?

ReviewingThis blog is called “Things I Tell My Mom” for a reason. These are all things that I really do talk to my Mom about.  In fact, after posting about the trials and tribulations of publishing a paper, my Mom asked who these reviewers are anyway.

Reviewers are usually (hopefully) in the same field as the researcher so they have a background knowledge that will allow them to evaluate the research carefully and thoughtfully based on what is already known in the field. The editors select several (usually 3) reviewers who each review the manuscript independently. The good news is that in an ideal world, these well-informed reviewers will be in the best position to provide the journal with insightful feedback.  The bad news is that they may like what you’re working on so much that they provide suggestions for lots of additional experiments, steal the ideas in the paper and then quickly publish them before you get a chance. Is this “scooping” ethical? Nope. Does it happen? Yup.  Often? Probably not that often.

You also have to keep in mind that the reviewers know who the authors of the paper are, but the reviewers comments are anonymous. So if you get a poor review, you don’t always know if it’s because the manuscript is terrible or if the reviewer is someone who you are competitive with professionally or don’t get along with. As my Dad aptly said, “That system sucks.” This is in part why some journal are starting to offer double blind review (described in more detail here).

Beside this apparent conflict of interest, reviewers are also active researchers and therefore super busy people. If the reviewer doesn’t take the responsibility of reviewing seriously, this can mean one of two things – it will take a lot of time for them to get to reviewing the paper (dragging out the waiting) or they will look through it quickly and provide a crappy review. Crappy reviews can reject great papers or accept terrible papers. It’s an imperfect system. Some of this imperfection is highlighted on this hilarious and depressing website S**t My Reviewers Say Tumblr.

Also, keep in mind that reviewers don’t get compensated in any way for reviewing – it’s part of a scientist’s service to the scientific community. I have been a reviewer many times, and I take the job very seriously and try my best to provide a fair, complete review in a timely manner – and I expect to receive the same when I submit manuscripts as well. This is the ideal, but not always the reality.

For more Sci Snippets, click here.

How do scientific papers get published?

I had a busy week last week.  Besides doing experiments, I was also working on a manuscript about sustainability in biobanking. I’ve talked in other posts about what a scientific paper generally looks like (see here), but not what it takes to get from experiments in the lab to publication.  This is what I’m going to talk about today.

First, you have to know enough of the field to have an understanding of what is known and not known.  Second, you have to identify a hole in that knowledge that you could fill by doing experiments to test hypotheses. Third, you needed to do well-controlled experiments that will hold up to careful analysis.  These will be the basis of your manuscript’s results section.


From the NYT article

Let’s talk about this point a bit more, but first, go to the New York Times and take this little puzzle (it’ll only take a minute or so). This puzzle boils down the essence of doing a good experiment.  You start with a hypothesis (e.g., the numbers are all even). You test that hypothesis and see the result. If you get a negative result, you come up with a new hypothesis and test that.  If it’s a positive result, you can do a few things.  You can decide that your hypothesis is correct and try to guess the answer or, in the case of the laboratory, publish a paper about your new amazing result.  However, it won’t get published because you could be wrong. You will have to find other ways to confirm your hypothesis using different or complimentary experiments.  You should also do your best to disprove your hypothesis. Even in the NYT puzzle, you should try to find negative results. This will help you better understand the limitations of your hypothesis and the results you obtain.  In the lab, it will likely also lead you down roads of abject failure that will never see the light of day, but that’s the reality of science.

Now that you have your experiments completed, your forth step is to actually write the paper.  My undergraduate research adviser always had a draft outline of the paper in process as experiments were being done. That way, he could fill in results as he went along. Most researchers aren’t that organized and they get to writing the paper when their adviser (or the head of the lab) looks at them essentially tells them to “get their butt in gear and write the damn paper, already!” Why is starting a manuscript so difficult? Mostly because it involves synthesizing information. Results on their own are beautiful pieces of success that could go anywhere and do anything.  Synthesizing these into a “story” and discussing the implications of these results on the broader field is hard work. And the wrong analysis can affect the chances of the paper being published or your reputation in the field.

The other challenge when starting to write is figuring out what journal to submit the manuscript to – and keep in mind, there are thousands of possibilities.  Depending on the scientific significance, novelty of the results, and general scientific interest, you may submit your paper to one of the top journals (like Nature, Science or Cell) or you may submit to one that isn’t.  Maybe you’re wondering why this matters?  Each journal has an impact factor, which indicates how often papers published in that journal get cited by other authors.  The more prestigious the journal, the more citations your paper gets, indicating generally, that your research is more influential.  This affects how likely it is for you to (fill in the blank): get a job, get tenure, get invited to give talks at meetings around the world, get a raise, get more grants, get more students, be more successful overall etc. Therefore, the higher the impact factor, the more likely it is for you to become a wildly famous (in the ideal) or successful (in reality) scientist.


The first page of a publication I wrote about the DNASU plasmid repository published in the journal Nucleic Acids Research

Once the journal is decided upon, and the manuscript is reformatted to fit the requirements of that journal (which is a whole other pain in the butt sometimes requiring you to cut the manuscript length by 60%) you submit your manuscript and wait. And you may wait a long time.  I’m not a journal editor so I can’t provide all the insider details, but once at the journal there are  levels of review.  Internal reviews by the journal editors sending the manuscript out for review by other scientists in the field, additional editorial review, etc.  At each stage the manuscript can be rejected and each step takes time.

The manuscript reviewers take on an essential role in the scientific process. They help to ensure that the research is high quality and that the claims made in the paper are valid and supported by evidence. Any scientist can be a reviewers (I’ve reviewed a half a dozen manuscripts), but they should ideally have knowledge of the scientific topic being addressed in the manuscript. Typically each manuscript that is sent out to review to three scientists, and they each anonymously provide their comments back to you and to the editors.  This is where the process can get messy.  The reviewers could just flat out suggest your paper should be rejected or accepted.  Then again (and more likely), they may ask for edits – these can be as simple as fixing typos and adding a few clarifying sentences to doing years worth of additional experiments. And the reviewers may not agree – one reviewer could accept the manuscript, one could reject it and one could ask for edits.  It’s then up to the journal editor to decide what to do. For more about reviewers, check out this Sci Snippet about who these reviewers actually are and the crazy things they sometimes do and say.

When you get the reviews back, if your paper hasn’t been immediately accepted, you can choose to make edits, do the suggested experiments and resubmit to the same journal.  If this is your choice, just by re-submission, you are not guaranteed to have your paper accepted.  You may spend months doing experiments, just to have your manuscript rejected at the end of the day.  If the reviewer comments are too negative or the paper was rejected, you may realize that you set your sights too high and you should submit to a different journal that may have a higher likelihood of acceptance.  At that point, you start the review process all over again.

If this sounds frustrating to you, it is.  I have friends who have had papers in the review, edit, review process for over a year.  I have friends who have submitted papers over and over and haven’t been able to get them published. I also have friends that submit to a high profile journal and get accepted right out of the gate.  However, with the stakes being so high and affecting you scientific career so directly, the publishing process is a necessary frustration and if done correctly, can make your science and your research better.


What does a scientific paper look like?

When I was in college, I read my first scientific paper.  I don’t think I can recall the EXACT nature_humangenomefirst paper, but I remember a project where we had to learn about a topic of our choice and then write about what is known, what isn’t know, and what experiments we would do next.  It was so traumatic that to this day I remember that I read and wrote about G-protein coupled receptors (GPCRs).  These GPCRs are a diverse set of genes that make proteins that transport molecules into and out of the cell, but they only do this in response to a particular signal.  It’s as if the GPCR is a door, but a door that only let’s people in or out when someone pushes a button.  But I digress.  In college, reading these papers was like reading James Joyce – I knew that it was in English and that the words made up sentences, but it was incredible difficult to figure out what all of the sentences meant.  It was even harder to grasp the “big picture” ideas in the paper, and just forget about understanding all of the data!

But like all things, understanding a scientific paper takes practice, but isn’t impossible.  It’s all about learning enough of the vocabulary and scientific background to start to understand what the sentences mean, and then getting an idea of how a paper is structured to better find the big picture ideas.  The whole point of a scientific paper is for the researchers who performed the research to explain what they did (the experiments), show in detail the results of these experiments, and then describe what it all means.

So what does a scientific paper look like?  It has six main parts

  1. Abstract: although I’m not sure why it’s called an “abstract” because sometimes it’s the most clear part of the paper.  This is essentially a summary of the question the researcher asked (the hypothesis), why the hypothesis is interesting or important to know the answer to, what they did to test this hypothesis, what the results are and why the results are important.  And you have to do all of this in ~500 words.
  2. Introduction: this section talks about what is already known in the field.  This is also where the authors specifically say what isn’t known and what they are trying to figure out through the experiments in the paper.
  3. Materials and Methods: This section describes in detail how the experiments were done.  As an analogy, if a scientific paper is the menu at a restaurant, the Materials and Methods section is a list of all of the recipes. There are a few reasons that this section exists – first, scientists reading the paper need to know if the experiment was done correctly so they know whether or not to trust the results.  Second, if a researcher ever wants to repeat the experiment, these recipes should provide enough information to do so.
  4. VolcanioResults.  This is the meat of the paper, or in the restaurant analogy, the meal itself.  Results are described in words.  For example, in the famous elementary school experiment making a volcano, the results would say “when we added 1 cup of vinegar to 1 tablespoon of baking soda, 2 cups of foaming liquid erupted from the volcano.” Results are also shown using figures and tables.  A figure would be a picture of the erupting volcano and a table may include the amount of liquid that erupted from the volcano each of the 3 times that you did the experiment.  In scientific papers, the number of tables and figures varies, but usually there are 3 or more figures, each of which may have multiple parts.  An example of a figure from a journal called PLOS ONE is below. Accompanying all tables and figures is a legend describing what the table and figure means.plosoneHairPaper
  5. Discussion.  After all of the results have been presented, the authors then discuss what they might mean.  In the case of the volcano experiment, the discussion may include a sentence like “More foamy liquid was produced when more baking soda and vinegar were used but not when baking soda was added to water, indicating that the chemical reaction with the vinegar is critical to creating the foamy eruption.”  Usually the discussion is where you can find the “big picture” meaning of the research and get an idea of what may be missing.  Scientists typically say what the limitations of their study are and indicate what kind of experiments come next.
  6. References. In addition to all of the text, scientific papers include links to other scientific papers that were used as a basis for their work or to support the results or conclusions.  These are a great resource to read to find out what’s important in that specific field and most of the time when I read papers, I end up in a rabbit hole of additional reading!

To give you an idea of what a paper looks like, see here.  You can also check out some open access journals that give 100% free access to all of the published papers, like PLOS Biology.  Scientific papers are not written for the public.  They will look to you a lot like they looked to me when I first started reading scientific papers over 15 years ago. But in future posts, I will provide information on where to find information about current research in journals that is more accessible.