The Reproducibility Problem in Research

recipe

Image credit Pixabay

When you’re baking a cake, you follow a recipe that uses specific ingredients, added in a particular order, mixed in a specific way, and baked for a certain time at an exact temperature. But what if you made the cake twice?  Or three times? Will it be the same each time? Will your cake be “reproducible”? What if your baking powder is old?  The cake might not rise as much as normal.  Or you buy a different brand of flour? What if you’re baking at your sister’s house in Oregon at a higher elevation than normal?  Do you change the time or temperature that the cake bakes? How much? And what if you’re using your Grandmother’s handwritten recipe for her famous spice cake?  It’s filled with phrases such as a “pinch” of cinnamon or “about” 3 cups of flour.  Do you think it will taste the same as when your Grandmother makes it?  And what if two people try to make the same cake? On the Great British Baking Show, one of the challenges each week has each contestant follow the same recipe with the same ingredients to make the same baked item.  They NEVER come out the same because there is variability built until every step, even with the same instructions, equipment, and ingredients.

Science is sometimes a lot like baking.  Instead of a recipe, scientists’ follow a protocol (or standard operating procedure – shortened to “SOP” because scientists like acronyms). We purchase or make ingredients, typically called “reagents.”  Often a single reagent can be bought from many different companies or made in the lab by you or maybe by a technician, or maybe you were in a rush and you borrowed some from someone down the hall. In biology, biological materials like cells or enzymes are often involved.  These could be new or old.  You could have tested and “validated” your cells or antibodies or you could have relied on someone telling you that they are okay. Once you have all of your reagents pulled together, you do the experiment.

Experiments are funny little things.  You follow your SOP, but maybe one day, the 5 minute incubation turns into 10 because you were in the middle of answering an email.  Maybe another day you’re in a rush to get to a seminar so you skip a step.  Or maybe you’re training a new undergraduate how to do the experiment and you let them do a few steps on their own.

This variability is part of the reason why scientists repeat their experiments multiple times. Three, as you may expect, is often the magic number.  These data are then presented (for example, in a grant or a paper) either as a representative experiment, where only one of the three or more experiments are shown, or as an average of the experiments with “error bars” that often show how much the data differed between experiments.  This type of careful presentation gives other researchers more confidence that the result is real, as opposed to something that happened just because the new grad student messed something up.

The Reproducibility “Problem”

However, even with all this careful planning, there is a lot of chatter these days about the failure of scientists to be able to reproduce experiments.  One of the earliest papers about this topic came from researchers at Amgen who found that they couldn’t reproduce 47 out of 53 studies from cancer research labs. This has led to a snowball of studies and reports of the failure to reproduce data from various fields including biology and psychology. The most recent is a Nature Article surveying 1,500 scientists about their  ability to reproduce their own and others’ results in their own labs. 52% of these scientists felt that there was a reproducibility “crisis” and words like “bleak” and “discomfiting” were thrown around to express the severity of the problem.

So is there really a problem? Lots of papers have discussed this already, but I figured why not add my own opinion to the mix.  In part, yes, these likely is a bit of a problem.  This problem stems from using reagents you aren’t sure of.  For example, imagine that you think you’re studying prostate cancer and you think you’re using a cell line from a prostate cancer patient, but actually you’re using a super common cervical cell line? It happens all the time! An effort to make publishers and grantors enforce cell line authentication and other types of reagent confirmation before beginning experiments is gaining steam.  Not a bad idea and not too expensive.

Image credit Pixabay

Image credit Pixabay

Other efforts are also underway where a third party can be “hired” to authenticate results such as the Reproducibility Project. This is expensive, and time consuming, and one might wonder where the value lies in having someone else repeat your experiments? The value lies in having confidence in the result…but as we’ve discussed here already, the minute you move your experiment to another lab with new reagents and new people, you add more variability. If the experiment fails, how do you know it’s because the result was wrong or because someone else did it wrong?

This is where the issue lies, and I think it all comes back to the central goal of science and scientists. Scientists want to uncover what’s really happening in nature. Every experiment is done to test a hypothesis, and these results lead to more experiments and on and on. Even if an experiment doesn’t get identical results each time or can’t be reproduced in another lab, the fundamental question is whether or not the biological hypothesis is correct or not.  No matter what, scientists should always do multiple different kinds of experiments and follow-up experiments to confirm or refute their hypothesis. This all assumes that scientists are ethical and follow the scientific method – as opposed to folks who publish fabricated or modified data just to get a paper published (but that’s a topic for a whole other post!!)

So I guess the question may not be whether or not an experiment is reproducible, but whether or not the hypothesis is true. And if scientists focus on THAT as opposed to reproducibility, per se, then I think science is moving forward in a productive direction!

 

Stem Cells to Treat Spinal Cord Injury

Journal Club: Intravenous multipotent adult progenitor cell treatment decreases inflammation leading to functional recovery following spinal cord injury

Written by DePaul, et al. in the journal Scientific Reports. The complete article is here.

Background: Stem cells are unique in that they can divide indefinitely and can turn into multiple different types of cells (see image below).  For example, during human development, embryonic stem cells have the ability to turn into all of the different cell types throughout the body – such as liver cells, lung cells, and skin cells. Embryonic stem cells are called pluripotent stem cells for this reason.  Stem cells have also been found in adults, but they are usually able to turn into only one or several types of cells (called multipotent stem cells).  Scientists have thought that this ability for stem cells to continually grow and turn into different cell types could be harnessed to treat a number of diseases that involve cell death, including spinal cord injury.

Stem cells - wikipedia

By OpenStax College [CC BY 3.0], via Wikimedia Commons


The authors of this paper are interested spinal cord injury (SCI).  When the spinal cord (the bundle of nerves that goes down your back) is injured when something disrupts the vertebrae (like it being hit during a car accident) this can tear or push on the spinal cord.  This causes damage to the nerves and prevents electrical signals between the body and the brain from being transmitted properly, often resulting in paralysis. The authors were interested in studying whether stem cells could help reverse the damage that’s caused by this type of injury.

Results: In this publication, the authors isolate stem cells called multipotent adult progenitor cells (MAPCs) from the bone marrow. To break this down, “multipotent” means that the stem cells can turn into several different cell types. “Progenitor” cells are actually more specified than stem cells – stem cells can divide forever, but progenitor cells can only divide a certain number of times. “Adult” means that these cells are taken from the adult bone marrow.

Because you can’t just inject cells into humans with spinal cord injury (that would have to be a clinical trial WAY down the road once there is evidence that it works in animals), the authors use a “model” of spinal cord injury in rats. Technically – they crush the rat’s spine (eek).  In case you’re interested, here’s the device they use to do this. The rat then has decreased mobility and inability to urinate on it’s own – like what might happen in the case of a paralyzed person with a spinal cord injury.

The researchers injected multipotent adult progenitor cells into the rat after injury to see if they reverse these mobility and urination effects.  Interestingly, when MAPCs are injected into a rat vein the day of injury, nothing happened. However, if the MAPCs are injected 1 day after injury, the rats recover some mobility and the ability to urinate on their own compared to rats without treatment.

One might assume that the MAPCs do this by going to the injured area of the spinal cord and re-growing nerve cells.  However, the authors found that this wasn’t the case.  In fact, the MAPCs moved to the outside edges of the injury and even more to the spleen (see image below).  The spleen is where many of the body’s immune cells are stored. In spinal cord injury, the immune system is a double-edged sword.  The immune system cleans up the damage from the injury itself but also attacks the injury and makes more damage.  There is evidence from this paper that the MAPCs in the spleen decrease the damaging effect on the spinal cord injury from the immune cells in the spleen.

figure 6 SCI

Figure 6 from the paper. The green dots are labeled multipotent adult progenitor cells (MAPCs). They are labeled green so that you can see where they are located in the rat’s body.

Conclusion: This paper presents a promising result that provides hope for this type of therapy in spinal cord injury patients.

What’s been going on?

It’s been a while since I’ve been able to blog.  I have lots of reasons, but one of the big ones is that I’ve been writing and editing for some other sites!

In mid-January, I wrote an article for the ISBER News Blog (ISBER stands for the International Society for Biological and Environmental Repositories – the premiere biobanking society in the world). The article entitled “Have you joined the ISBER social network? Facebook!” is all about how to get started on Facebook and how to get more involved with ISBER through that social media outlet.  If you aren’t on Facebook already, you may find the first half of the article useful.  Once you join, you can follow the “Things I Tell My Mom” Facebook page where I post often about interesting science I find around the web.

From this experience, I was named the new Assistant Editor for the ISBER News Blog. This was announced in a very funny article from the current editor Rick Michels called “Bringing in Backup.” I’m super excited about this new role and have jumped right in to help edit lots of interesting articles from the biobanking world.  I hope many of them will be interesting to the public, and I’ll be sure to share them either through this blog or on my Facebook page.

As part of my job, I endeavor to educate the public on the importance of biobanking in enabling cancer research. To both work towards this goal as well as to talk about the Biobank’s support of World Cancer Day, I wrote an article for the Barrow Neurological Institute blog about what our Biobank does to help cancer research. You can find that article here.

And because that’s not nearly enough, my first article for GotScience.org was published today. GotScience is a fabulous website with the goal of increasing the public awareness about science – a perfect fit for my goals and dreams! I adapted an article that I first published on this blog for GotScience about “What is a biobank?” (I think you can see a theme emerging). Please check it out along with the other articles on GotScience!

Finally, I’ve been helping out an editor friend of mine at PN Online – a magazine to help people who are wheelchair bound. I contributed to an article about this great new ALS research called “Early ALS Treatment.” I adapted my work for this article to a journal club blog post that you can read here.

I promise I’ll be back to writing on this blog soon, but until then, enjoy the articles I shared above!!

How not to die when working in the lab – Biosafety

What’s your biggest concern for your safety when walking into work each morning? That you’ll get a papercut or burn your tongue on your morning coffee? Maybe if you’re in a higher risk profession (like a contractor) you’re worried about something falling on your head.  As a scientist, may researcher enter the lab every day and have to worry about being infected by what they are researching!

blood tubes

Blood tubes collected for our biobank

Human tissue and blood may contain viruses like HIV or hepatitis that’s just needs entry into your body through a cut or in the mucus membranes of your nose. Or maybe you’re working with viruses like the flu to see how they infect cells. And what of those people who are studying the deadliest diseases like Ebola – how do they study these diseases or work to develop vaccines or treatment drugs without getting Ebola themselves? BIOSAFETY!

Wow. Boring. Right? Well, kind of. I spent this last week taking a refresher course on biosafety and the reading was DULL, but sitting down to think about all of the implications it becomes fascinating.

The thing is, anything that would infect you is likely invisible.  If you’re working with human tissue or blood, you can see the blood but not what might be inside of it (bacteria or a virus) that could infect you.  And if you drop a blood tube on the floor, you can see the pool of blood but not the microscopic aerosolized droplets in the air just waiting to be inhaled. So how does a scientist take care of these hazards not just so they don’t get sick but so they don’t infect the public as well?

Good news is that scientists think about this a lot.  First, depending on what you’re working on depends on how careful you have to be. Compare this to the difference between giving a 4-year-old plastic play-doh scissors versus sharp surgical scissors – they have different risks associated with them and you’d treat the kid using them in different ways.  In the same way, working with things that are not known to infect humans and cause disease (like bacterial cells) don’t need to be handled as carefully as those that cause deadly diseases that are spread through the air (like Ebola).  This is what defines the “risk group” on a scale of 1-4, where 1 is the lowest risk and 4 is the highest. The play doh scissors or bacterial cells would be Risk Group 1 and the sharp surgical scissor or Ebola virus would be Risk Group 4.

This then helps researchers figure out what protective measures need to be taken – also called Biosafety Levels (abbreviated as BSL) 1-4.  For example, level 1 can be done in the laboratory out in the open wearing a lab coat and gloves.  Level 2 requires research to be done in a biosafety cabinet so that anything that spills or is aerosolized is contained.  Level 3 is for agents that cause moderate to severe disease and in this case the experiment needs to be completely contained in a glove box or in a special room with controlled air flow. Level 4 is for agents that cause lethal disease that has no treatment or cure (like Ebola). In this case the BSL-4 is what you may see on TV or movies where researchers are fully enclosed in a “space suit” to prevent any contact between the person and the agent.

biosafety

In my lab, we regularly work with human tissue and blood and because the tissue or blood may be infected with certain biological agents, they are considered risk group 2. We always wear a lab coat and gloves and work inside a hood.  When moving these tubes from one place to another, we make sure that there are at least two layers of containment – the blood tube is the first layer and this is inside a plastic bag as the second layer. We also get re-trained every year to make sure we remember how to handle spills (just in case).

Now because we’re all morbid creatures, I’m sure you’re wondering what’s happened when this hasn’t worked. Here is an article about the most common infections acquired in the lab and how they were acquired.  For more newsy stories, here is a story about a researcher who contracted plague in the lab. Eek! Definitely reinforces the importance of being careful in the lab.

Journal Club – A mutation in mice to study ALS

Amyotrophic Lateral Sclerosis (ALS) is a debilitating neurodegenerative disease that affects neurons in the brain, brain stem, and spinal cord. When these neurons die, it affects the connections that they have to muscles throughout the body resulting in muscle weakness that affects speaking, swallowing and breathing leading to paralysis and eventual dead.  The cause of ALS isn’t known in 90-95% of cases.  However, recently scientists have identified a mutation in a gene nondescriptly called “chromosome 9 open reading frame 72,” which is abbreviated to C9ORF72.  This mutation results in six nucleotides, GGGGCC, being repeating up to 1000 times within this gene in ALS patients.  Even though this is the most common mutation found in ALS patients, it’s still unclear exactly how these repeats affect neurons or the progression of the disease. The two hypotheses that are most studied are that the mutated C9ORF72 makes mutated RNA transcripts (RNA is the molecule that usually helps DNA be translated into proteins) or makes unusual proteins called dipeptide repeat proteins (DRPs), each of which can aggregate (clump) together and disrupt the normal activity of the neurons leading to neurodegeneration.

neuron_als_c9ORF72

The pink dots show clumped up RNA from the many many repeats in the C9ORF72 gene. Each panel shows these clumps in different areas of the mouse brain. Taken from the article Peters et al. 2015 Neuron

In two recent publications in the journal Neuron, researchers have used mice as a model system to look at how these large GGGGCC repeats in C9ORF72 affect the mouse nervous system.  Why use a mouse? First, scientists know how to experimentally change the mouse genome in order to add hundreds or thousands of repeats to a single gene, like C9ORF72.  Second, mouse and human genes are about 85% identical, so if scientists can understand how a gene like the mutated C9ORF72 affects neurons in mice, it may also help scientists understand how it works in humans. Third, by creating a mouse “model” of ALS, scientists can use this model to better understand ALS and to test potential future therapies (it’s worth noting that only one drug currently exists for ALS and it typically extends life only by a few months).

In each article, the mouse model was slightly different – one had a mutated C9ORF72 with 500 GGGGCC repeats and the other varied between 100-1000 repeats. However, both sets of researchers found the same results.  The mice had aggregated RNA transcripts and DRPs in their neurons just like what are found in human patients with ALS, but none of the mice had behavioral changes or neurodegeneration that are seen in human ALS patients.  So why didn’t the clumped up RNA and proteins cause neurodegeneration in mice like they do in humans?  There are lots of potential reasons – including the fact that even though mice and humans are similar, there are still lots of differences and mice may respond to these aggregated RNA and proteins differently than humans.  However, the authors of these papers suggest that other environmental and/or genetic factors along with the aggregates caused by the C9ORF72 mutation must be involved in developing the neurodegeneration.  It may also mean that getting rid of these aggregates before neurodegeneration occurs may prevent development of ALS.  Now that these mice are available to study, they should help in identifying the other factors involved in ALS development along with developing possible treatments for this debilitating disease.

Want to read the articles?  Unfortunately, they are behind a paywall, but you can see the abstracts here:

O’Rourke et al. (2015) C9orf72 BAC Transgenic Mice Display Typical Pathologic Features of ALS/FTD. Neuron. Volume 88, Issue 5, p892–901, 2 December 2015 Article

Peters et al. (2015) Human C9OFR72 Hexanucleotide Expansion Reproduces RNA Foci and Dipeptide Repeat Proteins but Not Neurodegeneration in BAC Transgenic Mice. Neuron. Volume 88, Issue 5, p902–909, 2 December 2015 Article

Read the Article from PN News that I contributed to here

What is Informed Consent for Research Subjects?

informed_consentAs I mentioned in my last post, the Institutional Review Board (or IRB) has a responsibility to review and monitor all human subject research to help ensure that the subjects are treated ethically.  But how does the patient know what they are getting into if they are interested in participating in  a research study?

One of the key parts of the Belmont Report (that report that sets the rule for human subject research) says that subjects must be treated with respect.  A huge part of this is to inform research subjects about the research, the risks, and the possible benefits. Because of the importance of informing the subject, most research studies involve providing the potential participant with an Informed Consent Form (ICF) that the they must read, understand, and sign before joining a clinical trial.  This informed consent form is reviewed by the IRB and usually much discussion goes into whether it is clear (ideally 6th-8th grade reading level), accurate, and informative.

So how do investigators or the IRB know what to inform the potential participant about in this informed consent? Well the government tells us – of course! The intuitively named Title 45 CFR 46 subparts A, B, C and D outline the rules and regulations for human subjects research.  Because this is the government’s stupid way of naming things, the easier name used for these rules is the Common Rule. This rule outlines what’s needed in an informed consent form (paraphrased directly from the Common Rule):

  1. A statement that the study involves research, an explanation of the purposes of the research and the expected length of the subject’s participation, a description of the procedures to be followed, and identification of any procedures which are experimental.
  2. A description of any reasonably foreseeable risks or discomforts.
  3. A description of any benefits to the subject or to others which may reasonably be expected from the research.
  4. A disclosure of appropriate alternative procedures or courses of treatment, if any, that might benefit the subject.
  5. A statement describing the extent, if any, to which confidentiality of records identifying the subject will be maintained and that notes the possibility that the Food and Drug Administration may inspect the records.
  6. For research involving more than minimal risk, an explanation as to whether any compensation and an explanation as to whether any medical treatments are available if injury occurs and, if so, what they consist of, or where further information may be obtained.
  7. An explanation of whom to contact for answers to pertinent questions about the research and research subjects’ rights, and whom to contact in the event of a research-related injury to the subject.
  8. A statement that participation is voluntary, that refusal to participate will involve no penalty or loss of benefits to which the subject is otherwise entitled, and that the subject may discontinue participation at any time without penalty or loss of benefits to which the subject is otherwise entitled.

The intent of the informed consent is for the subject to actually be informed.  This means that the investigator needs to clearly and honestly explain the research to the subject.  For example, when we provide informed consent for patients to provide tissue samples for the biobank, we tell them that their tissue may be used for any type of future research, it may be used to create commercial products but won’t financially benefit them, and that we won’t be able to provide any research results back to them.  In this context, participating in the biobank may sound like a raw deal.  However, we also explain how the tissue they donate will be used by researchers to better understand diseases and develop new treatment. Therefore, even though they may not directly receive benefit, they may benefit others with their disease in the future.

The potential participant is also given the chance to ask all the questions that they have about the study.  They can ask to bring the consent home to read it.  They can ask their doctor about it. Then after they are fully informed, they decide if they want to sign it or not. If not, that is okay.  It’s up to the patient if they want to participate and the investigator cannot do anything to coerce the patient to sign or participate.  At the same time, if the patient does decide to participate, they may leave the research study at any time with no penalties. Their participation is entirely voluntary.

One other quick note about making sure the patients are fully informed…the Common Rule has added additional safeguards in the case “when some or all of the subjects are likely to be vulnerable to coercion or undue influence.” These vulnerable subjects include children, prisoners, pregnant women, handicapped, or mentally disabled persons, or economically or educationally disadvantaged persons. For example, you don’t want to force imprisoned people to participate in research just because they are in jail or poor people to participate just because you are paying them a lot of money to participate.

The US Office of Human Research Protection is working on changes to the Common Rule right now (it’s called a Notice of Proposed Rulemaking – these silly government names).  The original re-write was sent out to the public for review in 2011.  The updated version was made available in September of this year (look at that for the glacial swiftness of the government!) for even more review.  It has many proposed changes that will affect human subject research and research using human tissue samples. To learn more about what this means for clinical and other research, check out the recent story from NPR.

Overall, the purpose of informed consent is to make sure anyone volunteering to participate in a research study knows what they are getting into. If you are interested in a clinical research study, can you find more at clinicaltrials.gov or talk to you physician.

Institutional Review Board (IRB) – Keeping Research Subjects Safe

Xmas

I was working over the holiday weekend, but at least I was working in my decorated living room with a fire going (the high in Arizona was only 66 today!!)

Hope you had a fabulous Thanksgiving weekend! Four day weekends are great, and even I took some time off to enjoy the holiday with my husband and the puppies.  And then I got back to work because I have an Institutional Review Board (shortened usually to the acronym IRB) meeting in two weeks and the committee has eight new protocols to review. This likely means very little to you, but the IRB is what ensures that the rights and welfare of humans participating as subjects in a research study are adequately protected. And here’s why that’s important…

In a previous post, I explained clinical research.  Clinical research studies new drugs or devices to determine if they are safe or effective. As you can imagine, at a world-class hospital, we have hundreds of clinical trials. You can check out information about these clinical trials by following the links for the Barrow Neurological Institute, St Joseph’s Hospital and Medical Center and the University of Arizona Cancer Center at Dignity Health.  Physicians and surgeons are studying new treatments for many different cancers, devices like the NovoTFF for glioblastoma, and comparing new drugs or combinations of drugs to current treatments to see if the new regime works better.

Now if a  company wants to test a new drug, they can’t just pick up the phone and ask their physician buddy if they could just use a few of their patients to test some stuff out. But why not?  Honestly, it’s because at one point researchers did some pretty crappy things in the name of science.  Thins like Nazis studying prisoners against their will and in the US and scientists who studied the untreated progression of syphilis in black patients in Tuskegee, Alabama from the 1930s-1970s. In these and other cases, the welfare of the patient (who is called a subject once they are part of a research study) wasn’t considered AT ALL and what the subjects had to endure was truly awful.

To avoid this from happening in the future, in 1974 the government passed the National Research Act, which resulted in the Belmont Report. From this, three ethical principles were developed in the treatment of research subjects:

  • Respect for persons.  This respect includes allowing them to make their own informed decisions about participating in the research.  This also means that the researcher conducting the study needs to be honest and not try to deceive or coerce patients into participating in the study. For example, the researcher can’t tell the patient that the research will be painless and cure their disease if they know it won’t.
  • Beneficence: Basically this ensures that the researchers do no harm to the research participants – for example, like the harm done during WWII or in untreated syphilis patients.
  • Justice:This is to avoid taking advantage of the patient or a vulnerable patient population.  For example, there are special rules to prevent taking advantage of prisoners or children. This principle also tries to make sure that all research participants receive benefit equally.

protocol for IRB reviewThese ethical principles have been developed into processes that are regulated by the Food and Drug Administration (FDA) and Department of Health and Human Services (specifically Office for Human Research Protections). How does the government make sure these regulations are followed?  Any institution that is performing human subject research has to obtain a Federalwide Assurance, which essentially registers the hospital or university with the government and assures the government that the hospital will follow the ethical rules and guidelines to conduct this research.

For each project or clinical trial involving human subjects, the investigator needs to put together a proposal – what we call a protocol.  This protocol includes information about exactly what is going to be done to the subjects, what the risks are, what the alternatives are for treatment, and how the subject’s safety and confidentiality will be safeguarded. This protocol is the sent to the IRB along with LOTS of other documents about how the patient will be informed about the research (in Informed Consent Form), whether or not the investigators have been trained to perform the research, and information about the drug or device being used.

The IRB is responsible at individual institutions for making sure that patients who become subjects in human subjects research are treated with respect, beneficence and justice while also decreasing the potential risks and letting the patient know what these risks are. The IRB reviews each new protocol (which is exactly what I am doing this weekend!) and at the IRB meeting (which is once a month from 7-9AM), the investigators present their protocol.  The IRB members then ask questions to the investigator and discuss the research after the investigators leave the room.  What do we discuss? It’s confidential for individual studies, but we may talk about how the study is being performed and identify possible problems with the study. We may also talk about the informed consent (what the patient reads to learn about the study – more on that in the next post) and if it accurately explains the research and the risks. We then can vote to approve the study, to send the protocol back to the investigator to answer questions or modify the protocol or to reject the study.

After the study has been approved, the IRB is also responsible for monitoring active research projects.  For example, we receive annual reports that let us know how many people have decided to participate in the study. We also monitor “adverse events.” Adverse events (or AEs) are any event that isn’t anticipated.  This can be anything from nausea to a broken leg to a rash to a missed appointment to death (death is considered a serious adverse event). Whenever an AE happens, the IRB is informed so that if it seems like there are too many of one type of AE, we can take measures to avoid them or tell the subject about an additional risk or shut the research project down.

My participation on the IRB is a responsibility I take seriously because I want any patient who comes to the facilities I work at to understand the research that may be made available to them.  And this understanding includes knowing what the research is all about and what risks the research entails. This is why I’m spending my holiday weekend reviewing research protocols for the IRB.

The difference between basic, translational and clinical research

When I started as a researcher, I had no idea that there were different types of research.  I don’t mean that some scientists study cancer and some scientists study Alzheimer’s disease.  I mean entirely different kinds of research that have fundamentally different methods, sources of funding, and purposes. Today’s post is going to outline three main types of research in the biological sciences: basic, translational and clinical research.

Basic Research:

science_image

By en:User:AllyUnion, User:Stannered (en:Image:Science-symbol2.png) [CC BY 3.0 or GFDL], via Wikimedia Commons

 Right off the bat, I need to be super clear that basic research is NOT research that’s easier to do or simpler than any other type of research.  It is just as complex and just as hypothesis-oriented as other types of research.  However, the goal of basic research is to  understand at a very basic level some aspect of biology.  Also called fundamental research, basic research doesn’t require that the outcome of the research can cure a disease or fix a problem.  That being said, basic research often does create the foundation that is required for other researchers to apply to solving a problem. I like how basic research is described on WIkipedia as “Basic research generates new ideas, principles, and theories, which may not be immediately utilized but nonetheless form the basis of progress and development in different fields”  This research can be in biology, physics, math, environmental sciences or any other scientific field. So what are some examples of basic research in biology?

  • Understanding the proteins and pathways that result in cells dying by apoptosis
  • Developing technology to better determine the 3D structure of proteins.
  • Creating mathematical models representing population growth in cities over time
  • Studying how leaf litter affects the ecosystem (an actual active funded grant at TGen here in Arizona)

This research is often funded by the government, specifically the National Institutes of Health, which funds 50,000 grants to more than 300,000 researchers at more than 2,500 institutions around the world, and the National Science Foundation, which funds 24% of all federally-funded basic science research in the United States.

Translational Research:

mouse_for_research

By Maggie Bartlett, NHGRI. [Public domain], via Wikimedia Commons

Translational research is how basic research and biological knowledge is translated into the clinic.  Often called “bench-to-bedside” or research (referring to the research bench and the patient’s bedside) or “applied” research (of applying basic research to solve a real-world problem), this research is needed to show that a drug or device works in some living system before it is used on humans. This is the research that happens after the results from basic research are obtained and before clinical research.

For example, if a drug is found in the lab that targets a protein that is thought to cause a disease like cancer, the drug will first be tested on animal models.  The animal model may be a mouse that has been genetically altered so that it develops that specific kind of cancer or a mouse that has human cancer cells injected into it (like the patient derived xenografts I described in a previous post). The drug will then be used on the animal to see if it is safe or if low doses are so toxic that the animal dies. Whether or not the drug hits the targeted protein or cell type can also be tested in mice.  For example, if the drug is supposed to kill brain tumor cells, researchers would want to make sure the drug was able to pass the blood brain barrier of the mouse.  Finally, if the drug is supposed to kill tumor cells, researchers would want to check that the tumor shrinks, the cells die, and/or that the survival of the mouse is extended from using this treatment. Often, drugs are “weeded out” at the translational research stage saving millions of dollars and years worth of time and effort in clinical trials.

Translational research isn’t just for drug development.  It is also useful for devices. For example, to develop a device that can diagnose diseases in third world countries, where access to electricity and high tech labs is more difficult.

Clinical Research:

blood_tube_for_research

By Tannim101 [CC BY 3.0, GFDL or CC BY 3.0], via Wikimedia Commons

Clinical research is what is performed in a healthcare environment to test the safety and effectiveness of drugs, diagnostic tests, and devices that could be used in the detection, treatment, prevention or tracking of a disease.  The cornerstone of clinical research is the clinical trial.  There are 4 basic phases to a clinical trial.  Each phase is performed sequentially to systematically study the drug or device.

  • Phase I: This is the first time the drug or device has been in humans and it is used on a small number of patients in low doses to see whether or not it is safe and what the side-effects may be. At this point, the clinicians are not trying to determine if the treatment works or not.
  • Phase II: In this phase, more patients are treated with the device or drug to test safety (because more side effects may be identified in a larger, more diverse population) and whether the drug or device is effective (in other words, does it work?).
  • Phase III: This is the phase that focuses on whether the drug or device is effective compared to what is typically already used to treat patients.  It’s used on a large group of people and “end points” like increase in survival or decrease in tumor size are used to evaluate its effectiveness.
  • Phase IV: These trials are done after the drug has gone to market to see if it works in various populations .

There are several different types of clinical trials depending on who is funding them. Some clinical trials can be initiated by a doctor or group of doctors.  These are call “physician-initiated” or “investigator -initiated”  studies and are often used to determine which type of treatment works better in patient care.  For example, there may be two treatments that are commonly used to treat a disease. Investigators may initiate a study to figure out what treatment works better in what patient population.

The kind of clinical research you may be more familiar with are drug companies who are working to develop a drug or device.  These companies will “sponsor” (aka “pay for”) a clinical trial.  They work with clinicians at one or more medical institutes to use their drug or device in a particular way (depending on the phase of the trial) and the clinicians report back the results, including whether there were any side effects to the treatment. At the end of the clinical trial, if the treatment or device was a success, the drug company can apply to the Food and Drug Administration (FDA) for approval to use the drug in the general population.  Bringing a drug to market is a timely and extremely expensive process estimated at over 10 years and $1.3 Billion dollars per drug. Much of this time and cost is due to high cost of conducting the clinical trials.

If you are interested in what clinical trials are currently available in the United States, all clinical trials are registered on ClinicalTrials.gov.  Anyone can search this database to see if trials are available for them to participate in.

Overall, each type of research needs to understand the other, and researchers need to work together to successfully understand our world and to come up with solutions to prevent, diagnose and cure disease.

What’s it like getting a science PhD?

By AdmOxalate (Own work) [CC BY 3.0 (http://creativecommons.org/licenses/by/3.0)], via Wikimedia Commons

Cold Spring Harbor Laboratory by AdmOxalate (Own work) CC BY 3.0, via Wikimedia Commons. This is where I went to grad school.

In my last post, I talked about how to get into graduate school.  This post will be about how PhD programs in the sciences are structured and how they work, because I’ve realized from lots of conversations with my non-scientist friends and family – no one really knows much about this!

There are fundamental differences between getting a PhD in the sciences and getting one in anything else. The first main difference is that you don’t have to pay for a PhD in the sciences, and in fact, they pay you.  Don’t get excited – they don’t pay much. The current NIH stipend rate is $22,920 per year (only about $2900 more in 2015 than what I received in 2001).  Tuition and this stipend are paid for in different ways depending on the school.  Some schools have endowments that support graduation positions. For example, I was supported by an institutional endowment made by the Beckman Foundation for my first two years of graduate school. Some schools rely on the students working as Teaching Assistants (TAs) helping to teach undergraduate courses to support some or all of their tuition or stipend.  In many cases, the research laboratory that the student works in pays for the tuition and stipend using their grants. Graduate students themselves also can apply for funding, which along with helping fund their position, is a prestigious resume entry.  I applied for and was awarded a National Science Foundation (NSF) Graduate Research Fellowship that supported my last few years of graduate school.

The second main difference between a science and non-science PhD is that there is NO WAY that you can work and get your PhD at the same time. Don’t get me wrong, you work. You work your butt off every day all day, but not while making money at another job. With the nature of scientific research, there isn’t time to have another job, and in most cases, it isn’t allowed by the institution anyway.

What is a graduate student so busy doing?  The graduate program at the WSBS, where I went to school, was designed to be very different from the traditional American graduate school model.  I’ll start by describing, generally (since all grad schools are different) traditional programs and then describe my program. Most PhD programs are expected to last between 4-7 years. The first two years are filled with a few key activities:

  • First two years: Traditional classes at the graduate level that cover scientific topics more deeply than an undergraduate program
  • First year: Rotations. These are short (usually 3 month) stints in a laboratory to figure out if you like what the research that lab is doing and whether or not you’d want to do your PhD thesis research there. This is also the chance for the head of that lab (also called the Principal Investigator or PI) to figure out if they want to have you in the lab for the next 4-6 years.
  • End of second year: Qualifying Exam. This exam, also called the comprehensive exam at some schools, is an enormous exam that is like the trigger for the institution to determine if you go forward in the PhD program or not. Usually held at the end of the second year, if you pass, you move on to nearly exclusively doing research in the lab to complete your thesis.  If not… well, I don’t think I know anyone who didn’t pass after at least a few tries.
  • Third year until you graduate: After the first few years, most of the time is spent in the lab. There may be required Teaching Assistant responsibilities or other required seminar classes (like Journal Club), but this varies by school. Then there are the thesis committee meetings.  Pretty early on in each student’s research project, a committee of 3-5 faculty at the university are invited to participate on your thesis committee.  Their job is to provide a set of eyes (other than the PI of your lab) to make sure you’re moving in the right direction. They approve the thesis proposal and meet with you regularly (in a traditional program, this might be yearly) to keep you on track. They are also the committee that reads and evaluates your thesis dissertation and holds your defense (more on that shortly).

As I mentioned, this traditional system is a bit different from what I went through at CSHL.  The philosophy of WSBS is to shorten the time frame from matriculation to graduation to 4 years while also maintaining academic excellence.

  • First semester (4 months): This is the only time I took core courses – what my mom called “Science Boot Camp”.  These classes were unique because instead of learning facts out of textbooks we learned how to critically think about, write about, and present science. The classes focused on reading journal articles, scientific exposition and ethics, and particular scientific topics in depth like neuroscience and cancer.
  • Second semester (4 months): After the first semester, we had three one month rotations that allowed us to explore our scientific interests to help decide on a thesis laboratory or just allow us to try something new. I did rotations in a lab that used computers to understand lots of scientific data, a lab that used microscopy to figure out how a cell worked, and a lab that studied apoptosis (where I ended up doing my thesis research). Also during this time, we did our one required teaching experience at the DNA Learning Center. Here we taught middle and high school students about biology and DNA.  The idea was that if we could explain science to kids, we could explain it to anyone.
  • End of year one:  After the first year, we took the Qualifying Exam.  For my QE, I had two topics assigned to me (Cancer and Cell-Cell Communication) and I had to learn everything about these two topics in one month. A panel then grilled me for nearly 2 hours on these topics, and fortunately, I passed.
  • Years 2-4: The classes are only held in the first semester and the rotations only held in the second semester so that we could focus on what we were doing at all times. No excuses. So after the qualifying exam we were expected to focus on all research all the time. The one exception being the Topics in Biology courses held each year.  The Topics in Biology courses were held for an entire week (7am-11pm) and gave you the chance to interact with experts in various fields both to extend your scientific knowledge and to critically think about new problems.
Photo Nov 15, 9 24 23 PM

My thesis. It’s about 1.5 inches thick. Or as my hubby said “That’s your thesis? Impressive, baby”

Doing research was intense lab work punctuated by intense meetings.  FYI – intense lab works mean 8am-7pm (or later) Monday through Friday and usually the weekend too (and by weekend, I do mean both Saturday and Sunday).  And let’s not forget the 4am time points when you have to go into the lab just to check on your experiments every 4-6 hours for 24 hours straight. But back to the intense meetings…The first intense meeting was the thesis proposal defense, which was held in the second year. This was where you told a committee of 4-5 researchers what you were going to research for the rest of grad school, they quizzed you for 1-2 hours and then gave you the go ahead (or not) to do that work. The next set of intense meeting were the thesis committee meetings every 6 months to keep each student was on track. Again, 1-2 hours of presenting and critical evaluation of your work by committee.  At some point, the committee gives you the “green light” to start writing your thesis, you take all of the work from the past 3-4 years and put it in a massive document called a dissertation. The thesis committee reads it, you present the work in front of them and all of your family and friends, and then again, you spend 2 hours in a room with your committee answering every question they can think of – aka “defending” your thesis.

Cathy_graduation

My PhD graduation day with two of my classmates. I’m in the center

As I write this, I realize that my thesis defense was 9 years ago next week. How time flies. After the defense, you have your PhD and officially graduate whenever the ceremony is held – in my case in May of 2007. I graduated 5 years after I started – just slightly longer than the expected 4 years for the Watson School. Was it easy? Nope, not even a little bit (ask my mom). Would I do it again? In a heartbeat.

This post is dedicated to my classmates and my friends in graduate school – you know who you are.  Without you, I wouldn’t have made it. And to my mom, who convinced me at least twice, not to quit.

How do you get into a PhD program in science?

When I was very young, my uncle died from lung cancer. I wasn’t allowed to see him before he died (his wishes). There was a part of me that thought it was my fault that he dies because he didn’t listen to my pleas that he should stop smoking. That’s when I decided that I should cure cancer. At the time, I had no idea how to do that, but by the time I was in high school, I realized it would involve getting a PhD.  Other than a great uncle (on the other side of the family) that I barely knew, no one else in my family had a PhD, so I was the trailblazer in figuring out how it all works. In this post and my post on Thursday, I’ll write about how to get into graduate school and then what the program is like once you get there. More accurately, I’ll write about how I got  into grad school and what grad school was like for me since I know that everyone’s experience is different.

So how do you get into a PhD program? Let’s skip the fact that you’ll need an interest in science, good grades in college and likely do undergraduate research. Also, one difference between science PhDs and other PhDs is that you aren’t expected to get your Master’s degree first. You can apply straight from undergrad, and the idea is that you get your Master’s degree on your way towards the PhD.  If you leave the PhD program at a certain point (usually after you take a qualifying exam), you’ll leave with a Master’s degree. In fact, other than maybe having more research or other experience, there isn’t much of an advantage to getting a Master’s before your PhD degree versus not.

The first step needed before applying for grad school is to take the general GREs exam along with a subject-based GRE exam.  These are standardized tests like the SAT or ACT but for graduate school.  The subject-based exam feels like the biggest and longest test you’ve ever taken for a particular subject.  I took the Biology subject test (I could have taken the Biochemistry subject test, but I heard it was a lot harder, so I just studied by butt off for the Biology one instead). For most grad schools, these exam scores are critical.  Just like if you get a good score on the SAT you can get into high ranking colleges, high GREs scores help you get into grad programs at the Harvards and Yales of the world.

Just like undergrad, you have to send in your applications with the ever-important personal statement.  This statement has to talk about why you want to go to grad school, but also why that school and the researchers at that institution are of interest to you.  When I advise current undergrads about choosing a PhD program, the most critical part is to apply to schools that have research labs that do the research that you are interested in.  Once you get into the graduate program, as I’ll talk about in detail in my post on Thursday, you spend years of your life in this research lab so if there isn’t a research lab you like, don’t even bother applying to that school.

phdAfter applying, the graduate schools interested in you invite you for an interview.  This isn’t a one hour, chat with a guidance counselor type of interview.  This is a weekend of interviews with distinguished faculty grilling you about your undergraduate research (assuming you had some) and asking critical questions to determine how clever you are and whether you’d be a good fit for the school. I went on three interview weekends at Harvard Medical School, Johns Hopkins and the Watson School of Biological Sciences (WSBS) at Cold Spring Harbor Laboratory (CSHL)(where I eventually attended). The CSHL interview by far was the most intense with over a dozen interviews in one day including one with Nobel Laureate Jim Watson who was the chancellor of the lab at the time. My favorite “words of wisdom” from Dr. Watson at that interview were to always select research projects with a 30% chance success. Less than that, you’d be wasting your time and more than that, the project is too obvious and wouldn’t make a big impact on the field. This may sound a bit masochistic – setting yourself up for likely failure – but this is the life of a scientist!

Usually there are dozens of candidates invited for the interview weekends so the schools also plan bonding time among the candidates and the current grad students. This could be a dinner out, a party thrown by one of the current grad students, or a trip to NYC to see a Broadway show.  To this day I’m still friends with people that I interviewed with even though we both chose other grad schools.

After the interview, the waiting game begins. I remember the evening that I received the call saying that I was accepted into the CSHL program (the one I really wanted to attend). I was in my dorm room at Boston University and I get a phone call – keep in mind this is before cell phones so they called the landline in my room. I thought it was a prank call from my friend Greg and I told him (more than once) that this wasn’t a funny joke. No joke – the Dean of the school was called to let me know about my acceptance. I received the official acceptance letter in an email minutes later.

wsbs_2001

My WSBS Class entering in 2001. I’m the one sitting on the double helix

I actually got into all of the graduate programs that I applied to, which caused a bit of a problem because my dream had always been to attend Harvard. My decision, then, to attend the Watson School was confusing to my parents, who had heard of Harvard but never Cold Spring Harbor Laboratory.  Why was this my choice? The research at CSHL was incredible  – every scientist was engaged with their work like I had never experienced in my undergraduate career. It was inspirational to think about being a part of that. CHSL had also just started their graduate program – I would be in the third entering class – and their program focused on learning how to learn and how to think in a way that was different than any other graduate program out there (more on that in the next post). I wanted to be a pioneer in this program. And finally, the culture suited me. I went to a large undergraduate institution with classes of 300 people and anonymity amongst thousands of classmates. In graduate school, I wanted to be part of a small class where I could really be challenged and learn from a close-knit group of peers. My WSBS class had six students, including myself, that constantly challenged me to think faster and smarter and become the best scientist that I could be.