Mouse study findings prove contentious among scientists
[The Pitt News, Mar. 1, 2013, pg. 1 print edition]
A celebrated virtue of science is its ability to produce objective results. Interpretation of the results, however, often breeds controversy — especially when the sensitive public opinion of animal research is involved.
Such controversy has rocked scientific circles since mid-February when The New York Times published a story about a scientific study — which involved Pitt faculty — that questions the usefulness of certain mouse models for research in the understanding and treatment of three human inflammatory diseases. Scientists across biomedical fields disagree over how far to extend the study’s results, and they worry about whether the millions of Times readers could find within its pages a fair interpretation of the results.
“There’s a lot of discussion after this [study],” said Bill Yates, a Pitt neuroscience professor who has chaired the Animal Care and Experimentation Committee of the American Physiological Society.
A tale of two articles
The study in question represents the latest installment in a long series of academic papers since 2001 coming out of the Inflammation and the Host Response to Injury research program, a collaboration between 18 institutions funded by the National Institutes of Health — the federal funding agency for biomedical research. Motivated by the failure of nearly 150 clinical trials of drugs built to control excessive immune responses, the study set out to systematically evaluate mouse models of disease from which the drugs emerged.
Published online in the Proceedings of the National Academy of Sciences on Feb. 11, the study looked into the activation of inflammation-related genes of white blood cells in response to three conditions: burns, trauma and a model of sepsis (a whole-body inflammatory condition typically caused by severe bacterial infection). The main conclusion: No relationship exists between mice and humans in terms of which genes activate in response to the damage.
Titling their paper “Genomic responses in mouse models poorly mimic human inflammatory diseases,” the 39 co-authors concluded, “The prevailing assumption — that molecular results from current mouse models developed to mimic human diseases translate directly to human conditions — is challenged by our study.”
Online that day and in print the next, The New York Times published an article by science writer Gina Kolata that tried to translate the findings in the PNAS paper for a general audience. In the opening paragraph, Kolata wrote: “But now, researchers report evidence that the mouse model has been totally misleading for at least three major killers — sepsis, burns and trauma. As a result, years and billions of dollars have been wasted following false leads, they say.”
Shortly after the Times article was published, animal-rights organizations took the opportunity to speak out against animal research in general.
The National Anti-Vivisection Society ran with the Times story in its emailer that week, and the group’s science director, Pam Osenkowski, wrote on the NAVS website, “This ‘game-changing’ study reinforces the message that NAVS has long voiced: Scientific evidence shows that the use of animals in science can mislead researchers while it wastes valuable time and resources.”
On Feb. 20, the Humane Society of the United States President Wayne Pacelle, who represents the country’s largest animal-rights organization, expressed a similar sentiment in The Huffington Post, which, according toQuantcast.com, reaches 60 million people per month. Citing the Times article, he titled the post “A Scientific Indictment of Animal Research and Testing.”
Concerns over public impact
Given the size of the newspaper’s readership and influence, the Times’ presentation of the results of the PNAS paper worries scientists.
“I think [Kolata] went way beyond what this paper actually said,” said Cliff Deutschman, critical-care physician and sepsis researcher at the University of Pennsylvania, whom Kolata quoted in the Times article.
“First of all,” Deutschman said, referring to the Times article’s claim that mouse models in the respective areas of inflammation research have been “totally misleading” and wasteful, “you can always learn something from a bad example.”
“To suggest that valuable human therapies have not come out through mouse models is absolutely false,” Timothy Billiar, chair of the UPMC Department of Surgery and a co-author of the PNAS paper, said of the Times article.
Billiar refers to drugs that block TNF, a regulatory molecule produced by white blood cells, since these anti-TNF drugs were developed in mouse models of sepsis but proved effective for human sufferers of rheumatoid arthritis.
“Now they’ve gone on to be blockbuster drugs and have helped thousands of patients.”
The chance that the public may gather from the Times article that scientists suddenly realized they “can’t equate” mice to humans also concerns Billiar. “We’ve always known the limitations of mouse studies,” he said. “You cannot make sweeping conclusions based on mouse studies.”
Wenzhong Xiao, a primary author of the PNAS paper and a genome scientist associated with Massachusetts General Hospital and Stanford University, said of the Times article, “It’s not really a faithful portrait of what we wrote.”
Kolata said that her article accurately represented her sources. The two other PNAS-paper authors the Times article quoted were not available to comment on the record to The Pitt News.
“I am not a scientist, so I cannot reply to scientific quibbles,” Kolata said in an email. “I spoke to experts and accurately reported what they said.”
The scientific debate
Outside of the media attention, interpreting the results themselves divides scientists.
For Olivera Finn, chair of Pitt’s Department of Immunology, the PNAS paper offers little more than confirmation that mouse models carry limitations. “The paper provided hard data for what all of us already assumed to be the case,” she said in an email. “It is easier to argue with data than with assumptions.”
Similarly, Billiar of UPMC thinks the well-publicized PNAS paper pales in significance next to other papers that have emerged from the Inflammation and the Host Response to Injury collaboration, for which he led a clinical site at Pitt. For the group’s magnum opus, he points to a different academic paper published in 2011, which found that injury produces a “genomic storm” within white blood cells.
“A small subcomponent of [the NIH funding] was meant to address the comparison with mouse models,” Billiar said.
But it goes beyond institutional context for Billiar, who did not edit the PNAS paper. He thinks the article may have overstepped itself in interpreting the data.
For one, “I don’t think the title serves the paper well,” he said, similarly criticizing the paper’s discussion section, where results are interpreted. He doesn’t think these parts adequately reflected the paper’s own limits.
One limit is the difference in subjects — the study only used one mouse strain, whereas human samples were pulled from heterogeneous populations (not controlled for age, race, pre-existing conditions, etc.).
“We’ve known that mouse responses vary considerably by strain,” Billiar said. “Some [strains] would exhibit responses closer to humans, some would exhibit responses different from humans.”
Nonetheless, Xiao said, the study chose the specific strain because it was the one “used in this particular field” during its long history of developing drugs that failed.
Deutschman, who would have preferred more “rigorous” gene activation assays than those used in the study, emphasized that the results apply narrowly.
The study’s conclusion is “legitimate up to a point,” he said. “It’s legitimate in talking about global white-cell behavior. It’s not legitimate in talking about lung, liver, heart, brain and anything else.”
Deutschman thinks it’s an important finding that “one organ system — the white cells — doesn’t behave the same way [in mice as in humans],” but that “doesn’t mean that in one subset on the white cells, that you won’t see homology there,” he said.
Deutschman also noted the difference between endotoxemia, in which the subject’s immune system responds to a single part of a pathogen experimentally injected into the bloodstream, and human sepsis, which occurs in patients who often have real infections and other medical conditions. The study focused on the endotoxemia model of sepsis.
“Endotoxemia is not correct,” Deutschman said. “The best model [of sepsis] is cecal ligation and puncture.” Deutschman refers to a method of inducing infection by puncturing the mouse’s colon.
But again, Xiao — a primary author on the PNAS paper — asserted the relevance of all models considered in the study, including endotoxemia. “Those were the popular models that were used in the field of inflammation,” he said, referring to the models that produced the 150 failed clinical trials of sepsis drugs.
“What we’re trying to say here is that the current animal models need to be carefully re-examined,” Xiao said. “We think a lot of studies so far maybe conveniently sidestepped this issue.”
Going forward
“It won’t change my research practices at all,” Billiar, who works with mouse models in inflammation research, said of the PNAS paper.
But that’s not because Billiar clings to an equality between people and laboratory animals. “Mice aren’t really models — they’re assays for very specific mechanistic questions. They really represent the mammalian response, but not a human disease,” he said.
Others see a need for a more aggressive transformation of the field of inflammation research, without giving up animal models.
“I think there’s a lot of value in questioning the relevance of these animal models,” said John Kellum, professor of critical-care medicine at Pitt.
Kellum thinks the PNAS paper provides cause to develop animal models that more closely match the variability of human patients.
“We’ve never looked at groups of human beings that all have the same infection, all with the same antibiotics, all with the same timing, all with the same peritonitis [abdominal infection],” he said, “yet we somehow expect the animal model is going to be like what happens in humans. It’s obvious that that’s a big leap of faith.”
Xiao puts it bluntly. “The bottom line is that we need to choose something else,” he said. “If we don’t talk about this ever, then things will still be in the old way.”
How the PNAS paper might affect funding for research remains an open question. Francis Collins, director of the NIH, commented in his blog on Feb. 19, “This study provides more reason to develop better and more sophisticated models of human disease.” In the blog post, Collins emphasized the NIH’s recent investment in “tissue chips” — small 3-D models of human organs that feature living cells — which he hopes could fulfill part of this desire.
But an NIH official would not directly specify whether research projects using mouse models in the areas of trauma, burns and sepsis would now find it harder to acquire funding.
Daniel Rotrosen, director of the Division of Allergy, Immunology and Transplantation at the National Institute of Allergy and Infectious Diseases, said that the PNAS paper “provides a framework to characterize and potentially validate existing mouse models and to do the same for new and improved models as they are developed,” and he hopes that it will catalyze more focus in human studies.
Given the concerns over the Times report and the activation of animal-rights groups, scientists are considering how best to communicate their findings to general audiences in the future.
“I do believe very strongly that one thing we don’t do in the scientific community is outreach. We don’t talk to people about why we do what we do, why it’s important, etc., and that’s a big, big problem,” said Yates, professor of neuroscience at Pitt.
“The bottom line about this [PNAS] article is it shouldn’t be over-interpreted,” Yates said. “Animal research is still important, but it does point out that you always have to consider the limitations in models and methods.”
A celebrated virtue of science is its ability to produce objective results. Interpretation of the results, however, often breeds controversy — especially when the sensitive public opinion of animal research is involved.
Such controversy has rocked scientific circles since mid-February when The New York Times published a story about a scientific study — which involved Pitt faculty — that questions the usefulness of certain mouse models for research in the understanding and treatment of three human inflammatory diseases. Scientists across biomedical fields disagree over how far to extend the study’s results, and they worry about whether the millions of Times readers could find within its pages a fair interpretation of the results.
“There’s a lot of discussion after this [study],” said Bill Yates, a Pitt neuroscience professor who has chaired the Animal Care and Experimentation Committee of the American Physiological Society.
A tale of two articles
The study in question represents the latest installment in a long series of academic papers since 2001 coming out of the Inflammation and the Host Response to Injury research program, a collaboration between 18 institutions funded by the National Institutes of Health — the federal funding agency for biomedical research. Motivated by the failure of nearly 150 clinical trials of drugs built to control excessive immune responses, the study set out to systematically evaluate mouse models of disease from which the drugs emerged.
Published online in the Proceedings of the National Academy of Sciences on Feb. 11, the study looked into the activation of inflammation-related genes of white blood cells in response to three conditions: burns, trauma and a model of sepsis (a whole-body inflammatory condition typically caused by severe bacterial infection). The main conclusion: No relationship exists between mice and humans in terms of which genes activate in response to the damage.
Titling their paper “Genomic responses in mouse models poorly mimic human inflammatory diseases,” the 39 co-authors concluded, “The prevailing assumption — that molecular results from current mouse models developed to mimic human diseases translate directly to human conditions — is challenged by our study.”
Online that day and in print the next, The New York Times published an article by science writer Gina Kolata that tried to translate the findings in the PNAS paper for a general audience. In the opening paragraph, Kolata wrote: “But now, researchers report evidence that the mouse model has been totally misleading for at least three major killers — sepsis, burns and trauma. As a result, years and billions of dollars have been wasted following false leads, they say.”
Shortly after the Times article was published, animal-rights organizations took the opportunity to speak out against animal research in general.
The National Anti-Vivisection Society ran with the Times story in its emailer that week, and the group’s science director, Pam Osenkowski, wrote on the NAVS website, “This ‘game-changing’ study reinforces the message that NAVS has long voiced: Scientific evidence shows that the use of animals in science can mislead researchers while it wastes valuable time and resources.”
On Feb. 20, the Humane Society of the United States President Wayne Pacelle, who represents the country’s largest animal-rights organization, expressed a similar sentiment in The Huffington Post, which, according toQuantcast.com, reaches 60 million people per month. Citing the Times article, he titled the post “A Scientific Indictment of Animal Research and Testing.”
Concerns over public impact
Given the size of the newspaper’s readership and influence, the Times’ presentation of the results of the PNAS paper worries scientists.
“I think [Kolata] went way beyond what this paper actually said,” said Cliff Deutschman, critical-care physician and sepsis researcher at the University of Pennsylvania, whom Kolata quoted in the Times article.
“First of all,” Deutschman said, referring to the Times article’s claim that mouse models in the respective areas of inflammation research have been “totally misleading” and wasteful, “you can always learn something from a bad example.”
“To suggest that valuable human therapies have not come out through mouse models is absolutely false,” Timothy Billiar, chair of the UPMC Department of Surgery and a co-author of the PNAS paper, said of the Times article.
Billiar refers to drugs that block TNF, a regulatory molecule produced by white blood cells, since these anti-TNF drugs were developed in mouse models of sepsis but proved effective for human sufferers of rheumatoid arthritis.
“Now they’ve gone on to be blockbuster drugs and have helped thousands of patients.”
The chance that the public may gather from the Times article that scientists suddenly realized they “can’t equate” mice to humans also concerns Billiar. “We’ve always known the limitations of mouse studies,” he said. “You cannot make sweeping conclusions based on mouse studies.”
Wenzhong Xiao, a primary author of the PNAS paper and a genome scientist associated with Massachusetts General Hospital and Stanford University, said of the Times article, “It’s not really a faithful portrait of what we wrote.”
Kolata said that her article accurately represented her sources. The two other PNAS-paper authors the Times article quoted were not available to comment on the record to The Pitt News.
“I am not a scientist, so I cannot reply to scientific quibbles,” Kolata said in an email. “I spoke to experts and accurately reported what they said.”
The scientific debate
Outside of the media attention, interpreting the results themselves divides scientists.
For Olivera Finn, chair of Pitt’s Department of Immunology, the PNAS paper offers little more than confirmation that mouse models carry limitations. “The paper provided hard data for what all of us already assumed to be the case,” she said in an email. “It is easier to argue with data than with assumptions.”
Similarly, Billiar of UPMC thinks the well-publicized PNAS paper pales in significance next to other papers that have emerged from the Inflammation and the Host Response to Injury collaboration, for which he led a clinical site at Pitt. For the group’s magnum opus, he points to a different academic paper published in 2011, which found that injury produces a “genomic storm” within white blood cells.
“A small subcomponent of [the NIH funding] was meant to address the comparison with mouse models,” Billiar said.
But it goes beyond institutional context for Billiar, who did not edit the PNAS paper. He thinks the article may have overstepped itself in interpreting the data.
For one, “I don’t think the title serves the paper well,” he said, similarly criticizing the paper’s discussion section, where results are interpreted. He doesn’t think these parts adequately reflected the paper’s own limits.
One limit is the difference in subjects — the study only used one mouse strain, whereas human samples were pulled from heterogeneous populations (not controlled for age, race, pre-existing conditions, etc.).
“We’ve known that mouse responses vary considerably by strain,” Billiar said. “Some [strains] would exhibit responses closer to humans, some would exhibit responses different from humans.”
Nonetheless, Xiao said, the study chose the specific strain because it was the one “used in this particular field” during its long history of developing drugs that failed.
Deutschman, who would have preferred more “rigorous” gene activation assays than those used in the study, emphasized that the results apply narrowly.
The study’s conclusion is “legitimate up to a point,” he said. “It’s legitimate in talking about global white-cell behavior. It’s not legitimate in talking about lung, liver, heart, brain and anything else.”
Deutschman thinks it’s an important finding that “one organ system — the white cells — doesn’t behave the same way [in mice as in humans],” but that “doesn’t mean that in one subset on the white cells, that you won’t see homology there,” he said.
Deutschman also noted the difference between endotoxemia, in which the subject’s immune system responds to a single part of a pathogen experimentally injected into the bloodstream, and human sepsis, which occurs in patients who often have real infections and other medical conditions. The study focused on the endotoxemia model of sepsis.
“Endotoxemia is not correct,” Deutschman said. “The best model [of sepsis] is cecal ligation and puncture.” Deutschman refers to a method of inducing infection by puncturing the mouse’s colon.
But again, Xiao — a primary author on the PNAS paper — asserted the relevance of all models considered in the study, including endotoxemia. “Those were the popular models that were used in the field of inflammation,” he said, referring to the models that produced the 150 failed clinical trials of sepsis drugs.
“What we’re trying to say here is that the current animal models need to be carefully re-examined,” Xiao said. “We think a lot of studies so far maybe conveniently sidestepped this issue.”
Going forward
“It won’t change my research practices at all,” Billiar, who works with mouse models in inflammation research, said of the PNAS paper.
But that’s not because Billiar clings to an equality between people and laboratory animals. “Mice aren’t really models — they’re assays for very specific mechanistic questions. They really represent the mammalian response, but not a human disease,” he said.
Others see a need for a more aggressive transformation of the field of inflammation research, without giving up animal models.
“I think there’s a lot of value in questioning the relevance of these animal models,” said John Kellum, professor of critical-care medicine at Pitt.
Kellum thinks the PNAS paper provides cause to develop animal models that more closely match the variability of human patients.
“We’ve never looked at groups of human beings that all have the same infection, all with the same antibiotics, all with the same timing, all with the same peritonitis [abdominal infection],” he said, “yet we somehow expect the animal model is going to be like what happens in humans. It’s obvious that that’s a big leap of faith.”
Xiao puts it bluntly. “The bottom line is that we need to choose something else,” he said. “If we don’t talk about this ever, then things will still be in the old way.”
How the PNAS paper might affect funding for research remains an open question. Francis Collins, director of the NIH, commented in his blog on Feb. 19, “This study provides more reason to develop better and more sophisticated models of human disease.” In the blog post, Collins emphasized the NIH’s recent investment in “tissue chips” — small 3-D models of human organs that feature living cells — which he hopes could fulfill part of this desire.
But an NIH official would not directly specify whether research projects using mouse models in the areas of trauma, burns and sepsis would now find it harder to acquire funding.
Daniel Rotrosen, director of the Division of Allergy, Immunology and Transplantation at the National Institute of Allergy and Infectious Diseases, said that the PNAS paper “provides a framework to characterize and potentially validate existing mouse models and to do the same for new and improved models as they are developed,” and he hopes that it will catalyze more focus in human studies.
Given the concerns over the Times report and the activation of animal-rights groups, scientists are considering how best to communicate their findings to general audiences in the future.
“I do believe very strongly that one thing we don’t do in the scientific community is outreach. We don’t talk to people about why we do what we do, why it’s important, etc., and that’s a big, big problem,” said Yates, professor of neuroscience at Pitt.
“The bottom line about this [PNAS] article is it shouldn’t be over-interpreted,” Yates said. “Animal research is still important, but it does point out that you always have to consider the limitations in models and methods.”
