Impact of Data and Methods on Comparative Effectiveness Research Conclusions, Information Dissemination & Stakeholder Decisions
December 10, 2009
10:00 a.m. - 3:30 p.m.

With more than 18,000 randomized controlled trials (RCTs) published each year, it’s challenging for health care providers and patients to figure out which outcomes and information are valid or most relevant to them. A new study, published this month by the National Pharmaceutical Council, is now available to help by offering a series of easy-to-use checklists to evaluate RCTs and other types of comparative effectiveness research (CER).

The study, Demystifying Comparative Effectiveness Research: A Case Study Learning Guide, and the steps involved in evaluating various CER study designs were presented at a December 10 “town hall” meeting hosted by AstraZeneca at its Wilmington, Del., headquarters. During the meeting, study author Robert W. Dubois, MD, PhD, chief medical officer of Cerner LifeSciences, described the three main types of CER to the nearly 200 town hall participants:

• Randomized controlled trials (RCTs), wherein subjects in a population are randomly allocated into study and control groups to receive or not to receive an experimental intervention, such as a new therapeutic biopharmaceutical or medical device;
• Meta-analysis, which synthesizes data across a series of similar studies, generally RCTs; and
• Observational studies, which follow participants over a period of time to examine the potential associations between patients’ exposure to treatment and health outcomes.

Dr. Dubois also pointed out the pitfalls of CER that could lead to poor or biased studies or even incorrect conclusions. For RCTs, Dr. Dubois described such factors as whether the “RCT meets your real world needs.” For meta-analysis, “just combining studies because they are there may not be appropriate. It’s all about the studies you include, or don’t include.” Ignoring negative studies, for example, could bias the meta-analysis. As for observational studies, “this is a great way to look at the real world,” but for the study to be applicable, “make sure the population looks like yours,” Dr. Dubois cautioned. To make it easier for CER end-users to avoid those pitfalls, Demystifying includes basic, “yes or no” checklists to help determine the validity of a study.

According to Dr. Sean Tunis, head of the Center for Medical and Technology Policy, some of the challenges in understanding and evaluating CER are related to “the failure of the research enterprise to answer important questions” needed by "real-world" decision makers.

“Some of the reasons for the gaps in evidence are that the subjects [in RCTs] are highly selected and don’t look like those seen in practice, research settings aren’t typical of clinical practice, and there are missing or incorrect comparators,” said Dr. Tunis.

Following presentations by Drs. Tunis and Dubois, panelists discussed the difficulties of producing data and recognized that most studies are not relevant to the ultimate end-user of the information: the patient. “We have too much information being generated, but too many [studies] that aren’t meaningful to us,” said Dr. Mark Fendrick, co-director of the Center for Value-Based Insurance Design.

Jean Slutsky, head of the Agency for Healthcare Research and Quality, concurred. “It’s important to reproduce health outcomes that are important to patients’ endpoints. For researchers, it’s mortality. For patients, their endpoints are different – can they lift the kids? How will they feel?”

For researchers and even policy makers, these are important questions to consider. More than $1.1 billion has been invested in federal CER, and as part of health reform legislation, Congress is currently considering the establishment of a permanent "Institute" to oversee CER. With this kind of commitment to CER, speakers at the town hall acknowledged the importance of developing sound and useful data.

To “get CER right,” Dr. Marcus Wilson, president of HealthCore, suggested that in developing the questions to be answered, key decision makers must be at the table, including health care providers and representatives from health care facilities. It’s a model he uses when developing such studies at HealthCore, and the diverse input has helped his organization better understand certain medication compliance trends.

It is also critical to recognize how payers, providers and patients can understand and apply CER study results, and how those results can be communicated by the media. This topic was addressed during an afternoon panel discussion featuring Marc Boutin, Executive Vice President and Chief Operating Officer of the National Health Council, Nielsen Hobbs, Editor of The Pink Sheet, Brian Sweet, Chief Clinical Pharmacy Officer of Health Solutions at WellPoint, Dr. Fendrick, and Dr. Les Paul, NPC’s Vice President for Clinical and Scientific Affairs, who moderated the panel.

“CER is a step in the right direction,” said Sweet, “But we need better health outcomes data.”

A videocast of the town hall is available.

AGENDA

Welcome and Introductions

• Rich Fante, President, AstraZeneca US
• Dan Leonard, President, National Pharmaceutical Council

Comparative Effectiveness Research: Evaluating the Evidence

• Sean S. Tunis, MD, MSc, Director, Center for Medical Technology Policy
• Demystifying Comparative Effectiveness Research: A Case Study Learning Guide
• Robert W. Dubois, MD, PhD, Chief Medical Officer, Cerner LifeSciences

Panel #1: Evidence Generation

• Jean R. Slutsky, Director, Center for Outcomes and Evidence, Agency for Healthcare Research and Quality
• Marcus Wilson, PharmD, President, HealthCore
• A. Mark Fendrick, MD, Co-director, Center for Value Based Insurance Design
• Doug Levine, MD, Executive Director, Strategic Development, AstraZeneca
• Moderator: Frank Casty, MD, Vice President, External Medical Relations, AstraZeneca

Panel #2: Evidence Utilization

• Marc Boutin, JD, Executive Vice President and Chief Operating Officer, National Health Council
• A. Mark Fendrick, MD, Co-director, Center for Value Based Insurance Design
• Nielsen Hobbs, Editor, The Pink Sheet
• Brian Sweet, MBA, Chief Clinical Pharmacy Officer, Health Solutions, WellPoint
• Moderator: Les Paul, MD, MS, Vice President, Clinical and Scientific Affairs, National Pharmaceutical Council

Discussion of Checklist

• Robert W. Dubois, MD, PhD, Chief Medical Officer, Cerner LifeSciences

Closing Remarks

• Les Paul, MD, MS, Vice President, Clinical and Scientific Affairs, National Pharmaceutical Council