Defining High-Quality Observational Studies

Published

Defining High-Quality Observational Studies

Observational Studies Can Inform Health Decisions—When Done Correctly. TO IMPROVEHEALTH CAREQUALITY & REDUCE HEALTH CARECOSTSTHERE IS A GROWING DEMAND FOR AND SUPPLY OF OBSERVATIONAL STUDIES* *Sometimes called real-word evidence orevidence from clinical experience.

OBSERVATIONAL STUDIES HAVE SEVERAL STRENGTHS

Provide information about diverse populations. Improve understanding of patient-centered outcomes. Tell us how treatments work in real-world settings. Can be conducted more quickly for timely results.

BUT ALSO HAVE LIMITATIONS THAT NEED TO BE ACCOUNTED FOR

Many factors drive treatment decisions Real-world patients can have complex clinical conditions. Some studies must address unbalanced groups, differential follow-up and missing data.

WHAT MAKES OBSERVATIONAL STUDIES HIGH QUALITY?

New research gives us an idea.

RESEARCHERS

EVALUATED

9 standards and guidelines for observational studies

23 items that observational studies should include

  • AHRQ
  • CER Collaborative
  • ENCePP Checklist
  • ENCePP Guide
  • FDA
  • GRACE Checklist
  • GRACE Principles
  • ISPOR
  • PCORI

WHAT DID THEY FIND?

2 items are addressed and have agreement across standards and guidelines

  • Defining study objectives and research questions
  • Including details on data sources

12 items are addressed but disagree across standards and guidelines, such as

  • Describing study limitations
  • Handling missing data
  • Addressing bias

9 items are addressed by some but not all standards and guidelines, such as

  • Reviewing prior research
  • Involving stakeholders in research
  • Assessing heterogeneity of treatment effects

LACK OF COMMON STANDARDS FOR HIGH-QUALITY OBSERVATIONAL STUDIES CAN IMPACT

  • What Studies FUNDERS Support
  • What Methods RESEARCHERS Apply
  • What StudiesJOURNAL EDITORS Publish
  • What Evidence PATIENTS, PROVIDERSand PAYERS Use

HOW DO WE AGREE ON WHAT “GOOD” LOOKS LIKE?

AUTHORS RECOMMEND 4 STEPS

  1. GAIN ALIGNMENT: Perspectives may differ, but what items are included should not
  2. AGREE ON LEVEL OF CONSENSUS: Minimum standards rather than best practices are most achievable
  3. GARNER CONSENSUS: Stakeholders should convene in an iterative process to gain consensus
  4. ENCOURAGE CONSENSUS: Voluntary adoption is most likely for success

Source: Morton SC, Costlow MR, Graff JS, Dubois RW. Standards and Guidelines for Observational Studies: Quality Is in the Eye of the Beholder. The Journal of Clinical Epidemiology. 2015. Online ahead of print: November 5, 2015.

Embed this infographic on your site

Download the Infographic PDF

Defining High-Quality Observational Studies PDF