Comments on “The NCCN Comparative Therapeutic Index TM as a Paradigm

The National Pharmaceutical Council (NPC) appreciates this opportunity to comment on the proposed “The NCCN Comparative Therapeutic Index ™ as a Paradigm for Near Term Comparative Effectiveness Analyses of Existing Data in Oncology” White Paper. This white paper addresses a critical component of the comparative effectiveness research infrastructure. It informs the translation and dissemination of hematology/oncology knowledge to optimize the use of drugs, biologics, procedures and techniques in cancer care. National Comprehensive Cancer Network (NCCN) is recognized as a source of high-quality information for clinicians, payers and policymakers through their various materials including NCCN Clinical Practice Guidelines in Oncology™, NCCN Chemotherapy Order Template™, and NCCN Drugs & Biologics Compendium™. To maintain this stature we recommend including robust methodologies and processes to develop the Comparative Therapeutic Index (CTI), incorporating stakeholder preferences , and acknowledging that the CTI is to inform the riskversus- benefit ratio of products and not for coverage and policy decisions for individual patients.

To further the credibility of the CTI and the improve the care of patients, NPC recommends the following comments for consideration:

1) Methodology for Evaluating Data

1a) Selection of Data for Consideration in the CTI Paradigm

The NCCN White paper notes that it will utilize scientific comparative effectiveness analyses from “credible organizations and authors”. While using analyses from credible organizations and authors offers a trusted source, NPC believes that greater emphasis should be placed upon credibility of data and analyses. NCCN recognizes that “other methodologies, such as outcomes databases, registries, and practical clinical trials can contribute substantially”, however it is not clear if and under what conditions such studies would be included in the data review. Explicit guidelines on what types of studies, conducted under what circumstances, and what requirements for methodological conduct and interpretation must be met for inclusion in CTI review. Without explicit guidelines for considering and evaluating data, the CTI paradigm may overly rely on expert opinion of oncology thought leaders and less on evidence-based medicine. By further defining the types of evidence and criteria included in the analyses and noting the limitations of such evidence, the perception of the CTI Paradigm as a credible and trusted source will be enhanced.

1b) Selection of Oncology Thought Leaders for Panel Members

NCCN offers an opportunity to identify multidisciplinary experts through the NCCN member institutions in the fields of medical, radiation, and surgical oncology. In addition to the clinical expertise, NPC recommends a broader representation adding epidemiologists, statisticians, and technical experts familiar with appropriate methodologies and interpretation of secondary data. Further specifying the minimum expertise and requirements for panel members as well as the composition requirements for the CTI panels would be beneficial to ensure credible, trusted information is considered by clinical and methodology experts.

1c) Explicit Criteria Definitions in the Proposed Scoring Grids

NPC commends the step-wise pilot testing, and testing for reliability and validity of the CTI process through inter-rater and intra-rater reliability as well as construct validity.

Additional consideration and quantitative information could be included in the Proposed Scoring Charts to plan proactively for the best potential reliability and to minimize individual panel expert subjectivity. As an example, the differences in the “Overall survival” dimension range from “0-No Impact” (defined as no meaningful impact on relevant disease related endpoints), “1-2 Minimal” (defined as no impact on survival/cure, but sometimes provides control of disease) to “3-Low” (defined as little or no impact on survival/cure but often provides control of disease). This allows for differences in interpretation among “meaningful” “sometimes” and “often” resulting in differences within a panel of experts but also between panels.

1d) Expansion of additional treatment options to include delivery options

NPC welcomes the step-wise pilot process that is proposed. While certain parameters must be limited to effectively pilot a program, it may be pre-emptive for the evaluations to initially focus on treatments. Others including the recent IOM Initial National Priorities for CER report, noted that CER may best inform patient and physician treatment when considering various healthcare strategies are considered, and was further highlighted by the prioritization of research in the oncology and hematology field.

2) Application of CTI findings

One of the strengths of NCCN undertaking translation of CER knowledge is “the NCCN Clinical Practice Guidelines in Oncology are widely recognized and used as the standard of care in oncology in both the academic and community settings and significantly influence appropriate practice patterns, prescribing behavior and coverage policy”. This adds additional responsibility for the CTI to remain true to the the stated purpose of the process “to compare “the riskversus- benefits of different treatment options” and include additional caveats to ensure that this information will not be used explicitly for coverage decisions.

3) Consideration and Effective Translation of Patient-Specific Characteristics

NPC welcomes the recognition that the NCCN CTI process should not be in conflict with “personalized medicine”. In the current form, appropriate delineation between productspecific data rather than patient specific information may be further clarified. These differences among patient groups based upon genetics, biomarkers, or other criteria may best be explicitly outlined in the CTI paradigms and graphical depictions detailed in Appendices A and B.

4) Incorporation of Resource Utilization

If the NCCN CTI is to be a “evidence-based, systematic, comprehensive, and transparent process using explicit criteria for evaluating and then comparing the risk-versus-benefits of different treatment options” the focus should remain on the comparative effectiveness of treatment options. Furthermore, NCCN has recognized that there is no agreed upon definition for what constitutes appropriate value, resource utilization or modest benefits. Resource utilization mentions only the price of therapy, administration setting, monitoring intensity and other special needs in the current proposal, but does not incorporate the holistic cost of overall treatment. Furthermore, as NCCN represents a broad group of stakeholders among the community oncology, academic and policy community, the acquisition costs (including regional or local cost variations) and stakeholder preferences may not be appropriately considered. Finally, experts in economic evaluations are not included in the panel composition. NPC recommends that the CTI process first consider risk-versus benefit in terms of effectiveness and toxicity based upon these limitations without the inclusion of robust methodology to consider holistic cost and comparative value.

5) Translation of Data to Additional Audiences Beyond Physicians

NCCN has a unique ability to inform and improve practice patterns of oncologists and hematologists. If data is truly to improve patient care additional opportunities to expand dissemination to additional decision makers including patients, caregivers and families may best assist the care for patients and families that NCCN member institutions serve.

NPC supports the NCCN CER white paper and the initiative to further translate and disseminate information for providers at the point of care through the NCCN Comparative Therapeutic Index. NCCN is a credible and recognized organization within the oncology community. By increasing the transparency in the types of data, data analyses, and experts included in the development of CTI paradigm, acknowledging the appropriate utilization of CTI is to inform physicians and patients rather than making coverage decisions, and informing additional stakeholders beyond the physician community with patient rather than product specific information may best enhance and improve patient care in the hematology/oncology setting.

INaik AD, Peterson LA. The Neglected Purpose of Comparative-Effectiveness Research. New England Journal of Medicine 2009; 360(19)1925-1927.
IIInstitute of Medicine of the National Academies Committee on Comparative Effectiveness Research Prioritization. Initial National Priorities for Comparative Effectiveness Research. June 2009; Accessed November 18, 2009