- Population: Patients with potentially resectable stage III lung cancer
- Intervention: Surgery followed by adjuvant systemic therapy
- Comparison: Definitive chemoradiotherapy ICI
- Outcome: Mortality, survival, quality of life
PICO questions frame the evidence search, and not all studies will address each outcome
GRADE process includes rating of recommendations
- Study design: Are the studies well-designed (e.g., randomized controlled trials)?
- Study limitations: Qualitative flaws in the studies that could affect the results?
- Consistency: Do the results of different studies agree?
- Directness: Does the evidence directly answer the clinical question?
- Precision: Are the results precise, or is there a lot of uncertainty?
Factors That Downgrade RCT Certainty
- Risk of Bias: lack of blinding, inadequate randomization
- Inconsistency: multiple RCTs show widely varying results
- Indirectness: surrogate outcomes (e.g., biomarkers) instead of centered outcomes (e.g., mortality)
- Imprecision: the width of the confidence intervals
Factors that can Upgrade Observational Studies
- Large and consistent treatment effects
- Dose-response relationship
- Plausible confounding factors addressed
- Animal or in vitro studies support the findings
- No plausible alternative explanations
Evidence to Decision Framework
The problem
1. Is the problem a priority? The severity and importance of the health issue are assessed.
Effects of the intervention
2. How substantial are the desirable or undesirable effects of the intervention?
3. What is the overall certainty of the evidence for the effects (rated from high to very low)?
Values, costs, and feasibility
4. Values and preferences: Is there important uncertainty or variability in how people value the main outcomes?
5. Balance of effects: Do the desirable effects outweigh the undesirable effects?
6. Resource use: How large are the resource requirements (costs)?
7. What is the certainty of the evidence for resource use?
8. Cost-effectiveness?
9. Equity: What would be the impact on health equity?
10. Acceptability & Feasibility: Is the intervention acceptable to and feasible for key stakeholders?
Criticisms and counter arguments
- The GRADE process can be slow, tedious and contentious
- All good science is difficult. Slow and tedious is balanced by transparency, enhancing trust
- So many recommendations are of low strength, or based on low quality evidence
- Where certainty is high, guidance may not be as important (likely to be less practice variability)
- More guidance is needed where the evidence is gray
- "Garbage In, Garbage Out"
- GRADE is framework for assessing the quality of evidence, but can't make bad studies good. If the foundational research is flawed, GRADE should reflect that
- Bigger problem when dealing with rare conditions, where research is sparse
- Subjectivity (GRADE and EtD)
- Different ‘experts’ rate the same evidence differently.
- Values and preferences are subjective. Particularly in complex clinical scenarios or when patient values are diverse.
- Highlights the importance of diverse guideline panels
- The "Moving Target" Challenge
- Medicine is constantly evolving. GRADE process is valuable for its transparency but takes time. Evidence for recommendations might already be outdated.
- Rapid updates and living guidelines require significant resources
- Speed of scientific advances >> Speed of guideline creation. Lung cancer epitomizes this (good) problem
- The Implementation Gap (EtD):
- Getting guidelines into practice is difficult. Providers might not be aware of the guidelines, or they might face barriers to implementing them
- Guidelines are only effective if they're actually used
- De-implementation of bad habits is also a very difficult problem
|
