My Real-World Evidence

“To the man who only has a hammer, everything he encounters begins to look like a nail.”
Abraham Maslow

man in black jacket standing near table with toolbox — Photo by Andrea Piacquadio on Pexels.com

Analytic tools

Methods for external control groups for single arm trials. In this article, Seeger et al. present examples of external control groups, discussing some of the issues and making recommendations to address them through careful assessment of the RWD source, the study design, and key analysis steps.
The European Medicines Agency (EMA) published on September 2022 a qualification opinion for Prognostic Covariate Adjustment (PROCOVA™), a statistical methodology, developed by Unlearn, intended to improve the efficiency of Phase 2 and 3 clinical trials, by using trial subject´s prediceted outcomes on placebo (prognostic scores) in linear covariate adjustment.
Population Health Methods, by the Columbia University Mailman School of Public Health. Descriptions and additional resources regarding important and emerging population health techniques and the tensions that may arise in the selection and application of appropriate techniques.
An Introduction to Propensity Score Methods for Reducing the Effects of Confounding in Observational Studies. In this article, Austin introduces the concept of the propensity score and describes how methods based on it can be used to reduce or eliminate the effects of confounding when using observational data to estimate treatment effects. A tutorial and case study by the same author illustrates the application of propensity score methods to estimate the reduction in mortality due to provision of in-hospital smoking cessation counseling to current smokers who had been hospitalized with a heart attack
R is a language and environment for statistical computing and graphics. R provides a wide variety of statistical and graphical techniques, and is highly extensible.
Observational Health Data Sciences and Informatics (OHDSI) program. A multi-stakeholder, interdisciplinary collaborative to bring out the value of health data through large-scale analytics. OHDSI offers a wide range of open-source tools to support various data-analytics use cases on observational patient-level data.
Epi Info. A public domain suite of interoperable software tools designed for the global community of public health practitioners and researchers.

Books

Causal Inference: What If. A book by Hernán & Robins, from the Harvard T.H. Chan School of Public Health, that helps scientists generate and analyze data to make causal inferences that are explicit about both the causal question and the assumptions underlying the data analysis.
Hands-On Machine Learning with R. An book that provides a practical and applied approach to learning into today’s most popular machine learning methods. A practitioner’s guide to the most popular machine learning methods.

Courses

Boston University, MPH Online Learning Modules. The MPH online learning modules help students learn about public health concepts. They are designed to complement MPH courses and address concepts and skills that cut across a range of disciplines.
Elements of Artificial Intelligence (AI). A series of free online courses created by Reaktor and the University of Helsinki. To learn what AI is, what can (and can’t) be done with AI, and how to start creating AI methods.

Opinion

Real-World Evidence – Where Are We Now? – A NEJM (May 5, 2022) must read that reviews what RWD and RWE mean today from the perspective of the Food and Drug Administration. The article highlights two widespread misconceptions about both terms, which are worth remembering.
Integrated evidence generation: A paradigm shift in biopharma. In this 2021 article, Amin et al., from McKinsey & Company, take up the up the torch from Olson asserting that biopharma companies should consider a new, integrated approach to evidence-generation strategies to better demonstrate the value of therapies to all stakeholders.
Developing an integrated strategy for evidence generation. In this visionary Editorial from 2017 in the Journal of Comparative Effectiveness Research, Olson described a new strategy that entails a complete change in mindset and way of working from the traditional models of evidence generation, typifying a new way of doing business in biopharma.
Raising the bar for using surrogate endpoints in drug regulation and health technology assessment. Surrogate endpoints provide no guarantee of clinical benefit, and Dalia Dawoud and colleagues argue they should be used only as a last resort in drug trials.
Why representativeness should be avoided? Do you think your sample should be representative? Maybe, but read this article by Rothman et al. before making a decision.

Reference

Principles of Epidemiology in Public Health Practice. An introduction to Epidemiology and Biostatistics, by the Centers of Disease Control and Prevention.
National Cancer Institute dictionaries. A dictionary of cancer terms, a dictionary of genetic terms, and a cancer drug dictionary.
Mendeley . A desktop and web program for managing and sharing research papers.

Standards and Guidance

NICE framework (published 23 June 2022)- The RWE Framework Describes best practices for the planning, conduct, and reporting of RWE studies to inform NICE guidance.
MHRA guideline on randomised controlled trials using real-world data to support regulatory decisions
FDA’s Guidance – Considerations for the Use of Real-World Data and Real-World Evidence To Support Regulatory Decision-Making for Drug and Biological Products
FDA’s Guidance – Submitting Documents Using Real-World Data and Real-World Evidence to FDA for Drug and Biological Products
FDA’s Draft Guidance – Real-World Data: Assessing Registries to Support Regulatory Decision-Making for Drug and Biological Products Guidance for Industry
FDA’s Draft Guidance – Data Standards for Drug and Biological Product Submissions Containing Real-World Data
FDA’s Draft Guidance – Real-World Data: Assessing Electronic Health Records and Medical Claims Data To Support Regulatory Decision-Making for Drug and Biological Products
European Medicines Agency´s Guideline on registry-based studies. The objective of this Guideline is to provide recommendations on key methodological aspects that are specific to the use of patient registries by marketing authorisation applicants and holders planning to conduct registry-based studies.
Developing a Protocol for Observational Comparative Effectiveness Research (CER): A User’s Guide. The guide serves as a resource for investigators and stakeholders when designing CER studies, particularly those with findings that are intended to translate into decisions or actions.
Good Practices for Real-World Data Studies of Treatment and/or Comparative Effectiveness. recommendations from ISPOR and ISPE regarding good procedural practices for RWD studies.
European Network of Centres for Pharmacoepidemiology and Pharmacovigilance (ENCePP). A network coordinated by the European Medicines Agency. Members are public institutions and contract and research organisations involved in research in pharmaco-epidemiology and pharmacovigilance. Access to a checklist for study protocols and a guide on methodological standards.
Improving Transparency to Build Trust in Real-World Secondary Data Studies for Hypothesis Testing. In this article an ISPOR/ISPE joint task force recommended that investigators preregister their RWE studies and post their study protocols in a publicly available forum before starting studies in order to reduce bias.
Artificial Intelligence: The Key to Unlocking Novel Real-World Data? This ISPOR article by Michele Cleary explores how artificial intelligence (AI) may improve clinical research through its ability to better translate RWD into RWE.