Evidence-Based Process

Evidence-Based Methodology

The DynaMed evidence-based methodology is the foundation underlying all our content. An evidence-based tool must derive conclusions based on the best available evidence. This can only occur if the evidence is consistently and systematically identified, selected, summarized, synthesized, and interpreted. Conclusions must also be adjusted as new evidence is added to the ever-growing evidence ecosystem.

The DynaMed editorial processes include the following steps:

To ensure that DynaMed provides the best available evidence, an extensive set of current literature is monitored daily. A state-of-the-art Systematic Literature Surveillance program continually monitors medical research as it is published, including journals, journal review services, review collections, guideline collections and other relevant sources.

To select the best available evidence, screen published research and clinical practice guidelines are screened for relevance and potential impact on clinical decision-making and patient care. Each relevant article is further assessed for validity relative to what is already known. The most valid articles are summarized and integrated into existing DynaMed content, which may trigger updates to overview statements and the overall outline structure.

Article selection is completed by a team of practicing physicians and methodologists with clinical expertise and/or training in scientific analysis. Relevance is determined primarily by results affecting clinical outcomes and clinical decision-making, and secondarily by consideration of popular or frequent clinical questions.

Every medical research paper identified and selected for inclusion in DynaMed undergoes an objective critical appraisal process involving consistently applied, systematic identification of potential sources of bias. A simple Level of Evidence rating system makes it easy for users to quickly understand the quality of the evidence being reported. Learn more about the editorial process.

Critical appraisal is completed by an editorial team that is rigorously trained in evidence-based medicine and critical appraisal of scientific/medical research.

The evidence is objectively reported using data from the original study publication, with a focus on clinical outcomes and absolute risk data when available. Levels of evidence are applied according to an explicit protocol. Clinicians review all content for validity and relevance at the point-of-care.

Objectively reporting the evidence for an individual study is necessary but insufficient for a comprehensive point-of-care reference. Understanding the best current evidence requires synthesizing multiple evidence reports. When evaluating multiple evidence reports, preference for inclusion and content organization is based on both clinical relevance and the quality of evidence.

When data of lesser quality is identified but does not add any substantially new or different information, this data is excluded. Lesser quality data is also removed when it is superseded by higher quality evidence. Evidence is also synthesized with clinical practice guidelines, and areas of inconsistency are presented. Clinicians review all content for validity and relevance at the point-of-care.

Multiple evidence reports are organized such that the overall conclusions quickly provide a synthesis of the best available evidence. Overviews (including the Overview and Recommendations, Testing Overview, and Management Overview) are based upon and linked directly to the supporting evidence.

The final step in evidence-based methodology is adjusting conclusions as new evidence is added to the ever-growing evidence ecosystem. This step is crucial because new evidence is published every day.

As soon as new evidence is evaluated using the six steps governing systematic processing, it is integrated into the appropriate DynaMed topic(s). This process allows immediate and complete access to the best evidence as it becomes available.

Editorial Processes

There are approximately 50 million medical and scientific publications available in public databases, and one new medical or scientific publication is released every 30 seconds. With new information coming out at such a staggering rate, it is virtually impossible for a clinician to keep up with all the newest evidence available daily. Our rigorous editorial process does the work for you.

A state-of-the-art Systematic Literature Surveillance (SLS) program supports DynaMed’s evidence-based methodology by continually monitoring medical research as it is published. A team of practicing physicians and methodologists identify, objectively appraise, and summarize clinical trials and clinical practice guidelines that impact patient care.

To identify the evidence the Systematic Literature Surveillance (SLS) program monitors 250 medical journals and 200 guideline organizations publishing the most clinically relevant research and clinical practice guidelines across 28 specialties.

DynaMed’s partnership with McMaster University adds the power of the McMaster PLUS resource to its monitoring program. McMaster University’s Health Information Research Unit uses its Critical Appraisal Process to identify studies and systematic reviews that are scientifically sound from approximately 120 top journals.

To select the best available evidence DynaMed works with > 50 medical specialists who screen published research and clinical practice guidelines for relevance and potential impact on clinical decision-making and patient care. In addition, McMaster PLUS provides ratings and comments from their global network of more than 8,000 frontline clinicians through the McMaster Online Rating of Evidence (MORETM) system.

Every medical research paper identified and selected for inclusion in DynaMed undergoes an objective appraisal process by an editorial team that is rigorously trained in evidence-based medicine and critical appraisal of scientific/medical research. Critical appraisal is the process of systematically assessing the outcome of scientific/medical research to evaluate its trustworthiness, relevance, and value. It facilitates the application of research into practice, allowing a clinician to understand whether research is reliable and applicable to their patients.

DynaMed’s critical appraisal process involves consistently-applied, systematic identification of potential sources of bias in order to enable objective communication of the relevant clinical message in the context of study’s limitations. A team of practicing physicians and methodologist objectively appraise the most valid clinical trials and clinical practice guidelines and integrate the findings into the existing clinical framework.

A simple Level of Evidence rating system makes it easy for users to quickly understand the quality of the evidence being reported in an evidence summary, and the primary rationale behind the rating.

Level 1 (likely reliable) eevidence represents research results that address clinical outcomes and meet an extensive set of quality criteria that minimizes bias.

Level 2 (mid-level) evidence represents research results that address clinical outcomes and demonstrate some method of scientific investigation but do not meet the quality criteria to achieve Level 1.

Level 3 (lacking direct) evidence represents either of the following:

Reports that are not based on scientific analysis of clinical outcomes (e.g., case series, case reports, conclusions extrapolated indirectly from scientific studies)

Research results that do not address clinical outcomes, regardless of the scientific rigor

Read the full Levels of Evidence criteria.

In the Overview and Recommendations, DynaMed synthesizes current evidence, current clinical practice guidelines, and clinical expertise to provide recommendations to support clinical decision-making. The internationally-accepted Grading of Recommendations Assessment, Development and Evaluation (GRADE) system is used to classify synthesized recommendations as Strong or Weak.

  • Strong recommendations are used when, based on the available evidence, clinicians (without conflicts of interest) consistently have a high degree of confidence that the desirable consequences (health benefits, decreased costs and burdens) do, or do not, outweigh the undesirable consequences (harms, costs, burdens).
  • Weak recommendations are used when, based on the available evidence, clinicians believe that desirable and undesirable consequences are finely balanced, or appreciable uncertainty exists about the magnitude of expected consequences (benefits and harms). Weak recommendations are also used when the range of patient values and preferences suggests that informed patients are likely to make different choices.

DynaMed synthesized recommendations are determined with a systematic methodology:

  • Recommendations are initially drafted by clinical editors (including ≥ 1 with methodological expertise and ≥ 1 with content domain expertise) who understand the best current evidence for benefits and harms and relevant recommendations from clinical practice guidelines.
  • Recommendations are phrased to match the strength of the recommendation. Strong recommendations use “should do” or “should not do” language or phrasing that implies an expectation to perform (or not perform) the recommended action for most patients. Weak recommendations use “consider” or “suggested” language.
  • Recommendations are explicitly labeled as Strong recommendations or Weak recommendations when a qualified group has explicitly deliberated on making such a recommendation. Group deliberation may occur during guideline development. When group deliberation occurs through DynaMed-initiated groups:
    • Clinical questions will be formulated using the PICO (Population, Intervention, Comparison, Outcome) framework for all outcomes of interest that are specific to the recommendation to be developed.
    • Systematic searches will be conducted for any clinical questions where systematic searches were not already completed through the DynaMed content development process.
    • Evidence will be summarized for recommendation panel review including for each outcome the relative importance of the outcome, the estimated effects comparing intervention and comparison, the sample size, and overall quality rating for the body of evidence.
    • Recommendation panel members will be selected to include at least three members that together have sufficient clinical expertise for the subject(s) pertinent to the recommendation, methodological expertise for the evidence being considered, and experience with guideline development.
    • All recommendation panel members must disclose any potential conflicts of interest (professional, intellectual, and financial), and will not be included for the specific panel if a significant conflict exists for the recommendation in question.
    • Panel members will make Strong recommendations if and only if there is consistent agreement in a high confidence in the likelihood that desirable consequences outweigh undesirable consequences across the majority of expected patient values and preferences. Panel members will make Weak recommendations if there is limited confidence (or inconsistent assessment of dissenting opinions) that desirable consequences outweigh undesirable consequences across the majority of expected patient values and preferences. No recommendation is made if there is insufficient confidence to make a recommendation.
    • All steps in this process (including evidence summaries that are shared with the panel, and identification of panel members) will be transparent and accessible in support of the recommendation.
  • Recommendations are verified by ≥ 1 editor with methodological expertise, not involved in recommendation development, with explicit confirmation that Strong recommendations are adequately supported.
  • Recommendations are published only after consensus is established with agreement in phrasing and strength of recommendation by all editors.
  • If consensus cannot be reached then the recommendation can be published with a notation of “dissenting commentary” and the dissenting commentary is included in the topic details.
  • If recommendations are questioned during peer review or post-publication by a qualified individual, or re-evaluation is warranted based on new information identified through the Systematic Literature Surveillance program, the recommendation is subject to additional internal review

New topics in DynaMed are developed under the direction of a Deputy Editor and undergo a rigorous writing and review process that includes methodologists, physician experts or specialists, and EBM experts. The topic development process leverages the SLS program for identifying evidence and selecting the best available evidence and the critical appraisal process.

DynaMed topics are developed to meet the information needs of clinicians and provide guidance to support effective and timely clinical decision-making. As a result, Topic and Section Editors in active clinical practice play an integral role in the topic development process and are listed on each topic they review.

The typical Topic/Section Editor is a physician who is active in academic and clinical practice and is board-certified in the appropriate specialty or devotes a portion of time caring for patients with the clinical entity in questions. Other experts (such as pharmacists) may be appropriate for selected topics to support a multidisciplinary approach to meeting the needs of clinicians. The Topic/Section Editor provides an initial clinical framework to guide the editorial team and then reviews the topic once it is developed to ensure that it is comprehensive, clinically relevant, and based on the best available evidence.

Prior to review by the Topic/Section Editor, the editorial team applies DynaMed’s evidence-based methodology to develop the topic, from performing a comprehensive literature search to integrating the best available evidence into the clinical framework. The content is reviewed for accuracy and application of critical appraisal by an editor as well as clinical relevance and usability at the point-of-care by a physician editor.

The final steps in the topic development process consist of an independent review by an EBM expert and review by the Deputy Editor. The EBM expert evaluates the Overview and Recommendations for adherence to sound recommendations and consistency.

DynaMed’s living reviews are updated regularly as new evidence is identified through the SLS program. In addition, topics undergo regular updating and review by the editorial team, Topic/Section Editors, and EBM experts. This ensures that DynaMed content keeps pace with the ever-growing evidence ecosystem and evolving standards of care.

Content Sources

DynaMed editors systematically monitor journals, journal review services, systematic review collections, guideline collections, drug information sources and other relevant sources.

DynaMed’s Systematic Literature Surveillance process includes:

  • Cover-to-cover surveillance of highest-yield content sources
  • Targeted MEDLINE searches for systematic reviews and randomized controlled trials for high-yield journals
  • Targeted MEDLINE searches for selected subject areas (e.g., complementary therapies)
  • Comprehensive MEDLINE searches for guidelines
  • Academic Emergency Medicine
  • Allergy
  • American Journal of Obstetrics and Gynecology
  • American Journal of Ophthalmology
  • American Journal of Psychiatry
  • American Journal of Respiratory and Critical Care Medicine
  • American Journal of Sports Medicine
  • American Journal of Transplantation
  • Anesthesiology
  • Annals of Emergency Medicine
  • Annals of Internal Medicine
  • Annals of Neurology
  • Annals of Surgery
  • Annals of the Rheumatic Diseases
  • Archives of Diseases in Childhood. Fetal and neonatal edition
  • Arthritis and Rheumatology
  • Arthroscopy: The Journal of Arthroscopic & Related Surgery
  • BJOG: An International Journal of Obstetrics and Gynaecology
  • BJU International
  • Blood
  • BMJ
  • British Journal of Anesthesia
  • British Journal of Dermatology
  • British Journal of Haematology
  • British Journal of Ophthalmology
  • British Journal of Psychiatry
  • British Journal of Sports Medicine
  • British Journal of Surgery
  • Canadian Medical Association Journal
  • Chest
  • Circulation
  • Clinical and Experimental Allergy
  • Clinical Gastroenterology and Hepatology
  • Clinical Infectious Diseases
  • Clinical Reviews in Allergy and Immunology
  • Clinical Journal of the American Society of Nephrology (CJASN)
  • Cochrane Database of Systematic Reviews
  • Critical Care
  • Critical Care Medicine
  • Diabetes Care
  • Emerging Infectious Diseases
  • European Heart Journal
  • European Respiratory Journal
  • European Urology
  • Gastroenterology
  • Gut
  • Head and Neck
  • Hepatology
  • Hypertension
  • Intensive Care Medicine
  • International Journal of Obesity
  • JAMA
  • JAMA Cardiology
  • JAMA Dermatology
  • JAMA Internal Medicine
  • JAMA Neurology
  • JAMA Oncology
  • JAMA Ophthamology
  • JAMA Otolaryngology - Head and Neck Surgery
  • JAMA Pediatrics
  • JAMA Psychiatry
  • JAMA Surgery
  • Journal of the American College of Cardiology
  • Journal of the American Academy of Child and Adolescent Psychiatry
  • Journal of the American Academy of Dermatology
  • Journal of the American Society of Nephrology
  • Journal of Adolescent Health
  • Journal of Allergy and Clinical Immunology
  • Journal of Bone and Joint Surgery. American volume
  • Journal of Clinical Endocrinology and Metabolism
  • Journal of Clinical Oncology
  • Journal of Hepatology
  • Journal of Neurology, Neurosurgery and Psychiatry
  • Journal of Pain
  • Journal of Pediatrics
  • Journal of the American Medical Directors Association
  • Journal of Urology
  • Journals of Gerontology. Series A Biological sciences and medical sciences
  • Kidney International
  • Lancet
  • Lancet. Child & Adolescent Health
  • Lancet. Diabetes & Endocrinology
  • Lancet. HIV
  • Lancet. Hematology
  • Lancet. Infectious Diseases
  • Lancet. Neurology
  • Lancet. Oncology
  • Lancet. Psychiatry
  • Lancet. Respiratory Medicine
  • Laryngoscope
  • Morbidity and Mortality Weekly
  • Neurology
  • New England Journal of Medicine
  • Obstetrics and Gynecology
  • Ophthalmology
  • Pain
  • Palliative Medicine
  • Pediatrics
  • Phytomedicine
  • PLoS Medicine
  • Radiology
  • Resuscitation
  • Sleep
  • Sleep Medicine
  • Surgery for Obesity and Related Diseases
  • Stroke
  • Thorax
  • Thrombosis and Haemostasis
  • Thyroid
  • American Academy of Neurology (AAN)
  • American Academy of Pediatrics (AAP)
  • American Academy of Clinical Endocrinologists (AACE)
  • American Association for the Study of Liver Diseases (AASLD)
  • American College of Cardiology (ACC)
  • American College of Chest Physicians (ACCP)
  • American College of Obstetricians and Gynecologists (ACOG)
  • American College of Physicians (ACP)
  • American Diabetes Association (ADA)
  • British Thoracic Society (BTS)
  • Canadian Cardiovascular Society (CCS)
  • Canadian Medical Association (CMA)
  • Canadian Task Force of Preventive Health (CTFPHC)
  • Centers for Disease Control and Prevention (CDC)
  • European Respiratory Society (ERS)
  • European Society for Medical Oncology (ESMO)
  • European Society of Cardiology (ESC)
  • Global Initiative for Chronic Obstructive Lung Disease (GOLD)
  • Infectious Diseases Society of America (IDSA)
  • International Society on Thrombosis and Haemostasis (ISTH)
  • National Comprehensive Cancer Network (NCCN)
  • National Health & Medical Research Council (NHMRC) [Australian guidelines]
  • National Institute for Health and Care Excellence (NICE)
  • Pan American Health Organization (PAHO)
  • Royal College of Physicians (RCP)
  • Scottish Intercollegiate Guidelines Network (SIGN)
  • World Gastroenterology Organisation (WGO)
  • World Health Organization (WHO)