Evidence-Based Process

Evidence-Based Methodology

The DynaMed evidence-based methodology is the foundation underlying all our content. An evidence-based tool must derive conclusions based on the best available evidence. This can only occur if the evidence is consistently and systematically identified, selected, summarized, synthesized, and interpreted. Conclusions must also be adjusted as new evidence is added to the ever-growing evidence ecosystem.

The DynaMed editorial processes include the following steps:

To ensure that DynaMed provides the best available evidence, an extensive set of current literature is monitored daily. A state-of-the-art Systematic Literature Surveillance program continually monitors medical research as it is published, including journals, journal review services, review collections, guideline collections and other relevant sources.

To select the best available evidence, published research and clinical practice guidelines are screened for relevance and potential impact on clinical decision-making and patient care. Each relevant article is further assessed for validity relative to what is already known. The most valid articles are summarized and integrated into existing DynaMed content, which may trigger updates to overview statements and the overall outline structure.

Article selection is completed by a team of practicing physicians and methodologists with clinical expertise and/or training in scientific analysis. Relevance is determined primarily by results affecting clinical outcomes and clinical decision-making, and secondarily by consideration of popular or frequent clinical questions.

Every medical research paper identified and selected for inclusion in DynaMed undergoes an objective critical appraisal process involving consistently applied, systematic identification of potential sources of bias. A simple Level of Evidence rating system makes it easy for users to quickly understand the quality of the evidence being reported. Learn more about the editorial process.

Critical appraisal is completed by an editorial team that is rigorously trained in evidence-based medicine and critical appraisal of scientific/medical research.

The evidence is objectively reported using data from the original study publication, with a focus on clinical outcomes and absolute risk data when available. Levels of evidence are applied according to an explicit protocol. Clinicians review all content for validity and relevance at the point-of-care.

Objectively reporting the evidence for an individual study is necessary but insufficient for a comprehensive point-of-care reference. Understanding the best current evidence requires synthesizing multiple evidence reports. When evaluating multiple evidence reports, preference for inclusion and content organization is based on both clinical relevance and the quality of evidence.

When data of lesser quality is identified but does not add any substantially new or different information, this data is excluded. Lesser quality data is also removed when it is superseded by higher quality evidence. Evidence is also synthesized with clinical practice guidelines, and areas of inconsistency are presented. Clinicians review all content for validity and relevance at the point-of-care.

Multiple evidence reports are organized such that the overall conclusions quickly provide a synthesis of the best available evidence. Overviews (including the Overview and Recommendations, Testing Overview, and Management Overview) are based upon and linked directly to the supporting evidence.

The final step in evidence-based methodology is adjusting conclusions as new evidence is added to the ever-growing evidence ecosystem. This step is crucial because new evidence is published every day.

As soon as new evidence is evaluated using the six steps governing systematic processing, it is integrated into the appropriate DynaMed topic(s). This process allows immediate and complete access to the best evidence as it becomes available.

Editorial Processes

With new information coming out at such a staggering rate, it is virtually impossible for a clinician to keep up with all the newest evidence available daily. Our rigorous editorial process does the work for you by systematically monitoring and performing targeted searches across the thousands of journals in PubMed and the hundreds of clinical practice guidelines worldwide. Currently, over 100,000 evidence and guideline summaries are presented within DynaMed from over 3,500 journals and guideline organizations from across the globe.

To identify new evidence and guidance, the DynaMed Editorial Team monitors over 450 medical journals and guideline organizations publishing the most clinically-relevant research and clinical practice guidelines across 28 specialties. DynaMed’s team of practicing physicians and methodologists identify, objectively appraise, and summarize clinical trials and clinical practice guidelines that impact patient care.

The Systematic Literature Surveillance (SLS) Team supports DynaMed’s evidence-based methodology by continually monitoring medical research as it is published. To select the best available evidence, the SLS team works with over 50 medical specialists who screen published research and clinical practice guidelines for relevance and potential impact on clinical decision-making and patient care. In order to optimize its literature surveillance, DynaMed has partnered with McMaster University. DynaMed utilizes PubMed searching and filtering strategies from McMaster University’s Health Information Research Unit to identify studies and systematic reviews that are scientifically sound. In addition, DynaMed uses McMaster’s PLUS database, which provides ratings and comments on newly published articles from their global network of more than 8,000 frontline clinicians through the McMaster Online Rating of Evidence (MORE™) system.

As part of DynaMed’s evidence-based methodology, the Editorial Team continually monitors clinical practice guidelines as they are released by American and International guideline organizations. DynaMed’s guideline monitoring process includes a novel automated website surveillance for guideline organizations developed in partnership with McMaster University. Practicing physicians and methodologists in the Editorial Team critically appraise and summarize clinical practice guidelines for integration into DynaMed topics across 28 specialties.

Every study identified and selected for inclusion in DynaMed is rigorously and objectively appraised by an editorial team trained in evidence-based medicine. Each study undergoes a process of critical appraisal, which assesses the research to evaluate its trustworthiness, relevance, and clinical value. The appraisal process enables the application of research into practice.

DynaMed’s critical appraisal process involves systematic identification of potential sources of bias in order to objectively communicate the relevant clinical takeaway in the context of a study’s limitations. A team of practicing physicians and methodologists objectively appraise the most valid clinical trials and clinical practice guidelines and integrate the findings into DynaMed.

A simple Level of Evidence rating system makes it easy to quickly understand the quality of the evidence being reported in an evidence summary, and the primary rationale behind the rating.

  • Level 1 (likely reliable) evidence represents research results that address clinical outcomes and meet an extensive set of quality criteria that minimizes bias.
  • Level 2 (mid-level) evidence represents research results that address clinical outcomes and demonstrate some method of scientific investigation but do not meet the quality criteria to achieve Level 1.
  • Level 3 (lacking direct) evidence represents either of the following:
    • Reports that are not based on scientific analysis of clinical outcomes (e.g., case series, case reports, conclusions extrapolated indirectly from scientific studies)
    • Research results that do not address clinical outcomes, regardless of the scientific rigor

Read the full Levels of Evidence criteria.

In the Overview and Recommendations, DynaMed synthesizes evidence, clinical practice guidelines, and clinical expertise to provide recommendations to support clinical decision-making. The internationally-accepted Grading of Recommendations Assessment, Development and Evaluation (GRADE) system is used to classify synthesized recommendations as Strong or Weak.

  • Strong recommendations are used when, based on the available evidence, clinicians (without conflicts of interest) consistently have a high degree of confidence that the desirable consequences (health benefits, decreased costs and burdens) do, or do not, outweigh the undesirable consequences (harms, costs, burdens).
  • Weak recommendations are used when, based on the available evidence, clinicians believe that desirable and undesirable consequences are finely balanced, or appreciable uncertainty exists about the magnitude of expected consequences (benefits and harms). Weak recommendations are also used when the range of patient values and preferences suggests that informed patients are likely to make different choices.

DynaMed synthesized recommendations are determined with a systematic methodology:

  • Recommendations are initially drafted by clinical editors (including ≥ 1 with methodological expertise and ≥ 1 with content domain expertise) who understand the best current evidence for benefits and harms and relevant recommendations from clinical practice guidelines.
  • Recommendations are phrased to match the strength of the recommendation. Strong recommendations use “should do” or “should not do” language or phrasing that implies an expectation to perform (or not perform) the recommended action for most patients. Weak recommendations use “consider” or “suggested” language.
  • Recommendations are explicitly labeled as Strong recommendations or Weak recommendations when a qualified group has explicitly deliberated on making such a recommendation. Group deliberation may occur during guideline development. When group deliberation occurs through DynaMed-initiated groups:
    • Clinical questions will be formulated using the PICO (Population, Intervention, Comparison, Outcome) framework for all outcomes of interest that are specific to the recommendation to be developed.
    • Systematic searches will be conducted for any clinical questions where systematic searches were not already completed through the DynaMed content development process.
    • Evidence will be summarized for recommendation panel review including for each outcome the relative importance of the outcome, the estimated effects comparing intervention and comparison, the sample size, and overall quality rating for the body of evidence.
    • Recommendation panel members will be selected to include at least three members that together have sufficient clinical expertise for the subject(s) pertinent to the recommendation, methodological expertise for the evidence being considered, and experience with guideline development.
    • All recommendation panel members must disclose any potential conflicts of interest (professional, intellectual, and financial), and will not be included for the specific panel if a significant conflict exists for the recommendation in question.
    • Panel members will make Strong recommendations if and only if there is consistent agreement in a high confidence in the likelihood that desirable consequences outweigh undesirable consequences across the majority of expected patient values and preferences. Panel members will make Weak recommendations if there is limited confidence (or inconsistent assessment of dissenting opinions) that desirable consequences outweigh undesirable consequences across the majority of expected patient values and preferences. No recommendation is made if there is insufficient confidence to make a recommendation.
    • All steps in this process (including evidence summaries that are shared with the panel, and identification of panel members) will be transparent and accessible in support of the recommendation.
  • Recommendations are verified by ≥ 1 editor with methodological expertise, not involved in recommendation development, with explicit confirmation that Strong recommendations are adequately supported.
  • Recommendations are published only after consensus is established with agreement in phrasing and strength of recommendation by all editors.
  • If consensus cannot be reached then the recommendation can be published with a notation of “dissenting commentary” and the dissenting commentary is included in the topic details.
  • If recommendations are questioned during peer review or post-publication by a qualified individual, or re-evaluation is warranted based on new information identified through the Systematic Literature Surveillance program, the recommendation is subject to additional internal review

New topics in DynaMed are developed under the direction of a Deputy Editor and Topic Editor. There is a rigorous writing and review process that includes clinical experts and methodologists. The editorial team performs subject area-specific targeted queries across the over 5,000 journals indexed in PubMed. The topic development process leverages DynaMed’s methodology to identify and select the best available evidence which is then critically appraised.

DynaMed topics are developed to meet the information needs of clinicians and provide guidance to support effective and timely clinical decision-making. As a result, Topic and Section Editors in active clinical practice play an integral role in the topic development process and are listed on each topic they review.

The typical Topic/Section Editor is a physician who is active in academic and clinical practice and is board-certified in the appropriate specialty or devotes a portion of time caring for patients with the clinical entity in questions. Other experts (such as pharmacists) may be appropriate for selected topics to support a multidisciplinary approach to meeting the needs of clinicians. The Topic/Section Editor provides an initial clinical framework to guide the editorial team and then reviews the topic once it is developed to ensure that it is comprehensive, clinically relevant, and based on the best available evidence.

Prior to review by the Topic/Section Editor, the editorial team applies DynaMed’s evidence-based methodology to develop the topic, from performing a comprehensive literature search to integrating the best available evidence into the clinical framework. The content is reviewed for accuracy and application of critical appraisal by an editor as well as clinical relevance and usability at the point-of-care by a physician editor.

The final steps in the topic development process consist of an independent review by an EBM expert and review by the Deputy Editor. The EBM expert evaluates the Overview and Recommendations for adherence to sound recommendations and consistency.

DynaMed’s living topics are updated regularly as new evidence is identified through the SLS process. In addition, topics undergo updating and review by the editorial team. This process includes performing subject area-specific targeted queries across the over 5,000 journals indexed in PubMed. This ensures that DynaMed includes the most current evidence for use at the point of care.

Content Sources

DynaMed editors systematically monitor journals, journal review services, systematic review collections, guideline collections, governmental health agencies, drug information and other relevant sources. The result is more than 3,300 journals and over 200 guideline organizations cited within DynaMed across over 100,000 evidence and guideline summaries.

View Full List of Content Sources

View Full List of Guideline Organizations Monitored