Home > Requirements > Accreditation Requirements Cme Providers > Examples Of Compliance And Noncompliance

Examples of Compliance and Noncompliance

The ACCME is sharing examples of providers’ practices, programs, strategies or procedures – as determined from the three data sources reviewed during the accreditation process (self-study report, interview and performance-in-practice documentation review). The goal of these examples is to enable providers and CME stakeholders to learn from each other and to understand how the ACCME determines compliance and noncompliance with its requirements.

The  reader should understand that these are examples only, not prescribed practices, nor do they represent a list of options from which providers must choose. They are what some providers have chosen to do to fulfill the expectation of the ACCME Accreditation Criteria.

For your convenience, we have created a PDF of the Examples of Compliance and Noncompliance.

 

Criterion 1

The provider has a CME mission statement that includes all of the basic components (CME purpose, content areas, target audience, type of activities, expected results) with expected results articulated in terms of changes in competence, performance, or patient outcomes that will be the result of the program.

ACCME Note

The ACCME is looking for explicit information on the five components of the CME mission, in order to understand how the organization intends to change their learners' (competence and/or performance and/or patient outcomes) through an overall CME program oriented to a stated purpose. The manner in which the provider intends to achieve these expected results is described in terms of the content areas the CME will address, who the target audience of their educational efforts will be, and what types of activities they will pursue. Compliance is determined when all the components are mentioned and the expected results are 'articulated in terms of changes in competence, performance, or patient outcomes that will be the result of the program.'

Examples of Compliance:
  • Criterion 1: Example 1

    "The [Name deleted] School of Medicine CME program seeks to see beyond the traditional administrative and crediting functions of medical education, and challenges itself to become a catalyst, advocate for, and a provider of education that promotes change, development and improvement. Purpose: Our CME program sees its purpose as providing high-quality, evidence-based educational opportunities that are designed to advance physician competence, enhance practice performance, promote patient safety, and, where possible, improve patient outcomes in the populations served by the healthcare providers we educate. Target Audience: We seek to serve not only the educational needs of healthcare professionals (physicians, allied health personnel, basic scientists, and researchers) within our institution but the needs of local, regional, and national healthcare providers and colleagues as well. Types of Activities Provided: As a CME provider at an academic tertiary-care medical center, our educational activities include department-specific, interdepartmental and specialty-specific conferences, symposia and seminars. These events take the form of live courses, live regularly scheduled series, and live internet activities. The educational design, instructional method and learning format for each event is chosen to best serve the educational needs and learning objectives of the planned educational activity. Content: Included among our educational offerings are updates in clinical medicine and basic science research, reviews of current or best practice recommendations for clinical care, learning modules in quality improvement, procedural and communication skills development exercises, as well as professional development in bioethics, leadership and patient safety. Expected Results: We expect that they will report greater confidence in their approach to clinical problems or express their intent to change their behavior and apply newly acquired strategies in their practice. We expect that when observed our learners will demonstrate competence and the effective use of targeted skills. Finally, if evaluated within the setting of clinical practice, we expect performance parameters to show improvement or a favorable impact on targeted patient outcomes to be demonstrated."

  • Criterion 1: Example 2

    Excerpt from expected results section of the provider's mission statement: "The expected result of our educational activities is that participating physicians enhance their knowledge and skills in the subject area(s) offered, and apply the knowledge and skills to improve performance and patient outcomes in their practice settings."
Criterion 2

The provider incorporates into CME activities the educational needs (knowledge, competence, or performance) that underlie the professional practice gaps of their own learners.

ACCME Note

Provider identifies gaps between current practice or outcomes and desirable or achievable practice or outcomes (i.e., professional practice gaps). The provider deduces needs as the 'knowledge causes,' 'strategy causes,' or 'performance causes' of the professional practice gap(s). The key for compliance is to be able to show ACCME that planning included the identification of a professional practice gap from which needs were identified. A common theme in the noncompliance descriptions is that the ACCME could not find in the description any evidence that a professional practice gap was identified. Professional practice is not limited to clinical, patient care practice but can also include, for example, research practice and administrative practice.
 

Examples of Compliance:
  • Criterion 2: Example 1

    The provider identifies professional practice gaps from national data from peer-reviewed published literature, databases such as Cochrane and registries such as www.clinicaltrials.gov. The provider interviews recognized thought leaders in the content area related to a practice gap to review the data to determine the underlying needs that are relevant to the target audience and then develops CME activities to address these needs.

  • Criterion 2: Example 2

    After a broad subject category is chosen, the provider identifies professional practice gaps in the area through review of new practice guidelines, national data, professional society/college data, and government publications. Participants also complete pre-activity surveys to define their own practice gaps and identify their underlying needs related to the gaps. CME activities then focus to address these needs.

  • Criteron 2: Example 3

    Practice gaps are identified by reviewing reports from the CDC, the IOM, daily news clippings related to infectious disease, and from their membership input. The provider also conducts surveys of its membership to assess their needs as related to identified gaps.

  • Criterion 2: Example 4

    Examples in the self-study narrative describe several CME activities implemented to address a training gap in the use of digital mammography; the gap is based on a study published in the journal "Radiology."

  • Criterion 2: Example 5

    The provider developed and demonstrates the use of a "Gap Analysis" worksheet that identified for their physician learners Best Practice, Current Practice, Resulting Gaps, Gap Cause Deduced (Knowledge, Competence, Performance), Learning Objective, Outcome Indicated (Competence, Performance, Patient Outcomes), and Outcomes Questions.

  • Criterion 2: Example 12

    The provider has developed an activity proposal system that is required on an institutional basis for planners to document professional practice gaps and their underlying educational needs. The system is based on the planners asking themselves a series of questions such as “What patient problems or professional challenges is the target audience unable to meet?” and “Why are they unable to address the patient problems or challenges articulated above?” and “What evidence, data, or sources were consulted in the identification of the professional practice gaps?” The provider gave evidence in the performance-in-practice files that this process is routinely followed. For its Regularly Scheduled Series (RSS), the provider identifies the professional practice gaps on an annual basis, using a similar application process.

  • Criterion 2: Example 13

    Professional practice gaps and educational needs of learners are identified by the provider’s education committee. The committee reviews the results of board self-assessments, interviews with trustees, physician leaders, healthcare executives, and faculty. The practice gaps are converted into an educational agenda that drives learning objectives, content, and faculty, all of which reflect the educational needs of the learners. In one example, the provider identified as a professional practice gap the fact that board members and physicians are not prepared for changes that will result from healthcare reform because executives and management teams have not been proactive in board development to prepare for the changes. To address this, the provider designed a leadership conference to confer strategies for achieving optimal board performance.

  • Criterion 2: Example 14

    The provider utilizes a variety of data sources, which it categorizes as “subjective” and “objective,” that include feedback from learners, risk consultant staff, claims adjustors and underwriters, as well as claim report data, risk assessment outcomes data, literature review of peer-reviewed journals, practice guidelines and other sources. In most cases, the underlying educational needs of their learners are developed specific to the individual physician who is participating in their primary CME activity, which comprises 98% of their current CME program.

  • Criterion 2: Example 15

    The provider describes a comprehensive process that includes data from their members and resident self-assessments, national sources, literature searches, and experts serving on their committees. They describe and demonstrate a comprehensive process of establishing a gap and underlying educational need.

  • Criterion 2: Example 16

    The provider makes extensive use of its own claims data to assess professional practice gaps of its own learners. The provider performs claims analyses of physicians with “multiple”, “single”, “high”, or “trending” increases in claims. Through analysis of medical coding and other factors, the provider discerns underlying educational needs, which are then compared to the medical literature.

  • Criterion 2: Example 17

    The provider uses data from scientific literature, conference evaluations, and focus groups to determine the educational needs that underlie the professional practice gaps of the learners. The provider typically addresses educational needs that underlie gaps stemming from how to apply new developments in a variety of specialty and subspecialty fields to practice.

  • Criterion 2: Example 18

    Professional practice gaps are identified by the director who uses surveys of patients and physicians, data from peer-reviewed publications, direct interactions with physicians, and the introduction of new techniques or procedures. Educational needs of learners that underlie the professional gaps are identified through prior course surveys, expert opinion, recent data from public health sources, reported morbidity/mortality gaps, and evidence of misdiagnosis or mistreatment. The course planners develop objectives for the activity that support the identified need.

  • Criterion 2: Example 19

    In the activities reviewed, the provider describes population-specific health issues, mortality rates, and the difference between current and ideal practice. The self-study report describes that activity planners identify areas where improvement is needed by identifying prevalent and serious medical problems in its local region or state, from peer-reviewed literature, specific hospital data, etc., then ask what can be done to eliminate the gap. The provider then turns the educational needs into learner-centered, behavioral objectives.

  • Criterion 2: Example 20

    Practice gaps and needs are identified through staff and management discussions, data from other organizations with similar membership and audience, member opinion surveys, online member forums, activity evaluations and pre- and post-test scores, input from consultants and topic experts, and from government regulations with which physicians must comply. These gaps and needs are incorporated into their activities through learning objectives that articulate knowledge, competence, and/or performance expectations for the learner following participation in the activity.

  • Criterion 2: Example 21

    Gaps are identified through physician self-observation, observed performance in the patient care setting, referral patterns, quality data generated from the institution's affiliated hospitals, and national performance measures. Face-to-face meetings with the Activity Medical Director serve to articulate the gap and compare the current practice to improved practice guidelines. Gaps are validated through peer-reviewed journals, literature and medical databases, and expert opinion. An activity planning worksheet is designed to identify the gaps and link them to the needs, objectives, and outcomes of an activity.

  • Criterion 2: Example 22

    The provider uses its own strategic plan combined with population health data from local, regional, and national sources, research findings, news media, national and international peer-reviewed journals, Health People 2010-2020, the Centers for Medicare and Medicaid Services (CMS), and patient and health care provider surveys. This data is then used with information received from the provider’s institutional leaders and discussions with clinical and research Department Chairs, Chairs of Regularly Scheduled Series (RSS), CME Course Directors, the CME Advisory Board, as well as other community stakeholders to develop activities and educational interventions based on identified practice gaps of the provider’s learners.

  • Criterion 2: Example 23

    The provider reviews articles in over 470 peer-reviewed journals to determine if new information should lead to changes in its topic-based curriculum. The provider also identifies gaps and needs through the questions that learners ask and the topics that they search. Information from learners’ practice-based questions and search data is then discussed and translated by the provider’s editorial staff into new articles used for Journal Based CME activities.

Examples of Non-Compliance:
  • Criterion 2: Example 6

    The provider designs courses to assist learners to pass board review courses. However they do not provide evidence of how the board requirements are either a gap in physicians' professional practice or are linked to or derived from a gap.

  • Criterion 2: Example 7

    The provider provided specimens of information-gathering tools (surveys, evaluation forms, statistical data, and national trends) as evidence of professional practice gaps. These examples did not show that the provider identified knowledge, competence or performance educational needs that underlie any of these gaps.

  • Criterion 2: Example 8

    The information presented describes that the provider uses information gathered from past participant evaluations, follow-up surveys, and literature. The evidence does not, however, demonstrate that the provider links this information to professional practice gaps of their learners or identify the needs that underlie those gaps.

  • Criterion 2: Example 9

    The provider stated, "We offer CME that is federally or state-mandated for physician re-licensure. The fact that education is mandated indicates that the state or federal health agency has conducted an evaluation and determined there is a gap in professional performance or patient outcomes." However, the provider did not connect these mandates to professional practice gaps of the provider's learners or identify the needs that underlie those gaps.

  • Criterion 2: Example 10

    The provider describes in its self-study report the incorporation of educational needs (knowledge, competence, or performance) that underlie the professional practice gaps of their own learners. However, evidence presented for the activities reviewed did not demonstrate that this occurred consistently in either regularly scheduled series (RSS) or non-RSS activities. RSS comprised over 90% of the provider's total physician participants during its current term.

  • Criterion 2: Example 11

    The provider described in its self-study report a process to identify professional practice gaps by referencing national data regarding patient safety and medical ethics. However, the provider did not link the professional practice gaps to their learners nor articulate educational needs underlying professional practice gaps in terms of knowledge, competence, or performance in the planning process.

  • Criterion 2: Example 24

    The provider does not identify professional practice gaps. It identifies educational needs through post-activity surveys and requests from physicians, institutions, and healthcare professional groups. The provider referenced the National Healthcare Quality and Disparities Report which discusses "differences in health care quality and access associated with patient race, ethnicity, income, education, and place of residence," but did not relate this report to their learners' professional practice gaps.
Criterion 3

The provider generates activities/educational interventions that are designed to change competence, performance, or patient outcomes as described in its mission statement.

ACCME Note

This criterion is the implementation of Criterion 2 in the provider's overall program of CME. In the planning of its program of CME activities, the provider must attempt to change physicians' competence, performance, or patient outcomes, based on what was identified as needs (that underlie a professional practice gap). The ACCME's expectation is that the education will be designed to change learners' strategies (competence), or what learners actually do in practice (performance), or the impact on the patient or on healthcare (patient outcomes). The ACCME affirms that 'knowledge' is acceptable content for accredited CME. With respect to Criteria 3 and 11, even if the preponderance of a provider's activities is focused solely on changing knowledge, the provider must still show how these activities contribute to the overall program's efforts to change learners' competence, or performance or patient outcomes.
 

Examples of Compliance:
  • Criterion 3: Example 1

    Activities are designed to change competence through the use of case-based scenarios and an Audience Response System that poses questions about what the learners would do when presented with the case.

  • Criterion 3: Example 2

    Activities are designed to change performance (surgical skills lab utilizing models, cadavers) with the ultimate goal of improving patient outcomes.

  • Criterion 3: Example 3

    The provider designs activities that translate gaps and needs into educational activities intended to change competence, performance and patient outcomes. The provider's planning document juxtaposes identified gaps with a desired result and content to focus meeting a need of knowledge, competence, or performance.

  • Criterion 3: Example 4

    The provider provides examples of their activities designed to change knowledge, competence, and performance and provided faculty with tools and instructions on how to incorporate clinical cases into their course curriculum in an effort to move learners beyond changes in knowledge to changes in competence.

  • Criterion 3: Example 5

    The provider utilizes didactic lectures and an annual, case-based slide survey for practical skills training in clinical pediatric xxxxx.

  • Criterion 3: Example 6

    As part of its planning process, the provider ensures that each activity is designed to change physician strategies that can be applied to practice. The provider measures this outcome by including an evaluation component that asks, "Will you make changes in your practice as a result of this activity?" and "Please describe a specific change you will make to your practice."

  • Criterion 3: Example 7

    The provider directly links objectives to a single professional practice gap it identified as "mission critical" for its entire CME program. These objectives are consistent with, and support achievement of the CME mission to, "improve physician performance with regard to communicating with patients."

Examples of Non-Compliance:
  • Criterion 3: Example 8

    The provider described in its self-study report the generation of activities designed to change competence, performance, or patient outcomes as described in its mission statement. However, there was no evidence of the implementation of this process in the activities reviewed.

  • Criterion 3: Example 9

    The provider describes in its self-study report and presents evidence that its activities were solely designed to change knowledge. In addition, the provider did not demonstrate that it collected and analyzed data and information to assess the compliance of its program of regularly scheduled series (RSS) for Criterion 2. RSS comprise over 50% of the provider's total hours of instruction during its current term.

  • Criterion 3: Example 10

    Although the provider describes in its self-study report the generation of activities designed to change patterns of care and the application of new information, the examples presented in the self-study report and in the activities reviewed show evidence only of activities designed to change knowledge, not competence, performance, or patient outcomes.
Criterion 4

The provider generates activities/educational interventions around content that matches the learners' current or potential scope of professional activities.

ACCME Note

The provider demonstrates that the content of its CME activities are related to what the learners actually do, or may one day do, in their professional practice. The ACCME allows considerable latitude in this area for material that could be considered, for example, 'just-in-case' CME or theoretical, or 'discovery', presentations on new knowledge.

Examples of Compliance:
  • Criterion 4: Example 1

    The program's New Course Proposal includes a description of how the content of the activity is aligned with the targeted learners' current or potential scope of practice.

  • Criterion 4: Example 2

    The provider designs CME activities that match learners' scope of practice. One activity, given as an example, was designed based upon the request of a specialty society for a training course to enable its members to engage in evidence analysis and grading of recommendations for the purpose of writing clinical guidelines.

  • Criterion 4: Example 3

    The provider's Annual Meeting Program Committee plans its educational activities according to "tracks" which match the learners' current or potential scope of practice with the content.

  • Criterion 4: Example 4

    The provider reports in its self-study report that on average, over 91% of their learners are members of the specialty society for its clinical discipline and 93% are affiliated with a cleft/craniofacial team--which is what their CME program is about.

  • Criterion 4: Example 5

    The provider develops educational offerings to match the scope of practice of its broad constituency by using a content-coding system to assist program planners in their work and learners in organizing their learning based on areas of practice.

  • Criterion 4: Example 6

    The provider's process for ensuring that educational activities at its annual meeting are matched to learners' scope of practice includes having gap and needs data reviewed by an expert "Education Program Committee." The provider shared meeting minutes as evidence that this process was applied to their program's activities.

Examples of Non-Compliance:
  • Criterion 4: Example 7

    Description and evidence presented by the provider for its non-regularly scheduled series (RSS) activities demonstrate Compliance with Criterion 4. However, the provider did not demonstrate that it has collected and analyzed data and information to assess the Compliance of its program of RSS in this area. RSS comprise over 50% of the provider's total hours of instruction during its current term.
Criterion 5

The provider chooses educational formats for activities/interventions that are appropriate for the setting, objectives, and desired results of the activity.

ACCME Note

All activity formats (eg, didactic, small group, interactive, hands-on skills labs) are perfectly acceptable and must be chosen based on what the provider hopes to achieve with respect to change in competence, performance, and/or patient outcomes. The ACCME is looking for information to demonstrate that the choice of educational format took into account the setting, objectives, and desired results of the activity.
 

Examples of Compliance:
  • Criterion 5: Example 1

    The program designs activities in a number of formats, including, but not limited to, lectures, online programs, home study, small group and panel discussion, case study, simulation, and lab courses. Formats are based on participant feedback or the nature of the content to be delivered.

  • Criterion 5: Example 2

    The provider utilizes a variety of formats including live activities built on didactic presentations, enduring materials (some of which are delivered via the Internet), hands-on training, interactive sessions utilizing an audience response system, journal clubs, and moderated morbidity & mortality sessions. The provider has recently established a simulation suite.

  • Criterion 5: Example 3

    The provider described the different learning formats it uses, rationales for format choices, strategies used to focus on competence and performance, and the guidance provided by its Education Council in choosing formats.

  • Criterion 5: Example 4

    In the self-study report, the provider shared its rationale for choosing grand rounds-style speaker programs via podcast, stating, "a dynamic interview format gives listeners multiple perspectives to consider and apply to their own practices, making this a valuable educational intervention for oncology healthcare professionals."

  • Criterion 5: Example 5

    The provider utilizes its annual meeting to educate members which come from several disciplines. Formats include lectures, keynote addresses, platform presentations, panel discussions, case studies, ethics forums, debates, hands-on study sessions, and "eye openers" for beginning, intermediate, and advanced learners.

  • Criterion 5: Example 6

    In the survey interview, the provider explained that it uses scientific, didactic and case-based presentations in its annual meeting to address different aspects of its learners' professional practice.

  • Criterion 5: Example 7

    The provider gives planners a catalog of educational formats and their rationale for appropriate use in a particular type of CME activity.

Examples of Non-Compliance:
  • Criterion 5: Example 8

    Description and evidence presented by the provider for its non-regularly scheduled series (RSS) activities demonstrate Compliance with Criterion 5. However, the provider did not demonstrate that it has collected and analyzed data and information to assess the Compliance of its program of RSS in this area. RSS comprise over 50% of the provider's total hours of instruction during its current term.

  • Criterion 5: Example 9

    In its self-study report, the provider indicated that its journals are included in searchable databases, and in activities reviewed, the provider included a copy of a Table of Contents. The provider did not demonstrate with this information, or with any other information, that it chooses educational formats that are appropriate for the setting, objectives and desired results of its activities.
Criterion 6

The provider develops activities/educational interventions in the context of desirable physician attributes [eg, Institute of Medicine (IOM) competencies, Accreditation Council for Graduate Medical Education (ACGME) Competencies].

ACCME Note

The ACCME is looking for an active recognition of “desirable physician attributes” in the planning process (eg, “We have planned to do a set of activities that touch on professionalism and communications to address our patients’ concerns that they are not receiving complete discharge instructions – which is the identified professional practice gap.”). The simple labelling of an activity with a 'competency' is a start and provides the learner with information with which to choose an activity and potentially will be important for reporting purposes within Maintenance of Certification™.
 

Examples of Compliance:
  • Criterion 6: Example 1

    Activities are developed in terms of competencies (medical knowledge) and specialty-specific competencies.

  • Criterion 6: Example 2

    The provider designs activities/educational interventions in the context of a desirable physician attribute on both an individual activity level and a programmatic level.

  • Criterion 6: Example 3

    Activities developed based on medical knowledge, evidence-based practice, quality improvement, patient-centered care, interpersonal and communication skills.

  • Criterion 6: Example 4

    In its self-study report, the provider describes how the themes of its annual meeting are mapped to the Accreditation Council for Graduate Medical Education (ACGME) Competencies.

  • Criterion 6: Example 5

    In its self-study report and examples, the provider explains that its Education Committee solicits input from workshop section experts to plan each annual meeting session, where, "the experts will assess and assure that workshop content will include information on the various Accreditation Council for Graduate Medical Education (ACGME) Competencies: medical knowledge, patient care, systems-based practice and practice-based learning in particular. For example, the 3-day Perinatal Pathology Course addresses all of these relevant core competencies."

  • Criterion 6: Example 6

    In its activity files, the provider documented the link between desirable physician attributes [Institute of Medicine (IOM), American Board of Medical Specialties (ABMS), and Association of American Medical Colleges (AAMC) competencies] and the content related to those competencies.

  • Criterion 6: Example 7

    A table in the provider's self-study report maps Accreditation Council for Graduate Medical Education (ACGME) competencies against learning objectives from each regularly scheduled series (RSS).

  • Criterion 6: Example 8

    The provider uses a "new activity application" that lists the six Accreditation Council for Graduate Medical Education (ACGME) and American Board of Medical Specialties (ABMS) Competencies, and planners are required to indicate which competencies will be addressed.

Examples of Non-Compliance:
  • Criterion 6: Example 9

    Description and evidence presented by the provider for its non-regularly scheduled series (RSS) activities demonstrate Compliance with Criterion 6. However, the provider did not demonstrate that it has collected and analyzed data and information to assess the Compliance of its program of RSS in this area. RSS comprise over 50% of the provider's total hours of instruction during its current term.

  • Criterion 6: Example 10

    Evidence presented in the self-study report and in the activities selected for review does not demonstrate that the provider has developed activities in the context of desirable physician attributes.

  • Criterion 6: Example 11

    The provider described a list of documents, including standards of training, a suggested outline for technical courses, procedural skills, and technique proficiencies for a specific clinical discipline. However, the provider did not describe a link between these documents and the development of their program of CME. The evidence did not demonstrate that the provider developed activities in the context of desirable physician attributes.

  • Criterion 6: Example 12

    The provider's self-study report contained a paragraph on how the provider "will be collaborating with its medical board organization in the creation of educational modules that focus on professionalism." However, at the time of review for reaccreditation, the provider had not begun development of those modules.

  • Criterion 6: Example 13

    The provider described in its self-study report that its specialty board has developed an exam and has approved a variety of training courses. However, the provider did not demonstrate with this information, or with other information presented, that it develops activities in the context of desirable physician attributes.
Criterion 7

The provider develops activities/educational interventions independent of commercial interests. (SCS 1, 2, and 6).

ACCME Note

Accredited continuing medical education is always designed and presented in a manner whereby the accredited provider retains control of the content of CME. Providers are expected to ensure that activity planning and implementation is in the hands of the provider. The provider must obtain information from all those in control of content (eg, planners, teachers, and authors) so as to allow for the management and resolution of potential conflicts of interest. The provider must disclose to learners the relevant financial relationships of all those who control the content of CME.
 

Examples of Compliance:
  • Criterion 7: Example 1

    The information submitted describes a planning process that clearly delineates the roles and responsibilities of the provider, its planners and faculty. The provider ensures that there is no control or input from commercial interests. All planners' and teachers' conflicts of interest have been identified and resolved. Disclosure of relevant, or no, financial relationships to learners has occurred.

Examples of Non-Compliance:
  • Criterion 7: Example 2

    The provider's commercial support policy, presented as evidence in the self-study report, states "the provider may request suggestions for presenters or sources of possible presenters from a commercial supporter." This is inconsistent with the ACCME's requirement that a provider must ensure such decisions are made free of the control of a commercial interest. (SCS 1)

  • Criterion 7: Example 3

    The provider did not demonstrate that the following decisions were made independent of commercial interests: the identification of CME needs, the determination of educational objectives, the selection and presentation of content, the selection of persons and organizations that will be in a position to control the content of the CME, the selection of educational methods, and the evaluation of the activity. For example, the provider describes a planning process that involves planners and editors from ACCME-defined commercial interests. At the interview, the provider discussed how some activity topics come from individuals who work for an ACCME-defined commercial interest that shares office space with the provider. In addition, the provider indicated that it offered commercial supporters a courtesy review of its CME content in order to get supporter feedback.  (SCS 1)

  • Criterion 7: Example 4

    The provider did not demonstrate independence in its CME activity development. Evidence presented to the ACCME points to a planning process influenced by commercial interests. A potential speaker is identified as preferred for several attributes, including the fact that she may have a relationship with a commercial interest (the same company from which commercial support would be solicited).  (SCS 1) 

  • Criterion 7: Example 5

    In its self-study report, performance-in-practice files, and interview, the provider demonstrates that it identifies relevant financial relationships of faculty. However, the provider does not identify relevant financial relationships of planning committee members who are also involved in the content development of its CME activities. Without identifying this information from everyone who is in control of the content of the CME activities, the provider is unable to identify and resolve potential conflicts of interest. In addition, the provider shows that in its CME activities it discloses to learners, "only significant financial relationships between faculty speakers and commercial interests." This is not consistent with the ACCME's definition of "relevant financial relationships."  (SCS 2)

  • Criterion 7: Example 6

    For its test-item writing activities, the provider did not demonstrate the implementation of a mechanism to identify and resolve conflicts of interest for all persons in control of content, including for example, all faculty, reviewers, and CME committee members. Therefore, not all conflicts of interest could be identified or resolved prior to the activity.  (SCS 2)

  • Criterion 7: Example 7

    The provider describes a mechanism to resolve conflicts of interest, but the implementation of a mechanism to resolve conflicts of interest was not consistently documented in the activities reviewed. In addition, evidence was not presented to demonstrate that all individuals in control of CME content disclose relevant financial relationships to the provider.  (SCS 2)

  • Criterion 7: Example 8

    The provider did not have evidence of consistently implementing a mechanism to resolve conflicts of interest when persons in control of content reported relevant financial relationships. The only evidence provided was an attestation form signed by the speaker/planner/staff that stated, "I will ensure that any financial relationship that I have with a commercial interest will not effect the recommendations I make about clinical care." Attestation alone is not a mechanism to resolve conflicts of interest.  (SCS 2)

  • Criterion 7: Example 9

    Both in the self-study report and the activity files reviewed, the evidence demonstrates that all persons in control of content, including planners and staff, for example, do not consistently disclose the presence or absence of relevant financial relationships to the provider. In addition, the provider defines a commercial interest as "any proprietary entity producing health care goods or services," which is not consistent with the ACCME's current definition of a commercial interest. For these reasons, not all conflicts of interest could be identified or resolved.  (SCS 2)

  • Criterion 7: Example 10

    The evidence presented did not demonstrate that disclosure to learners included the presence or absence of relevant financial relationships for all persons in control of content, including, for example, journal editors or content reviewers.  (SCS 6)

  • Criterion 7: Example 11

    In both its live activities and enduring materials, the provider did not consistently disclose to learners the presence or absence of relevant financial relationships of all who control CME content, for example, planners. The provider did not consistently disclose to learners the presence of relevant financial relationships that had been shared with the provider. In addition, when disclosure occurred verbally at the activity, the provider did not consistently have evidence regarding what disclosures were made.  (SCS 6)

  • Criterion 7: Example 12

    The provider did not disclose to learners relevant financial relationships of all persons who control content. The provider's disclosure to learners did not include the name of the commercial interest with which the individual had a relevant financial relationship.  (SCS 6)

  • Criterion 7: Example 13

    The provider did not disclose to learners the relevant financial relationships of all persons who control content, including for example, all faculty, test-item writers and reviewers, and CME committee members. In addition, the provider did not consistently disclose commercial support.  (SCS 6)

  • Criterion 7: Example 14

    The provider uses a "Documentation Review Form for Verbal Disclosure," which lists the "name and role of individual discloser," but in the two activities presented, this is the name of a staffer and not the speaker. It is unclear from these forms what, exactly, is being disclosed to the learner.  (SCS 6)
Criterion 8

The provider appropriately manages commercial support (if applicable, SCS 3 of the ACCME Standards for Commercial SupportSM).

ACCME Note

If they chose to accept commercial support, providers are expected to solicit, accept, and use commercial support appropriately and in accord with the parameters of Stanrdard 3 of the ACCME Standards for Commercial Support. Even if the provider does not accept commercial support, the provider is still expected to have policies and procedures in place that govern how (if) they pay honoraria and reimburse expenses for those involved in the planning and presentation of their CME activities.
 

Examples of Compliance:
  • Criterion 8: Example 1

    The provider included a narrative description (supported by performance-in-practice materials) to evidence a comprehensive approach to ensuring the appropriate management of commercial support. The evidence included not only policies and forms that are used, but also examples of the processes being implemented within the provider's commercially-supported activities (eg, communications between the provider and commercial supporter, signed letters of agreement, accounting of activity-related expenditures.

Examples of Non-Compliance:
  • Criterion 8: Example 2

    For the activities reviewed that accepted commercial support, some written agreements were not present, and some written agreements did not include the signature of the commercial supporter.

  • Criterion 8: Example 3

    The provider did not consistently have all written agreements for commercial support signed by both the provider and commercial supporter prior to the activity.

  • Criterion 8: Example 4

    The evidence presented for the activities reviewed did not demonstrate that the provider paid honoraria and expenses in compliance with its own policies. In several instances, the provider indicated/assumed the honoraria policy was not applicable because commercial support was not accepted.

  • Criterion 8: Example 5

    For its commercially supported activities, the provider had several written agreements that were signed only by the commercial supporter, and not by the provider.
Criterion 9

The provider maintains a separation of promotion from education (SCS 4).

ACCME Note

Providers must ensure that their learners can participate in educational activities without seeing, reading or hearing promotional or marketing information from commercial interests. Further, accredited providers must ensure that the selling of advertising or exhibit space is a business transaction entirely separate from the acceptance of commercial support for accredited CME.
 

Examples of Compliance:
  • Criterion 9: Example 1

    In the self-study narrative, the provider described its processes for ensuring that promotional events at its annual meeting (eg, exhibit halls) are kept distinct and separate from CME activities by not only their location and time in the program schedule, but also in how these events are clearly described as "Promotional."

Examples of Non-Compliance:
  • Criterion 9: Example 2

    It was not clear that the provider appropriately maintained a separation of promotion from education. The provider's evidence demonstrated that meetings between learners and industry representatives in a commercial exhibitors' hall was considered part of their educational activity. Although the exhibits were not in the same room as lectures and video demonstrations, discussion with representatives of the commercial interest at the exhibit was considered by the provider to be part of the learner's CME experience.
Criterion 10

The provider actively promotes improvements in health care and NOT proprietary interests of a commercial interest (SCS 5).

ACCME Note

Providers are expected to ensure that their CME programs and activities advance the public interest without bias that would influence health professionals to overuse or misuse the products or services of a commercial interest.
 

Examples of Compliance:
  • Criterion 10: Example 1

    The provider demonstrated that all scientific content and clinical recommendations made within CME activities are reviewed by a three member "content review council." The criteria for that "clinical data and recommendations" address valid public health issues as defined by government resources like the Center for Medicare and Medicaid Services and the Agency for Healthcare Research and Quality.

Examples of Non-Compliance:
  • Criterion 10: Example 2

    The provider presented at least one activity that promoted proprietary interests of a commercial interest. An enduring material CME activity presented in its self-study report focused on the clinical trials of a single drug made by the commercial supporter. The activity did not present a discussion of other therapeutic options.
Criterion 11

The provider analyzes changes in learners (competence, performance, or patient outcomes) achieved as a result of the overall program's activities/educational interventions.

ACCME Note

The provider is asked to analyze the overall changes in competence, performance, or patient outcomes facilitated by their CME program using data and information from each CME activity. Providers who only measure change in knowledge in all their activities will not have any data on change in competence, performance, or patient outcomes to analyze.

Examples of Compliance:
  • Criterion 11: Example 1

    The provider collects data about the change in learners' competence by using audience response from case studies and skills workshops. The provider uses these data to draw conclusions about its CME program's activities impact on changing physicians' competence.

  • Criterion 11: Example 2

    The provider conducted an analysis of learner change data. In the activities reviewed, the provider asked learners if they will make a change in practice and for them to describe what they will do differently--an explicit expression of a change in their approach (competence). These data are then aggregated and analyzed by the CME Committee.

  • Criterion 11: Example 3

    The provider collects and analyzes data about learner change through participant evaluations that happen immediately after the event (by asking what the learners will do differently in their practices) and 12 weeks after the event (by asking what the learners have done). The provider uses these data to draw conclusions about the changes in learners' competence and performance that have been supported by its CME program's activities.

  • Criterion 11: Example 4

    The provider documented pre- and post-activity changes in competence related to multiple CME activities by using clinical case vignettes, and other mechanisms. The provider also analyzes overall impact of regularly scheduled series (RSS) on such patient outcomes as heart failure management, Vermont Oxford neonatal performance data, and the 5 million lives campaign. The provider measures and documents changes in physician competence through a number of mechanisms including pre- and post-tests and clinical case vignettes. In addition, they presented data on changes on a number of quality and safety gaps as a result of their RSS.

  • Criterion 11: Example 5

    The provider uses three month follow-up surveys at the activity level to measure change in competence and performance. The provider included summary data in the self-study report across all learners. In addition, the provider produces an annual executive summary of data by therapeutic area.

  • Criterion 11: Example 11

    The provider has moved from a focus on specific educational activities to a focus on disease-state initiatives. The provider’s overall program evaluation focused on the effectiveness of the disease state model in terms of how well this model worked to change physician competence. Performance on post-test clinical vignettes showed that changes in competence have occurred for every audience and for every progress indicator tested. The provider presented two summary reports, one for performance improvement initiatives and one for its annual symposium. The provider concluded that participating physicians enhance their knowledge and competence in the subject areas offered.

  • Criterion 11: Example 12

    The provider includes a standard two-part question on its on-site evaluation forms, asking participants, “Will this meeting impact your practice? (Yes/No)” and “If yes, please describe how will this meeting impact your practice?” The provider presented a summary and analysis of 2 years of evaluation data, concluding that, “Learners consistently communicate that important topics are presented and discussed, that they bring this new information back to their [practice] groups, and that they implement new practices.”

  • Criterion 11: Example 13

    The provider uses pre- and posttests and “3 month outcome” evaluations of every activity to assess learner changes, most specifically in performance. For example, the provider collected data on changes in competence on-site, then did a follow up “Outcomes Survey,” asking specific questions about changes in practice. The survey asked learners to reflect on the patient care they had delivered by asking, “Was there a significant change in a patient’s health status because of what you learned in this course? Please describe the changes.” The provider presented supporting data and analysis concluding that its activities are achieving their expected results.

  • Criterion 11: Example 14

    The provider analyzes a variety of data to evaluate changes in learners’ competence and performance, including self-reported learner feedback from on-site course evaluations and post-activity ‘outcomes’ evaluations that include intended and actual changes made to practice as a result of participation in the provider’s CME activities. The provider also collects, tabulates, and analyzes CME Proctor Results (observed changes in learner performance made during CME activities). Further, the provider utilizes editorial boards to evaluate the quality of the reviews conducted by participants in Manuscript Review CME activities.

  • Criterion 11: Example 15

    The provider uses quantitative and qualitative feedback from learners regarding changes in their knowledge, skills, competence, performance, and attitudes that have occurred as a result of the educational activity. The provider measures their activities’ impact by soliciting feedback about learners’ commitment to change and by asking learners to share plans for how they intend to make changes to their clinical practice to improve patient outcomes. For selected activities, the provider also conducts pre- and 3-6 months post-activity surveys to measure changes in the utilization of new tools, processes, and procedures in learners' clinical practices. The provider conducts surveys of learners to determine their knowledge of clinical practice guidelines as well as the extent to which they have incorporated the guidelines into their practices. The provider analyzes trends from the End Stage Renal Disease (ESRD) Clinical Performance Measures (CPM) Project from the Centers for Medicare and Medicaid Services (CMS) to assess changes (improvements) in patient outcomes, reflecting increased learner competence and clinical performance.

  • Criterion 11: Example 16

    The provider analyzes changes in learner’s competence and performance achieved through its Risk Assessment CME Activity, which comprises 98% of its CME program. Changes in learner risk exposure are measured using a six-month follow-up visit where specific practice changes that were discussed in the activity are assessed. The provider includes a quantitative analysis of change across the aggregate of these activities. For its other activities, which are a small part of its CME program, the provider measures changes in physician competence and presents a summary and analysis of the data.

  • Criterion 11: Example 17

    The provider uses pre- and posttests to measure changes in competence and measures for changes in performance using post-activity focus groups of participants. The provider describes a comprehensive analysis of its evaluation data and summarizes achievements in changing learner competence and performance.

  • Criterion 11: Example 18

    The provider evaluates its activities via a variety of methods including changes in medical malpractice claims data, pre- and posttests administered at the time of the activity, data on participants’ intent to change practice, post-activity surveys, and onsite reviews where changes in practice are demonstrated. The provider performs an analysis based on a review of its data, and reports changes in competence and performance that inform its ongoing educational planning.

  • Criterion 11: Example 19

    The provider described that surveys conducted after its CME courses show that 92% of participants demonstrate increased knowledge of the subject and 75% of the participants attest to a “commitment to change” by describing the specific practice-based changes that they will make as a result of the course. The provider also collected data indicating a 15% reduction in GI bleeds in transplant patients and an 80% reduction in thromboembolic events. In its analysis, the provider concluded that, taken together, these data show the effectiveness of its educational activities in promoting learner change that impacts patient outcomes.

Examples of Non-Compliance:
  • Criterion 11: Example 6

    The provider evaluated learner satisfaction but not changes in competence, performance, or patient outcomes. In some activities, the provider asks "if" the activity will enhance professional effectiveness, which does not allow for the explicit expression of "what" will change. In addition, the provider did not collect or review any other information about changes in learners' competence, performance, or in patient outcomes. The provider, therefore, did not have data to conduct an analysis of changes in learners' competence, performance, or changes in patient outcomes that resulted from the provider's program's activities.

  • Criterion 11: Example 7

    The provider did not conduct analysis of learner change data. The provider only measures changes in learners' knowledge in the evaluation tool utilized. In addition, there is no analysis of the changes in learner competence, performance or patient outcomes across the Program.

  • Criterion 11: Example 8

    While the provider evaluates its individual activities for changes in learners' competence or performance, the provider did not use these or any other change data in an overall analysis of the changes (physician competence, performance, or patient outcomes) that resulted from the provider's overall program's activities.

  • Criterion 11: Example 9

    In its self-study report and the activities reviewed, the provider evaluates learner satisfaction and not changes in competence, performance, or patient outcomes. Therefore, the provider did not have data to conduct an analysis of change in learners' competence, performance, or patient outcomes. [The provider has started to include a question on its evaluation form asking learners to list a specific change they will make in practice; this may be one way to collect data which will allow for analysis of learner change in the future.]

  • Criterion 11: Example 10

    The evidence presented in the self-study report and in the activities reviewed demonstrated that the provider evaluated learner satisfaction and whether learning objectives were met. The provider did not present data related to, or an analysis of, changes in physician competence, performance, and/or patient outcomes.
Criterion 12

The provider gathers data or information and conducts a program-based analysis on the degree to which the CME mission of the provider has been met through the conduct of CME activities/educational interventions. 

ACCME Note

The provider is asked to integrate C11 information with a broader view of the CME program and organization – to determine the program's success at meeting all components of its own CME mission as described in C1 (i,e., purpose, content areas, target audience, type of activities, and expected results). Providers that review only activity measures of change (expected results) without looking at the other components of the mission will not be found in compliance by the ACCME. Providers should consider that there are five components to an ACCME-defined mission statement and therefore there are at least five components to an ACCME-defined "program-based analysis on the degree to which the CME mission of the provider has been met.” There are clear relationships between C11, C12, and C13-15 which relate to improvement plans based on this program-based analysis.
 

Examples of Compliance:
  • Criterion 12: Example 1

    The provider conducted an overall program evaluation and concluded that all components of its mission statement were being met. The provider demonstrated that it has fulfilled its purpose of enhancing professional development, achieved learning objectives and changed physicians' competence, reached its target audience, provided CME in the content areas and with the types of activities stated in its mission statement.

  • Criterion 12: Example 2

    In reviewing the extent to which its CME program addressed the five components of its mission statement, the provider recognized that it had been designing activities to change learner knowledge, but not competence, performance, and/or patient outcomes (as required by Criterion 1, "Expected Results" and Criterion 3). It also found that it did not know if the activities designed to change knowledge contributed to changing competence or performance or patient outcomes as stipulated in its mission statement. In its self-study narrative and survey interview, the provider acknowledged this issue and articulated changes it would make to its planning processes to ensure that activities are designed to change learners' competence, performance, and/or patient outcomes.

  • Criterion 12: Example 3

    In its self-study report, the provider concluded that it had fulfilled its purpose, "to improve interdisciplinary care of persons with clefts and craniofacial anomalies;" had been successful in, "making professionals aware of new clinical and research information," and achieved a portion of its expected results of, "updating the knowledge and skills of physicians in both their own disciplines and in the other disciplines with which they provided care."

  • Criterion 12: Example 4

    The provider conducted an overall program evaluation and concluded that all components of its mission statement were being met. The provider demonstrated that it has fulfilled its purpose of enhancing professional development, achieved learning objectives and changed physicians' competence, reached its target audience, provided CME in the content areas and with the types of activities stated in its mission statement.

  • Criterion 12: Example 5

    The provider excerpted minutes from the annual meeting of its CE Committee in which the group evaluated the degree to which the organization had met each of the elements of its mission statement.

Examples of Non-Compliance:
  • Criterion 12: Example 6

    Data and information was not presented to demonstrate that the provider conducts a program-based analysis on the degree to which the CME mission has been met. The provider did not address the degree to which the program has achieved expected results as articulated in terms of competence, performance, or patient outcomes.

  • Criterion 12: Example 7

    Data and information presented did not relate to the CME mission. The provider's method for overall program analysis is a survey of their membership as to their satisfaction with the CME program. Information was provided about the organization's operational function--but this was not related to an evaluation of the extent to which the CME mission was fulfilled. For example, the organization does a "SWOT" (strengths-weaknesses-opportunities-threats) analysis and has a comprehensive strategic planning process associated with its annual retreat, but this does not address the CME mission or CME program.

  • Criterion 12: Example 8

    The provider's program-based analysis on the degree to which the CME mission has been met did not address expected results based on data that would constitute an analysis of changes in learners' competence, performance or patient outcomes.

  • Criterion 12: Example 9

    The provider did not present, and was unable to describe, any data from which it could determine whether or not it was meeting its purpose as described in the organization's mission statement.

  • Criterion 12: Example 10

    The evidence provided fails to demonstrate that the provider presently performs program analyses to evaluate how it is meeting its mission. In particular, the provider has not provided information or analysis on the extent to which it has achieved the expected results outlined in its mission statement.
Criterion 13

The provider identifies, plans and implements the needed or desired changes in the overall program (eg, planners, teachers, infrastructure, methods, resources, facilities, interventions) that are required to improve on ability to meet the CME mission.

ACCME Note

The provider identifies its own 'professional practice gaps' in terms of its performance as a CME provider - and creates a strategic plan for organizational improvement, based on the insights from C11 and 12.
 

Examples of Compliance:
  • Criterion 13: Example 1

    The provider has described several changes including an electronic disclosure and conflict of interest resolution system, a moderator evaluation process, expanded use of the audience response system, a restructuring of the skills model lab, and additional improvements in the evaluation system.

  • Criterion 13: Example 2

    In its self-study narrative, the provider recognized that it failed to measure changes in learners' competence, performance, and/or patient outcomes in its CME activities, making it impossible to analyze changes in competence, performance, and/or patient outcomes as required by Criterion 11. The provider described in its self-study narrative and survey interview an "improvement plan" that it created to rectify the issue in future activity planning and evaluation.

  • Criterion 13: Example 3

    In its self-study report, the provider describes an improvement plan based on reports that determine if they are maintaining attendee levels from primary disciplines in team care, activity evaluations that determine whether to continue with certain educational formats and exhibits, debriefing meetings after the annual meeting to identify process or logistics areas for improvement, and membership surveys to determine why non-participants do not attend the annual meeting.

  • Criterion 13: Example 4

    The provider describes the identification, planning, and implementation of a list of changes designed to better achieve its mission. Examples include: improved evaluation forms that capture change data; post-meeting surveys; accessing quality improvement databases; outreach efforts to general pathologists to include them in their target audience for CME activities.

  • Criterion 13: Example 5

    The provider described the following five changes it will implement as a result of its analysis on how it was meeting its CME mission: (1) purchasing an association management system to track learners; (2) using new software to connect with learners before and after CME activities; (3) identification of a "best practice" planning document; (4) having the CME/MOC Committee review the overall CME program to include how well the organization was achieving its CME mission; and (5) changing the evaluation rating scale.

Examples of Non-Compliance:
  • Criterion 13: Example 6

    The provider did not present information to demonstrate that they have identified changes required to improve their ability to meet the CME mission.

  • Criterion 13: Example 7

    The provider indicated that upon review of its effectiveness, no changes or improvements were required. However, noncompliance findings in other Criteria indicate there are, in fact, improvements that could be made.
Criterion 14

The provider demonstrates that identified program changes or improvements, that are required to improve on the provider's ability to meet the CME mission, are underway or completed.

ACCME Note

The provider demonstrates the implementation of the change plans described in Criterion 13.
 

Examples of Compliance:
  • Criterion 14: Example 1

    The provider has demonstrated a number of changes that are underway or have already been completed. These changes include improved pre-activity questionnaires, inclusion of Q&A sessions, expansion of master's classes, and a searchable web page of scholarly works.

  • Criterion 14: Example 2

    The provider has demonstrated the institution of a learning management system and provided results of the beta testing of the system.

  • Criterion 14: Example 3

    The provider states that to expand visibility, it has introduced online programming that will be available soon. The provider also developed new standards for interdisciplinary team care and conducted a pilot program with five teams. Two of the teams have already reported some changes in patient outcomes.

  • Criterion 14: Example 4

    The provider included a "CE Advisory Council Educational Effectiveness Checklist" in a few of its activity files in support of Criterion 14. This reflects a change made at the programmatic level that is demonstrated in its activity files. CE Council members are asked to observe and evaluate an activity in real time and provide feedback to the provider.

Examples of Non-Compliance:
  • Criterion 14: Example 5

    The provider did not present evidence to demonstrate that identified changes or improvements required to improve the provider's ability to meet the CME mission are underway or completed.

  • Criterion 14: Example 6

    Program improvements to help the provider better meet its CME mission have not been identified, so changes are not underway or completed.
Criterion 15

The provider demonstrates that the impacts of program improvements, that are required to improve on the provider's ability to meet the CME mission, are measured.

ACCME Note

This criterion is analogous to C12 – and is focused on the results of the specific changes implemented in C14. What impact have the improvements that were implemented in C14 had on the ability of the provider to meet its mission?
 

Examples of Compliance:
  • Criterion 15: Example 1

    The provider described a number of ways that it is measuring the impact of the improvements that it identified through its program-based analysis. Some of these improvements were also demonstrated in the activity files, eg, revised planning forms, evidence of disclosure to learners.

  • Criterion 15: Example 2

    As a result of improvements the provider implemented to its evaluation and assessment practices, the provider has observed a higher rate of completion for online surveys and more comprehensive data to track changes in learners against professional practice gaps.

  • Criterion 15: Example 3

    The provider determines the impact of its program improvements through activity and learner evaluations, comprehensive activity summaries, committee analysis, external data and member surveys.

Examples of Non-Compliance:
  • Criterion 15: Example 4

    The provider did not demonstrate that it measured the impact of program improvements, but indicates that information will be gathered, in the future, to determine if changes made have fulfilled the intended purpose.

  • Criterion 15: Example 5

    During the interview, the provider stated that it has only just begun to work on the elements they are trying to improve. At the time of its accreditation review, the provider had not yet measured the impact of any of these changes.

  • Criterion 15: Example 6

    The provider did not demonstrate that it measured the impact of program improvements, but indicated that the (future) measuring instrument will be a survey of its readership overseen by its editorial leadership.
Criterion 16

The provider operates in a manner that integrates CME into the process for improving professional practice.

ACCME Note

The onus is on the provider to show that they have inserted CME into the processes to improve professional practice. Providers need to show that their CME program has a presence, influence, or contributory role in practice improvement. The provider goes beyond activity planning to show that CME is used as one of the tools to improve professional practice. C16 can also be about the use of CME in facilitating systems based quality improvement activities if the quality improvement activity is about changing professional practice.
 

Examples of Compliance:
  • Criterion 16: Example 1

    The provider's organizational structure has positioned CME to be a change agent within the organization. CME leaders of the institution have been strategically placed within key organizational groups that have as goals and work initiatives to improve physician's professional practice. For example, CME is integrated with case management, performance improvement and patient services. The provider has created a link from CME to all professional practice improvement activities through the nesting of CME within the organizational structure of all committees that address professional practice issues and representation on all primary clinical and medical components of the provider's institution.

  • Criterion 16: Example 2

    The provider's integration of CME into the process of improving professional practice involves identifying and addressing educational needs from practice and point of care setting and then using educational methods such as simulation to link teaching points to that practice and point of care setting. Such topics include adherence to evidence-based guidelines for practice, disclosure of adverse outcomes to patients and families, and patient safety.

  • Criterion 16: Example 3

    The provider develops practice guidelines and evidence-based clinical position statements, and produces CME activities to educate participants on these guidelines and recommendations. The provider has piloted and is implementing a comprehensive strategy for quality improvement within a specific clinical discipline, and has integrated CME into this strategy. The provider promotes efficient medical practice management as the cornerstone for the delivery of high quality patient care and offers a wide range of practice management education to help improve physicians' professional practice.

  • Criterion 16: Example 4

    The priorities of a CME provider within the U.S. military are set by federal government oversight. As a result, their CME program is focused on improving the professional practice of the physicians providing care to military personnel and their families. An example given was that the military relies on the CME provider to help ensure that forward surgical training teams deployed to combat environments are prepared for such duties. In addition, when government reports highlighted a need to improve the trauma care provided to soldiers in wartime settings, the provider's CME program was utilized to help make these improvements in professional practice.

  • Criterion 16: Example 5

    The provider has utilized CME to improve professional practice and patient outcomes within the institution. The CME Committee has representation from the Practice Improvement teams in the hospital. These teams, using internal quality improvement data, have developed CME activities to improve performance in areas such as MRSA surveillance and screening, surgical site-marking, "door to balloon" timing for angioplasty, use of guidelines related to dosing, as well as monitoring of glucose control in hospitalized patients.

  • Criterion 16: Example 6

    The provider's leadership has integrated the CME unit into the organizational structure in order to position CME as a change agent to help improve the professional practice of its physicians. For example, working in concert with quality improvement, the education department targets physician behaviors that negatively impact professional practice. After determining the practice problem, the education department develops and then conducts follow-up to assess practice improvements. When the expected level of improvement in professional practice is not achieved, the provider has implemented additional education. As another example, the provider is included in broad efforts to improve professional practice. After integrating new cardiac services to the hospital, the provider developed a weekly heart center performance-improvement conference to create a forum for continuous practice-based improvement related to cardiac care.

  • Criterion 16: Example 10

    The provider develops curricula designed to "improve professional practice on a broad scale," which is in keeping with their mission statement.  The goal of their signature CME activity is to raise awareness and drive implementation of national, specialty-specific practice guidelines for managing acute coronary syndromes.  In addition, they utilize expert leaders who serve on national guideline committees and who are known for developing professional practice standards in their fields.

  • Criterion 16: Example 11

    The provider gave several examples of CME activities, including a self-directed performance improvement platform that allows participants to access a variety of learning formats, tools, and best practice guidelines specific to the optimal care and management of patients with irritable bowel syndrome and constipation.

  • Criterion 16: Example 12

    The  provider has developed performance improvement  activities for venous thromboembolism and reduction mammoplasty based on evidence-based clinical practice guidelines for reduction mammoplasty that it has developed. In addition, the provider develops self-assessment tools that assist members to improve and modify their practice.

  • Criterion 16: Example 13

    The  provider works with its Risk Management department on the development of its regularly scheduled series activities as part of the solution for the prevention of sentinel events.  Other examples include performance improvement activities in six clinical areas, including ADHD, osteoporosis, and improving outcomes of acute coronary syndrome.

     

  • Criterion 16: Example 14

    The provider works with faculty and activity planners to adopt educational practices that support physician professional development and improve professional practice. For example, performance improvement activities were developed to improve chronic disease management, childhood immunizations, mammography, and the secondary prevention of coronary artery disease. These activities utilized the principles of Six Sigma and identified deficiencies and established strategies for improvements. An ambulatory care quality improvement activity led to improved care through the implementation of flow sheets and other process changes.

     

  • Criterion 16: Example 15

    The provider develops and publishes consensus statements and position papers, along with exercise prescription guidelines and other standards recognized across the field of sports medicine.  These standards are incorporated into and disseminated through the accredited education they produce.

     

  • Criterion 16: Example 16

    The provider incorporates American Board of Internal Medicine self-improvement modules (in support of ABMS Maintenance of Certification) into its annual symposia.

     

  • Criterion 16: Example 17

    The provider’s Board and staff collaborate with educational partners, including rural hospitals , clinics, and independent physician offices in the region, as well as national organizations, to obtain and use information from their quality improvement and quality assurance departments to identify practice issues that become the basis for CME designed to disseminate research findings that support evidence-based practices related, for example, to the implementation of electronic health records systems, HIPAA education, and tobacco cessation. In addition, the provider sponsors a performance improvement project in chronic obstructive pulmonary disease for rural primary care providers, offering an expert consult for physician participants to address difficult cases in the care of patients with chronic obstructive pulmonary disease and other specific case questions related to their practice.

     

  • Criterion 16: Example 18

    The provider’s relationship with its state medical board gives the organization the opportunity to serve as content experts/educational consultants on a variety of initiatives.  The staff of the provider sits on the Maintenance of Licensure (MOL) pilot state committee to ensure that CME is an integral part of MOL for physicians in its state.

     

  • Criterion 16: Example 19

    The provider established "Practice Improvement Education” as a means to link the organization's risk management, quality and safety and surgical safety units. The program uses quality data based on national benchmarks and institutional goals to address improving physician professional practice. The provider presented examples in the areas of pain management, stroke measures, and physician communication and bedside manner.

     

  • Criterion 16: Example 20

    The provider has developed practice improvement projects, which are CME activities eligible for Part IV of the maintenance of certification (MOC) requirements for its specialty board.  Additional opportunities for the integration of CME into the process for improving professional practice are created through overlapping membership among its various committees, including Education, Quality, Practice, Policy and Research.

     

  • Criterion 16: Example 21

    The provider performs practice risk assessments for its learners on request and also invites learners to participate in risk management telephone consultation.  Specific CME activities are suggested for the learner as a result of these assessments.  The provider also reports that one of their related activities has prompted many physician learners to seek support around the issue of electronic medical records from the provider as a result of participation.

     

  • Criterion 16: Example 22

    The provider integrates CME into the process for improving professional practice through direct practice assessment and measurement of data from within its own institution, the local community, and its statewide partnership with medical schools.  The provider described several examples of CME activities designed to support practice and quality improvement issues, such as patient adherence, falls in the elderly, inter-professional care of chronic obstructive pulmonary disease patients, and improving geriatric outcomes in primary care.

     

  • Criterion 16: Example 23

    The provider’s CME program is closely tied to performance improvement initiatives in the organization. For example, a recent hospital performance improvement and quality initiative was developed to reduce readmission rates for pneumonia and heart failure patients. Chart audits are being incorporated into a database that will be used for Joint Commission Ongoing Professional Practice Evaluation (OPPE) and performance improvement. This model will be refined and provide a model for future performance improvement activities. In addition, analysis of CME events is performed to determine the impact of each event on physician practice and to measure the multiyear trends in the number of activities addressing (1) changing and sustaining physician competence, (2) clinical performance, and (3) optimizing patient outcomes.

     

  • Criterion 16: Example 24

    The provider integrates CME into the practice improvement process by participating on committees in the institution that focus on quality, like the Nursing Magnet Team, for example, and developing CME in response to identified issues.  In addition, a CME liaison is assigned to each area of key therapeutic importance across the institution for the purpose of addressing practice gaps that are identified.  An example of integration was the online sepsis education program that was identified to improve practice around sepsis.

     

  • Criterion 16: Example 25

    The provider coordinated a system-wide initiative on making a "just culture," fostering networking among sites to share common concerns and interests. The provider conducted CME activities as part of a multi-year collaborative for long term care facilities to help improve common quality improvement issues and develop teams to address them. Live and enduring material CME activities were conducted as part of a coordinated effort for the implementation and meaningful use of an electronic health record, for compliance with patient care guidelines, for improving immunization rates, and for advancement of the "medical home " model.

     

  • Criterion 16: Example 26

    The provider has created four targeted practice improvement modules for their members to use in the ABMS Maintenance of Certification process: carpal tunnel syndrome, metacarpal fractures, arthritis, and flexor tendon.
     

Examples of Non-Compliance:
  • Criterion 16: Example 7

    In its self-study report and supplemental materials, the provider describes that its CME planning and design processes are, "designed to help facilitate changes in physician performance." The provider did not demonstrate with this information, or in other descriptions and examples, that it operates in a manner that integrates CME into the process for improving professional practice.

  • Criterion 16: Example 8

    The provider presented an example of one activity that integrates CME into the process for improving professional practice, but did not demonstrate that the CME program operates in a manner that integrates CME into the process for improving professional practice.

  • Criterion 16: Example 9

    The provider describes integrating CME into point of care settings using a number of different educational methods. However, the provider did not demonstrate with this information, or with other information presented, how CME is integrated into the process for improving professional practice.
Criterion 17

The provider utilizes non-education strategies to enhance change as an adjunct to its activities/educational interventions (e.g., reminders, patient feedback).

ACCME Note

The ACCME is looking for evidence of the use of strategies such as, but not limited to, rewards, process redesign, peer review, audit feedback, monitoring, reminders as tools to enhance, or facilitate, change. Some providers are concerned that some of these may be considered 'educational' as they potentially change what people 'know' or because they inform learners (eg, “It may be time for you to call back your patients with…”). In C17, the ACCME is looking for tactics that go beyond the educational activity or intervention. Essentially, the ACCME is looking for providers to be broadening the range of tools they use to facilitate change. We are providing specific examples of what the ACCME has been accepting as 'non educational strategies'.
 

Examples of Compliance:
  • Criterion 17: Example 1

    The provider has used adjunctive strategies such as physician reminders, safety flip charts, order sheets with parameters of care, a pain-scale assessment tool, and progress note templates to enhance the expected results of its CME activities.

  • Criterion 17: Example 2

    The provider utilized several adjuncts to health center based CME, including the use of a "SWOT" (strengths-weaknesses-opportunities-threats) analysis in a non-CME strategic planning session to improve colorectal cancer screening and the creation and distribution of a diabetes flow sheet used in ambulatory medical records. The provider also facilitated team-building for the entire health care staff.

  • Criterion 17: Example 3

    The provider supplements its CME activities with updates and reminders on aspects of patient care, post-conference e-mails, electronic newsletters, patient education materials, and a mechanism for peer review of clinical practice.

  • Criterion 17: Example 4

    For its diabetes care activities, the provider used a comprehensive care guide for health care professionals with tools such as standing orders, care flow sheets, self-management checklists, eye exam reports, foot exam forms, clinical guidelines, and practical tools for patients on diet and exercise.

  • Criterion 17: Example 5

    To support changes in physicians' approaches to overweight and obese patients, the provider developed a wall chart, a poster featuring a body mass index (BMI) table, links to online and downloadable BMI calculators for hand-held devices. A range of activities were developed in support the patient-focused National Colorectal Cancer Awareness Month. At an annual meeting, the provider also developed a laminated reference card with Center for Disease Control (CDC) infection control guidelines to support infection-control practices of attendees.

  • Criterion 17: Example 6

    The provider uses non-educational strategies to enhance change, including the distribution and use of an interactive DVD on sexual assault, algorithms on the exam and workup of hip and shoulder trauma, a quality improvement assessment tool for vaccination optimization, tip cards for suicide prevention, patient surveys to facilitate appropriate medical and social support services, and a handbook with resources available at the medical center.

  • Criterion 17: Example 7

    The provider uses hard copy reminders and pocket cards distributed in handouts for some programs. At some activities, they also distribute flash-memory thumb drives containing reminders and pocket cards that can be printed. Practice guidelines that are discussed during the activity are uploaded into the electronic health record for immediate use by physicians.

  • Criterion 17: Example 8

    The provider uses registry reports, individual medical record (IMR) alerts, clinical decision support tools and algorithms, patient education resources, standardized clinical protocols, standing orders, and shorthand documentation templates.

  • Criterion 17: Example 12

    The provider develops tools such as facial diagrams to assist surgeons in communicating realistic expectations to patients, a resource guide as a supplement to patient safety activities, a resource on informed consent in plastic surgery, patient handouts on venous thromboembolism prevention for surgeons as part of the patient safety series, patient education material that surgeons may use on their own websites (in English and Spanish) for common procedures. In addition, the provider launched a public portal for information on plastic surgery on the Internet.  The provider also has a monthly column, in print and online, dedicated to medical coding to supplement information covered in CME activities on coding and reimbursement.

     

  • Criterion 17: Example 13

    The provider offers a Course Companion booklet which includes chapters from a related textbook.  These resources refer back to key teaching points covered in several courses.  In addition, the provider sends an email reminder to participants post-activity which contains a link to a Surgical Reference Guide to reinforce learning and enhance change.

     

  • Criterion 17: Example 14

    The provider has utilized Facebook to promote pearls from its activities.  The provider has also created over 400 “Let’s talk about…” handouts on various topics for patient and family education.  Other examples include ready reference cards, screen savers, and newsletters.

     

  • Criterion 17: Example 15

    The provider has utilized several non-education strategies as an adjunct to its activities to enhance change, including learner polls asking questions related to CME activities designed to stimulate further thought and discussion and an electronic newsletter that supports linking the education to practice.

     

  • Criterion 17: Example 16

    The provider utilizes non-education strategies to enhance change as an adjunct to its activities, including updates and newly released rules and regulations sent to its participants by e-mail and by electronic and mailed newsletters. In addition, the provider has developed a readiness assessment, from which the provider has created a development plan for healthcare boards, physician leaders, and their senior management to implement in their organizations.

     

  • Criterion 17: Example 17

    The provider utilizes non-educational strategies such as booklets on CPT coding , immunization DVDs, policy statements and most recently a Medication Management Manual.
     

  • Criterion 17: Example 18

    The provider has utilized a variety of non-educational strategies including laminated pre-brief posters to augment customer relationship management (CRM) training skills, a patient-friendly medication administration record to verify medication prior to administration, and evidence-based order sets.

     

  • Criterion 17: Example 19

    The provider utilized non-education strategies to enhance change, including online forums to enhance communication between practicing physicians and experts in the field and a quarterly newsletter that includes additional materials for learners.

     

  • Criterion 17: Example 20

    The provider’s website includes a library of meeting abstracts, a newsletter with reviews and analysis of trends in the field, guidelines, and surveys.  The website contains an e-Manual on Anesthesia for Ventricular Assist Device Implantation, which is developed by and for members. The site also hosts a searchable database of educational echocardiography information.

     

  • Criterion 17: Example 21

    The provider utilized a) website portals to allow access to toolkits, policies, and forms, b) articles and newsletters, and c) a smartphone application to assess whether level of care meets expectations.

     

  • Criterion 17: Example 22

    The provider offers online "face to face" consultation corners where participants can interact with subject matter experts, as well as a physician-only listserv, an image library, practice management webinars and a clinician-scientist development program where younger researchers can develop mentorship relationships with selected faculty.  The provider has developed over 40 patient education brochures for physicians to use in their practice, provides patient education on a public website, and issues press releases from its Public Awareness Committee on hand injury prevention.

     

  • Criterion 17: Example 23

    The provider employs guidelines that are adjuncts to activities, screening tools for patient care, and social media, such as Twitter, blogs, and Facebook, for connecting researchers and practitioners.  In addition, case-based decision trees and journal articles are directly incorporated into the educational activities.

     

Examples of Non-Compliance:
  • Criterion 17: Example 9

    The provider describes the use of non-physician outreach liaisons that visit 75-100 physicians per year. The liaison visits are primarily to build relationships with referring physicians, maintain referrals, and to describe new services at the medical center.

  • Criterion 17: Example 10

    Although the provider described a single activity in which laminated reminder cards were distributed to physicians as a prompt for best-practice taking of a family history, the provider failed to demonstrate non-educational approaches as an adjunct to any other activities within its CME Program.

  • Criterion 17: Example 11

    The provider described several initiatives such as the development of practice guidelines and technical standards and participation in a physician-focused consortium. However, the provider did not demonstrate that these initiatives were strategies to enhance change as an adjunct to its CME activities.
Criterion 18

The provider identifies factors outside the provider's control that impact on patient outcomes.

ACCME Note

The provider has data and information that explains patient outcomes, beyond the performance of their learners. Here the provider demonstrates knowledge of the factors contributing to the health care 'quality gap' about which they are concerned.
 

Examples of Compliance:
  • Criterion 18: Example 1

    The provider has identified and grouped factors outside of its control into categories of: (1) financial factors (eg, reimbursement issues, costs of pharmaceuticals), (2) human factors (eg, patient issues, heath care provider factors), and (3) state of science (eg, capabilities and resources of the institution to provide the needed care).

  • Criterion 18: Example 2

    The provider identified factors such as health insurance coverage and physician communication skills. In addition, the provider implemented a "SWOT" (strengths-weaknesses-opportunities-threats) analysis that yielded several additional factors that would impact outcomes related to colorectal cancer screening such as cost and patients transportation.

  • Criterion 18: Example 3

    The identification of factors is built into the provider's CME planning process. Cues to identify factors are listed on the provider's planning form and include items for selection, as relevant to the activity content, such as "lack of insurance reimbursement and lack of health care resources."

  • Criterion 18: Example 4

    The provider identified poverty and underserved communities as factors that impact patient outcomes, and established a foundation that supports the education of physicians from impoverished and underserved areas. The provider also identified patient fear and lack of factual information for the public as factors that impact patient outcomes. The provider has developed a public-access website with patient information about neuromuscular diseases to help provide the public with valid information.

  • Criterion 18: Example 5

    Follow-up activity evaluations asked learners what changes they implemented and if they encountered barriers to those changes. The responses showed "health insurance coverage," "lack of time to assess/counsel patients," "lack of administrative support," and "patient noncompliance" as barriers to implementing change. Adoption of electronic health records, a topic of several CME activities, was noted to be affected by barriers such as cost, time required to investigate systems, workflow changes, and training requirements.

  • Criterion 18: Example 6

    The provider identified the anticipation of a hepatitis C epidemic, improved tolerability of hepatitis C vaccine therapies for patients, the role of the Internet as a source of health information, and the need to support consumers to become better informed about digestive health care as factors beyond its control impacting patient outcomes.

  • Criterion 18: Example 7

    By asking learners about barriers, the provider has identified factors including, pervasive misinformation about anthrax vaccine, challenges with non-formulary drugs, cultural challenges associated with physicians working in various international settings with different policies in effect, and patients that die before they can get medical care.

  • Criterion 18: Example 8

    The provider identified differences in scope of practice between physicians and nurse anesthetists, on-call schedules, the healthcare team's workloads, relationships with colleagues, hospital policies, and state requirements as factors impacting patient outcomes.

  • Criterion 18: Example 10

    Factors outside of the organizations control that impact patient outcomes are identified from surveys and evaluations at the end of course and by meeting with the course directors.  "Time to evaluate and change behavior” and “patient autonomy for decision-making" are two such factors.
     

  • Criterion 18: Example 11

    The provider identified factors outside its control that impact on patient outcomes, including the destruction and devastation from hurricanes, the after effects on the economy, and the prolonged and complicated recovery process involving the Federal Emergency Management Administration (FEMA), insurers, and others.

     

  • Criterion 18: Example 12

    The provider identified the pressure to "do more with less," including the need for physicians to see more patients per unit of time; time and other pressures created by needing to switch to electronic health records and defensive medicine, including the tendency to order more tests and procedures to prevent malpractice.

     

  • Criterion 18: Example 13

    Barriers identified by the provider include impediments college students have in accessing care , negative environmental factors on college campuses, stigma, the prevalence of infectious diseases on college campuses, academic issues, and silos of care.

     

     

  • Criterion 18: Example 14

    The provider identifies factors outside its control on a program-wide basis through committee work and task forces.  These factors are communicated to the Cancer Education Committee so that they can develop content to address the issues identified.  Some factors include the cost  of cancer care, insurance coverage, health disparities, and workforce issues.

     

  • Criterion 18: Example 15

    The provider identified a number of factors outside its control that impact patient outcomes, including patient behavior factors related to Workers' Compensation/Occupational Medicine, economic factors that might have an effect on the medical care of injured workers, and several national policy issues related to physician reimbursement or limitation of number of office visits for care for injured workers.

     

  • Criterion 18: Example 16

    The provider identifies multiple factors outside its control that impact patient outcomes. Some of these factors are identified on a health needs assessment of the local community done every five years. Examples of these factors are crime, drug abuse, smoking, gun control, physical activity, and nutrition.

     

  • Criterion 18: Example 17

    The provider identified the current economic climate, particularly in very poor and violent communities, as well as a host of individual patient characteristics, including the underinsured or uninsured, communication and language barriers, social issues, e.g., addiction and violence, as well as culture , that have had an impact on patient outcomes.  This information is derived from a variety of sources, including responses on evaluation forms, direct learner feedback, conversations with clinicians, and electronic health record data reports.

     

  • Criterion 18: Example 18

    The provider uses a systematic way to identify barriers outside its control that impact patient outcomes.  One method used is to interview experts in the field about barriers they identify.  For example, when planning an activity in the area of contraception and the need for enhanced counseling abilities by physicians, the barrier that was identified was direct-to-consumer advertising which affects women’s attitudes.  In addition, the provider has added questions on barriers in its evaluation form, including “Which of the following do you perceive as being the primary barrier to achieving optimal patient outcomes?”  Responses to that question included side effects and economic factors that may limit patient access to treatment.

     

  • Criterion 18: Example 19

    The provider categorized factors as learner-related, patient-related, and systems-related. Learner-related issues included reimbursement, increased practice demands, the evolving healthcare environment, and inconsistent standards and approaches to care. Patient-related issues included cultural values, mistrust of the healthcare system, and economic circumstances contributing to adherence problems. Systems-related issues included complex processes in office and hospital settings, regulatory requirements, and physician-centric healthcare model.
     

Examples of Non-Compliance:
  • Criterion 18: Example 9

    In its self-study report, the provider indicated that it, "identified factors that impact on patient outcomes." However, no factors were described. Further review of activity files and discussion during the survey interview failed to provide more information to support the provider's compliance with Criterion 18.
Criterion 19

The provider implements educational strategies to remove, overcome or address barriers to physician change.

ACCME Note

The provider has data and information on barriers to change applicable to its own learners, and incorporates these insights into its CME program through activities. In C19, the provider shows that activities are included in their educational program that are focused on 'overcoming barriers to physician change.'

Examples of Compliance:
  • Criterion 19: Example 1

    The implementation of educational strategies to remove, overcome, or address barriers to physician change is a part of the provider's CME activity planning process. For example, a CME activity on the human papillomavirus vaccine included specific information on myths and misconceptions regarding the virus and the vaccine. In another example, a CME activity on cardiac issues in women covered content related to gender stereotypes which can delay diagnosis and treatment for women.

  • Criterion 19: Example 2

    The provider described the inclusion of content into CME activities that would address barriers to physician change, such as content related to how to more effectively utilize nurse assistants to help ensure body mass index calculation is always done and is reviewed by the physician and effective screening tools to assess geriatric patients in a busy practice.

  • Criterion 19: Example 3

    The provider's planning process integrates discussions into CME activities that address identified barriers to physician change. As an example, the provider identified physician resistance to changing the manner in which medications have historically been written as a barrier. The provider implemented an educational strategy to use real-life examples of poorly handwritten medical orders, and asked physician learners to interpret via an audience response system.

  • Criterion 19: Example 4

    The provider conducts a survey to identify perceived barriers to physician change, and CME activities are subsequently designed to address them. For example, technical incompetence to properly implement electrodiagnostic testing to make an appropriate diagnosis is a focus of educational efforts. Limited time for physicians in practice has also been identified as a barrier, and educational efforts at increasing efficiency of report writing have been delivered. The provider has striven to offer multiple asynchronous learning modalities to address issues related to "limited time for learning."

  • Criterion 19: Example 5

    The provider implemented an online electronic health record comparison tool to help physicians compare 23 certified systems based on functionality, cost, security and interoperability. To address barriers identified in a diabetes CME activity, the provider developed a care guide that includes information on a team-based model to provide administrative and patient care support, and how to implement changes. Several examples of CME activity agendas were provided that addressed the incorporation of topics to address barriers in health disparities, inefficiencies, insurance issues, communication, and health literacy.

  • Criterion 19: Example 6

    The provider has implemented strategies such as treatment protocols, education on patient and family centered care, education on overcoming barriers related to improper diabetes management and education to help physicians navigate ethical issues they confront.

  • Criterion 19: Example 7

    In its self-study report, the provider described several educational strategies it implemented to help physicians overcome barriers related to disclosing mistakes and apologizing to patients. These educational efforts included individual CME activities, content in grand rounds and seminars, content in new physician orientation and a website about disclosure and apology.

  • Criterion 19: Example 8

    The provider developed a web tool on their organizational intranet that facilitates communication and provides a number of resources, including: clinical information for conditions that require frequent consultation, educational sessions focused on effective and efficient methods of communicating with patients, and development of materials to enable health care professionals to help patients with their financial problems.

  • Criterion 19: Example 10

    The provider seeks to address and overcome barriers to physician change by understanding the nature of barriers through evaluation questions, culture surveys, and face-to-face meetings. When system issues are identified, the provider uses a Root Cause Analysis process to find a solution.  For example, the provider designed and offered specific CME activities to address the following identified barriers: 1) physicians’ failure to listen adequately to patients; 2) physicians’ testing of HIV disease based upon historical perspectives and a failure to consider that the disease has migrated to new populations; and, 3) the effect of media coverage of opioid prescribing on physicians’ treatment of patients.
     

  • Criterion 19: Example 11

    The provider’s CME planning process addresses barriers and considers strategies to overcome them. For example, a lack of competence in physicians leading interdisciplinary teams was overcome through a multifaceted educational and non-educational initiative around venous thromboembolism prophylaxis.

     

  • Criterion 19: Example 12

    The provider emphasizes lack of teamwork as a barrier to optimal emergency cardiac and stroke care and addresses team member roles and team skills in its activities. To address this, activities are designed to include key team members and discuss their potential roles in providing the needed care.

     

  • Criterion 19: Example 13

    The provider identified common barriers to physician change it encountered, including not having enough time to talk with patients about difficult issues such as medical errors, a belief that noncompliant patients are inaccessible, and the concern about medical liability litigation.  The provider describes specific educational strategies to overcome and address these barriers to change, including an internet enduring material with strategies to efficiently talk to patients about medical errors, and another activity that addressed the potential claims that can occur when noncompliant patients are not tracked.

     

  • Criterion 19: Example 14

    The provider includes questions related to barriers to physician change as part of its standard evaluation form.  Using this feedback, the provide analyzes the data and, based on the nature of the barrier, develops activities to address knowledge, attitudinal, behavioral, and system barriers to change.  Examples include interactive presentations to address physician concerns about the frequency and number of childhood immunizations and the development of activities using standardized patients to help physicians with communication issues in addressing issues of substance abuse.

     

  • Criterion 19: Example 15

    To address identified barriers to physician change, including the lack of patient adherence/compliance, lack of time, lack of evidence-based guidelines, as well as insurance / financial and organizational / institutional barriers, the provider has begun to incorporate educational offerings into its curriculum that focus on topics such as the effects of healthcare reform on patient care, practicing within a changing system, strategies for better patient adherence and patient outcomes, and the effective use of technology in medical practice. The provider also offers activities to address physician barriers in communicating with patients, dealing with ethical and quality of care issues, and incorporating clinical practice guidelines.

     

  • Criterion 19: Example 16

    The provider developed a series of educational activities that combined didactic sessions combined with hands-on workshops across the country to address primary care physicians’ inability to demonstrate the proper use of asthma medication inhalers to patients. In the workshops, the learners observed the proper instructional techniques demonstrated by trained clinicians on patient simulators. The physicians then applied those techniques while under observation until a new degree of competence was achieved.
     

Examples of Non-Compliance:
  • Criterion 19: Example 9

    The provider presented two examples of how to address factors that impact on patient outcomes. However, the provider did not demonstrate with this information, or with other information presented, that it implemented educational strategies to remove, overcome or address barriers to physician change.
Criterion 20

The provider builds bridges with other stakeholders through collaboration and cooperation.

ACCME Note

The provider allies itself with other organizations or components of its own organization in a purposeful manner to achieve common interests. These collaborations may support any aspect of the provider's CME program in service of achieving its mission. The ACCME does not consider joint sponsorship, in itself, as a collaboration that will guarantee compliance with C20. However, joint sponsorship can be a byproduct of a larger collaboration and if this larger collaboration is described for ACCME, then it could result in compliance with C20. In C20, the ACCME is looking for active engagement in collaborative and cooperative projects.

Examples of Compliance:
  • Criterion 20: Example 1

    The provider participates in a variety of collaborations and cooperative initiatives, including those with a university (on safety issues), an institute and a hospital association (on improving quality of care).

  • Criterion 20: Example 2

    The provider identified multiple stakeholders with which it partners, eg, a county department of health services (for colorectal cancer screening), a state bioterrorism preparedness organization, and a state quality improvement organization (to help identify professional practice gaps for CME).

  • Criterion 20: Example 3

    Examples include collaboration with other medical schools on an annual cancer treatment activity, with a national foundation for a multidisciplinary activity on disease care, with community organizations to help improve overall community health, and with a software company to bring new technology to e-learning.

  • Criterion 20: Example 4

    The provider builds bridges with stakeholders through collaboration and cooperation in the development of published clinical guidelines and clinical evidence reviews on HIV screening (with a medical association partner), dementia treatment (with a medical specialty society partner), low back pain (with another medical society partner), and management of venous thromboembolism (with a physician membership organization). For its performance improvement activities, it acted as a member of national quality improvement-focused consortia. The provider has individuals involved as members on various performance measurement workgroups and advisory boards. For its diabetes CME activities, the provider collaborated with the various national diabetes organizations. For its Maintenance of Certification activities, it collaborated with and provided feedback to a specialty board.

  • Criterion 20: Example 5

    The provider described several collaborations, including those with: a pediatric health system and community partners to help children grow up healthy; a community-based pediatric association to develop a yearly educational program created to improve the standard of pediatric care locally; a public-private commission of schools, community agencies, and families on a conference for children with special needs; and, a pediatric specialty society to develop a CME activity for pediatric surgeons from around the world with a special focus on the care of children with cerebral palsy.

  • Criterion 20: Example 6

    The provider identified several interdisciplinary stakeholders with which it partners, including a nurse specialty society and professional organization for physician assistants on an allied health symposium; a national specialty society and disease-focused association on a collaboration to reduce the risk of gastrointestinal problems and to improve public health. The provider also works with a diverse stakeholder group on national campaign to address obesity, including members of Congress and a wide range of consumer and disease advocacy groups.

  • Criterion 20: Example 7

    The provider collaborated with a physician membership organization to establish a committee on physician reentry, and with a disease-focused professional society to address issues related to injury and violence. In addition, the provider listed twenty organizations with which it collaborates to identify community health problems and works cooperatively on strategies and educational efforts that address various community health issues including: teen pregnancy, infant mortality, immunization, adolescent homicide, access to health care, and obesity. The results of these collaborations have led to educational activities, the development of protocols, and the identification of clinical initiatives.

  • Criterion 20: Example 8

    The provider describes collaboration with shared stakeholders including a national lung cancer partnership, a blood and marrow transplant program, and a national patient advocacy foundation. In the cases listed, the provider and the organizations identified shared goals for education and worked together over time to produce programs designed to improve patient care and patient outcomes.

  • Criterion 20: Example 10

    The provider   collaborated with non-governmental organizations on research studies, and convened a roundtable on workplace health and productivity as part of its collaboration with a national health policy organization. The provider also collaborates with national professional association on key issues related to workplace health and wellness.

     

  • Criterion 20: Example 11

    The provider described multiple partners and collaborators, including work with government agencies, quality improvement entities, and other medical societies, and coalitions.

     

  • Criterion 20: Example 12

    The provider collaborates with over 100 educational partners, including rural hospitals, clinics, and specialty interest groups. The provider’s collaborations focus on the joint identification of quality improvement data to support the development of CME activities that promote improved physician practice and patient outcomes, regarding, for example, safety issues/concerns with the use of anticoagulant therapy.

     

  • Criterion 20: Example 13

    The provider collaborates with regulatory and legislative bodies in support of health policy issues, specifically for expanded resources to improve drug safety.  The provider collaborated with the FDA on Risk Evaluation and Mitigation Strategies (REMS) and with The Joint Commission on development of medication reconciliation standards and on a sentinel events report related to certain medications associated with higher risk of suicide.

     

  • Criterion 20: Example 14

    The provider collaborates with other stakeholders based on common goals, including outpatient clinics to improve professional practice in the care of patients with cardiovascular disease on a national and international basis. The provider collaborates with the National Institutes of Health and other nonprofit organizations to share data, distribute diagnostic training and certification on a worldwide basis, and to develop educational tools.

     

  • Criterion 20: Example 15

    The provider is engaged in collaborative and cooperative relationships with several stakeholders, including academic medical centers, medical professional societies, community physician practices, private hospitals and city and municipal agencies. Examples included a collaborative effort on quality patient care and physician training with a local school of medicine and collaboration with a local hospital system and a municipal fitness program, designed to address some of the causes of childhood obesity.

     

     

  • Criterion 20: Example 16

    The provider collaborates with many stakeholders, including the federal agencies on disaster preparedness issues, specifically educating health care professionals how to respond to certain types of disaster-related injuries. The provider has worked with a local specialty society to share with health professionals the difficulties survivors have reintegrating into society.

     

  • Criterion 20: Example 17

    The provider is engaged in collaborative and cooperative relationships with local, regional, and national organizations to identify and address cardiovascular health disparities. The organization works closely with Congress and the U.S. Department of Health and Human Services and its agencies, e.g., the National Institutes of Health, Centers for Disease Control and Prevention, and especially with the Office of Minority Health and Health Disparities, to monitor the health status of minorities.

     

  • Criterion 20: Example 18

    The provider described multiple collaborations with healthcare professionals, medical societies and accredited and non-accredited educational partners. The provider has developed relationships with key physicians who participate on the organization’s clinical advisory board and planning committees. The provider collaborates with educational partners so that both parties can achieve the delivery of educational activities that would not have been possible otherwise. The provider’s collaboration with medical societies has helped to enhance educational opportunities for the societies’ members.

     

  • Criterion 20: Example 19

    The provider allies itself with several types of organizations in a purposeful manner to achieve common interests, including state government agencies, hospitals and medical societies.  In each of these cases, the provider’s collaborations demonstrate a shared commitment to education that improves patient safety and healthcare quality.
     

  • Criterion 20: Example 20

    The provider participates in multiple and purposeful collaborations within its state and internationally.  These include a statewide patient adherence initiative with a consortium of state medical schools, collaboration with state-based community advocacy organizations to reach a variety of health and law enforcement professionals, and an international collaboration to develop health care and public health learning opportunities for students.

     

  • Criterion 20: Example 21

    The provider is working collaboratively with a federal agency on meningitis and tuberculosis surveillance in college students.  The provider has also worked with the this agency to address influenza, mumps, sexual assault, and sexually transmitted infections.  Other strategic partners include national professional associations for mental health care providers.

     

  • Criterion 20: Example 22

    The provider has worked with national non-profit organization to convene a consensus conference with the goal of identifying strategies to overcome barriers and improve inpatient care for diabetes. The provider participates with a number of other non-profit organizations and advocacy groups focused on diabetes and related medical conditions.  In addition, the provider developed and promoted a legislative initiative to evaluate current federal activities related to the diabetes.

     

Examples of Non-Compliance:
  • Criterion 20: Example 9

    The provider lists its joint sponsorship relationships, but provides no description or evidence that these relationships are collaborative or cooperative.
Criterion 21

The provider participates within an institutional or system framework for quality improvement.

ACCME Note

The provider is focused on integrating and contributing to healthcare quality improvement. In C21, the provider has evidence that CME has become a part of institutional, or system, quality improvement efforts. 'System' can also include the network of other organizations in the health care 'system'. (Note: organizational self-assessment and improvement focused on improving the quality of the CME program are recognized and reward in C12-15, not in C21.)

Examples of Compliance:
  • Criterion 21: Example 1

    The framework of the institution is designed to promote quality improvement. The inclusion of multiple institutional stakeholders and departments on the performance improvement committee allows the provider to make decisions regarding the most appropriate ways to address identified quality gaps. When education is appropriate, CME stands ready to address the gap. The organization is also positioned to apply other appropriate interventions, such as technology changes and staffing modifications. In addition, performance improvement data is shared with the CME committee, which includes all specialty departments.

  • Criterion 21: Example 2

    Leaders of the CME program interact and are actively involved with the Hospital Quality Improvement Office and the Quality Coordination Group.

  • Criterion 21: Example 3

    Quality improvement is articulated in the CME mission statement. The CME office has a representative on the local college of medicine's quality and safety committee. In addition, evidence of the implementation of this framework through CME activities such as, a certificate program regarding risk management, an on-line risk management course, and a three-day course to help improve medical ordering.

  • Criterion 21: Example 4

    The provider has a quality improvement task force which is engaged in activities to improve patient outcomes. The provider works closely with a number of quality improvement associations, including the National Quality Forum and the American Medical Association's Physician Consortium for Performance Improvement.

  • Criterion 21: Example 5

    The provider's CME committee is comprised of subcommittee chairs and editors from the organization's Clinical Efficacy Assessment Subcommittee and Performance Measurement Subcommittee. These are quality improvement subcommittees of the organization whose work informs the evidence base and identification of practice gaps for the provider's CME activities. These subcommittees are liaisons to national quality coalitions. A key member of the CME team is the Director of Clinical Programs and Quality of Care who helped develop quality improvement activities, specifically the diabetes initiatives. Through these relationships, evidence was provided that the provider participates within its own internal as well as external frameworks for quality improvement.

  • Criterion 21: Example 6

    Several institutional quality improvement leaders serve on the primary CME committee and are active in planning activities. Multiple CME activities are directly focused on institutional quality improvement issues.

  • Criterion 21: Example 7

    The provider participates both within an internal institutional framework and an external system aimed at quality improvement. The provider has a quality task force and task force on the future of gastroenterology training. Externally, the provider is represented in various organizations and coalitions where quality issues are discussed and performance measures developed, including the American Board of Internal Medicine (ABIM), the American Medical Association (AMA), the American College of Physicians (ACP), and the National Quality Forum (NQF). Data and insights from these external efforts were integrated into various educational activities. The provider has been actively engaged with the Centers for Medicare and Medicaid Services' Physician Quality Reporting Initiative, along with an NQF workgroup in the development of peer reviewed quality measures. Once measures are adopted, the provider educates its membership and provides tools for integration into practice.

  • Criterion 21: Example 8

    The provider participates within a framework for quality improvement. The CME committee includes representatives from its quality improvement department who provide input into activity planning. Activities are developed to focus on clinical practice guidelines and quality initiatives. Weekly teleconferences occur between the quality department and the CME unit. Evidence was present of the quality improvement focus in many activities selected for performance-in-practice review.

  • Criterion 21: Example 11

    The provider undertook a quality improvement project to determine the impact on business of employee absenteeism due to influenza.

     

  • Criterion 21: Example 12

    The provider has established a protocol to insert departmental-specific quality improvement activities within RSS sessions. The CME Director sits on the patient safety committee and Medical Staff Review Board for the academic medical center, allowing the CME program to integrate educational strategies and CME activities with the process improvement practices stemming from adverse events within the hospital.

     

  • Criterion 21: Example 13

    The provider is involved in a project that will develop an online tool for the collection of validated patient-reported outcome measures from surgeons' offices.  The provider is collaborating with a local university to develop an online tool with functionality that allows the routine collection of validated outcomes scores in an medical clinics.  This system will collect data to determine expected outcomes for specific diagnoses.

     

  • Criterion 21: Example 14

    Four of the provider’s board members sit on the quality improvement committees at their respective organizations. These members are instrumental in communicating quality data, identifying professional practice gaps and providing expertise needed for planning CME activities. For example, one Board member is University faculty and has access to ‘de-identified’ clinical registry data that identifies problematic practices at a systems level. These partnerships enable the provider to focus on educational content that addresses and incorporates quality patient care and patient safety practices into its CME activities.

     

  • Criterion 21: Example 15

    The provider participates on the quality committee and works with the graduate medical education (GME) department to develop quality and performance improvement activities for residents and faculty.

     

  • Criterion 21: Example 16

    The leadership of the CME program is integrated in Quality Improvement Committees, the Operating Room Safety Committee, and the Performance Improvement and Patient Safety Committees. This integration is an active one that positions CME as an agent of change and improvement for the institution.

     

  • Criterion 21: Example 17

    The provider includes CME as part of its three-part Quality Improvement mission for its academic health center. Key staff members of the CME office are members of the institution’s Patient Safety and Quality Improvement committees.  A member of the institution’s patient safety unit is a member of the CME unit’s Advisory Group.
     

  • Criterion 21: Example 18

    The foundation of the provider’s CME program is a national initiative which offers hospital departments a system-based solution to improve patient safety and reduce medical errors and malpractice risk. The provider partners with the quality improvement efforts of its client organizations to develop CME activities to support the organizations’ overall quality improvement goals.

     

  • Criterion 21: Example 19

    The provider has created a patient registry to gather data on use and outcomes from treatment. A result of this is a benchmark study to track physician practice and progress in the delivery of office-based treatment, including information on morbidity, mortality, and drug use to establish a benchmark that others may use to measure their performance.

     

  • Criterion 21: Example 20

    The provider determines key priorities for clinical improvement at its institution, which are reflected on a care scorecard.  Project teams for each of these key priorities then work with the Office of Accreditation to develop an education plan.  Educational initiatives have focused on ‘physician communication’ and ‘breast cancer’, for example. Another initiative is aimed at reducing the number of elective procedures. An advanced training program includes education, practice and practical application of quality improvement theory, it incorporates quality improvement project processes, and advanced training program projects are aligned with system-level clinical improvement strategies and priorities.

     

  • Criterion 21: Example 21

    The CME program is closely tied to performance improvement initiatives in the organization and has outcomes and performance improvement managers on the CME advisory committee.  The Director of CME is also a member of the health system's Patient Safety Committee and Performance Improvement Council.  A focus group consisting of CME, performance improvement, and patient safety representatives developed a system framework to embed quality improvement and data-driven outcomes into all CME activities as well as to enhance ongoing performance improvement initiatives.  The provider requires that 25% of its RSS activities must be based on needs derived through the quality assurance /quality improvement process, and performance improvement activities provide a framework for quality improvement.

     

  • Criterion 21: Example 22

    The provider participates within a framework of quality improvement through education to reduce malpractice risks, and has recently retained an actuarial firm and will offer efficacy studies to those organizations that lack the internal resources to do so. In addition, the provider’s programs are woven into existing processes such as credentialing, insurance renewal, peer review and are used to drive improvement in patient outcomes.

     

  • Criterion 21: Example 23

    Committees interact with government, federal agencies, healthcare organizations, and other medical associations to influence and implement quality measures across the provider’s clinical discipline. A quality improvement workgroup participates with national quality and performance improvement groups such as the National Quality Forum to promote national standards and healthcare quality measures.

     

Examples of Non-Compliance:
  • Criterion 21: Example 9

    While mechanisms are in place for enhancing the quality of services and operations within the accredited provider, these mechanisms did not demonstrate that the provider participates within an institutional or system framework for health care related quality improvement efforts.

  • Criterion 21: Example 10

    The provider described satellite broadcasts into hospitals and cancer center networks and a local CME speaker program series. However, the provider did not demonstrate with this information, or with other information presented, that it participates within an institutional or system framework for quality improvement.
Criterion 22

The provider is positioned to influence the scope and content of activities/educational interventions.

ACCME Note

In C22, the ACCME expects the provider to play a meaningful role in the formation and direction of activities across its entire CME program.

Examples of Compliance:
  • Criterion 22: Example 1

    The provider's framework and processes ensure that the institution is positioned to influence the scope and content of its educational activities/interventions. Regarding the scope of its activities, the CME that is delivered is based on gaps as identified by the performance improvement and CME committees. Regarding the content, CME planning is always linked explicitly to an organizational component and/or practice area.

  • Criterion 22: Example 2

    Leadership for the provider's CME program is positioned to influence medical student education, graduate medical education, quality management, faculty and career development, and clinical care. The provider's CME program is also positioned within the community to influence local and regional CME. The local health department and quality improvement organization turn to the provider's CME office to help with their performance improvement initiatives.

  • Criterion 22: Example 3

    The CME office is strategically connected and initiates CME activities based on specific hospital and system requirements and responds to specific institutional quality and performance issues that impact the safety needs and translates those into CME interventions.

  • Criterion 22: Example 4

    The provider's framework and processes position the organization to influence the scope and content of its activities. In addition, the provider cooperates with other similarly-focused specialty societies to develop a core curriculum and ensure that the education obtained is consistent for the specialty. The content of the curriculum is shared with the planning committees for CME activities, so there is continuity regarding major issues and topics.

  • Criterion 22: Example 5

    The provider has positioned itself to influence its scope and content through its comprehensive planning and oversight processes, which include an online system that is used for the planning, execution, and evaluation of its activities around the world.

  • Criterion 22: Example 8

    Influence over the scope and content of activities is demonstrated by the systems of control in planning and review that the provider has in place.  In addition, the provider offers training to all the departments on CME, accreditation, and MOC to ensure the entire institution has access to accurate information in these areas.

     

  • Criterion 22: Example 9

    The CME program is integral to the planning and approval of all activities proposed by activity planners. CME policies and procedures along with dedicated review committees help position the organization to influence the scope and content of educational interventions. A recent change in the structure of the CME program whereby the CME office is integrated with the quality and safety activities within the University's medical center ensures that the CME activities offered meet the needs of the practicing physicians at the institution.
     

  • Criterion 22: Example 10

    The provider presented committee structures and collaborations that demonstrate its involvement and influence in the planning, development and evaluation of its CME activities and program.

     

  • Criterion 22: Example 11

    The provider’s leadership and position within the institution, as well as its association with state and national committees within its field of expertise, demonstrate that it is influential in and responsible for the scope and content of its activities/educational interventions.

     

  • Criterion 22: Example 12

    The provider applies transparent policies and processes for continuous control of the development of CME activities and retains control over all aspects. All activities are linked to the organization's Strategic Plan which is reviewed and revised regularly with input from various levels and committees of the organization. All activities are designed for its own member categories.

     

  • Criterion 22: Example 13

    The provider demonstrates that its educational program is driven in scope and content by its research agenda and its members who are well-positioned to develop education that is central to the advancement of its mission.

     

  • Criterion 22: Example 14

    Influence over the scope and content of activities is demonstrated through the provider’s policies stating the extent of its control of planning processes and, in practice, through its planners’ completion of a compliance checklist ensuring that the provider’s policies are met in educational activities.

     

Examples of Non-Compliance:
  • Criterion 22: Example 6

    Information collected from the review of activity files and the survey interview showed that the provider delegates many aspects of its CME program to third parties with no mechanism for oversight or reporting. Examples of these delegated functions include activity planning, interaction with planners and speakers, content development and review, and the management of commercial support.

  • Criterion 22: Example 7

    In its self-study report, the provider indicated that its CME program is primarily comprised of jointly sponsored activities in which joint sponsors identify educational needs for activities, develop educational curricula and conduct evaluation and assessment of the effectiveness of CME activities. In the survey interview, the provider indicated that it, "defers to the expertise," of its joint sponsors for most of the operation of its CME program.