President's Message: Stephen I. Wasserman, MD
The Accountability, Evidence and Assurance of the MOC Process
by Stephen I. Wasserman, MD
One of the most frequently asked questions I receive is “what is the evidence that Maintenance of Certification (MOC) is needed and is effective.” Coincidentally a supplement to the Journal of Continuing Education in the Health Professions was published late last year that addresses these questions in depth. Two articles in this publication are of particular interest to those of you who wish to take a “deep dive” into the question of evidence. I will try however to summarize the findings presented in this supplement which is the best compilation of the literature on this topic.
First, I believe it is essential to approach this question from the point of view of Medicine as a profession and with an understanding of the requisite responsibilities of this view. Cruess, et al. Med Teach. 2004;16:74-76 have defined a profession as:
- An occupation whose core element is based on mastery of complex knowledge and skill
- Founded on service to others
- Governed by codes of ethics
- Committed to competence, integrity, morality, altruism and promotion of public good.
They add that these principles form the basis of a contract between physicians and society which grant a great deal of self-governance to the medical profession, but we remain accountable to those we serve and to society.
Given this compact and accountability, is there evidence that something needs to be done to provide society with assurance that this covenant is being honored, and that the components of MOC are appropriate to that task? Are the four parts of MOC: Professional Standing, Self-Assessment, Cognitive Expertise and Practice Assessment/Quality Improvement appropriate to this goal? Hawkins, et al. J Contin Ed Health Prof. 2013;33(Suppl 1):S7–S19) addresses this question in depth. It is known that performance deficits are critical factors in medical errors, that physicians are aware of deficiencies in their colleagues, patients do not receive much of the care recognized as needed for their particular health issues, that skills and knowledge do decline with time after training (especially due to lack of acquisition of new information), and that adverse license/malpractice actions increase with time in practice.
How then to enable physicians to identify areas of opportunity for improvement and to assist them in that improvement. Peer and patient surveys address aspects of communication which is known to be an important aspect of medical practice and an area shown by many studies to be an opportunity for improvement. Data indicate that assessment in this domain can be associated with improvement that is meaningful to patient satisfaction and quality of life.
Part II of MOC (Lifelong Learning and Self-Assessment) covers activities that are widely accepted by physicians as an essential tenet of medical practice. One important aspect of such learning is that to be most effective it requires an external objective assessment to provide individual physicians a road map of areas that would benefit from extra attention. It has been shown that physicians are not good at accurately self-identifying areas of strength or of weakness (Davis DA, et al. JAMA. 2006;296(9):1094–1102), thus feedback from some objective assessment is essential. At present this is provided by the MOC Part III secure examination, but other less high stakes opportunities could be envisioned such as a non-failable examination like the in-training examination provided to Allergy-Immunology fellows. When educational efforts are directed to objectively identified learning opportunities, it has been shown that physician’s knowledge and performance improve.
In addition to helping to focus learning on those areas most in need, the secure examination is a valid instrument in identifying the knowledge and diagnostic reasoning of physicians; and, as noted above these skills have been shown to erode with time out of training and their assessment therefore is not only appropriate but essential. Overall MOC scores and pass vs. fail categories do identify important aspects of a physicians' skill set. Thus, as reviewed by Hawkins, scores on the ABIM MOC secure examination correlate with increased exposure to education and training, with decreased disciplinary actions, and with increased practice complexity. In addition, better care of patients with diabetes, hypertension and in screening practices have been associated with better performance on the MOC examination.
Finally, Part IV of MOC provides a platform for physicians to examine their own care in defined areas using their own data to uncover potential gaps in performance. Such activities provided by ABIM, termed Practice Improvement Modules (PIMs) have been shown to facilitate improvement in care. It is clear however that improvements in data capture and better mechanisms to capture the activity of teams and systems that effect care are essential.
In a separate paper, Lipner, et al. J Cont Ed Health Prof. 2013;33(Suppl 1):S20–S35, summarize the large number of studies dealing with the evidence-base underpinning certification and MOC. While there is more evidence in the literature dealing with initial certification there is an accumulating set of observations that support the conclusion that both certification and MOC identify aspects of physicians that are important to quality health care and to patients. It is clear that the certifying boards, ABAI included, need to do more to evaluate the strengths and weaknesses of each aspect of MOC. In this regard, constructive and candid feedback from diplomates is essential. You will be receiving a survey shortly from ABAI and I hope you will use that opportunity to help us help you by making MOC the most effective tool possible.
Please share your thoughts with me on this topic at abai@abai.org or swasserman@ucsd.edu.