ABSTRACT

As outcome measurement becomes an increasingly critical adjunct to learning activities for continuing medical education (CME), both supporters and providers need cost-effective ways to include robust outcome measurement not only of learning gains (Kirkpatrick Level 2) but also of corresponding changes in clinical practice (Level 3) and patient outcomes (Level 4). Healthcare providers (HCPs) are often surveyed and asked whether their clinical practices will change or have changed as the result of the educational activity in which they have participated. Because this type of self-reported survey information is not based on direct measurement or observation of actual behavior, it is not outcomes data per se but is rather the respondent’s opinion about past or future outcomes. As such, it is not a “measurement” but is an informed guess.

In spite of these inherent limitations, the self-reported survey is the only method commonly used to assess the clinical impact of a CME program, because the current methods for assessing changes in clinical practice and patient outcomes by means of direct observation are simply too costly and time-consuming for all but the occasional well-funded research project. These include the standard patient method (to assess clinical practice), which uses trained actors and actresses posing as real patients; and the chart analysis method (to assess patient outcomes), which typically involves review of paper charts and hand-keying of relevant data.

The interactive clinical case study simulation offers an extremely cost-effective method for measuring the impact of a CME activity on the learner’s clinical practice. Although it cannot directly observe the HCPs behavior in the actual clinical setting, it can directly measure the choices the HCP makes in diagnosis, treatment, and follow-up. By simulating the most common patient profiles for a particular disorder, the interactive clinical case study can record the HCPs diagnostic and treatment decisions as he or she works through the case, show the likely outcome of each decision, and provide individualized feedback to identify and correct inappropriate decisions.

Case studies have long been used in medical education, and their effectiveness as an instructional strategy is well established. In fact, clinicians’ scores on case study exercises have been found to be a more accurate reflection of actual clinical practice than traditional chart analysis [1,2]. Interactive simulations of clinical case studies can be created cost-effectively by loading specific case information into a pre-programmed software template that has been designed to mimic, in generic form, the clinical diagnostic and treatment process. Such simulations can be constructed as text-only modules for maximum cost effectiveness, as multimedia modules with audio, video, and animation for maximum educational impact, or as an intermediate blend of text and multimedia. Whatever the level of production, the interactive clinical simulation can be readily programmed to present a typical patient profile and then record every decision the learner makes in diagnosis, treatment, and follow-up. Administered prior to an educational program or event, the simulation can record the clinician’s baseline skills and knowledge. Administered again after the educational event, a parallel simulation can measure the learner’s improvement in making diagnostic and treatment decisions—in other words, the impact of the educational event on the learner’s clinical thinking.

In a more complex form of the interactive case study simulation, the learner is presented with feedback on the consequences of each clinical decision, given the opportunity to revise diagnostic and treatment decisions, and guided toward the optimal diagnosis and course of treatment. This type of simulation goes beyond the simple assessment of clinical competence and provides an individualized educational experience targeted at precisely those areas in which the learner needs remediation.

Modern computer technology has made possible the creation of simulated clinical experience and the precise measurement of patterns of clinical decisionmaking within a technology environment that can deliver quantitative outcomes data rapidly and at a lower cost than any other outcome measurement strategy. Because the interactive case study directly measures learner behavior, it generates data of intrinsically greater validity than the self-administered survey questionnaire; and because it can be distributed and administered entirely by computer, it can do so with a minimal commitment of time and resources.

Download a PDF of the full article