Learner-centered verification methods are at the core of Donna Wright's model for competency assessment. Using Wright's framework, an academic medical center studied the use of simulation as a verification method for their annual ongoing nursing competency assessment. Of the 10 pilot participants, 60% used simulation as a verification method to successfully show competence. Assuming adequate professional development practitioner and facility resources, simulation can be used as an option for ongoing competency assessment.

Neurosurgical cadaveric and simulation training is a valuable opportunity for residents and fellows to develop as neurosurgeons, further neuroanatomy knowledge, and develop decision-making and technical expertise. The authors describe the growth and development of Oregon Health & Science University (OHSU) Department of Neurological Surgery (NSG) resident hands-on simulation skull base course and provide details of course layout and setup.

A three-part surgical simulation series was created to provide training in cadaveric skull base procedures. Course objectives were outlined for participants. Residents participated in NSG hands-on simulation courses (years 2015-2020) and completed annual course curriculum and anonymous course evaluations, which included free text reviews. Courses were evaluated by Likert scale analysis within Python, and free text was quantified using Valence Aware Dictionary for sEntiment Reasoning (VADER). Descriptive statistics were calculated and plotted using Python's Seaborn and Matplotlib library modules.

Analysis included 162 skull base (anterior fossa, middle fossa and lateral, and endoscopic endonasal-based) simulation course evaluations. Resident responses were overwhelmingly positive. Likert responses demonstrated high average responses for each question (4.62 ± 0.56 and above). A positive attitude about simulation courses is supported by an average compound sentiment value of 0.558 ± 0.285.

This is the first time Likert responses and sentiment analysis have been used to demonstrate how neurosurgical residents view a comprehensive, multi-year hands-on simulation training program. We hope the information presented serves as a guide for other institutions to develop their own residency educational curriculum in cadaveric skull base procedures.

The level of performance of every clinician and of the overall multiprofessional team relies on the skills and expertise they have individually and collectively acquired through education, training, self-directed learning, and reflection. Simulation-based education (SBE) is playing an increasingly important role in that respect, and it is sometimes said that it is an art to facilitate. Many explanations can justify this assertion. Although there is generally an emphasis on making everything as realistic or "high-fidelity" as possible, it is often futile and this is where the art of simulation comes into play with an element of modulation of realism linked to the intended learning objectives. The atmosphere created by the educators; how the learners are made to engage and interact; how physical, technical, and contextual elements are simulated or represented; and what type of technology is used need to be appropriately adapted to contribute to the immersiveness of any SBE activity. Although it inevitably carries a negative connotation, some form of "deception" is more commonly used than one may think for the benefit of learners during SBE. High levels of realism are sometimes achieved by making learners believe something works or reacts as would be expected in real life, whereas it is achieved in a totally different manner. Learners do not need to know, see, or understand these "tricks of the trade", shortcuts, or artistic or technological aspects, and this can be considered a form of benevolent deception. Similarly, information may be withheld to recreate a realistic situation and push learners to demonstrate specific learning outcomes, but it needs to be practised with caution and be justifiable. These forms of "positive" deception are part of most SBE activities and are used to help learners bridge the reality gap so they can suspend disbelief more easily, exercise critical thinking, and treat the simulation more realistically without damaging the trust they place in their educators. This article will discuss how aspects of SBE activities are often manipulated, modified, or hidden from learners to facilitate the learning experience and present a simulation fidelity model encompassing the environmental, patient, semantical, and phenomenal dimensions.

Simulation-based assessment is poised for application in educational promotion and credentialing in gynecologic surgery. With high-stakes assessment, validation necessitates evidence, not just about the trustworthiness of decisions but of beneficial consequences to education and healthcare as well. In this paper, we unpack the modern conceptualization of validity as it pertains to surgical simulation and high-stakes competency assessment.

N/A SETTING: N/A PATIENTS: N/A INTERVENTIONS: N/A MEASUREMENTS AND MAIN RESULTS: N/A CONCLUSION: Validity in high-stakes simulation-based assessment necessitates evidence, not just about the trustworthiness of score-based decisions but of beneficial consequences to education and healthcare as well.

As the average age of surgeons continues to rise, determining when a surgeon should retire is an important public safety concern.

To investigate strategies used to determine competency in the industrial workplace that could be transferrable in the assessment of aging surgeons and to identify existing competency assessments of practicing surgeons.

We searched websites describing non-medical professions within the United States where cognitive and physical competency are necessary for public safety. The mandatory age and certification process, including cognitive and physical requirements, were reported for each profession. Methods for determining surgical competency currently in use, and those existing in the literature, were also identified.

Four non-medical professions requiring mental and physical aptitude that involve public safety and have mandatory testing and/or retirement were identified: Airline pilots, air traffic controllers, firefighters, and United States State Judges. Nine late career practitioner policies designed to evaluate the ageing physician, including surgeons, were described. Six of these policies included subjective performance testing, 4 using peer assessment and 2 using dexterity testing. Six objective testing methods for evaluation of surgeon technical skill were identified in the literature. All were validated for surgical trainees. Only Objective Structured Assessment of Technical Skills (OSATS) was capable of distinguishing between surgeons of different skill level and showing a relationship between skill level and post-operative outcomes.

A surgeon should not be forced to hang up his/her surgical cap at a predetermined age, but should be able to practice for as long as his/her surgical skills are objectively maintained at the appropriate level of competency. The strategy of using skill-based simulations in evaluating non-medical professionals can be similarly used as part of the assessment of the ageing surgeons' surgical competency, showing who may require remediation or retirement.

Continuing professional development (CPD) activities delivered by simulation to independently practicing physicians are becoming increasingly popular. At present, the educational potential of such simulations is limited by the inability to create effective curricula for the CPD audience. In contrast to medical trainees, CPD activities lack pre-defined learning expectations and, instead, emphasize self-directed learning, which may not encompass true learning needs. We hypothesize that we could generate an interprofessional CPD simulation curriculum for practicing pediatric emergency medicine (PEM) physicians in a single-center tertiary care hospital using a deliberative approach combined with Kern's six-step method of curriculum development.

From a comprehensive core list of 94 possible PEM clinical presentations and procedures, we generated an 18-scenario CPD simulation curriculum. We conducted a comprehensive perceived and unperceived needs assessment on topics to include, incorporating opinions of faculty PEM physicians, hospital leadership, interprofessional colleagues, and expert opinion on patient benefit, simulation feasibility, and value of simulating the case for learning. To systematically rank items while balancing the needs of all stakeholders, we used a prioritization matrix to generate objective "priority scores." These scores were used by CPD planners to deliberately determine the simulation curriculum contents.

We describe a novel three-step CPD simulation curriculum design method involving (1) systematic and deliberate needs assessment, (2) systematic prioritization, and (3) curriculum synthesis. Of practicing PEM physicians, 17/20 responded to the perceived learning needs survey, while 6/6 leaders responded to the unperceived needs assessment. These ranked data were input to a five-variable prioritization matrix generating priority scores. Based on local needs, the highest 18 scoring clinical presentations and procedures were selected for final inclusion in a PEM CPD simulation curriculum. An interim survey of PEM physician (21/24 respondents) opinions was collected, with 90% finding educational value with the curriculum. The curriculum includes items not identified by self-directed learning that PEM physicians thought should be included.

We highlight a novel methodology for PEM physicians that can be adapted by other specialities when designing their own CPD simulation curriculum. This methodology objectively considers and prioritizes the needs of practicing physicians and stakeholders involved in CPD.

The Accreditation Council for Graduate Medical Education (ACGME) has recently implemented milestones and competencies as a framework for training fellows in Pain Medicine, but individual programs are left to create educational platforms and assessment tools that meet ACGME standards.

In this article, we discuss the concept of milestone-based competencies and the inherent challenges for implementation in pain medicine. We consider simulation-based education (SBE) as a potential tool for the field to meet ACGME goals through advancing novel learning opportunities, engaging in clinically relevant scenarios, and mastering technical and nontechnical skills.

The sparse literature on SBE in pain medicine is highlighted, and we describe our pilot experience, which exemplifies a nascent effort that encountered early difficulties in implementing and refining an SBE program.

The many complexities in offering a sophisticated simulated pain curriculum that is valid, reliable, feasible, and acceptable to learners and teachers may only be overcome with coordinated and collaborative efforts among pain medicine training programs and governing institutions.

We developed simulator-based tools for assessing provider competence in transthoracic echocardiography (TTE) and vascular duplex scanning.

Psychomotor (technical) skill in TTE image acquisition was calculated from the deviation angle of an acquired image from the anatomically correct view. We applied this metric for formative assessment to give feedback to learners and evaluate curricula.Psychomotor skill in vascular ultrasound was measured in terms of dexterity and image plane location; cognitive skill was assessed from measurements of blood flow velocity, parameter settings, and diagnosis. The validity of the vascular simulator was assessed from the accuracy with which experts can measure peak systolic blood flow velocity (PSV).

In the TTE simulator, the skill metric enabled immediate feedback, formative assessment of curriculum efficacy, and comparison of curriculum outcomes. The vascular duplex ultrasound simulator also provided feedback, and experts' measurements of PSV deviated from actual PSV in the model by <10%.

Skill in acquiring diagnostic ultrasound images of organs and vessels can be measured using simulation in an objective, quantitative, and standardized manner. Current applications are provision of feedback to learners to enable training without direct faculty oversight and formative assessment of curricula. Simulator-based metrics could also be applied for summative assessment.

Healthcare providers who use peripheral vascular and cardiac ultrasound require specialized training to develop the technical and interpretive skills necessary to perform accurate diagnostic tests. Assessment of competence is a critical component of training that documents a learner's progress and is a requirement for competency-based medical education (CBME) as well as specialty certification or credentialing. The use of simulation for CBME in diagnostic ultrasound is particularly appealing since it incorporates both the psychomotor and cognitive domains while eliminating dependency on the availability of live patients with a range of pathology. However, successful application of simulation in this setting requires realistic, full-featured simulators and appropriate standardized metrics for competency testing. The principal diagnostic parameter in peripheral vascular ultrasound is measurement of peak systolic velocity (PSV) on Doppler spectral waveforms, and simulation of Doppler flow detection presents unique challenges. The computer-based duplex ultrasound simulator developed at the University of Washington uses computational fluid dynamics modeling and presents real-time color-flow Doppler images and Doppler spectral waveforms along with the corresponding B-mode images. This simulator provides a realistic scanning experience that includes measuring PSV in various arterial segments and applying actual diagnostic criteria. Simulators for echocardiography have been available since the 1990s and are currently more advanced than those for peripheral vascular ultrasound. Echocardiography simulators are now offered for both transesophageal echo and transthoracic echo. These computer-based simulators have 3D graphic displays that provide feedback to the learner and metrics for assessment of technical skill that are based on transducer tracking data. Such metrics provide a motion-based or kinematic analysis of skill in performing cardiac ultrasound. The use of simulation in peripheral vascular and cardiac ultrasound can provide a standardized and readily available method for training and competency assessment.

Objective measurement of simulation performance requires a validated and reliable tool. However, no published Italian language assessment tool is available. Translation of a published English language tool, the Ottawa Crisis Resource Management Global Rating Scale (GRS), may lead to a validated and reliable tool. After developing an Italian language translation of the English language tool, the study measured the reliability of the new tool by comparison with the English language tool used independently in the same simulation scenarios. In addition, the validity of the Italian language tool was measured by comparison to a skills score also applied independently. The correlation coefficient between the Italian language overall GRS and the English language overall GRS was 0.82 (adjusted 95 % confidence interval: 0.62-0.92). The correlation coefficient between the Italian language overall GRS and the skill score was 0.85 (adjusted 95 % confidence interval 0.68-0.94). This study demonstrated that the Italian language GRS has acceptable reliability when compared with the English language tool, suggesting that it can be used reliably to evaluate the performance during simulated emergencies. The study also suggests that the tool has acceptable validity for assessing the simulation performance. The study suggests that the Italian language GRS translation has reasonable reliability when compared with the English language GRS and reasonable validity when compared with the assessment of the skills scores. Data suggest that the instrument is adequately reliable for informal and formative type of examinations, but may require further confirmation before use for high-stake examinations such as licensing.

Determining clinical competency on the milestones requires direct observation of residents, which is difficult for faculty members who are also providing patient care. Simulation can potentially represent an effective standardized tool for high-stakes assessment. Using a longitudinal simulation curriculum with formative and summative components, we conducted a pilot investigation to examine whether (1) performance on the formative cases predicted performance on the summative cases and (2) performance on the summative cases correlated with the clinical competency committee's (CCC) milestone placement.

We developed 6 formative and 6 matched summative cases for pediatric interns that covered core pediatric topics. The interns progressed through the formative cases in pairs during the course of the academic year and then through the summative cases back to back individually at the end of the year. The interns were evaluated using a competency-based simulation evaluation. We determined the relationship between the formative and summative scores and between the summative scores and those from the CCC.

The relationship between formative and summative scores was not statistically significant. There was a statistically significant relationship between summative and CCC scores.

Significant methodological limitations preclude definite conclusions about the predictive power of simulation cases for Pediatric Milestones-based assessment. However, our work is an example of how simulation is a potentially useful tool for assessing residents' skill development on the Pediatric Milestones. More rigorous research is needed to determine the extent to which simulation can be used for high-stakes, milestones-based assessment.

To explore and categorise the state of existing literature for national programmes designed to affirm or establish the continuing competence of physicians.

Scoping review.

MEDLINE, ERIC, Sociological Abstracts, web/grey literature (2000-2014).

Included when a record described a (1) national-level physician validation system, (2) recognised as a system for affirming competence and (3) reported relevant data.

Using bibliographic software, title and abstracts were reviewed using an assessment matrix to ensure duplicate, paired screening. Dyads included both a methodologist and content expert on each assessment, reflective of evidence-informed best practices to decrease errors.

45 reports were included. Publication dates ranged from 2002 to 2014 with the majority of publications occurring in the previous six years (n=35). Country of origin--defined as that of the primary author--included the USA (N=32), the UK (N=8), Canada (N=3), Kuwait (N=1) and Australia (N=1). Three broad themes emerged from this heterogeneous data set: contemporary national programmes, contextual factors and terminological consistency. Four national physician validation systems emerged from the data: the American Board of Medical Specialties Maintenance of Certification Program, the Federation of State Medical Boards Maintenance of Licensure Program, the Canadian Revalidation Program and the UK Revalidation Program. Three contextual factors emerged as stimuli for the implementation of national validation systems: medical regulation, quality of care and professional competence. Finally, great variation among the definitions of key terms was identified.

There is an emerging literature focusing on national physician validation systems. Four major systems have been implemented in recent years and it is anticipated that more will follow. Much of this work is descriptive, and gaps exist for the extent to which systems build on current evidence or theory. Terminology is highly variable across programmes for validating physician competence and fitness for practice.

To determine which of the 21 general pediatrics milestone subcompetencies are most difficult to assess using traditional methodologies and which are best suited to simulation-based assessment.

We surveyed 2 samples: pediatric simulation experts and pediatric program directors. Respondents were asked about current use of simulation for assessment and to select 5 of the 21 pediatric subcompetencies most difficult to assess using traditional methods and the 5 best suited to simulation-based assessment. Spearman rank correlation was used to determine a correlation between how the 2 samples ranked the subcompetencies.

Forty-eight percent (29 of 60) simulation experts and 20% (115 of 571) program directors completed the survey. Few respondents reported using simulation for summative assessment. There are clear differences across the pediatric subcompetencies in perceived difficulty of assessment and suitability to simulation-based assessment. The 3 most difficult to assess subcompetencies were "recognize ambiguity," "demonstrate emotional insight," and "identify one's own strengths and deficiencies." The subcompetencies most suitable to assessment using simulation were "interprofessional teamwork," "clinical decision making," and "effective communication." Program directors and simulation experts had high agreement for both questions: difficult to assess (rho = 0.76, P < .001) and suitable to simulation-based assessment (rho = 0.94, P < .001).

Several general pediatrics milestone subcompetencies were identified by pediatric simulation experts and pediatric program directors as difficult to assess using current methodologies and as amenable to simulation-based assessment. The pediatric simulation community should target development of simulation-based assessment tools to these areas.

Providing adequate bag-mask ventilation (BMV) is an essential skill for neonatal resuscitation. Often this skill is learned using simulation manikins. Currently, there is no means of measuring the adequacy of ventilation in simulated scenarios. Thus, it is not possible to ascertain proficiency. The first aim of this study was to measure the pressure generated during BMV as performed by providers with different skill levels and measure the impact of different feedback mechanisms. The second aim was to measure the pressure volume characteristics of two neonatal manikins to see how closely they reflect newborn lung mechanics.

In Phase I to achieve the first aim, we evaluated BMV skills in different level providers including residents (n=5), fellows (n=5), neonatal nurse practitioners (n=5) and neonatologists (n=5). Each provider was required to provide BMV for 2-min epochs on the SimNewB (Laerdal), which had been instrumented to measure pressure-volume characteristics. In sequential 2-min epochs, providers were given different feedback including chest-wall movement alone compared to manometer plus chest-wall movement or chest-wall movement plus manometer plus laptop lung volume depiction. In Phase II of the study we measured pressure-volume characteristics in instrumented versions of the SimNewB (Laerdal) and NeoNatalie (Laerdal).

In Phase I, all providers are compared with the neonatologists. All measurements of tidal volume (Vt) are below the desired 5 ml kg(-1). The greatest difference in Vt between the neonatologists and other providers occurs when only chest-wall movement is provided. A linear relationship is noted between Vt and PIP for both SimNewB and NeoNatalie. The compliance curves are not 'S-shaped' and are different between the two models (P<0.001).

Phase I of this study demonstrates that the SimNewB with the feedback of chest-wall movement alone was the best method of distinguishing experienced from inexperienced providers during simulated BMV. Therefore this is likely to be the best method to ascertain proficiency. Phase II of the study shows that the currently available neonatal simulation manikins do not have pressure-volume characteristics that are reflective of newborn lung mechanics, which can result in suboptimal training.

Simulation-based medical education has been growing rapidly and becomes one of the most popular teaching methods for improving patient safety and patient care. The Simulation Subcommittee of the Hong Kong College of Emergency Medicine organized an educational program emphasizing the team training, clinical decision-making and communication skills. This study aimed to evaluate the attitude of the participants toward a new training program and the change in the knowledge on clinical performance in emergency physicians and nurses after attending the educational program.

A course evaluation form was filled in by the participants at the end of the workshop. An assessment of 20 multiple-choice questions with 5 options was administered to the participants before and after the 2-day simulation-based training workshop.

A total of 72 doctors and nurses working in the Accident and Emergency Department were enrolled. The average pretest and posttest scores were 12 and 14.3 respectively. The percentage improvement in the mean score of the pretest and posttest was 11.5%. The Chi-square test showed significant improvement in the pretest and posttest score grading (P=0.00). Paired t-test revealed significant difference between the mean scores of the pretest and posttest (P=0.00).

Participants had positive attitude toward this new training program. Significant improvement of the knowledge on clinical performance in healthcare professionals in the Accident and Emergency Department was observed after the participation in this simulation-based educational program.

Simulation-based training has been associated with reduced central line-associated bloodstream infection (CLABSI) rates. We measured the combined effect of simulation training, electronic medical records (EMR)-based documentation, and standardized kits on CLABSI rates in our medical (MICU) and surgical (SICU) intensive care units (ICU).

CLABSI events and catheter-days were collected for 19 months prior to and 37 months following an intervention consisting of simulation training in central line insertion for all ICU residents, incorporation of standardized, all-inclusive catheter kits, and EMR-guided documentation. Supervising physicians in the MICU (but not the SICU) also completed training.

Following the intervention, EMR-based documentation increased from 48% to 100%, and documented compliance with hand hygiene, barrier precautions, and chlorhexidine use increased from 65%-85% to 100%. CLABSI rate in the MICU dropped from 2.72 per 1,000 catheter-days over the 19 months preceding the intervention to 0.40 per 1,000 over the 37 months following intervention (P = .01) but did not change in the SICU (1.09 and 1.14 per 1,000 catheter-days, P = .86). This equated to 24 fewer than expected CLABSIs and $1,669,000 in estimated savings.

Combined simulation training, standardized all-inclusive kits, and EMR-guided documentation were associated with greater documented compliance with sterile precautions and reduced CLABSI rate in our MICU. To achieve maximal benefit, refresher training of senior physicians supervising practice at the bedside may be needed.

The accreditation of medical school programmes and the licensing and revalidation (or recertification) of doctors are thought to be important for ensuring the quality of health care. Whereas regulation of the medical profession is mandated in most jurisdictions around the world, the processes by which doctors become licensed, and maintain their licences, are quite varied. With respect to educational programmes, there has been a recent push to expand accreditation activities. Here too, the quality standards on which medical schools are judged can vary from one region to another.

Given the perceived importance placed by the public and other stakeholders on oversight in medicine, both at the medical school and individual practitioner levels, it is important to document and discuss the regulatory practices employed throughout the world.

This paper describes current issues in regulation, provides a brief summary of research in the field, and discusses the need for further investigations to better quantify relationships among regulatory activities and improved patient outcomes.

Although there is some evidence to support the value of medical school accreditation, the direct impact of this quality assurance initiative on patient care is not yet known. For both licensure and revalidation, some investigations have linked specific processes to quality indicators; however, additional evaluations should be conducted across the medical education and practice continuum to better elucidate the relationships among regulatory activities and patient outcomes. More importantly, the value of accreditation, licensure and revalidation programmes around the world, including the effectiveness of specific protocols employed in these diverse systems, needs to be better quantified and disseminated.

Over the past two decades, there has been an exponential and enthusiastic adoption of simulation in healthcare education internationally. Medicine has learned much from professions that have established programs in simulation for training, such as aviation, the military and space exploration. Increased demands on training hours, limited patient encounters, and a focus on patient safety have led to a new paradigm of education in healthcare that increasingly involves technology and innovative ways to provide a standardized curriculum. A robust body of literature is growing, seeking to answer the question of how best to use simulation in healthcare education. Building on the groundwork of the Best Evidence in Medical Education (BEME) Guide on the features of simulators that lead to effective learning, this current Guide provides practical guidance to aid educators in effectively using simulation for training. It is a selective review to describe best practices and illustrative case studies. This Guide is the second part of a two-part AMEE Guide on simulation in healthcare education. The first Guide focuses on building a simulation program, and discusses more operational topics such as types of simulators, simulation center structure and set-up, fidelity management, and scenario engineering, as well as faculty preparation. This Guide will focus on the educational principles that lead to effective learning, and include topics such as feedback and debriefing, deliberate practice, and curriculum integration - all central to simulation efficacy. The important subjects of mastery learning, range of difficulty, capturing clinical variation, and individualized learning are also examined. Finally, we discuss approaches to team training and suggest future directions. Each section follows a framework of background and definition, its importance to effective use of simulation, practical points with examples, and challenges generally encountered. Simulation-based healthcare education has great potential for use throughout the healthcare education continuum, from undergraduate to continuing education. It can also be used to train a variety of healthcare providers in different disciplines from novices to experts. This Guide aims to equip healthcare educators with the tools to use this learning modality to its full capability.

In an integrated curriculum such as problem-based learning (PBL), students need to develop a number of learning skills and competencies. These cannot be achieved through memorization of factual knowledge but rather through the development of a wide range of cognitive and noncognitive skills that enhance deep learning.

The aim of this article is to provide students and teachers with learning approaches and learning strategies that enhance deep learning.

We reviewed current literature in this area, explored current theories of learning, and used our experience with medical students in a number of universities to develop these tips.

Incorporating the methods described, we have developed 12 tips and organized them under three themes. These tips are (1) learn how to ask good questions, (2) use analogy, (3) construct mechanisms and concept maps, (4) join a peer-tutoring group, (5) develop critical thinking skills, (6) use self-reflection, (7) use appropriate range of learning resources, (8) ask for feedback, (9) apply knowledge learnt to new problems, (10) practice learning by using simulation, (11) learn by doing and service learning, and (12) learn from patients.

Practicing each of these approaches by students and teachers and applying them in day-to-day learning/teaching activities are recommended for optimum performance.

Competency-based assessment and an emphasis on obtaining higher-level outcomes that reflect physicians' ability to demonstrate their skills has created a need for more advanced assessment practices. Simulation-based assessments provide medical education planners with tools to better evaluate the 6 Accreditation Council for Graduate Medical Education (ACGME) and American Board of Medical Specialties (ABMS) core competencies by affording physicians opportunities to demonstrate their skills within a standardized and replicable testing environment, thus filling a gap in the current state of assessment for regulating the practice of medicine. Observational performance assessments derived from simulated clinical tasks and scenarios enable stronger inferences about the skill level a physician may possess, but also introduce the potential of rater errors into the assessment process. This article reviews the use of simulation-based assessments for certification, credentialing, initial licensure, and relicensing decisions and describes rater training strategies that may be used to reduce rater errors, increase rating accuracy, and enhance the validity of simulation-based observational performance assessments.