The Program for Medical Education Innovations and Research (PrMEIR), part of NYU Langone’s Division of General Internal Medicine and Clinical Innovation, is dedicated to advancing medical education scholarship and developing best practices for patient-centered, evidence-based medical education. Our research initiatives strengthen the links between physician training and patient health and wellbeing.

At PrMEIR, we’ve earned a national reputation for our medical education outcomes research, innovative use of standardized patients, and development of performance-based assessments to measure core competence in medical training programs.

Research on Medical Education Outcomes

How do we know that medical education prepares healthcare providers to deliver high-quality care? At NYU Langone, an interdisciplinary team of doctors, nurses, researchers, educators, and psychologists—known as Research on Medical Education Outcomes, or ROMEO—investigates this question by linking medical education strategies to patient health outcomes. Through weekly meetings that feature work-in-progress presentations, journal clubs, and targeted working groups, ROMEO has created and nurtured a vibrant community of medical education research practice.

Our goal is to ensure that medical students, residents, and fellows are equipped to deliver excellent care. ROMEO research informs innovative curricula, emphasizing the importance of communication skills, health literacy, patient activation, and professionalism in treatments for chronic conditions such as obesity and substance use. This collaboration led to the creation of the Educationally Sensitive Patient Outcome (ESPO) Theoretical Model, now recognized globally as a framework for identifying health outcomes tied to provider training, experience, and core clinical skills.

In 2025, ROMEO renewed its focus on ESPOs to leverage advances in competency-based assessment, electronic health record data, and analytics, including Resident Sensitive Quality Measures (RSQMs) and TRainee Attributable & Automatable Care Evaluations in Real-time (TRACERs). Five interdisciplinary working groups are charting new directions—updating the literature on the role and importance of ESPOs in medical education, exploring clinical learning environments, applying machine learning to existing data, harnessing unannounced standardized patient (USP) visits, and developing new ESPO measures. These efforts aim to create precise, learner-attributable outcomes that connect education to patient care. Ultimately, this work positions ESPOs as a cornerstone of evidence-based medical education research with broad implications for healthcare quality and system improvement.

To accomplish this, we are leveraging the Database for Research on Academic Medicine (DREAM), which collects assessment and performance data from medical students, residents, fellows, and practicing physicians. Results of these efforts will be integrated with existing information and prior efforts to expand our collective knowledge of the continuum of lifelong learning.

Database for Research on Education in Academic Medicine

Our Database for Research on Education in Academic Medicine (DREAM) is a Framingham-style longitudinal database for researching relationships between medical education, residency, fellowship programs, and patient outcomes.

DREAM has data from more than 5,000 medical students, residents, and fellows collected for more than 10 years; it was made possible by an innovative Institutional Review Board–approved medical education research registry. The database holds great promise for overcoming challenges that limit the development of evidence-based medical education and training programs.

DREAM’s strengths include the following:

• the ability to link information over the full span of medical education, from medical school through residency to actual practice
• data collection from a variety of sources and methods
• the use of gold standard performance-based measures, such as observed structured clinical exams and unannounced standardized patient visits
• integration with electronic medical records, patient charts, and patient exit interviews

More than 70 studies have been published using this data, including studies that assess medical school and training curricula, quality of patient care, provider competency, skills development, the transfer of skills to practice, and the effects of education on patient outcomes.

Unannounced Standardized Patient Program

As part of the Unannounced Standardized Patient (USP) Program, developed by Sondra R. Zabar, MD, skilled medical educators train actors to recognize certain hallmarks of high-quality healthcare that should be present during a standard patient encounter. The actors then go undercover as patients at ambulatory clinics where our general internal medicine residents train. The actors engage with medical teams, assess how responsive our healthcare providers are to patient needs, evaluate the experience from a patient perspective, and provide feedback to medical teams on performance.

In collaboration with our clinical partners, we have also deployed USP visits to evaluate quality improvement efforts, assess team functioning, test readiness of clinical units for handling public health threats, and monitor the effects of large-scale changes in healthcare delivery.

Special Pathogens Readiness

There is a constant need to prepare New York City hospitals and clinics to provide effective treatment to patients carrying special pathogens, or emergent infectious diseases. To meet one such need, PrMEIR’s USP team developed an innovative partnership with representatives from NYC Health + Hospitals and New York City Department of Health and Mental Hygiene (DOHMH) to ensure readiness for an avian flu virus disease (H5N1) case.

This collaborative partnership is engaging in a series of activities to ensure the healthcare systems across New York City, New York State, Puerto Rico, and the US Virgin Islands are ready to safely and successfully identify, isolate, assess, transport, and treat patients with H5N1 or patients under investigation for H5N1, and that it is well prepared for future emerging infectious disease outbreaks. These activities include case development, implementation of comprehensive USP visits at each site, and feedback for care teams about the engagement, both in the moment and after the visit.

Role Transitions in Medical Education

In collaboration with the New York Simulation Center for the Health Sciences (NYSIM), PrMEIR faculty developed Night onCall and First Night On-Call, two simulation tools designed to improve periods of role transition during medical education, which are known to be stressful for medical students and house staff. They also are investigating a new area, professional identity formation, in medical education research.

First Night On-Call

First Night On-Call (FNOC) is an immersive four-hour simulation that is part of orientation for incoming interns at NYU Langone to ensure they are ready for common patient safety challenges on day one of residency. During FNOC, new interns collaborate with one another as they rotate through a series of patient safety‒related role-playing scenarios in small groups. These scenarios focus on specific actions that enhance patient safety, such as escalation of care to a superior and the reporting of a medical error.

We developed and implemented FNOC in 2017 to address the need to improve escalation and cultivate a culture of safety at NYU Langone. FNOC has been consistently well received among the NYU Langone residents who complete the program each year.

Our institutional performance on the Agency for Healthcare Research and Quality’s Culture of Safety Survey has also demonstrated improvement. The program has received international, national, and local recognition, winning prestigious awards such as the Best Patient Safety and Quality Presentation at the Society of General Medicine and the Best Medical Education Research Poster at NYU Langone’s Medical Education Innovations and Scholarship Day.

Night onCall

Night onCall (NOC) is an immersive, multistation simulation designed to assess and prepare medical students who are about to graduate for the transition to residency. During the experience, learners assume the on-call responsibilities of a resident intern while rotating through a series of clinically authentic scenarios. Students receive standardized, multisource feedback from standardized patients, nurses, and attending physicians, with performance evaluated across the 13 Core Entrustable Professional Activities (EPAs). NOC supports and empowers graduating medical students to practice “putting it all together,” building confidence and readiness for residency.

Developed in 2016 at NYU Grossman School of Medicine as a research project, Night onCall has since been incorporated into the curricula of multiple medical schools in the United States in both in-person and virtual formats. Over time, NOC has been presented at various international and national conferences within the health professions education community.

Night onCall Research Consortium to Assess, Prepare, and Support Transition to Internship for Medical Students

Supported by funding from the Josiah Macy Jr. Foundation (2021 to 2023), the Night onCall Research Consortium was established to adapt and study the immersive Night onCall (NOC) multistation simulation across institutions. The consortium has grown to include partnerships across both undergraduate and graduate medical education, advancing the development, study, and implementation of NOC to support graduating medical students as they transition to residency. The consortium includes Medical College of Wisconsin–Milwaukee, Medical College of Wisconsin–Central Wisconsin, Medical College of Wisconsin–Green Bay, Texas Tech University Health Sciences Center–El Paso, UC Davis School of Medicine, CUNY School of Medicine, McGovern Medical School at UTHealth, Elson S. Floyd College of Medicine at Washington State University, and Kaiser Permanente Bernard J. Tyson School of Medicine.

In addition to the NOC simulation, its platform is being used to develop a suite of tools, including FeedbackAssist (AI-assisted clinical note evaluation and feedback), NOC Live (video-based encounter capture), and Coach Remy (data-informed coaching and remediation support). Ongoing validation and educational alignment guide the integration of these tools as development continues.

Professional Identity Formation

Being a doctor involves more than just providing clinical care. At NYU Grossman School of Medicine, professional identity formation (PIF)—the process of internalizing a profession’s core values and beliefs—is an explicit goal of medical education and key to the safety and quality of patient care.

The Division of General Internal Medicine and Clinical Innovation is pioneering a new area of research to better understand physicians in training and what we can do to assist them in developing a professional identity.

PrMEIR faculty and national medical education leaders are building culture change through open dialogue aimed at recognizing professional identity as part of the growth process necessary to become a physician. Efforts include re-establishing the role and importance of PIF in medical education, exploring the expansion of PIF to include residents, and linking PIF data with other performance-based assessment data to explore patterns of learner development and expand our evidence base.

Improving the Quality of Feedback in Medical Education Training and Assessment

Faculty and staff in the Program for Medical Education Innovations and Research (PrMEIR), have partnered with researchers at NYU Langone’s Institute for Innovations in Medical Education (IIME) to implement an innovative, Josiah Macy Jr. Foundation-funded project exploring the use of AI-enhanced assessment and feedback for resident trainees.

Yuliya N. Yoncheva, PhD, Jesse B. Rafel, MD, and PrMEIR staff are spearheading the effort to reinvent the current feedback framework. This system will use advanced AI to provide rigorous, high-quality, personalized feedback to trainees on their communication skills. Using AI and triangulation of data captured across multiple years of objective structured clinical examinations (OSCE), the randomized-trial-style project will ultimately test the impact of AI-powered feedback on educational outcomes and patient care.

Exploring Patterns of Performance and Quality of Telehealth Assessment

As part of a multiyear Association of Americal Medical Colleges (AAMC) grant, PrMEIR faculty and staff, including Dr. Zabar; Colleen Gillespie, PhD; and Lisa Altshuler, PhD, have been exploring patterns of performance among trainees who participated in telehealth-based OSCEs and expanding the use of a standardized telehealth skill checklist. The goal of these efforts include understanding variation in skills required to conduct telehealth versus in-person visits; identifying gaps in trainee skills across each level of the educational continuum, creating and disseminating a just-in-time educational tool kit to improve practices, and exploring the quality of the standardized assessment tool via a validation study. In its final year of study (2026), the team is proactively disseminating results of these efforts through a series of academic manuscripts.