Residency is often seen as the defining stage of medical education, but for new medical school graduates, the transition can be quite challenging. As new MDs, students are expected to put their learning into practice immediately. Medical students go through hundreds of hours of training, studying, exams, and preparation before they start their residencies. Even though they are closely supervised by attending physicians, the first night on call for almost every new resident is a harrowing and memorable experience. Lack of proper preparation, training, and practice can not only cause immense stress for residents but may also affect the safety and quality of care that patients receive.
Dr. Adina Kalet has a long history of identifying and understanding how to best train learners at different levels and recognized that preparation for incoming residents, particularly for their first night on call, was lacking. By exposing new residents to various types of patient encounters prior to the first night on call, Dr. Kalet believed that she could help ease the transition from medical school to residency. “Suddenly going from not being a physician in practice to having the responsibility for patients and being alone in the middle of the night, making assessments and decisions, and then knowing when to call for help, is daunting,” said Dr. Kalet. “We needed something to help students understand what that feels like so they can make that transition effectively.”
So, Dr. Kalet joined forces with Dr. Sondra Zabar, and together with support from the Josiah Macy Jr. Foundation, NYU Grossman School of Medicine, and Medical College of Wisconsin, they launched the Night-onCall (NOC) program, which eventually became a consortium. NOC assesses the readiness of soon-to-be residents and helps them prepare for the challenges they will encounter transitioning to residency.
The Night-onCall Program
Night-onCall simulates a typical night on call experienced by new resident physicians utilizing standardized patients, nurses, residents, attending physicians, and patient partners.
Night-On Call participants are provided information about the goals and expectations of the program and are encouraged to prepare for the three-hour immersive simulation experience by reviewing online course materials–the WISE-OnCall modules–which are part of the NYU Grossman School of Medicine’s Web Initiative for Surgical Education (WISE). This critical step ensures learners have confidence in their knowledge and handling of situations they will face in the simulations. After the initial preparation, learners go through the simulation with patient scenarios that present complex and authentic clinical challenges.
NOC started with just three clinical cases to prepare learners for routine clinical issues but has grown to six with funding from the Macy Foundation. Three of these cases are selected for each learner’s first night on-call simulation by their educational program leaders. While the initial three cases challenged learners to assess and address routine situations involving hospitalized patients, such as a decrease in urine production or elevated blood pressure, the new NOC cases provide learners with the opportunity to practice addressing emerging health issues and situations. For example, one case starts with the learner being paged by a nurse to come to the bedside of a patient, complaining that “they forgot to give me pain medication in the Emergency Room.” The learner is challenged to evaluate their patient’s pain and determine an appropriate treatment plan. Another case involves learners caring for a man who identifies as transgender and is experiencing abdominal pain. Learners are expected to to take a sexual history and perform an organ inventory and initial assessment. Night-OnCall has also added a simulation that engages learners in responding to explicit bias from a patient.
The activities that learners work on for each case are designed specifically to assess crucial components of practicing medicine independently. Depending on the case, learners might perform a literature search, give an oral presentation to an attending physician, or answer a series of questions related to ensuring a culture of safety. During the simulation, learners are evaluated by standardized patients, nurses, residents, attending physicians, and patient partners and demonstrate their clinical reasoning by writing a patient care note as they would in actual practice. At the end of every NOC experience, participants hand off each case to a resident to emulate a typical morning transfer of patient care when interns finish a shift.
Once a NOC simulation is completed, faculty use a collaborative AI tool, FeedbackAssist, to provide participating students with prompt feedback on their clinical reasoning along with other performance measures. Dr. Tavinder K. Ark is NOC’s data scientist, supporting sites in capturing data which is used to to report and interpret the results for students and curriculum leaders.
“A big push for NOC was to get data back to students in a timely manner so it would actually be helpful,” emphasized Dr. Ark. “Generating data was important, but translating it into meaningful visualization and outcomes for those using it was the other really critical step.” The data collected helps students identify areas of improvement and can also help medical schools identify trends across classes to see what might be missing from the curricula.
Learner feedback has been immensely positive–many proclaiming their relief at having that chance for realistic but low-risk preparation before being thrown into residency. “We realized this was a final exam of sorts, but without the high-stakes pressure,” said Dr. Kalet. “NOC created a safe environment where everyone is receiving meaningful feedback to support their growth into excellent physicians. This helps near-graduates come together and learn from each other.”
Night-onCall Consortium
After the initial success of the Night-onCall program, the team hoped to bring the simulation to a wider audience of medical schools. They developed the Night-onCall Consortium to bring various schools and partners together to help expand and improve the modules and simulations and ensure the program was responsive to the needs of learners.
The consortium now includes eight medical schools with a total of 1,620 eligible learners annually. It has allowed NOC to expand cases by gathering input from member schools and identifying needs. It has also created an open culture for any consortium member to pick up the phone and request adjustments to the program tailored to their own needs and unique contexts. Additionally, all members of the consortium have agreed to use two of the same patient cases, allowing for richer comparisons across the country. The consortium members meet several times a year to report on their experiences and contribute to improving the program.
This collaborative approach has created an even more effective simulation tool. Thanks to the vested interest consortium members have in the program, the NOC team has been able to respond to needs and opportunities as they arise.
For instance, by the spring of 2020, the NOC Consortium had grown to multiple schools across the country when the COVID-19 pandemic hit, mere weeks before medical school graduation. Medical students across the country suddenly faced the prospect of being unable to complete the requirements for their MD in time to graduate. The NOC team sprang into action, adapting the simulations for the remote environment and helped almost 150 students complete their graduation requirements in time to join their new residency programs in July. “We went to the accreditors and asked for special dispensation ‘if students can pass NOC, then with the approval of their faculty, can they graduate from medical school?’” said Dr. Kalet. “And we were given approval because we were able to show the scholarship around NOC and the engagement of the consortium.”
What’s Next for NOC
In November 2023, the NOC team was invited to share their work with all medical schools across the country via a presentation at the annual Association of American Medical Colleges (AAMC) meeting. The NOC team is hoping to expand the simulation with more cases and more consortium partners. They are collecting data with the goal of creating performance benchmarks so students can know where they stand compared with peers and to help refine standardization across medical schools so areas for curricular improvement are more accurately identified.
“The evolution of Night-onCall has been both exciting and effective,” said Dr. Zabar. “Our program has proven to have a significant impact on easing the transition from medical school to residency for learners. We hope the consortium will continue to grow and help ensure better quality care for patients.”
You can learn more about the program, the team, and how to become part of the consortium on the NOC website.
Interested in becoming a Macy Foundation grantee? Check out our grantee page for upcoming opportunities.