OBM Geriatrics is an Open Access journal published quarterly online by LIDSEN Publishing Inc. The journal takes the premise that innovative approaches – including gene therapy, cell therapy, and epigenetic modulation – will result in clinical interventions that alter the fundamental pathology and the clinical course of age-related human diseases. We will give strong preference to papers that emphasize an alteration (or a potential alteration) in the fundamental disease course of Alzheimer’s disease, vascular aging diseases, osteoarthritis, osteoporosis, skin aging, immune senescence, and other age-related diseases.

Geriatric medicine is now entering a unique point in history, where the focus will no longer be on palliative, ameliorative, or social aspects of care for age-related disease, but will be capable of stopping, preventing, and reversing major disease constellations that have heretofore been entirely resistant to interventions based on “small molecular” pharmacological approaches. With the changing emphasis from genetic to epigenetic understandings of pathology (including telomere biology), with the use of gene delivery systems (including viral delivery systems), and with the use of cell-based therapies (including stem cell therapies), a fatalistic view of age-related disease is no longer a reasonable clinical default nor an appropriate clinical research paradigm.

Precedence will be given to papers describing fundamental interventions, including interventions that affect cell senescence, patterns of gene expression, telomere biology, stem cell biology, and other innovative, 21st century interventions, especially if the focus is on clinical applications, ongoing clinical trials, or animal trials preparatory to phase 1 human clinical trials.

Papers must be clear and concise, but detailed data is strongly encouraged. The journal publishes a variety of article types (Original Research, Review, Communication, Opinion, Comment, Conference Report, Technical Note, Book Review, etc.). There is no restriction on the length of the papers and we encourage scientists to publish their results in as much detail as possible.

Publication Speed (median values for papers published in 2023): Submission to First Decision: 5.7 weeks; Submission to Acceptance: 17.9 weeks; Acceptance to Publication: 7 days (1-2 days of FREE language polishing included)

Current Issue: 2024  Archive: 2023 2022 2021 2020 2019 2018 2017
Open Access Short Report

The COVID-19 Geropsychiatry Rounds: A Curriculum for Healthcare Providers

Andreea L. Seritan 1,2,*

  1. Department of Psychiatry and Behavioral Sciences, University of California San Francisco, San Francisco, California, USA

  2. UCSF Weill Institute for Neurosciences, San Francisco, California, USA

Correspondence: Andreea L. Seritan

Academic Editor: Pedro Morouco

Special Issue: COVID-19: A Geriatric Emergency

Received: November 26, 2021 | Accepted: January 19, 2022 | Published: January 24, 2022

OBM Geriatrics 2022, Volume 6, Issue 1, doi:10.21926/obm.geriatr.2201187

Recommended citation: Seritan AL. The COVID-19 Geropsychiatry Rounds: A Curriculum for Healthcare Providers. OBM Geriatrics 2022; 6(1): 187; doi:10.21926/obm.geriatr.2201187.

© 2022 by the authors. This is an open access article distributed under the conditions of the Creative Commons by Attribution License, which permits unrestricted use, distribution, and reproduction in any medium or format, provided the original work is correctly cited.

Abstract

Older adults’ mental health needs significantly increased during the COVID-19 pandemic. Geriatric psychiatry is an area of extreme workforce shortage globally. A novel curriculum was developed to educate healthcare providers on COVID-19-related geriatric and geropsychiatry topics. Monthly lectures were presented from November 2020 to June 2021. Evaluations were collected after each lecture via an anonymous survey. Overall lecture quality and relevance for the participants’ clinical practices were rated on a 1-3 Likert-type scale. 217 participants attended lectures; 72 evaluations were collected (33% response rate). Overall lecture rating score was 2.82 ± 0.38 and relevance score was 2.77 ± 0.45. The curriculum was well received and relevant to participants. Future studies should collect more details regarding participants’ clinical practices. To our knowledge, this is the first COVID-19-related geriatric psychiatry curriculum developed to educate healthcare providers and empower them to care for older adults during the pandemic.

Keywords

Continuing education; COVID-19; curriculum; geriatric; psychiatry

1. Introduction

The coronavirus disease-2019 (COVID-19) pandemic took healthcare systems around the world by surprise. As the pandemic evolved, psychiatrists embarked on a steep learning curve together with their colleagues from other specialties, studying the pathophysiology of this novel disease, related neuropsychiatric symptoms, and pharmacological management strategies. Older adults, particularly the oldest-old (85 years of age and older) were the most vulnerable group during the pandemic [1]. Multiple neuropsychiatric manifestations associated with COVID-19 have been reported in adults of all ages, including anxiety, cognitive impairments, delirium, depression, psychosis, post-traumatic stress disorder, and sleep disturbances [2,3,4]. Older adults unfamiliar with technology were additionally affected by the quarantine and ensuing social isolation [5]. Across the globe, mental health care systems were overwhelmed by the demand, yet grossly understaffed [6,7,8]. Even pre-pandemic, geriatric psychiatry was an area of extreme workforce shortage [6,9,10,11]. The widespread use of telepsychiatry during the pandemic facilitated access for patients who might not have otherwise received care and has been generally well received by patients and providers [12,13]. However, this further compounded the shortage of mental health services. Geriatric psychiatrists had to step up to the plate urgently, caring for higher numbers of patients while developing and disseminating knowledge regarding COVID-19 neuropsychiatric manifestations in older adults.

Several efforts have been put forth to date to address geriatric psychiatry and/or medicine knowledge gaps related to COVID-19, although each targeted a specific discipline or group of learners. The American Association for Geriatric Psychiatry assembled a curriculum consisting of 30 online lectures for residents whose clinical rotations were disrupted by the pandemic [14]. A group of geriatricians offered a series of five virtual meetings, GERIAtrics Fellows Learning Online And Together (GERI-A-FLOAT) [15]. Fifty-five participants representing 14 fellowship programs attended the meetings. Overall evaluation scores ranged from 4.4 to 5.0 (on a scale of 1 to 5, with 1 being very poor and 5 being excellent). All respondents stated that they intended to change their practice following these sessions [15]. Neumann-Podczaska et al. [16] developed an interprofessional telemedicine geriatric consultation rotation for medical and pharmacy students. Thirteen students completed 49 consultations; this initiative was well received by both patients and learners [16]. Other educators converted existing inpatient geriatric psychiatry rotations to an outpatient format or developed new educational experiences during the pandemic, although their learning objectives may not have been COVID-19-specific [17,18,19,20].

In response to the increased mental health needs of older adults during the pandemic, a team at the University of California, San Francisco (UCSF) developed an innovative curriculum aiming to educate health care providers, particularly non-psychiatrists, on COVID-19-related geriatric psychiatry aspects and empower them to manage these complex, vulnerable patients.

2. Materials and Methods

2.1 Curriculum Description

The Geropsychiatry Rounds consisted of a series of monthly presentations on COVID-19-related topics. This curriculum was developed based on an informal needs assessment, gauging interest at two faculty and staff meetings, as well as ongoing trainee and participant feedback. Lectures were presented from November 2020 to June 2021 and advertised to a broad audience including providers from the UCSF Health Departments of Emergency Medicine, Family and Community Medicine, Internal Medicine, Neurology, and Psychiatry and Behavioral Sciences, the San Francisco Veterans Affairs Medical Center, and community clinics. Email recipients were asked to forward lecture announcements to any interested colleagues. Invited speakers included geriatricians, neurologists, psychiatrists, a family practice physician, and a palliative medicine specialist. Lectures were presented in Zoom (https://zoom.us/) and recorded. Each lecture lasted 50 minutes and covered one or two topics, with 10-15 minutes for discussion. All events were held at lunchtime, to accommodate busy clinicians’ schedules. Educational materials and video recordings were uploaded to a shared drive after each event.

The lectures highlighted practical aspects of geriatric and geropsychiatric care relevant to the COVID-19 pandemic. The main topics were selected based on clinical questions encountered during the pandemic, for example noting a high number of older adults with new-onset psychosis referred to the geriatric psychiatry clinic, consistent with previous case reports [3]. Telemedicine also allowed evaluation of patients in their homes or in long term care facilities. As such, patients with advanced neurocognitive disorders, who would not have been able to travel to an outpatient clinic pre-pandemic, could be seen in clinics. Frailty was another important topic for discussion, particularly since it has been identified as a poor prognostic factor and determinant of increased care needs after hospitalization in older adults with COVID-19 [21,22]. Furthermore, there was an influx of COVID-19 survivors with posttraumatic stress disorder (PTSD) after intensive care unit stays, as noted throughout the globe [2,4]. These referral patterns guided the development of the curriculum, aiming to help providers learn what they needed to know in order to best take care of their patients [23]. A few topics were requested by trainees (e.g., bipolar disorder in older adults, update on California End of Life Option Act, financial capacity evaluation) because they represented self-identified knowledge gaps and were missing from the residency curriculum. All speakers were asked to comment on COVID-19 relevant aspects for their topics. One topic (social isolation) was added based on participant request.

Topics covered are listed in Table 1.

Table 1 The COVID-19 Geropsychiatry Rounds topic list.

2.2 Evaluation Survey

Evaluations were collected by email after each lecture, with one reminder a week later. A 3-question anonymous Qualtrics survey was designed for this purpose (Qualtrics, Provo, UT). The first question, Please rate the following lecture, was rated on Likert-type scale from 1 to 3, with 1 = Poor, 2 = Good, 3 = Outstanding. The second question, How relevant was the lecture topic for your clinical practice? was also rated 1-3, with 1 = Not relevant at all, 2 = Somewhat relevant, 3 = Very relevant. The third question was open-ended, soliciting additional feedback and suggestions for future topics.

2.3 Data Analysis

Descriptive statistics were used to analyze survey responses: frequencies for attendance and number of evaluations, and means and standard deviations (SDs) for evaluation scores.

2.4 Ethics Statement

The UCSF Institutional Review Board (IRB) approved this project.

3. Results

Table 2 summarizes lecture attendance, number of evaluations collected, and evaluation scores. A total of 217 non-unique participants attended lectures (several attended more than one). Attendance varied (range, 9-50 participants per lecture), depending on availability and interest in the topic. Several providers requested access to materials after the events. Asynchronous access was not tracked. Participants included physicians, marriage-family therapists, nurse practitioners, psychologists, and social workers. Most attendees were faculty, staff, residents, and fellows from the UCSF Departments of Emergency Medicine, Internal Medicine, Neurology, and Psychiatry and Behavioral Sciences; there were also community providers and out-of-state clinicians. Topics presented together were evaluated together. 72 evaluations were collected (33% response rate).

Table 2 Lecture attendance and evaluation scores.

All lectures received high overall ratings, with the highest rated being the talk on long COVID and its impact given by a physician who is a COVID-19 survivor, the update on the California End of Life Option Act, and the presentation on post-COVID cognitive changes. The topic with the highest relevance score was cognitive assessment for diverse populations; participants also planned to make changes in their clinical practices based on this presentation.

Open-ended illustrative comments are presented in Table 3. Other topics suggested for future lectures included computerized cognitive assessment, psychotherapeutic treatment of bipolar disorder, and involuntary treatment and its impact. Several speakers were invited to give follow-up presentations to larger audiences, such as Grand Rounds in the Department of Psychiatry and Behavioral Sciences and neuropsychiatry lectures.

Table 3 Open-ended illustrative comments.

4. Discussion

This report presents an innovative geriatric psychiatry curriculum designed to educate health care providers and, ultimately, to improve the mental health care of older adults in non-psychiatry clinics during the COVID-19 pandemic. To our knowledge, this is the first COVID-19-related geriatric psychiatry curriculum designed for an interprofessional audience within a large healthcare system. Participants rated lectures highly and found the content to be relevant to their clinical practices. The curriculum also reached community and out-of-state providers, showing potential impact and dissemination. This educational effort can be distinguished from other geriatrics or geropsychiatry curricula developed during the pandemic through three novel aspects: interprofessional format, just-in-time content selection, and teaching methods.

The concept of interprofessional geriatric training is not new. In fact, most mental health care of older adults is delivered through primary care, collaborative care, or consultation models, due to the scarcity of geriatric psychiatrists [24,25]. Workforce shortages in behavioral health for older adults have been well documented in California and across the U.S., and continue to worsen [9,10,11]. As such, it is critical to support and educate non-psychiatrists, helping them feel better equipped to manage their geriatric patients, particularly in times of crisis such as these. The fact that participants requested so many non-COVID-19 topics shows the dearth of geriatric psychiatry didactics at our institution. Recent studies confirmed the benefits of interprofessional education in improving the care of older adults [26]. Ideally, we would have evaluated the impact of our curriculum on non-psychiatric providers’ practices and noted any changes in referral patterns (i.e., whether participants referred less older adults to the geriatric psychiatry clinic due to feeling more confident and able to manage them on their own). This information would be helpful in guiding the development of future curricula.

A distinctive feature of our curriculum was the fact that lectures summarized emerging knowledge and represented just-in-time learning. While most topics taught were specifically focused on COVID-19, several covered “bread and butter” content requested by trainees. For this reason, the curriculum was heterogeneous, pairing basic geriatric topics with cutting-edge data. Future iterations could focus on emerging COVID-19 updates. On the other hand, the flexible approach to assembling this mini-series of lectures mirrored best practices in collaborative self-directed learning, allowing participants to identify topics that addressed their knowledge gaps [27]. By making available the recordings for asynchronous access, we further supported self-directed learning, as viewers could “mix and match” topics of interest. Just-in-time learning has been shown to promote the transfer of acquired knowledge into practice in continuing education [23,28]. Nevertheless, performing a targeted needs assessment would have allowed us to tailor the content to the needs of all learners and is an essential element of curricular development [29].

Another unique aspect of our curriculum was the approach used, informed by andragogy principles: most lectures were 20 minutes long, unlike extant COVID-19 lectures which used the traditional 1-hour lecture format [14,15]. Online geriatric learning has become ubiquitous during the COVID-19 pandemic for both didactic and clinical experiences, although best practices have yet to be evaluated [17,18,19,20]. We purposely invited short talks, given the emerging nature of the knowledge base and in order to keep cognitively diverse groups of clinicians engaged. Previous studies have shown that learner attention lapses during 50-minute in-person lectures, starting in the first 5-15 minutes [30]. Similarly, longer online videos were associated with more dropouts (participants stopped watching partway through) [31] and there was a strong negative correlation between the length of a video and how much of the video students watched in one sitting [32]. Splitting video lectures into smaller chunks and adding interactive components can enhance audience engagement [33].

The evaluations focused on level 1 of the Kirkpatrick framework, measuring participant reaction after each lecture [34]. A pre- and post-curriculum evaluation design was initially planned to assess changes in participant knowledge and confidence regarding topics presented. Answer choices were designed on a 1-3 Likert-type scale, with 1 = Not enough, 2 = About right, 3 = OK, but I need a refresher for the knowledge question and 1 = Low, 2 = Good, 3 = High for the confidence question. However, the timing of the IRB approval precluded collecting pre- surveys for the first lecture. Due to this, the evaluation plan had to be modified to collect post- surveys only. On the other hand, the redesigned survey explored the lecture relevance for the participants’ practices, which is an important indicator in the revised (New World) Kirkpatrick model [35]. Asynchronous access to recordings was not tracked; this would have offered further insight into self-directed learning activities (e.g., which topics were preferred and by whom).

Strengths of our curriculum included high attendance, especially earlier during the series; high evaluation ratings and enthusiastic open-ended comments; high relevance and “just-in-time” learning for busy practitioners in multiple specialties. One strategy to increase attendance might be to offer continuing education credits.

Limitations included the lack of a targeted needs assessment and the fact that participants were self-selected, so not everyone with knowledge gaps attended the lectures. The overall survey response rate of 33% was low, although consistent with similar studies conducted during the pandemic [15]. It is possible that participants who were dissatisfied with the curriculum chose to not respond, although it is more likely that the main barriers were time constraints and competing demands. Future studies should collect participant demographic information (e.g., professional degree, specialty, number of years in practice) and systematically evaluate their pre-and post- knowledge, attitudes, and skills.

In summary, there is great need for education on all aspects of psychiatric care of older adults related to COVID-19. Interprofessional lectures can simultaneously increase knowledge among non-psychiatric providers and help improve the care of older adults in saturated systems. This curriculum can serve as a blueprint for other medical centers to consider similar initiatives.

Acknowledgments

The author thanks all the presenters and participants and Tammy Duong, MD for her help in organizing one lecture.

Author Contributions

The author designed and implemented the curriculum, obtained ethics approval, developed the evaluation survey, collected evaluations, analyzed the data, and wrote the entire manuscript.

Funding

This project received no funding support.

Competing Interests

The authors have declared that no competing interests exist.

References

  1. Shahid Z, Kalayanamitra R, McClafferty B, Kepko D, Ramgobin D, Patel R, et al. COVID‐19 and older adults: What we know. J Am Geriatr Soc. 2020; 68: 926-929. [CrossRef]
  2. Rogers JP, Chesney E, Oliver D, Pollak TA, McGuire P, Fusar-Poli P, et al. Psychiatric and neuropsychiatric presentations associated with severe coronavirus infections: A systematic review and meta-analysis with comparison to the COVID-19 pandemic. Lancet Psychiatry. 2020; 7: 611-627. [CrossRef]
  3. Parker C, Slan A, Shalev D, Critchfield A. Abrupt late-onset psychosis as a presentation of coronavirus 2019 disease (COVID-19): A longitudinal case report. J Psychiatr Pract. 2021; 27: 131-136. [CrossRef]
  4. Janiri D, Carfì A, Kotzalidis GD, Bernabei R, Landi F, Sani G, et al. Posttraumatic stress disorder in patients after severe COVID-19 infection. JAMA Psychiatry. 2021; 78: 567-569. [CrossRef]
  5. Brooks SK, Webster RK, Smith LE, Woodland L, Wessely S, Greenberg N, et al. The psychological impact of quarantine and how to reduce it: Rapid review of the evidence. Lancet. 2020; 395: 912-920. [CrossRef]
  6. Jeste DV. Coronavirus, social distancing, and global geriatric mental health crisis: Opportunities for promoting wisdom and resilience amid a pandemic. Int Psychogeriatr. 2020; 32: 1097-1099. [CrossRef]
  7. Serafini G, Bondi E, Locatelli C, Amore M. Aged patients with mental disorders in the COVID-19 era: The experience of Northern Italy. Am J Geriatr Psychiatry. 2020; 28: 794-795. [CrossRef]
  8. Yang Y, Li W, Zhang Q, Zhang L, Cheung T, Xiang YT. Mental health services for older adults in China during the COVID-19 outbreak. Lancet Psychiatry. 2020; 7: e19. [CrossRef]
  9. Abrams RC, Young RC. Crisis in access to care: Geriatric psychiatry services unobtainable at any price. Public Health Rep. 2006; 121: 646-649. [CrossRef]
  10. Bragg EJ, Warshaw GA, Cheong J, Meganathan K, Brewer DE. National survey of geriatric psychiatry fellowship programs: Comparing findings in 2006/07 and 2001/02 from the American Geriatrics Society and Association of Directors of Geriatric Academic Programs' Geriatrics Workforce Policy Studies Center. Am J Geriatr Psychiatry. 2012; 20: 169-178. [CrossRef]
  11. Frank JC, Kietzman KG, Palimaru A. California's behavioral health services workforce is inadequate for older adults. Policy Brief. 2019; 2019: 1-8.
  12. Chen JA, Chung WJ, Young SK, Tuttle MC, Collins MB, Darghouth SL, et al. COVID-19 and telepsychiatry: Early outpatient experiences and implications for the future. Gen Hosp Psychiatry. 2020; 66: 89-95. [CrossRef]
  13. Torales J, Vilallba-Arias J, Bogado JA, O’Higgins M, Almirón-Santacruz J, Ruiz Díaz N, et al. Satisfaction with telepsychiatry during the COVID-19 pandemic: Patients’ and psychiatrists’ report from a university hospital. Int J Soc Psychiatry. 2022. Doi: 10.1177/00207640211070762. [CrossRef]
  14. Conroy ML, Garcia-Pittman EC, Ali H, Lehmann SW, Yarns BC. The COVID-19 AAGP online trainee curriculum: Development and method of initial evaluation. Am J Geriatr Psychiatry. 2020; 28: 1004-1008. [CrossRef]
  15. Duggan MC, Goroncy A, Christmas C, Chippendale R. Staying afloat in the COVID‐19 storm: Geriatrics fellows learning online and together (GERI‐A‐FLOAT). J Am Geriatr Soc. 2020; 68: E54-E56. [CrossRef]
  16. Neumann-Podczaska A, Seostianin M, Madejczyk K, Merks P, Religioni U, Tomczak Z, et al. An experimental education project for consultations of older adults during the pandemic and healthcare lockdown. Healthcare. 2021; 9: 425. [CrossRef]
  17. Collier S. A geriatric psychiatry virtual rotation during COVID-19. Am J Geriatr Psychiatry. 2020; 28: 891. [CrossRef]
  18. Michener A, Fessler E, Gonzalez M, Miller RK. The 5 M ʼs and more: A new geriatric medical student virtual curriculum during the COVID‐19 pandemic. J Am Geriatr Soc. 2020; 68: E61-E63. [CrossRef]
  19. Ramaswamy R, Shah AA, Denson KM, Sehgal M, Syed Q, Powers BB, et al. Teaching geriatrics during the COVID‐19 pandemic: Aquifer geriatrics to the rescue. J Am Geriatr Soc. 2021. Doi: 10.1111/jgs.17169. [CrossRef]
  20. Chew QH, Sim K. Impact of COVID-19 pandemic on undergraduate psychiatry teaching, educational environment, and learning processes. Adv Med Educ Pract. 2021; 12: 1371-1377. [CrossRef]
  21. Hewitt J, Carter B, Vilches-Moraga A, Quinn TJ, Braude P, Verduri A, et al. The effect of frailty on survival in patients with COVID-19 (COPE): A multicentre, European, observational cohort study. Lancet Public Health. 2020; 5: e444-e451.
  22. Vilches-Moraga A, Price A, Braude P, Pearce L, Short R, Verduri A, et al. Increased care at discharge from COVID-19: The association between pre-admission frailty and increased care needs after hospital discharge; a multicentre European observational cohort study. BMC Med. 2020; 18: 408. [CrossRef]
  23. Yilmaz Y, Papanagnou D, Fornari A, Chan TM. Just-in-time continuing education: Perceived and unperceived, pull and push taxonomy. J Contin Educ Health Prof. 2021. Doi: 10.1097/CEH.0000000000000415. [CrossRef]
  24. Conn DK, Madan R, Lam J, Patterson T, Skirten S. Program evaluation of a telepsychiatry service for older adults connecting a university-affiliated geriatric center to a rural psychogeriatric outreach service in Northwest Ontario, Canada. Int Psychogeriatr. 2013; 25: 1795-1800. [CrossRef]
  25. Seritan AL, Haller E, Linde P, Orgera S, Fisher WS, Iosif AM, et al. The psychiatric assessment and brief intervention program: Partnering with primary care providers. Prim Care Companion CNS Disord. 2018; 20: 17m02221. [CrossRef]
  26. Bhattacharya SB, Jernigan S, Hyatt M, Sabata D, Johnston S, Burkhardt C. Preparing a healthcare workforce for geriatrics care: An Interprofessional team based learning program. BMC Geriatr. 2021; 21: 644. [CrossRef]
  27. Kemp K, Baxa D, Cortes C. Exploration of a collaborative self-directed learning model in medical education. Medical Sci Educ. 2022. Doi: 10.1007/s40670-021-01493-7. [CrossRef]
  28. Luhanga U, Chen W, Minor S, Drowos J, Berry A, Rudd M, et al. Promoting transfer of learning to practice in online continuing professional development. J Contin Educ Health Prof. 2021. Doi: 10.1097/CEH.0000000000000393. [CrossRef]
  29. Hughes M. Step 2: Targeted needs assessment. In: Curriculum development for medical education: A six-step approach. 3rd ed. Baltimore: Johns Hopkins University Press; 2016. pp.29-49.
  30. Bunce DM, Flens EA, Nelies KA. How long can students pay attention in class? A study of student attention decline using clickers. J Chem Educ. 2010; 87: 1438-1443. [CrossRef]
  31. Kim J, Guo PJ, Seaton DT, Mitros P, Gajos KZ, Miller RC. Understanding in-video dropouts and interaction peaks in online lecture videos. Proceedings of the first ACM Conference on Learning@ Scale Conference; 2014 March 4th-5th; Atlanta, GA, USA. New York: Association for Computing Machinery. [CrossRef]
  32. Nielsen KL. Students’ video viewing habits during a flipped classroom course in engineering mathematics. Res Learn Technol. 2020; 28. Doi: 10.25304/rlt.v28.2404. [CrossRef]
  33. Geri N, Winer A, Zaks B. Challenging the six-minute myth of online video lectures: Can interactivity expand the attention span of learners? Online J Appl Knowl Manag. 2017; 5: 101-111. [CrossRef]
  34. Kirkpatrick D. Revisiting Kirkpatrick's four-level model. Train Dev. 1996; 50: 54-59. [CrossRef]
  35. Moreau KA. Has the new Kirkpatrick generation built a better hammer for our evaluation toolbox? Med Teach. 2017; 39: 999-1001. [CrossRef]
Newsletter
Download PDF Download Citation
0 0

TOP