Collaborative Development of a Statistics Microlearning Course for Health Professionals

Introduction

Microlearning is learning delivered in short, focused bites. This educational strategy enhances knowledge retention, learner attention, and confidence (Gagne et al. 2019). It particularly benefits learners accustomed to frequent notifications and check-ins, akin to those used in mobile devices and social media platforms.

In a 19th-century study, Hermann Ebbinghaus characterized how learners forget information over time using a Forgetting Curve (study reprinted 2013). He showed that learners recall information more effectively through spaced repetition—small, incremental reviews that reinforce learning. This method ensures information remains retrievable even after extended periods of non-use (Wollstein and Jabbour 2022), making it a practical strategy to manage learning alongside other responsibilities.

Microlearning is highly adaptable, catering to each student's individual needs and preferences. It can be short, self-directed, ongoing, granular, relevant, and just-in-time (Alias and Razak 2024). The bite-sized learning modules facilitate easier scheduling and improved focus. Students can access these modules at their own pace and on mobile devices, enabling learning in their preferred environment and at their convenience. These quick-learning chunks provide more frequent moments of accomplishment.

The ongoing nature of microlearning means students can spread out or revisit learning bites, adapting the learning process to suit their needs. Updating this content can be less hassle for educators because only a few modules may need edits. Each module covers one learning objective, allowing students to locate the content that they require quickly. Learners can choose modules that align closely with their needs or professional roles, further personalizing their learning journey. Students can immediately apply the knowledge they have acquired, enhancing the practical relevance of their learning experience to their individual situations (Alias and Razak 2024).

The modern concept of microlearning refers to topical e-modules that are less than 20 minutes long. However, just like the Forgetting Curve, the principles behind microlearning are not new. Strategies aimed at minimizing cognitive load and enhancing information retention through frequent exposure, such as word-of-the-day calendars, flashcards, and email subscriptions, have been employed for decades. Microlearning expands upon these traditional methods using modern instructional technology. Features like push notifications, brief assessments, and mobile-friendly interfaces significantly enhance learner engagement. Bite-sized learning chunks are well suited for graduate medical education because they can reduce cognitive load and time demands (Manning et al. 2021). Platforms such as Anki (“Anki,” n.d.) and other eLearning apps leverage microlearning principles to enhance user engagement.

Why make a microlearning statistics course?

The genesis for this microlearning course came from a survey sent out by Northwell’s office of academic affairs to hospital medical residents and fellows. The office of academic affairs oversees both the Northwell Libraries and Northwell’s graduate medical education programs. The results of their survey showed that medical residents and fellows were requesting more support with statistical analysis in research.

Most medical residents and fellows have had some training in statistics from medical school, but often they have never applied that knowledge to an actual research project. Moving from the training they have received to implementation in their research is a major barrier to completing research projects for medical residents, who are often expected to publish several times over the course of their Northwell program. The feedback from this survey was communicated to the library director with the request to increase the support the library was providing in this area.

Northwell is a large New York City area hospital system with 27 hospital locations, and approximately 85,000 employees. Therefore, it was not practical for the librarians to do classes on statistics with every medical resident, especially given that only a few librarians were comfortable teaching on this topic. Hence, the decision was made to go for an asynchronous training course.

The library had already begun supporting medical residents’ statistics needs by providing free licenses to GraphPad Prism, a statistical software package that does not require coding knowledge to all Northwell employees (Dotmatics 2024). However, we had previously received feedback that employees who took advantage of the free software were overwhelmed with trying to learn to use it. Creating new instructional materials on statistics would need to build on the existing program to provide GraphPad Prism licenses and make GraphPad Prism more approachable to all of our users, medical residents included.

Once we had decided on an asynchronous, GraphPad Prism based course, we still needed to decide on a format for the proposed course. Together, we came up with a simple profile for our ideal user: a busy healthcare professional, probably a medical resident, who would not have time to take a formal statistical analysis course. This person probably had some previous statistics instruction but could use a refresher on statistical analysis concepts and practical examples to help them with data analysis for their first publication.

Microlearning seemed to fit this user base well. We wanted to create a course that would be practical and get the right information to them at the point they needed it. We wanted a tutorial where they could skip around or go back and replay content as needed.

As we moved through the development process, we came to realize that the course we were creating would be useful not only for medical residents, but also other kinds of novice healthcare researchers (for example, nurse residents). While our primary audience was medical residents and fellows, this became an “add-on” benefit.

Development process

The first part of creating this course was bringing together a team, which ultimately consisted of Lena Bohman, the data librarian; Regina Vitiello, another Northwell librarian; and Michael Kelly, then director of learning and organizational development. Each of us brought something different to the table: Lena had experience working with Northwell researchers on their data and statistics needs, Regina had previously worked as a scientist and so understood their perspective, and Mike had expertise in eLearning.

Next, we decided to use SharePoint as a platform for distributing the course. Northwell uses SharePoint for its intranet, so our target audience was familiar with using the platform as part of their job. The SharePoint page would be a collection of subpages, each with a different topic covered. In Figure 1, you can see a screenshot of the final course homepage from the Northwell intranet.

Figure 1: Screenshot of the final course homepage on Northwell’s intranet.

We worked together to brainstorm ideas for pages to include. After coming up with a number of possible pages, we realized that our ideas naturally fell into two categories: pages on specific statistical tests, and background information (such as how to install the software or plan an analysis). Once we had generated a tentative list of topics, we brought in the head of Northwell’s biostatistics unit to give her feedback. This led to our final list of pages:

• General information

  • Course overview

  • Study design & types of studies

  • Planning your statistical analysis

  • GraphPad Prism introduction

  • Importing your dataset

  • Inferential vs. descriptive statistics

• Statistical tests

  • Linear regression

  • T test

  • ANOVA

  • Mann-Whitney & Kruskal-Wallis tests

  • Chi square & Fisher’s exact tests

Following the microlearning model, each page is populated by short videos. We used a mix of videos we created and recorded ourselves using screen recording software, and videos we found online and in GraphPad Prism’s tutorials. Some pages also included links to more information or information on Northwell-specific resources, like how to request electronic health records data for research. We tried to avoid reinventing the wheel and focus our efforts on creating videos tailored specifically to our audience and which did not already exist. Figure 2 is a screenshot of an example statistical analysis page from the final course.

Figure 2: Screenshot of the final linear regression page from the course on Northwell’s intranet.

As part of fitting the course to our audience, we created example videos of GraphPad Prism analysis using simulated EHR data. We created a synthetic dataset of 1,000 New York patients, aged 24-90. The dataset was generated using Synthea (The MITRE Corporation 2024). Many of the hospital’s novice researchers base their first research project off of data they obtain from our EHR, and Northwell already has systems in place to safely supply EHR data to staff for research purposes. Using the simulated EHR dataset made our examples directly relevant to the user base’s likely project.

After we created the course, the next step was to publicize it to Northwell employees. To do this, we leveraged the connections we had made across the institution during the project. The office of academic affairs, who had provided the impetus for this project, sent out a message to all Northwell medical residents and fellows to let them know of the new resource. The biostatistics unit, who had consulted on creating the course, agreed to promote it in their classes and communication with medical residents. We created a flyer about the course, which we hung in public spaces and passed out at events. Finally, because the library controls access to GraphPad Prism licenses, we started sending a link to the course to everyone who requests access.

Results

Over the course of nine months, the 13 course modules have accumulated a total of 1,408 views. The most popular modules, the Course Overview and Study Designs modules, attracted 555 and 255 views, respectively. The page on linear regression, the most viewed statistical analysis page, received 71 views. Table 1 shows a detailed breakdown of the views by course page.

Table 1: Total views of course pages by page name, from the launch of the course in October 2023 to June 2024.

Due to the opt-in nature of microlearning and the need to protect user privacy on our institutional website, assessing this course proved challenging. We explored the possibility of integrating knowledge checks, quizzes, or surveys into the course structure but ultimately concluded that such assessments were unlikely to be completed and might even create barriers to access. In our literature review for this project, it seemed that there was no consensus on how to assess a microlearning project like this (Leeuw et al. 2019).

In future projects, we aim to incorporate short assessments within the microlearning modules at strategic points within the modules. This would allow us to reinforce key concepts and provide immediate feedback, helping learners to consolidate their understanding. These types of assessments would allow us gather valuable data on learner progress and comprehension and facilitate a more interactive learning environment, promoting active engagement with the material. However, one challenge for us is to create an assessment that will feel encouraging for our audience of medical residents, who are often perfectionistic about tests due to their experience in medical school. We are currently exploring investing in a license for Articulate 360 (“Articulate,” n.d.), an eLearning software package favored by instructional designers, which would give us many more tools to address this challenge than are available natively in SharePoint.

One metric we considered when developing the microlearning modules was to compare their view counts to those of other internal resources, such as libguides. Given the more technical content of the course, the modules received a significant number of views compared to our other resources. Moreover, the course was designed as a statistical learning resource rather than something that needed full completion.

To informally assess the course, we reviewed email feedback and followed up with several medical residents who accessed the course during their research rotations. Responses included appreciation for how the course breaks down statistics concepts into short modules and feedback indicating increased confidence in selecting appropriate data analyses. One resident remarked, "The library's instruction on statistics concepts is integral to our research process and tailored to our specific interests as medical residents," in a post-assessment of general library instruction during their research rotation. Such feedback underscores the effectiveness of microlearning in enhancing engagement, relevance, and ease of learning for participants.

The microlearning course and GraphPad licenses provided by the library have helped promote library services in statistics. We have seen spillover from promoting this course to interest in other, more informal library services in statistics. A notable example of this impact occurred when a librarian presented the microlearning course during a monthly meeting of one hospital's research & evidence-based practice council. Following this presentation, a nursing research team from the hospital scheduled a session with a librarian to receive training on using GraphPad Prism for statistical analysis.

Conclusions & Future Research

As we have finished the active phase of this project, we have had time to reflect on what made it successful. Part of our strategy was pulling in expertise from across the organization. The final project involved input from Northwell’s library, the office of academic affairs, the biostatistics unit, and the office of data strategy. Each of the groups had their own expertise to contribute, and then were able to promote the course to their unique internal networks.

We were also able to build on a preexisting program, the library program to supply GraphPad Prism licenses to employees. The librarians already had experience helping novice researchers use the software and were familiar with the common pain points. This experience helped direct the final design of the course.

We have also realized how this course dovetails with our literature search services. We often support research teams with searches at the outset of their projects. However, planning variables and statistical analysis from the project's start is also necessary to draw meaningful conclusions. Librarians can recommend the statistics microlearning course concurrently with their literature searches. Introducing this resource early in the process aligns with the research workflow and encourages teams to engage with the library again once data collection is complete.

References

Alias, Nurul Fitriah, and Rafiza Abdul Razak. 2024. “Revolutionizing Learning in the Digital Age: A Systematic Literature Review of Microlearning Strategies.” Interactive Learning Environments April: 1-21. https://doi.org/10.1080/10494820.2024.2331638.

“Anki.” n.d. Accessed July 16, 2024. https://apps.ankiweb.net.

“Articulate.” n.d. Accessed October 15, 2024. https://www.articulate.com/360.

Dotmatics. 2024. “GraphPad Prism.” 2024. https://www.graphpad.com.

Ebbinghaus, Hermann. 2013. “Memory: A Contribution to Experimental Psychology.” Annals of Neurosciences 20 (4): 155-156. https://doi.org/10.5214/ans.0972.7531.200408.

Gagne, Jennie Chang De, Hyeyoung Kate Park, Katherine Hall, Amanda Woodward, Sandra Yamane, and Sang Suk Kim. 2019. “Microlearning in Health Professions Education: Scoping Review.” JMIR Medical Education 5 (2): e13997. https://doi.org/10.2196/13997.

Leeuw, Robert de, Anneloes de Soet, Sabine van der Horst, Kieran Walsh, Michiel Westerman, and Fedde Scheele. 2019. “How We Evaluate Postgraduate Medical E-Learning: Systematic Review.” JMIR Medical Education 5 (1): e13128. https://doi.org/10.2196/13128.

Manning, Kimberly D., Jennifer O. Spicer, Lucas Golub, Mikhail Akbashev, and Robin Klein. 2021. “The Micro Revolution: Effect of Bite-Sized Teaching (BST) on Learner Engagement and Learning in Postgraduate Medical Education.” BMC Medical Education 21 (1): 69. https://doi.org/10.1186/s12909-021-02496-z.

The MITRE Corporation. 2024. “Synthea.” 2024. https://synthetichealth.github.io/synthea.

Wollstein, Yael, and Noel Jabbour. 2022. “Spaced Effect Learning and Blunting the Forgetfulness Curve.” Ear, Nose & Throat Journal 101 (9_suppl): 42S-46S. https://doi.org/10.1177/01455613231163726.