Patient-Reported Outcome Measures (PROMs) in Neurology for Web 3.0 and mHealth: A Conceptual Framework

Talha Nazir ^1,†,‡, Muhammad Umair ^1,†, Muhammad Mushhood Ur Rehman ^2,†, Reeda Saeed ^3,†, Osama O Zaidat ^4,†, Junaid S. Kalia ^5,6,†,*

1. Research Fellow, NeuroCare.AI, Dallas, TX, US

2. Chief Scientific Officer, NeuroCare.AI, TX, US

3. Chief Operating Officer, NeuroCare.AI, Dallas, TX, US

4. Mercy Health St. Vincent Mercy Hospital, Toledo, OH, US

5. Founder NeuroCare.AI, Dallas, TX, US

6. Neurologypocketbook.com, Dallas, TX, US

‡ Current Affiliation: NeuroCareAI

† These authors contributed equally to this work.

* Correspondence: Junaid S. Kalia

Academic Editor: Luc Jasmin

Special Issue: New Concepts and Advances in Neurotechnology

Received: June 02, 2023 | Accepted: January 07, 2024 | Published: January 09, 2024

OBM Neurobiology 2024, Volume 8, Issue 1, doi:10.21926/obm.neurobiol.2401206

Recommended citation: Nazir T, Umair M, Mushhood Ur Rehman M, Saeed R, Zaidat OO, Kalia JS. Patient-Reported Outcome Measures (PROMs) in Neurology for Web 3.0 and mHealth: A Conceptual Framework. OBM Neurobiology 2024; 8(1): 206; doi:10.21926/obm.neurobiol.2401206.

© 2024 by the authors. This is an open access article distributed under the conditions of the Creative Commons by Attribution License, which permits unrestricted use, distribution, and reproduction in any medium or format, provided the original work is correctly cited.

1. Introduction

The utilization of patient-reported outcomes (PRO) has been firmly established in both research and neurological practices [1]. Approximately 57% of office-based physicians use EHR systems. The increasing interest in using digital platforms for medical records makes them the natural place to incorporate standardized ePRO [2]. As per World Health Organization (WHO) data 2019, neurological conditions account for 1503.39 disability life years (DALY) per 100,000 population [3]. Despite the challenges of incorporating PRO into clinical practice, it is valuable. Implementing patient-reported outcomes for neurological conditions poses challenges due to the prevalence of elderly patients with comorbidities and cognitive impairments. Regulatory authorities have started to recognize that the existing tools are insufficient to capture all necessary endpoints hence there is a dire need for disease-specific Patient Reported Outcome Measurement (PROM) [1].

The United States Food and Drug Administration (FDA) defines Patient Reported Outcomes (PROs) as "any report of the status of a patient’s health condition that comes directly from the patient, without interpretation of the patient’s response by a clinician or anyone else" [4]. Although PROs on paper are manageable, however, incorporating them into clinical settings, making decisions with them, or using them in scientific processes can be challenging. This can also be accompanied by an added administrative burden associated with them [5].

With increasing use of Electronic Health Records (EHRs) systems, Health Information Technology (HIT) is becoming more integrated into the US healthcare system. The HIT enables the incorporation of standardized patient-reported data into clinical practices, improving data collection with fewer unanswered questions and greater repeatability. Moreover, clinicians can provide feedback on health status and response to treatments in real-time [6].

Electronic Patient Reported Outcome (ePRO) is defined as the use of electronic media for PRO data collection. Studies have shown that there is an improved completion rate for ePRO compared to the traditional paper-based administration of PRO [5]. A four-weeks pilot study was conducted with 12 patients and 6 providers to explore the role of ePROs in self-management of disease and shared decision-making. The study showed improved recognition of contextual factors that can impact patients' ability to self-manage and providers' ability to manage complex patients [7]. Similarly, Basch et al., reported statistically significant improvement in symptoms control, quality of life and physical function in cancer patients using weekly ePRO surveys as compared to the patients receiving standard treatment [8].

In an increasingly digital world, it is crucial to integrate practical and accessible healthcare solutions that are mobile-friendly and promote greater patient engagement. In addition, it will open up a new way to obtain high-quality ePROs that can be used for patient-centered care, improving quality, and performing decentralized clinical trials. Although the FDA has provided general guidelines for the development of PROs to support the labeling claims of a medical device, there is no specific framework or guidance available for the development of disease-specific Electronic Patient Reported Outcome Measures (ePROMs) in neurology [9]. Given the pressing need to develop validated electronic patient-reported outcomes (ePROs) for individuals with neurological disorders, we propose an adaptable and customizable modular ePRO framework specifically designed for creating ePROMs in the field of neurology.

1.1 Understanding Technological Standards for ePRO Development

The growing availability of handheld and wearable electronic devices, improved internet connectivity, and digital health technologies are enabling the development of customized ePROs that can be tailored to specific conditions and/or patient populations. Therefore, it is critical to understand the technological standards required for the development of disease-specific ePROs.

1.1.1 Governance

The implementation of ePRO in healthcare systems requires a robust governance plan. Governance establishes infrastructure standards, leverages best practices, and engages stakeholders in decision-making. Both the technical implications (i.e., how IT supports ePRO functionality) and clinical consequences (i.e., how ePROs support patient care) of ePRO implementation need to be balanced by the governance body. The governance of ePRO can follow different models (e.g., a single steering committee or multiple bodies), and they should align with existing organizational and leadership structures within the health system [10,11].

1.1.2 Integration

The successful integration of electronic patient-reported outcomes (ePROs) in clinical settings requires a comprehensive governance plan and collaboration between clinical teams and IT personnel. The guidelines for integrating ePROs are divided into five steps, which include clarifying the use cases of PROMs, developing an ePRO workflow, establishing the ePRO tool, leveraging existing IT infrastructure, and ensuring sustained ePRO use and continuous learning [12]. To guide effective implementation of ePROs, the Non-adoption, Abandonment, Scale-up, Spread, Sustainability (NASSS) framework can be used, and a multi-phased research approach should be adopted that is based on user-centered design principles and aligns with recommended core outcome sets [13]. Developers should also ensure that sufficient attention is given to integrating ePROs with current hospital information technology systems and electronic health records, which, otherwise, can be obstacles to successful implementation.

1.1.3 Reporting

As collecting data through a multimodal system is complex, the reporting process should prioritize end users: patients and clinicians. Patients should have multiple options for entering data, such as typing, voice-to-text, and gestures. Additionally, the UI/UX design should be customizable, visually enhanced, and provide necessary statistical, longitudinal, comparative, and contextual information.

Since the purpose of ePRO data is to aid clinicians in making clinical decisions, the report design and presentation process should follow the information ladder from data to wisdom: Data → Information→ Knowledge → Understanding → Insight → Wisdom [14].

2. Electronic Patient Reported Outcomes (ePRO) Framework

The FDA has recognized the use of PROs in clinical trials to measure the effectiveness of different treatment options. PROs should be reliable, valid, and able to detect change [9].

With the healthcare landscape shifting towards digital, we present a framework (Figure 1) for developing ePROs in neurology based on established guiding principles (Figure 2). Our framework is divided into the following steps:

• Development of ePRO questionnaire
  1. Identification of endpoints
  2. Identification of repeatable domains and subdomains
  3. Development of domains/sub-domains set
• Making ePRO mobile-compatible/user friendly
• Consensus Building on ePRO
• Validation of ePRO

Figure 1 Electronic Patient Reported Outcome Framework.

Figure 2 Guiding Principles to Develop Electronic Patient Reported Outcome.

2.1 Development of ePRO Questionnaire

It is important for electronic patient-reported outcomes (ePRO) capture to include domain-specific symptoms to establish a temporal relationship between symptoms. Therefore, the first step in the process should be to identify disease-specific endpoints and group them into core domains. Core domains refer to the fundamental aspects or dimensions of a patient's health and well-being that are considered essential for assessment and measurement. These domains capture the subjective experiences and perspectives of patients regarding their health status, treatment outcomes, and the impact of a particular aspect of their condition on daily lives. To make PROs mobile-compatible, building repeatable modules based on core domains will provide flexibility to repeat or make minor modifications in different symptom diaries and saving time.

2.1.1 Identification of Endpoints

According to the FDA, an "endpoint" refers to the measurement of outcomes or what happens to patients during a clinical trial. Endpoints can be clinical or surrogate. While clinical outcomes are the most reliable measurements since they identify factors that are of utmost importance to patients and can strengthen patient-centered healthcare infrastructure, surrogate endpoints can also measure clinical benefits and are acknowledged by the FDA as evidence to support claims [15].

To be considered a clinical endpoint, the measurement should be valid, reproducible, mobile-friendly, translatable into clinical practices and policies, have the ability to detect change, and be usable by clinicians to make decisions. Since patient-reported outcomes concentrate on what is important to patients, it's vital to involve all stakeholders, including patients, clinicians, and subject experts. Additionally, conducting a comprehensive literature review and coverage of both clinical and surrogate endpoints is essential in this process (Figure 3).

Figure 3 Identification of Endpoints and Development of Questionnaire.

2.1.2 Identification of Repeatable Domains and Subdomains

To streamline the collection of information and measure the impact of contextual information on data variation, endpoints should be categorized into relevant domains or subdomains. Domains can be generalized or specific such as disease-specific symptoms, triggers, quality of life, and therapeutics. It should cover the minimum set of areas of in the particular context.

The domain or subdomain can take various forms, such as a single question, a questionnaire, a score to measure quality of life, a score to quantify treatment impact, or something else entirely. In addition, domains and sub-domains should be created in a way that allows them to be repeated for multiple pathologies. For instance, the domain focused on quality of life can be applied to both stroke and epilepsy. Figure 3 illustrates how endpoints can be categorized into relevant domains.

2.1.3 Development of Domains/Sub-Domains Set

The next step is the development of domains set that covers all required endpoints required to monitor the progress of a disease. The set of domains should include at least one domain specific to each area and contain one valid endpoint.

2.2 Mobile-Friendly ePRO Design Considerations

The healthcare industry is undergoing a significant transformation due to the increasing availability of handheld and wearable electronic devices (such as Apple Watch, Fitbit Flex, and Pebble), improved internet connectivity, and digital health technologies. As digital technologies continue to flourish, clinical research and practice have new opportunities to develop and utilize ePROs. To make ePROs mobile friendly, we recommend following approach:

2.2.1 Developing a Standardized E-Response/Item Pool for ePROs

E-response is the user's answer recorded with the help of a digital device. This may include a slider to measure severity on a scale of 1 to 10, radio buttons to answer yes/no, multiple-choice questions, or a picture to select the affected body area, among other options. The item pool is the directory of e-responses that can be used in each ePRO by changing questions. Standardizing the questions and responses can help ensure consistency in data collection and improve the ability to detect change when used by a single user over time. Furthermore, the mode of data collection should be considered, as different methodologies can result in data inconsistencies. An example of e-responses/item pool is given in Figure 4.

Figure 4 Example of E-Responses in Neurology ePRO.

2.2.2 Interoperability Standards for ePROs: HL7-FHIR Questionnaire Resource

Health Language 7 (HL7) and Fast Healthcare Interoperability Resources (FHIR) have created standard resources that enable the connection, use, and sharing of EHRs. The Office of the National Coordinator for Health Information Technology (ONC) has also created a new interoperability law that establishes a path for shared information exchange [16]. This makes it possible for EHR systems to use HL7 standard resources and provide both patients and clinicians with access to their data through a standard Application Programming Interface (API).

HL7-FHIR has created a resource called "Questionnaire" specifically for creating questionnaires, including ePROs. FHIR® defines a questionnaire as a structured set of questions designed to generate timestamped e-responses based on pre-set rules. E-responses are the user's answers to specific questions at a particular time relevant to specific concepts. FHIR is a flexible standard that can be used to exchange a variety of data types, including text, images, and audio, making it possible to collect and exchange a wide range of PROs data, including patient-reported surveys, clinical notes, and images of wounds or lesions. It also has a rule-based engine that functions similarly to Computer Adaptive Testing (CAT), allowing for less fixed-item testing and potentially improving data quality and collection efficiency. This resource can help achieve interoperability and, more importantly, standardize implementation and validation [17].

2.2.3 Stakeholder’s Perspective

During the development of ePRO, it is equally important to consider the perspective of stakeholders. Since many neurology patients suffer from disabling conditions such as motor weakness and cognitive deficits, it is vital to ensure that the user interface is compatible with the target population [18]. All stakeholders can be involved in the ePRO development process by providing input on its various aspects.

Patients can share their preferences on the types, frequency, and format of questions, as well as receive their results accordingly. Providers can contribute their expertise to make ePROs user-friendly and informative for informed patient care. Researchers can leverage ePROs to capture patient outcomes and develop innovative treatments and interventions. Payers can utilize ePRO data to assess the value of healthcare services and make informed reimbursement decisions. IT staff can ensure the technical feasibility of the ePRO system and cater to the needs of all stakeholders [19]. Lastly, caregivers can help to add more information where it is needed, either to supplement existing information or to add new information.

By involving all stakeholders in the ePRO development process, a comprehensive approach can be taken to develop a feasible, accessible, and equitable digital solution.

2.3 Consensus Building on ePRO

Building consensus enables stakeholders to reach an agreement that aids in complex decision-making processes. There are several methods for building consensus, such as the Nominal Group Technique (NGT), Delphi Technique (DT), Consensus Development Conference (CDC), and RAND/UCLA Appropriateness Method (RAM). NGT and CDC use face-to-face discussion panels consisting of small private meetings and public forums, respectively. DT uses anonymous responses from participants. RAM uses a mix of NGT and DT [20].

As there is a lack of clear FDA guidelines on disease-specific ePRO development, and given that the concept is relatively new, it is essential to involve all relevant stakeholders in the consensus-building process. Doing so will improve the validity, transparency, and credibility of this approach. Additionally, involving all stakeholders will help identify challenges and improve the adoption of ePRO to enhance patient care.

2.4 Validation of ePRO

Validation is the ongoing process of demonstrating that an instrument effectively functions for a specific purpose within a particular population. Validation can be performed by examining convergent validity, divergent validity, content validity, criterion validity, construct validity, and ability to detect change. Additionally, evaluation reliability can be done by evaluating test-retest reliability and internal consistency is crucial. All these terminologies are defined in Table 1. Furthermore, assessing cross-sectional sensitivity enables the detection of changes at a specific point in time, while longitudinal sensitivity focuses on identifying temporal relationships.

Table 1 Definition Reliability and Validity Methodologies.

The ongoing process of clinical validation and data collection helps to improve and revise the disease specific endpoints, domains, and questionnaires. For example, the method of measuring severity could be changed from a slider with a range of 1-10 to a color-coded bar ranging from green to red. These changes can enhance the user experience [27].

3. Discussion

The digital paradigm shift in healthcare has made it easier to collect patient information on a day-to-day basis and establish a more comprehensive temporal relationship between prognosis and management plans. This has transformed healthcare data collection from paper-based PROMs to PROs [28].

The FDA has published non-binding guidelines for the development of a PRO instrument using an iterative process to support device claims [29]. Recommendations are available for the development of PROs in cancer clinical trials [30]. However, this paper focuses on the development of ePROs in neurology for the digital world, creating PRO that is mobile and web friendly. The FDA has funded pilot projects for the development of ePROs and Clinical Outcome Assessments (COAs) by collaborating with academia and other experts in the field [29]. Our framework will add value to existing literature and can help further refine and standardize the process.

In this paper, we describe the process of disease-specific ePRO development in neurology. This process includes identifying clinical endpoints, developing repeatable domains/subdomains, and creating the ePRO itself (Figure 3). We have presented a generalized framework that can be modified as needed for the target population.

To improve mobile and web compatibility in healthcare systems, we propose using the widely-used HL7-FHIR standards. This will address multiple issues, such as the flow of information between systems, data security, and data accessibility, while also providing a collaborative platform for modification according to specific needs. Furthermore, HL7-FHIR offers several advantages such as being freely accessible for use, seamless integration with wearable devices, a robust underlying framework, and readily available standardized code. It also provides user-friendly explanations for implementation, making it easier to comprehend and adopt [17].

Consensus and validation of ePRO are important steps in ensuring the quality and accuracy of data. Establishing consensus will enhance the trust and reliance on utilizing ePRO in primary care settings, while validation ensures that ePRO is dependable, precise, and capable of effectively measuring its intended targets. We propose using the Delphi method for consensus building and test-retest reliability and sensitivity, along with clinical studies, to validate ePRO.

The proposed ePRO framework offers a comprehensive approach to developing feasible and accessible ePRO in neurology. It has the potential to revolutionize how healthcare is delivered by enabling patient-centered care, improving quality, and performing decentralized clinical trials. By involving all stakeholders in the process, a comprehensive approach could be taken to develop a feasible, accessible, and equitable digital solution.

4. Limitations

Limitations include the fact that the proposed ePRO framework has not been tested or validated in a real-world setting. Additionally, the framework is primarily focused on the development of disease-specific ePROs in neurology and may not be applicable to other medical fields. Finally, while the framework provides a comprehensive approach to ePRO development, it may not account for all potential challenges or limitations that could arise during implementation and customization of ePRO based on individual needs.

5. Conclusion

In conclusion, the use of ePROs is an important development in healthcare that has the potential to improve patient care, data collection, and clinical research. This study has presented a framework for developing ePROs in neurology, including the identification of clinical endpoints, the development of repeatable domains/subdomains, and the actual creation of the ePRO instrument. The paper also explores the significance of mobile-friendly ePRO design considerations, governance, integration, and reporting, as well as stakeholder perspectives, consensus building, and validation. By following the proposed framework, future developers can create effective ePROs that are reliable, valid, able to detect change in a way that is mobile and web friendly, and easy to use for patients and clinicians alike.

Author Contribution

Talha Nazir contributed to literature search, concept formulation, and manuscript writing. Muhammad Umair, Muhammad Mushhood Ur Rehman, and Reeda Saeed contributed to literature search and manuscript writing. Osama Zaidat and Junaid Kalia provided expert opinions on the topic and supervised all other authors.

Competing Interests

The authors declare no competing interests.

Abbreviations