This white paper validates the Digital Experience Scale for Patients (DES/P) framework by comparing the scores obtained with DES/P to those obtained with SUS when evaluating the same digital healthcare properties.
Introduction
Understanding digital patient experience needs and preferences is crucial as many healthcare services today have a digital component. Recognizing the need to objectively evaluate these digital interactions, we developed the Digital Experience Scale for Patients (DES/P).1 This innovative framework is designed specifically to quantify and measure the modern digital patient experiencein healthcare.
The DES/P framework takes inspiration from the well-established System Usability Scale (SUS), a tool that has been a cornerstone in UX evaluation for nearly three decades. SUS has a proven track record of effectiveness and reliability in various sectors, and DES/P is modeled to offer similar insights specifically for patient digital experiences.
This white paper aims to validate the DES/P framework by comparing the scores obtained with DES/P to those obtained with SUS when evaluating the same digital healthcare properties. Our goal is to establish a clear correlation between the two, confirming DES/P's effectiveness as a tool for measuring and improving the digital experience in healthcare. This study is not just about proving DES/P's reliability, but also about highlighting its potential as a standard tool in the healthcare industry for enhancing patient digital interactions.
Adapting UX Principles to Healthcare with DES/P
With over 25 years of experience in patient studies, we've consistently recognized the need for a framework that accurately measures the digital experience of patients. The System Usability Scale (SUS), created in 1986, is a widely used tool for assessing user experience. However, it's not specifically tailored to healthcare or medical services and consists of ten general questions, like “I found the system unnecessarily complex” and “I felt very confident using the system.”2
DES/P was developed by UX researchers at Intechnic to satisfy an ever-growing need to quantify and measure digital patient experiences in the healthcare and pharmaceutical industry. It is based on in-depth user experience (UX) research, involving extensive patient interviews and user tests.
Affinity mapping was used to analyze patient feedback, ensuring that DES/P's criteria are both comprehensive and reflective of actual patient expectations and experiemces.3 This process led to the identification of 12 key criteria, which were then organized into three central categories crucial for a patient-centered experience: Value, Simplicity, and Connectivity (Figure 1).
Figure 1. The Digital Assessment Scale for Patients (DES/P) questionnaire.
In DES/P, each of these twelve criteria is rated on a scale from 1 (poor) to 10 (excellent), with patients providing scores for each. The average of these scores gives the overall DES/P score, reflecting the patient’s overall digital experience. Additionally, by grouping criteria into the above-mentioned categories, DES/P allows for a focused analysis of specific areas within the patient experience.
Similarly, the SUS methodology involves users rating ten items on a scale ranging from Strongly Agree to Strongly Disagree. These responses are converted to a numerical value, totaled, and then multiplied by 2.5 to shift the scale from 0-40 to 0-100. In this system, a SUS score above 68 is considered above average, while a score below 68 indicates a below-average user experience.
Statistical Correlation Analysis of DES/P and SUS
In our study, we applied both the DES/P and SUS frameworks to evaluate the same three digital properties, aiming to explore the correlation betweenthe scores obtained with these two scales. This comparative analysis included a total of 21 patients(age: 44-73; 57% male / 43% female).
By running a statistical correlation on the scores obtained from these two distinct frameworks, we sought to understand the degree of alignment between DES/P, a tool specifically designed for the digital healthcare experience, and SUS, a more general user experience scale. The objective was to validate the DES/P model by examining how closely its results matched those of the widely recognized and established SUS, thereby establishing DES/P’s reliability and relevance in assessing patient digital experiencesin the healthcare sector.
Although the two scoring systems are based on a different number of questions and produce a different scale of outputs, they are both representations of Likert scales, which produce a linear continuum of outputs. To make a comparison between the SUS and DES/P score, the responses to the 12 DES/P questions were given equal weighting and summed. This total score was compared to the overall calculatedSUS score to producethe correlation value.
Reporting a Strong Correlation between DES/P and SUS Scores
Our analysis revealed a strong correlation between the top-line scores obtained with DES/P and SUS frameworks. In a combined analysis of multiple studies where both DES/P and SUS scores were gathered, we found a correlation coefficient (r) of 0.73. This strong correlation underscores the effectiveness of DES/P in mirroring the established SUS metrics, particularly in the context of digital healthcare.
Further analysis of the individual group scores within DES/P revealed varying degrees of correlation with the overall SUS score. The group of questions within the Simplicity category showed the highest correlation (r = 0.89). This is a logical outcome, considering the Simplicity category in DES/P is modeled closely after SUS questions and reflects universal UX principles.
The Connectivity and Value categories of DES/P showed moderate correlations with the SUS score, with r equal to 0.51 and 0.47, respectively. This moderate correlation is indicative of these categories addressing aspects more specific to healthcare and the patient-centric user experience, which are not as prominently featured in the SUS framework.
On an individual level, questions in DES/P focusing on Navigation, Ease-of-Use, Language, and Completeness showed high correlations with the overall SUS score (r= 0.75-0.84). In contrast, questions related to Benefits, Empathy, Trust, Social Proof, and Omnichannel correlated lower with the total SUS score (r= 0.24-0.36). This disparity highlights the unique aspects of patient-centric experiences captured by DES/P that are not as prevalent in the general UX tenets of SUS.
Additionally, within each DES/P category, questions were found to be correlated with one another, with intragroup correlations varying between 0.55 and 0.62. All reported correlations in our study were significant at or beyond the 95% confidence level.
Conclusion
As healthcare technologies evolve, the need for better patient user experience becomes paramount, especially with the increasing role of digital tools in patient care. Consumers expect digital technologies to streamline and enhance their interactions with healthcare professionals and organizations, making patient engagement technologies crucial for patient satisfaction and effective communication and care.
These findings not only affirm the validity of the DES/P framework but also shed light on its alignment and divergence with the SUS. The strong correlation, particularly in the Simplicity category, validates DES/P's effectiveness in capturing universal UX principles relevant to patient experiences. The moderate correlations in other categories underline areas that are uniquely tailored to healthcare, providingnuanced insights into the patient-centric UX. The varying degrees of correlation across individual questions further highlight the distinct and specialized nature of the patient experience in digital healthcare.
Ultimately, these insights offer a valuable guide for healthcare providers and healthcare organizations. Customizing and enhancing digital platforms in ways that resonate with needs and expectations of healthcare consumers, underscores the potential of DES/P as a pivotal tool in the evolution of healthcare UX.
References
1. Digital Experience Scale for Patients (DES/P). Intechnic. https://www.intechnic.com/desp. Date accessed: March 29, 2024.
System Usability Scale (SUS). Usability.gov. https://www.usability.gov/how-to-and-tools/methods/system-usability-scale.html. Date accessed: March 29, 2024.
White Paper. Mapping Digital Patient Experience Preferences and Expectations. Intechnic. 2024. https://www.intechnic.com/blog/mapping-digital-patient-experience-preferences-and-expectations. Date accessed: March 29, 2024.
