Comparing User Experiences on the Search-based and Content-based Recommendation Ranking on Stroke Clinical Guidelines- A Case Study

Researchers have proposed methods and tools in order to translate the guidelines to a computer interpretable format in order to generate patient-specific recommendations and highlight the most relevant recommendation to clinicians, known as a clinical decision support system (CDSS). Although there are many evaluation aspects that have been considered to assess the impact of CDSS, however investigating the users’ feedback with respect to recommendation ranking has been neglected. Hence, in this study we compared the search-based and content-based recommendation rankings for the guideline users. We first developed two prototypes and then conducted a case study. A survey, usability test and questionnaire were administered in order to achieve more accurate results. Our results showed that users were more successful in finding the right recommendation for treatment using a content-based prototype. 75% of participants preferred the content-based prototype. Furthermore, we concluded that the less experienced user preferred the content-based prototype. Although usability test scores in the searched-based prototype were higher compared to content-based module, our interview results revealed the opposite outcome regarding user satisfaction. Our results indicated that users are mostly concerned with credibility of recommendations; hence providing additional information with the graded recommendation would increase their confidence in choosing recommendations. In addition, participants stated that understanding the recommendation ranking increased their confidence when choosing recommendations. Regarding the search-based prototype, users mostly searched for an acronym rather than the full length term. Participants suggested a combination of search-based and content-based approaches when finding the guidelines and relevant treatments.