Evaluating user interaction with a web-based group decision support system: A comparison between two clustering methods

• System utilization is crucial for the assessment of the performance of DSS.
• An exploratory analysis of user interaction with web-based GDSS is presented.
• Clusters of user interaction from two clustering methods are derived and compared.
• Multiple sequence alignment analysis resulted in non-reliable clusters.
• Hierarchical cluster analysis proved reliable clusters for overall interaction duration.

Task-Technology Fit theory and the Technology Acceptance Model identify system utilization as an important indicator for the performance of complex software systems. Yet, empirical evaluations of user interaction with group decision support systems are scarce and often methodologically underdeveloped. For this study we employed an exploratory evaluation of user interaction in the context of web-based group decision support systems. Specifically, we used information-rich server logs captured through a web-based platform for participatory transportation planning to identify groups of users with similar use patterns. The groups were derived through multiple sequence alignment and hierarchical cluster analysis based on varying user activity measures. Subsequently, we assessed the reliability of the classifications obtained from the two clustering methods. Our results indicate limited reliability of classifications of activity sequences through multiple sequence alignment analysis and robust groupings from hierarchical cluster analysis for user activity initiations and durations. The presented work contributes a novel methodological framework for the evaluation of complex software systems that extends beyond the common approach of soliciting user satisfaction.