Decision Quality Measures in Recommendation Agents Research

Electronic recommendation agents provide a way for online marketers to gather information about consumer preferences and assess the quality of consumer decisions. Much of the literature on recommendation agents, however, employs divergent measures to assess consumer decision quality. Moreover, decision quality measures are dictated by the type of agent employed. This article provides a review of the decision quality measures used in the recommendation agent research to date and proposes novel measures. We classify and examine the assumptions of–and relationships among–preference-dependent, preference-independent, and subjective measures of decision quality. The analysis of data from an experiment that simulates a broad spectrum of recommendation agents shows that the relative utility and the sum of attribute values of the chosen alternative capture the majority of variance in objective decision quality. Although subjective decision quality measures turn out to be poor proxies for objective measures, they provide important incremental information. Managerial implications for deploying electronic recommendation agents to gather information and measure consumer decision quality under different conditions are discussed.

Research Highlights
► Provides review of decision quality measures used in recommendation agent research.
► Classifies 12 preference-dependent, preference-independent, subjective measures.
► If consumer preferences are available, relative utility is most informative.
► If consumer preferences are not available, attribute sum is most informative.
► Subjective measures provide important insights beyond objective measures.