Uncovering the message from the mess of big data

User-generated content, such as online product reviews, is a valuable source of consumer insight. Such unstructured big data is generated in real-time, is easily accessed, and contains messages consumers want managers to hear. Analyzing such data has potential to revolutionize market research and competitive analysis, but how can the messages be extracted? How can the vast amount of data be condensed into insights to help steer businesses’ strategy? We describe a non-proprietary technique that can be applied by anyone with statistical training. Latent Dirichlet Allocation (LDA) can analyze huge amounts of text and describe the content as focusing on unseen attributes in a specific weighting. For example, a review of a graphic novel might be analyzed to focus 70% on the storyline and 30% on the graphics. Aggregating the content from numerous consumers allows us to understand what is, collectively, on consumers’ minds, and from this we can infer what consumers care about. We can even highlight which attributes are seen positively or negatively. The value of this technique extends well beyond the CMO's office as LDA can map the relative strategic positions of competitors where they matter most: in the minds of consumers.