A prosody-based vector-space model of dialog activity for information retrieval

• Prosodic information can support search in dialog archives.
• We represent prosodic-context information with a vector space model.
• Proximity in this space reflects dialog-activity similarity and topic similarity.
• Weighted-distance measures outperform city-block distance and Euclidean distance.
• Prosodic information provides less value for search than lexical information, but can usefully complement it.

Search in audio archives is a challenging problem. Using prosodic information to help find relevant content has been proposed as a complement to word-based retrieval, but its utility has been an open question. We propose a new way to use prosodic information in search, based on a vector-space model, where each point in time maps to a point in a vector space whose dimensions are derived from numerous prosodic features of the local context. Point pairs that are close in this vector space are frequently similar, not only in terms of the dialog activities, but also in topic. Using proximity in this space as an indicator of similarity, we built support for a query-by-example function. Searchers were happy to use this function, and it provided value on a large testset. Prosody-based retrieval did not perform as well as word-based retrieval, but the two sources of information were often non-redundant and in combination they sometimes performed better than either separately.

Figure optionsDownload as PowerPoint slide