Effective adaptation of multimedia documents with modality conversion

Besides content scaling, modality conversion is an important aspect of content adaptation. In this paper, we study modality conversion of a multimedia document under the constraints of available resource and human factor (user preference). We first formulate the content adaptation process of a multimedia document as a general constrained optimization problem and then extend it to effectively support modality conversion. To represent conversion boundaries between different modalities, we propose the overlapped content value (OCV) model that relates the content value of different modalities with resources. Also, the human factor is specified in the form of modality conversion preference, and then integrated into the framework by modifying the OCV model. We apply the Viterbi algorithm of dynamic programming and its fast approximation to the optimization problem with a practical resource constraint, namely the total datasize. Experiments demonstrate that modality conversion (in combination with content scaling) brings a wider range of adaptation for QoS support. Moreover, the proposed approach is shown to be effective to apply in practice.