An approach towards an event-fed solution for slowly changing dimensions in data warehouses with a detailed case study

From the point of view of a data warehouse system, collecting and receiving information from source systems is crucial for all subsequent business intelligence applications. Incoming information can generally be classified into two types: (1) the state-oriented data and (2) event-oriented data or transactional data, which contains information about the change performed by processes on the instances of information objects. On the way towards achieving the goal of a full-fledged active data warehouse it becomes more and more important to provide data with minimal latency. In this paper we focus on dimensional data which is provided by general data warehouse applications. The information transfer is performed via messages containing the change of information on the dimension instances. The proposed approach is able to validate the event-messages, reconstruct the complete history of the dimension and provide a well applicable “comprehensive slowly changing dimension” (cSCD) interface for queries on the historical and current state of the dimension. A description of the prototype implementation for this kind of an “active integration” in a data warehouse and a case study at T-Mobile conclude the paper.