Rolling Off Data from Data Targets: Which Process Is Right for You?

Evaluate several processes that allow you to delete data in BW, including archiving, selective deletion, delete and change log requests, and deletion of Persistent Staging Area data and master data.

---

Key Concept

Rolling off data from a data target means deleting data from a data target in a rolling manner. Instead of simply deleting all the data at once, you can roll off the oldest parts at a set interval of time.

Sizing is one of the most important aspects of BW design. When a data target grows too big, performance often deteriorates. You can manage performance for data targets with large volumes by partitioning, compressing, or aggregating. However, you still need to manage the size of data targets because BW does not have enough room to keep the history of your source systems. Therefore, you must devise a strategy to roll off data from your data targets.

In a typical data warehouse environment, you must keep historical data from three months to 10 years depending on the data target’s granularity. For a data target with calendar day granularity, usually the retention period is three months to a year. For a data target with calendar month granularity, the retention period can be longer — three to five years. For a data target with calendar year granularity, you may keep up to 10 years of data.

BW contains no clear-cut functionality to automate the process of rolling off data from data targets. I will show you how to use BW’s tools to design a solution to roll data off from data targets seamlessly and automatically. There are no specific requirements or prerequisites other than those I’ll explain later.

The process you choose depends on your data target (master data, ODS, or Info-Cube) and your update mode. Possible update modes include delta, pseudo-delta (full, but with selections to force delta), or full. Use pseudo-delta when it is impossible to create a generic delta extractor — instead, write code in the InfoPackage selections to get updated data only. Let’s explore the functionality that BW provides to roll off data from data targets.