Articles

SAP NetWeaver BW Near Line Storage solution (NLS) based on Sybase IQ

Improve decision-making capabilities by bringing unprecedented performance to SAP NetWeaver Business Warehouse with SAP HANA for current data and SAP Sybase IQ for historical data— achieving the perfect balance of cost and performance. The adapter for Sybase IQ as a near-line solution is delivered with the BW system. Integration of Sybase IQ makes it possible for you to separate data that you access frequently from data that you access rarely, thus making less demands on the resources in the BW system. The near-line data is stored in compressed form. You can thus reduce the costs incurred by data that is accessed less frequently. Please Note that the SAP-NLS Solution can be used with all supported Database Versions supported by SAP NetWeaver BW 7.3x. SAP Near-line-storage for SAP BW is general available with SAP NetWeaver BW 7.3  Support Package 9 and Sybase IQ 15.4.

SAP is pursuing a three-step color-coded persistence management concept.

Active data that has to be accessible on a permanent basis for read and write processes is referred to as ‘hot’ data. As would be expected, this data is stored in the main memory of the HANA DB.
Warm data, which is not accessed quite so often, is best stored in the HANA DB file system. Whenever this data is loaded into the main memory for processing, it is displaced with higher priority once processing has finished.To achieve this, the BW development team has developed a non-active data concept in order to optimize the displacement strategy for BW tables in the HANA DB. Here, tables with warm data are flagged and are prioritized for displacement from the main memory whenever they are used. These tables are normally partitioned in the default setting, and only the partitions of the table that are affected by the transaction are loaded to the main memory. This makes it possible to make more efficient use of the priceless commodity that is main memory space.

While the non-active data concept manages the passage of warm data between the main memory and the disk, NLS management is concerned with the next data layer. This is ‘cold’ data that is rarely used and for which no more updates are anticipated in BW. This data is removed in time slices to the Sybase IQ DB. All affected time intervals of an InfoProvider are locked for load processes in BW.
Apart from this restriction, the removed NLS data remains available at any time for all other processes in BW, for read accesses quite naturally but also for ETL processes or individual look-ups.
Strategic queries, which analyze data over longer periods of time and thus inevitably also include cold data, are split and procure data both from HANA DB and from Sybase IQ.
The removal process itself is highly complex and must comply with transactional requirements. All this can be easily managed in both directions using BW Data Archiving Processes (DAP) which can be combined in Process chains. Finally, the consistency of the entire data pool must be ensured at all times, even if this is split into two disjoint parts in the case of NLS.

The interface to the Sybase IQ database required for this was implemented by SAP developers in Walldorf, on the basis of the existing NLS interface and its SDK.