ODP based data extraction from S/4HANA via OData client

Introduction

For data extraction scenarios from S/4HANA the following requirements have typically to be met:

Logical data models abstracting the complexity SAP source tables and corresponding columns
The data source must be delta-/CDC-enabled to avoid full delta loads
Open interfaces and protocols to support the customer demand for cloud-based architectures.
Support of frequent intraday data-loads instead of nightly batches
Supports the extraction of transactional and master data

The updated ODP-OData feature in SAP NW 7.5 is the enabling technology for achieving the requirements describe above. Further references: New ODP feature in SAP NetWeaver 7.5

This document will describe the required step to enable the OData-based consumption of changed records in a SAP S/4HANA system.

High level scenario description and architecture

This tutorial will describe a scenario which consists of the following implementation steps

Extend an existing ABAP CDS view for data extraction
Expose the ABAP CDS view as ODP-enabled ODATA service
Implement a prototype ODATA client which subscribes to the delta queue.

To reimplement this scenario for education purposes, the S/4HANA fully activated appliance is recommended to be deployed on SAP CAL (S/4HANA fully activated appliance). From a high-level perspective the S/4HANA implementation consists of the following components:

ABAP Core Data Services (CDS) based data provisioning (ABAP CDS based ODP context)

The data provisioning mechanism used in this tutorial is typically known as SAP BW extractors or SAP BW business content extractors. The API utilized for providing these data extraction functionalities is referenced as Operational Data Provisioning (ODP). With S/4HANA and NW 7.5 the ODP technology was updated and has now to option to leverage SAP HANA virtual data models(CDS-Views) for data extraction in combination with OData. This approach applicable for S4/HANA public cloud too. Some fundamentals regarding ABAP CDS based ODP-extraction is the prerequisite for this tutorial and this wiki and blogs are a good starting point:

Operational Data Provisioning (ODP) and Delta Queue (ODQ)

Data Provisioning Supportability of SAP S/4HANA On-Premise Edition 1709

How to create delta-enabled BW DataSource based ABAP CDS views

The shown CDS-View customization is based on the blog of Maksim Alyapyshev. In his example the sales document CDS views(I_SalesDocument) gets extended for data extraction.

Extending the sales document CDS View “I_SalesDocument”

Based on the example of Maksim’s blog the CDS view had to be slightly adjusted by commenting out some lines which prevented the CDS- view from being activated. The column 'LastChangeDateTime' is used as timestamp based identifier for CDC.

@AbapCatalog.sqlViewName: 'ZRB_ISALESDOC_1'
@AbapCatalog.compiler.compareFilter: true
@AccessControl.authorizationCheck: #CHECK
@EndUserText.label: 'CDS for Extraction I_SalesDocument'
@Analytics:{dataCategory:#DIMENSION ,
            dataExtraction.enabled:true}
@Analytics.dataExtraction.delta.byElement.name:'LastChangeDateTime'
@Analytics.dataExtraction.delta.byElement.maxDelayInSeconds: 1800
@VDM.viewType: #BASIC

            

define view ZRB_I_Salesdocument as select from I_SalesDocument {
key SalesDocument,

      //Category
      SDDocumentCategory,
      SalesDocumentType,
      SalesDocumentProcessingType,

      CreationDate,
      CreationTime,
      LastChangeDate,
     //@Semantics.systemDate.lastChangedAt: true
     LastChangeDateTime,

      //Organization
      SalesOrganization,
      DistributionChannel,
      OrganizationDivision,
      SalesGroup,
      SalesOffice,
      
      //Pricing
      //TotalNetAmount,
      TransactionCurrency,
      PricingDate,
      RetailPromotion,
      //PriceDetnExchangeRate,
      SalesDocumentCondition
    
}

This HANA SQL statement can be used to identify extraction enabled CDS-Views:

SELECT * from CDSVIEWANNOPOS WHERE 
ANNOTATIONNAME = 'ANALYTICS.DATAEXTRACTION.ENABLED'

Remark: Currently only timestamp-based CDC-flags are supported by extraction-enabled CDS-views. HANA HASH_SHA256 functions could be evaluated for delta calculation in addition to timestamps.

Generating the ODATA service

After successfully activating the CDS-view for data-extraction, the OData service must be created. The required steps for activating the OData service are described in this documentation for the ODP OData client

2550286 - Unable to register service in SEGW Transaction

Briefly summarized the following steps are required to create the service:

Transaction SEGW: Create project
Redefine model based on ODP extraction: Right click --> “Data model”
Search for technical name of SQL CDS view
Finish Wizard
Generate Runtime Object
Register service

[X] Done. Now the service can be tested using the NW Gateway ODATA-client.

Using the ODP OData service with the gateway OData client.

Via the transaction /IWFND/GW_CLIENT you will be able to test the generated service with the gateway client. The tests for the prototype will implement the following cases:

Check if the OData client is already subscribed to the ODP queue
Subscribe to the ODP queue and initialize the delta processing
Get delta links
Update sales document in VA02
Fetch updated record from delta queue

Check if the OData client is already subscribed to the ODP queue

URL: 
/sap/opu/odata/sap/ZRB_ODP_ODATA_SRV_01/SubscribedToAttrOfZRB_ISALESDOC
Result: 
SubscribedFlag:false

Subscribe to the ODP queue and initialize the delta processing Initialize delta processing by setting the HTTP header: “Prefer/odata.track-changes” , then the delta queue gets associated with the authenticated user.

URL: /sap/opu/odata/SAP/ZRB_ODP_ODATA_SRV_01/AttrOfZRB_ISALESDOC?
Set HTTP header values: Prefer/odata.track-changes

Get the delta links

URL: 
/sap/opu/odata/sap/ZRB_ODP_ODATA_SRV_01/DeltaLinksOfAttrOfZRB_ISALESDOC
Result: 
DeltaLinksOfAttrOfZRB_ISALESDOC('D20190708134907_000128000')/ChangesAfter

Update sales document in VA02
Fetch updated record from delta queue

URL: 
/sap/opu/odata/sap/ZRB_ODP_ODATA_SRV_01/DeltaLinksOfAttrOfZRB_ISALESDOC('D20190708134907_000128000')/ChangesAfter

Summary

This example implements the first basic interactions of the OData client with the ODP interface.

Further enhancements and improvements are required to fully implement the OData client and the lifecycle of data extraction processes.

Therefore, is intention of the document to be a starting point and inspiration to further implement and improve the described scenario.

Further references: Requirements for the OData client

How to use ABAP CDS for Data Provisioning in BW

SAP S/4HANA and SAP Analytics (Focus on SAP Analytics Cloud): Where to Do What?

ROBROICH / SAP_ODP_ODATA_CLIENT