Skip to main content

Celonis Product Documentation

SAP Extractor new version (RFC Module: 2.1, Extractor: 3.0/2021-02-04)

Contents

The new SAP Extractor (RFC Module: 2.1, Extractor: 3.0.0.-2021-02-04) has been released and is available for download on this page..

Warning

Starting from this version, for all new installations and upgrades The SAP Java Connector (SAP JCo) should be downloaded and set up separately. JCo is a development library that enables a Java application to communicate with on-premise SAP systems via SAP's RFC protocol.

For more details on how to set up the JCo please check the Setup Instructions or see the notes below.

Changes in the Setup Process
The SAP Java Connector (SAP JCo) setup is done separately

The JCo library should be downloaded and setup separately in order for the connection to SAP to work properly. This involves 2 steps:

This step should be performed by someone who has an SAP Service User (S User), which authorizes to download software from SAP portals. Usually, a customer's SAP BASIS has this access.

  • Copy the Library files into the dedicated folder in the Extractor Package

The Library contains 2 files - a Java part “sapjco3.jar”, and an operating system-specific part, for example, sapjco3.[.dll | .so | .sl ].

After downloading the files make sure to copy them to the following folder on the extractor server "Extractor path\jco". The extractor is configured to read the Library from this directory.

The Change Log Tables are included in the Full RFC Transport

One of the most time-consuming steps of the Real-Time Extension setup has always been the creation of the Change Log tables. To streamline this process we have included the relevant tables for the 4 main processes (O2C, P2P, AR, AP) in the transports. Below is the list of the Change Log tables.

Change Log Tables

Source Table

Change Log Table

BKPF

ZCLC65D8C41B7AAC

BSAD

ZCL2E79AB74B559C

BSAK

ZCLFF485AF12D5D8

BSID

ZCL38C23C7E64902

BSIK

ZCL490377AF61B0C

EBAN

ZCLFFEDDB7DC764C

EINA

ZCL8F4BE2E0D87DE

EINE

ZCLC95BD159685F7

EKBE

ZCL61EAEB6374BA9

EKES

ZCLA55590C994AC2

EKET

ZCL14C93F01619F3

EKKO

ZCLCAAB4FCEEC56D

EKPO

ZCL0B35D88E9735E

JCDS

ZCL143C851F455C6

KNA1

ZCLEC274559318E5

KNB1

ZCL627E0D0D849AD

KNB5

ZCLCFC1B35FFED85

KNKK

ZCLB58F73A232E57

LFA1

ZCL404CF03151AE5

LFB1

ZCL019E9472DE275

LFM1

ZCL8D04B9F91AED5

LIKP

ZCL231F0663E39D4

LIPS

ZCL3242B9B1DAE61

MAKT

ZCLC0A31EA85AD30

MARC

ZCLB7A54DF536E6A

MKPF

ZCL78325B9FC7F1A

MSEG

ZCL0176A3CFF90DC

NAST

ZCLA40604D1DAF01

QALS

ZCL69DE41C355146

QAVE

ZCL1ECFA9BF081DC

RBKP

ZCL887ED7E542127

RSEG

ZCLFF8D1E8AA2E9B

USR02

ZCL0FCCCB498BEEF

VBAK

ZCL9E31C185F0A09

VBAP

ZCL0004673655AE5

VBEP

ZCL7D1284281DA08

VBFA

ZCL8EBE1300527B2

VBKD

ZCLC6ADBEE46EC0C

VBRK

ZCL8CE3FE7C19305

VBRP

ZCL2760F869D4D73

VBUK

ZCLDB51F86DA0771

VBUP

ZCLF2FD74E3B56FD

VTTK

ZCL7DE685A68549A

VTTP

ZCLCBD143A4F12FA

New Features
Turn the chunking mode of the clean up job on/off, and define the chunk size

When the real-time mode is activated for a table in SAP, the triggers continuously populate the Log Tables with the IDs of the modified records. These log records are cleaned up after each extraction.

To avoid deadlocks on the Change Log tables, the clean-up is done via chunks. Before the chunk size was hardcoded to 1000, but starting from this new version it will be possible to:

  • Turn the chunking off (boosts the cleanup performance)

  • Modify the chunk size

This can be done either:

  • in the file application-local.yml, located in the on-premise Extractor package, by adding the respective statement in the file, under the namespace celonis=>sap

  1. Uncomment the line and set it to true/false to turn on/off the chunking mode.

  2. Uncomment the line and set the desired size (50000 is the recommended setting)

41194559.png
  • or in the Data Integration=> Connection Advanced Settings (only the chunk size can be setup here)

41194560.png
Define how the deleted records are handled at the table level

The real-time extension enables to also capture of the deleted records. With this feature, users can define how the deleted records should be handled for each table.

  1. Define whether the deleted records should be extracted or ignored

  2. If extracted, you can either remove the records from the mirror table in Celonis, i.e. USR02, or stage the records into a separate staging table to remove them later via a script.

The feature is available both in the Data Jobs and Replication Cockpit, at the table level.

41194561.png
Define how the records generated from other Clients are handled during the extraction

SAP systems are usually multi-client environments, where different clients are writing to the same database/tables. The real-time extension triggers are actually client independent, meaning that they capture changes by all clients and log them in the same Change Log table. This may create complications if 2 separate clients want to extract from the same system. Therefore, we have implemented a feature that allows defining how the data generated by separate clients is handled. Two modes are possible:

  • Extract the data only for the client that is defined in the connection

  • Extract all the data, i.e. client independent

This option can also be setup either in application-local.yml, or in the Data Integration => Connection Advanced Settings

  • In the application-local.yml file, add/uncomment the parameter clientDependent under the namespace celonis=>sap, and set it to:

    1. false, if you want to extract all the records from the table irrespective of the client that has generated it

    2. true, if you want to extract only the records generated by the client that is defined in the connection (this is applicable in scenarios when 2 clients are extracting in real time from the same SAP System).

41194562.png
  • or in the Data Integration=> Connection Advanced Settings

41194564.png
Bug Fixes
  • The Last Extracted date of a table was not updated in the Celonis Management UI in SAP.