Skip to main content

Celonis Product Documentation

Replication Cockpit

 
Pre-requisites
EMS Deployment Options

Real Time Connection to SAP can be implemented both for Full Cloud and Hybrid (LYDIP) scenarios.

Supported Databases

The following database are currently supported:

  • Oracle

  • DB2

  • MS SQL (2012 and higher)

  • SAP HANA

  • Sybase

Step 1: Installation Process (RFC Module & Extractor)

The real time extension requires the following minimum versions of the Celonis RFC Module and Extractor:

  • RFC Module: 2.0.0

  • SAP Extractor: 2020-10-19

Please install/upgrade these components before moving forward with the setup.

For clean installation of the RFC Module and Extractor Service follow the steps as described in the this and this help pages respectively.

For update follow the steps as described in the this help page.

Related Content

Step 2: Setup and Configuration of Real-Time Extractor

The Real-time extractor setup mainly includes the following 2 actions:

  1. Create the Log tables to store the changes in SAP

  2. Install triggers to monitor the changes in SAP

Step 1 has been simplified by including the Change Logs of the standard templates (O2C, P2P, AR, AP) in the transports of the Celonis RFC Module. This means that the customer who are installing the RFC Extractor the first time, can easily activate the Change Log tables for the respective process with a minimal effort, as described below.

a) Creating Log Tables

The standard SAP landscape is comprised of 3 systems - DEV, QA and PROD. The Log tables can usually be created only in the DEV system, because the QA and PROD systems are locked for these kinds of changes.After the tables are added in DEV, they should be transported to QA and then to PRD via a transport. However, the triggers are not transportable, and should be installed in QA (for testing) and then subsequently in PRD (for productive use). The diagram below describes a typical setup process.

41191831.png

To add a Change Log table:

  1. Log in into SAP and navigate to Transaction Code /CELONIS/CLMAN_UI

  2. Click the button “Add Change Log Tables”

  3. In the pop up select the process you want to activate. The respective tables will be populated in the screen.

  4. Click "Check Tables" to dry run the creation process. If everything is green, click "Save", and the Change Log tables will be added

60363355.png
60363356.png
60363357.png

After the tables are created, you will need to transport them to the next system as described inn the next sesction.

Default tables for the main 4 processes

You can find the recommended tables for main 4 processes in the following page, or in this google sheet: https://docs.google.com/spreadsheets/d/19UUv__IqBHLcdGZ1FPEtvg_HPj-sMO-aCrSczvjOHIg/edit?usp=sharing.

Required SAP rights to add log tables and triggers

The SAP User should have minimum the following rights to be able to create the tables and add triggers. However, usually this action is performed by SAP BASIS who has full admin rights.

To access /CELONIS/CLMAN_UI and install triggers

  • Object: S_TCODE

    • TCD: /CELONIS/CLMAN_UI

To create/delete Change Log tables

  • Object: S_DEVELOP

    1. ACTVT: *

    2. DEVCLASS: ZCELONISCL

    3. OBJNAME: *

    4. OBJTYPE: TABL

b) Transporting the Change Log Tables Forward

To streamline the transfer of CL tables from DEV to either of QAS or PRD, the T code /CELONIS/CLMAN_UI allows the user to generate a transport that carries forward the setup configurations to the next system in the landscape.

Before creating the transport in DEV, the SAP BASIS admin should change the Transport Layer of the package ZCELONISCL. Make sure to select a Layer that contains a Consolidation Route connecting DEV to QAS. This will ensure that the log tables can be assigned to a transportable request, and therefore, be pushed forward from DEV to QAS. To change the layer:

  1. Call SE80 to open the Object Navigator

  2. In the object list selection, choose Package.

  3. Enter ZCELONISCL and choose Display.

  4. Double-click the name of the package.

  5. Switch to edit mode

  6. Select a Transport Layer that contains a Consolidation Route connecting DEV to QAS

41191827.png

Tip

After saving the Transport Layer change, you may be prompted to transport that modification too. However, this change is not needed in QA/PRD so you don't need to transport it.

After performing the steps above, you are ready to create a transport request for the Change Log tables. To do that:

  1. Call the transaction code /CELONIS/CLMAN_UI

  2. Click the Transport button

  3. Select a Request where to place the transport, or create a new one

    1. If creating a new Request, define a description and click save.

  4. The all Change Log tables and the configuration table will be added to the transport request, which you can release to the next system.

c) Creating Triggers

A trigger should be installed to track the changes in the source tables, and capture them in the respective Change Logs. To do that:

  1. Log in into SAP and navigate to Transaction Code /CELONIS/CLMAN_UI

  2. Find the table(s) in the grid (check this section for the steps on how to add a table)

  3. Click on Install Trigger

  4. After the trigger is installed successfully, its name will be displayed in the column “Trigger Name”

After these steps are completed successfully, the Delta Extractions for the specified tables will be executed in real time mode. The date and time of latest successful execution will be displayed in the column “Last Extracted Date”.

41191829.png

Warning

Triggers cannot be transported. They should be installed in QA (for testing) and then subsequently in PRD (for productive use).

d) Activating the Clean Up Background Job

In the CL Management UI you can set up a recurring background job in SAP which will run on a daily basis and clean up the records from the Log tables.

To setup the job click the button "Clean Up Settings". A dialogue will pop up where you should define:

  • Number of days to keep records. All records older than the specified date will be removed from the log tables.

  • Where to run the background job. If the "Run Anywhere" option is selected, SAP system will decide on which server to execute the job. Otherwise, the job will be run on the same server where the settings have been defined.

41191828.png

Warning

The clean up background job is an important element to control the size of the Log tables. It prevents the tables from growing in case they are not being extracted.

Further Maintenance
Removing Triggers and Tables

The Triggers can be removed via the transaction code /CELONIS/CLMAN_UI. The triggers are not DDIC objects, so to remove the trigger in the production you don't need to open the client for changes. After you remove the triggers, you can remove the change Log tables too. Please make sure to remove the triggers before you start removing the Change Logs.

The Change Log tables, unlike triggers, cannot be removed directly in the PRD, so you need to delete the tables manually in DEV and assign them to a Transport. This transport can be afterwards moved to QA and PRD and remove the tables from there too.

  • Navigate to SE80

  • Select the package ZCELONISCL

  • Delete all the Change Log tables that you don't need anymore and write them in a transport

  • In the target system, i.e. QA or PRD, double check that the triggers are uninstalled for the tables that are bing removed

  • Move the transport further

  • After the transport is imported, navigate to the t code /CELONIS/CLMAN_UI, and remove the tables from there too by selecting them and clicking "Delete Change Log". This will remove the mapping from the UI, rather than the Change Logs which have already been removed via the transport import.

Adding New Tables

To activate the real-time extractions for new tables, you basically have to follow the same approach as when you are setting up new ones.

Warning

Before you start adding a new table, make sure that all systems - DEV, QA and PRD - have the same setup. You can do this via the t code /CELONIS/CLMAN_UI. Its content should be the same across all the systems.
  1. Add the Change Log tables in the DEV system

  2. Create a transport via the /CELONIS/CLMAN_UI, and import it to QA. NOTE: This will transport not only the newly added tables, but also the existing ones. However, since the tables already exist in the target system, they will be simply updated during the import.

  3. After the import is completed, check the Transport Logs to ensure that no anomaly has been observed.

  4. In QA system activate the triggers for the new tables and run test extractions.

  5. Again, create a transport in the QA, and import it into the PRD system. Make sure to check the transport logs again to ensure that it was successful.

  6. Activate the triggers for the new tables in PRD too.

Temporary Suspension of the Triggers

There may be situations where it is necessary to temporarily suspend the triggers, for example during a massive data migration. This can be done by deleting the trigger, and then re-installing it later for the respective tables.

Appendix
Checking the Table Type Before Activating Logs for it

SAP tables from which the data is extracted by Celonis usually fall into one of these classes:

  • Transparent Table

  • Cluster or Pool Table

Transparent tables physically exist in the database level, and there is a 1-to-1 mapping between the logical table in SAP and its physical equivalent in SAP. Most of the tables fit into this category, i.e. EKKO, EKPO, BKPF, VBAP, etc...

Pool and Cluster tables, on the contrary, exist only in the SAP logical layer, and do not have a physical counterpart in the database. This makes it impossible to install triggers for them, because there is simply no equivalent base table in the database. The most popular Cluster tables are BSEG and CDPOS.

If you want to activate real time mode for a table which is not present in the standard script, you should first check whether it is transparent or not. Table classes are stored in the SAP table DD02L. This table should be extracted and then the following query run, to check the class of the tables for which triggers are desired. Replace XXXX with the desired table name.

select * from "DD02L"

where "DD02L" . "TABNAME" in ( 'XXXX' )

Replacing BSEG with Secondary Index Tables

BSEG is not a Transparent table and therefore we cannot install triggers on it and extract it in real-time mode. Meanwhile, given its size and importance we cannot simply exclude it from the real-time extraction, so it is strongly recommended to replace BSEG with the alternative Transparent tables. You can find a detailed documentation on how to do it here.

FAQ

The trigger creation command is executed from a SAP GUI (transaction code /CELONIS/CLMAN_UI), however, the trigger is created only at the database level executing native SQL command. In other words, the trigger does not exist in the SAP Data Dictionary. Therefore it cannot be transported and should manually be created in each environment.

Triggers generate a logging entry for each transaction in SAP, so there is some impact. However, the impact is very small and does not slow down operations.

Triggers are a native database technology and therefore they are very effective. They are not a technology created by Celonis, and we just instruct the trigger when and how to react.

Before starting the data migration you can delete the triggers, and then re-install them once the migration is over.

riggers operate at the database level, and whenever the database record is inserted/updated/deleted the trigger works. The trigger responds to all field updates, and it does not distinguish between the standard SAP and custom (Z) fields. For any update a log record will be generated.

When setting up the triggers you should also create the Clean Up job, which is basically an SAP Recurring Background Job that runs on a daily basis and removes the extracted records that are older than X days, where X is defined by you. This makes sure that the tables are regularly purged and consume minimal space.

The real time extractor uses its own triggers which will not interfere with those of SLT. Both can be run in parallel.

Recommended Tables for Real Time Extraction Mode

System

Process

Table

SAP ECC

AP

BKPF

SAP ECC

AP

LFA1

SAP ECC

AP

LFB1

SAP ECC

AP

USR02

SAP ECC

AP

BSIK

SAP ECC

AP

BSAK

SAP ECC

AR

BKPF

SAP ECC

AR

KNA1

SAP ECC

AR

KNB1

SAP ECC

AR

KNB5

SAP ECC

AR

LIKP

SAP ECC

AR

LIPS

SAP ECC

AR

MKPF

SAP ECC

AR

USR02

SAP ECC

AR

VBAK

SAP ECC

AR

VBAP

SAP ECC

AR

VBFA

SAP ECC

AR

VBRK

SAP ECC

AR

VBRP

SAP ECC

AR

BSID

SAP ECC

AR

BSAD

SAP ECC

O2C

BSAD

SAP ECC

O2C

BKPF

SAP ECC

O2C

JCDS

SAP ECC

O2C

KNA1

SAP ECC

O2C

KNB1

SAP ECC

O2C

KNKK

SAP ECC

O2C

LIKP

SAP ECC

O2C

LIPS

SAP ECC

O2C

MAKT

SAP ECC

O2C

MARC

SAP ECC

O2C

MKPF

SAP ECC

O2C

MSEG

SAP ECC

O2C

NAST

SAP ECC

O2C

USR02

SAP ECC

O2C

VBAK

SAP ECC

O2C

VBAP

SAP ECC

O2C

VBEP

SAP ECC

O2C

VBFA

SAP ECC

O2C

VBKD

SAP ECC

O2C

VBRK

SAP ECC

O2C

VBRP

SAP ECC

O2C

VBUK

SAP ECC

O2C

VBUP

SAP ECC

O2C

VTTK

SAP ECC

O2C

VTTP

SAP ECC

P2P

BKPF

SAP ECC

P2P

EBAN

SAP ECC

P2P

EINA

SAP ECC

P2P

EINE

SAP ECC

P2P

EKBE

SAP ECC

P2P

EKES

SAP ECC

P2P

EKET

SAP ECC

P2P

EKKO

SAP ECC

P2P

EKPO

SAP ECC

P2P

LFA1

SAP ECC

P2P

LFB1

SAP ECC

P2P

LFM1

SAP ECC

P2P

MAKT

SAP ECC

P2P

MARC

SAP ECC

P2P

MKPF

SAP ECC

P2P

MSEG

SAP ECC

P2P

NAST

SAP ECC

P2P

QALS

SAP ECC

P2P

QAVE

SAP ECC

P2P

RBKP

SAP ECC

P2P

RSEG

SAP ECC

P2P

USR02

S/4 HANA

AP

BKPF

S/4 HANA

AP

BSEG

S/4 HANA

AP

LFA1

S/4 HANA

AP

LFB1

S/4 HANA

AP

SKB1

S/4 HANA

AP

USR02

S/4 HANA

AR

BKPF

S/4 HANA

AR

BSEG

S/4 HANA

AR

KNA1

S/4 HANA

AR

KNB1

S/4 HANA

AR

KNB5

S/4 HANA

AR

KNKK

S/4 HANA

AR

LIKP

S/4 HANA

AR

LIPS

S/4 HANA

AR

USR02

S/4 HANA

AR

VBAK

S/4 HANA

AR

VBAP

S/4 HANA

AR

VBFA

S/4 HANA

AR

VBRK

S/4 HANA

AR

VBRP

S/4 HANA

O2C

BKPF

S/4 HANA

O2C

BSEG

S/4 HANA

O2C

JCDS

S/4 HANA

O2C

KNA1

S/4 HANA

O2C

KNB1

S/4 HANA

O2C

KNKK

S/4 HANA

O2C

LIKP

S/4 HANA

O2C

LIPS

S/4 HANA

O2C

MAKT

S/4 HANA

O2C

MARA

S/4 HANA

O2C

MATDOC

S/4 HANA

O2C

NAST

S/4 HANA

O2C

USR02

S/4 HANA

O2C

VBAK

S/4 HANA

O2C

VBAP

S/4 HANA

O2C

VBEP

S/4 HANA

O2C

VBFA

S/4 HANA

O2C

VBKD

S/4 HANA

O2C

VBRK

S/4 HANA

O2C

VBRP

S/4 HANA

O2C

VTTK

S/4 HANA

O2C

VTTP

S/4 HANA

O2C

VTTS

S/4 HANA

P2P

BKPF

S/4 HANA

P2P

BSEG

S/4 HANA

P2P

EBAN

S/4 HANA

P2P

EKBE

S/4 HANA

P2P

EKES

S/4 HANA

P2P

EKET

S/4 HANA

P2P

EKKO

S/4 HANA

P2P

EKPO

S/4 HANA

P2P

LFA1

S/4 HANA

P2P

LFB1

S/4 HANA

P2P

LFM1

S/4 HANA

P2P

MAKT

S/4 HANA

P2P

MARA

S/4 HANA

P2P

MATDOC

S/4 HANA

P2P

NAST

S/4 HANA

P2P

RBKP

S/4 HANA

P2P

RSEG

S/4 HANA

P2P

USR02