Skip to main content

Celonis Product Documentation

SAP Data Intelligence (prev. Data Hub) Connection

New data integration content available

You're currently viewing a data integration topic that has been replaced and will eventually be removed from our documentation.

For the latest data integration content, see: Data Integration.

Limited Availability

This extractor is currently in limited availability mode and can be added to your Celonis Platform environment by request only.

To request access to this extractor, please create a support ticket on Celopeers.

Rather than focusing on specific connectors, Celonis Data Intelligence operators empower users to leverage Data Intelligence to provide data from source systems such as S/4HANA to the Celonis Platform for various use cases.

Transforming your Data Intelligence graph into the data pipeline that powers SAP Process Mining by Celonis requires two Celonis operators:

  • The Celonis Push Job Creator initiates the data push to the Celonis Platform.

  • Subsequently, the Celonis Cloud Upload pushes extracted data chunk by chunk.

For details about the operators and information on how to set them up, see Operators Detail, SAP Data Intelligence.

FAQ

  • Currently, Celonis operators for Data Intelligence are compatible with json and parquet data input. What if another operator's output data is in a different format?

    Celonis operators compatibility with additional formats, such as csv, is on the roadmap for our operators. For a temporary workaround, see below.

Setting up Celonis Operators in SAP Data Intelligence Graphs

  1. Please contact Celonis Support to request the current version. The Celonis operators will be provided as respective pushJobCreater and pushToCelonisIBC tarball-file.

  2. Import the operators into Data Intelligence.

    1. Log into your user in SAP Data Intelligence System Management and select the File tab at the top of the screen.

    2. Make sure no folder is selected in the File window.

    3. For current user: select Import File > Import Solution File in the My Workspace section.

      For all users in your current tenant (only possible for tenant admin): selectImport File > Import Solution Filein theTenant Workspacesection.

    4. In the popup window, select the Celonis operators' tarball-file which will then be uploaded to the selected workspace.

  3. Go to the Data Intelligence Modeler where the Celonis operators will be included in the left-hand tab.

  4. Drag/drop the operators into the graph and connect them in a logical way (Celonis Push Job Creator always before Celonis Cloud Upload; see below visualization as reference).

  5. In the Celonis operator configuration, define specifics, such as the Celonis Platform realm and team domain to push data to (see below operator sections for more detail).

  6. Save the graph and run it to query relevant data from the source system and push it to the Celonis Platform.

A high-level overview how Celonis operators work in SAP Data Intelligence graphs:
35555092.png
Workaround: Configuring Graphs to Ensure Data Format Compatibility

Data graphs can be flexibly adjusted to convert non-json/parquet data outputs from other operators. For example, the Cloud Data Integration operator's output is in csv format. Therefore, its output has to be converted to parquet before providing it to the Celonis Cloud Upload operator. That can be achieved using the following operator flow: Cloud Data Integration >> Flowagent CSV Producer >> Format Converter. Note that those three have to be specifically grouped out in a Flow Agent section because Celonis operators do not run on Flow Agent. In that exceptional case, Celonis provides an additional Python operator passing the push job's ID from the Celonis Push Job Creator to the Celonis Cloud Upload through a dedicated channel.

About Data Intelligence

SAP Data Intelligence provides all integration, orchestration, metadata management, connectivity and other services of SAP Data Hub with the services of SAP Leonardo Machine Learning in the cloud. The Data Hub part of the Data Intelligence solution represents a data orchestration and management component running on Kubernetes and leveraging open source/embedding machine learning capabilities. It was released by SAP in 2017 to deal with big data and complex data orchestration working across distributed landscapes and processing engines to enhance developer productivity and extract value from distributed data.

So-called graphs are the core concept of SAP Data Intelligence. Graphs are a network of operators connected to each other using typed input ports and output ports for data transfer. Users can define and configure operators in a graph to create data pipelines. Operators communicate via messages (SAP object) passed between them. Messages are composed of attributes and body – in the example of the Celonis operators: a json incl. metadata, such as the Celonis Platform realm and team domain, and a json incl. payload data to be pushed to the Celonis Platform.