Skip to main content

Celonis Product Documentation

Operators Detail, SAP Data Intelligence

Operator Detail: Celonis Push Job Creator (pushJobCreator)

Creates a data push job in the EMS as configured.

Configuration Parameters
  • realm (string): EMS realm

    (in URL: team.realm.celonis.cloud)

  • team (string): EMS team domain

    (in URL: team.eu-1.celonis.cloud)

  • key (string): API or app key to authenticate at the EMS

  • use_api_key (bool): true if API key/false if app key should be used

  • pool_id (string): ID of the data pool to push data to

    (in URL: https://team.realm.celonis.cloud/integration/ui/pools/ poolId)

  • connection_id (optional; string): none if data should be pushed to data pool's global schema, else ID of connection to push data to

    (in URL: https://team.realm.celonis.cloud/integration/ui/pools/poolId/system-

    administration/data-connections/connectionType/connectionId/edit)

  • is_upsert (optional; bool): true to merge new data with existing data, false to delete old data before importing new data

  • table_name (string): name for table which is pushed

  • primary_keys (list[string]): column(s) used as primary key(s)

Input

Input parameters with the same name as configuration parameters overwrite the configuration variables for one upload. All input parameters (attributes and body) will be passed unchanged to the output.

  • input (message):

    • attributes:

      • realm (string): EMS realm

      • team (string): EMS team domain

      • port (int): the port to use; default: 443

      • use_tls (bool): whether to use https; default: true

      • key (string): API or app key to authenticate at the EMS

      • use_api_key (bool): true if API key/false if app key should be used

      • pool_id (string): ID of the data pool to push data to

      • connection_id (optional; string): none if data should be pushed to data pool's global schema, else ID of connection to push data to

      • is_upsert (optional; bool): true to merge new data with existing data, false to delete old data before importing new data

      • table_name (string): name for table which is pushed

      • primary_keys (list[string]): column(s) used as primary key(s)

Output
  • output (message):

    • attributes:

      • all input attributes

      • job_id: ID of the data job used by Celonis Cloud Upload operator to push data

      • status (string): final data push job status; one of

        • DONE: data push job created successfully

        • C ANCELLED: error prior to creating the data push job

        • ERROR: data push job creation failed with unknown error

    • data: all message data that was received, unmodified

  • debug (string): debug messages

Operator Detail: Celonis Cloud Upload (pushToCelonisIBC)

Receives a job ID from the Celonis Push Job Creator, as well as data in json/parquet format and pushes it to the EMS as configured.

(If data is received as json, the operator transforms it to parquet before pushing.)

Configuration Parameters
  • realm (string): EMS realm

    (in URL: team.realm.celonis.cloud)

  • team (string): EMS team domain

    (in URL: team.eu-1.celonis.cloud)

  • key (string): API or app key to authenticate at the EMS

  • use_api_key (bool): true if API key/false if app key should be used

  • pool_id (string): ID of the data pool to push data to

    (in URL: https://team.realm.celonis.cloud/integration/ui/pools/ poolId)

  • job_id (string): ID of the data job, usually created by Celonis Push Job Creator

    (alternatively in URL:https://team.realm.celonis.cloud/integration/ui/pools/

    poolId/data-push/job Id)

  • file_path (optional; string): path to file to be pushed to the EMS

  • column_type_dict (optional; dict()): pairs of column, column_type *

  • message.batchIndex (optional; int) index of batch to be pushed, often generated by SAP HANA Client

  • message.lastBatch (optional; bool): flag to indicate if job should be submitted

  • polling_timeout_seconds (optional; int): max number of seconds before job is considered dead by timeout; default: 7200

  • polling_interval_seconds (optional; int): interval which defines how often job status is polled in seconds; default: 5

Input

Input parameters with the same name as configuration parameters overwrite the configuration variables for one upload. Note that this operators' input is usually provided by preceeding operators.

  • input (message):

    • attributes:

      • realm (string): EMS realm

      • team (string): EMS team domain

      • port (int): the port to use; default: 443

      • use_tls (bool): whether to use https; default: true

      • key (string): API or app key to authenticate at the EMS

      • use_api_key (bool): true if API key/false if app key should be used

      • pool_id (string): ID of the data pool to push data to

      • table_name (string): name for table which is pushed

      • primary_keys (list[string]): column(s) used as primary key(s)

    • body: data to be pushed as json

Output
  • output (message):

    • attributes:

      • all input attributes

      • status (string): final data push job status; one of

        • DONE: data push job created successfully

        • C ANCELLED: error prior to creating the data push job

        • ERROR: data push job creation failed with unknown error

  • debug (string): debug messages

*: Custom Type Information

By default, the Celonis Cloud Upload operator determines the type of a column by looking at the data types of the supplied column entries. Columns can be forced to a particular data type by providing column_type_dict in the message.

Sample Syntax

column_type_dict = {'column0':'date', 'column1':'datetime64', 'column2': 'int64', 'column3': 'float64', 'column4': 'float', 'column5': 'bool', 'column6':'int', 'column7':'str'}

Note

  • Supported data types: bool, date, datetime64, float, float64, int, int64, str.

  • Quotes around data types are required.