Operators Detail, SAP Data Intelligence
Operator Detail: Celonis Push Job Creator (pushJobCreator)
Creates a data push job in the EMS as configured.
Configuration Parameters - Operators Detail
realm (string): EMS realm
(in URL: team.realm.celonis.cloud)
team (string): EMS team domain
(in URL: team.eu-1.celonis.cloud)
key (string): API or app key to authenticate at the EMS
use_api_key (bool): true if API key/false if app key should be used
pool_id (string): ID of the data pool to push data to
(in URL: https://team.realm.celonis.cloud/integration/ui/pools/ poolId)
connection_id (optional; string): none if data should be pushed to data pool's global schema, else ID of connection to push data to
(in URL: https://team.realm.celonis.cloud/integration/ui/pools/poolId/system-
administration/data-connections/connectionType/connectionId/edit)
is_upsert (optional; bool): true to merge new data with existing data, false to delete old data before importing new data
table_name (string): name for table which is pushed
primary_keys (list[string]): column(s) used as primary key(s)
Input - Operators Detail
Input parameters with the same name as configuration parameters overwrite the configuration variables for one upload. All input parameters (attributes and body) will be passed unchanged to the output.
input (message):
attributes:
realm (string): EMS realm
team (string): EMS team domain
port (int): the port to use; default: 443
use_tls (bool): whether to use https; default: true
key (string): API or app key to authenticate at the EMS
use_api_key (bool): true if API key/false if app key should be used
pool_id (string): ID of the data pool to push data to
connection_id (optional; string): none if data should be pushed to data pool's global schema, else ID of connection to push data to
is_upsert (optional; bool): true to merge new data with existing data, false to delete old data before importing new data
table_name (string): name for table which is pushed
primary_keys (list[string]): column(s) used as primary key(s)
Output - Operators Detail
output (message):
attributes:
all input attributes
job_id: ID of the data job used by Celonis Cloud Upload operator to push data
status (string): final data push job status; one of
DONE: data push job created successfully
C ANCELLED: error prior to creating the data push job
ERROR: data push job creation failed with unknown error
data: all message data that was received, unmodified
debug (string): debug messages
Operator Detail: Celonis Cloud Upload (pushToCelonisIBC)
Receives a job ID from the Celonis Push Job Creator, as well as data in json/parquet format and pushes it to the EMS as configured.
(If data is received as json, the operator transforms it to parquet before pushing.)
Configuration Parameters - Operators Detail
realm (string): EMS realm
(in URL: team.realm.celonis.cloud)
team (string): EMS team domain
(in URL: team.eu-1.celonis.cloud)
key (string): API or app key to authenticate at the EMS
use_api_key (bool): true if API key/false if app key should be used
pool_id (string): ID of the data pool to push data to
(in URL: https://team.realm.celonis.cloud/integration/ui/pools/ poolId)
job_id (string): ID of the data job, usually created by Celonis Push Job Creator
(alternatively in URL:https://team.realm.celonis.cloud/integration/ui/pools/
poolId/data-push/job Id)
file_path (optional; string): path to file to be pushed to the EMS
column_type_dict (optional; dict()): pairs of column, column_type *
message.batchIndex (optional; int) index of batch to be pushed, often generated by SAP HANA Client
message.lastBatch (optional; bool): flag to indicate if job should be submitted
polling_timeout_seconds (optional; int): max number of seconds before job is considered dead by timeout; default: 7200
polling_interval_seconds (optional; int): interval which defines how often job status is polled in seconds; default: 5
Input - Operators Detail
Input parameters with the same name as configuration parameters overwrite the configuration variables for one upload. Note that this operators' input is usually provided by preceeding operators.
input (message):
attributes:
realm (string): EMS realm
team (string): EMS team domain
port (int): the port to use; default: 443
use_tls (bool): whether to use https; default: true
key (string): API or app key to authenticate at the EMS
use_api_key (bool): true if API key/false if app key should be used
pool_id (string): ID of the data pool to push data to
table_name (string): name for table which is pushed
primary_keys (list[string]): column(s) used as primary key(s)
body: data to be pushed as json
Output - Operators Detail
output (message):
attributes:
all input attributes
status (string): final data push job status; one of
DONE: data push job created successfully
C ANCELLED: error prior to creating the data push job
ERROR: data push job creation failed with unknown error
debug (string): debug messages
*: Custom Type Information
By default, the Celonis Cloud Upload operator determines the type of a column by looking at the data types of the supplied column entries. Columns can be forced to a particular data type by providing column_type_dict in the message.
Sample Syntax
column_type_dict = {'column0':'date', 'column1':'datetime64', 'column2': 'int64', 'column3': 'float64', 'column4': 'float', 'column5': 'bool', 'column6':'int', 'column7':'str'}Note
Supported data types: bool, date, datetime64, float, float64, int, int64, str.
Quotes around data types are required.