Skip to main content

Setting up an on-premise extractor

Celonis on-premise extractors are designed for deployment on physical local servers or within secure private cloud environments (VPCs/VNets) to bridge the gap between your protected data and the Celonis Platform.

Important

Before starting the setup, ensure your environment meets the required system requirements.

Extractor setup on a Windows/Linux server

Set up the connection in Celonis Platform

  1. Go to your team settings, and then system/Uplink integrations.

  2. Create a new uplink with the type Connector (independent of which Connector type you would like to connect).

  3. Copy the Client ID and Client Secrect keys, and save them to a secure location, as they are required in future steps.

    Important

    This is the only time the keys are shown. If you do not copy and save them, you must create a new uplink to complete follow-on steps.

Download and configure the extractor

  1. Download the extractor package.

    Note

    Unless you're using an older version of SAP, 4.6C or earlier or you use PI/PO, we recommend using our newer on-prem clients. For the instructions to install the new client, see On-premises clients (OPC). The on-prem clients run mostly in the Celonis Platform cloud, so you'll get these benefits compared to the uplink-based on-premise extractors.

    If you still wish to get the on-prem extractor, contact the Celonis support team at servicedesk@celonis.com.

  2. Configure the following uplink values in the application-local.yml:

    YAML indentation and syntax

    When making any changes to the application-local.yml, ensure you maintain the exact whitespace and nesting hierarchy provided in the template. Additionally, when adding any new configurations, ensure you follow YAML syntax. Failing to do so may cause the extractor to fail on startup.

    uplink:
  enabled: true
  url: https://[team].[realm].celonis.cloud/uplink/api/public/uplink
  clientId: id
  clientSecret: secret

    1. uplink.url: Update with your team and realm information.

    2. uplink.clientId: Provide the Client ID key you saved when creating the uplink.

    3. uplink.clientSecret: Provide the Client Secrect key you saved when creating the uplink.

Managing log files

To prevent logs from consuming unnecessary disk space, you can configure a rolling policy in the application-local.yml. To do so:

  1. Open your application-local.yml, and locate the logging values.

  2. Add the following values in the logging section:

    logging:
...
  logback:                     
    rollingpolicy:
      max-history: 7          # Retains logs for 7 days
      total-size-cap: 1GB     # Limits total log storage to 1GB

  3. Ensure the logback values are correctly indented and nested, and save the file.

Note

If you are adding this configuration to an existing extractor, you must restart the extractor using your preferred method as described in Run the extractor.

Tip

To use an optional proxy, see the Proxy settings for on-premise extractors.

Set up the JCo (applicable only for SAP extractor)

To communicate with the Celonis RFC module, the extractor needs to establish a connection to the SAP system and be able to make RFC calls. For this purpose, we require the SAP Java Connector (JCo) 3.1 or later to be installed.

The JCo library should be downloaded and set up separately. This involves the following steps:

  • Download the respective package from the SAP Marketplace

    This step should be performed by someone who has an SAP Service User (S User), which authorizes to download software from SAP portals. Usually, a customer's SAP BASIS has this access.

  • Copy the Library files into the dedicated folder in the Extractor Package

    The Library contains two files - a Java part “sapjco3.jar”, and an operating system-specific part, for example, sapjco3.[.dll | .so | .sl ].

    After downloading the files make sure to copy them to the following folder on the extractor server "Extractor path\jco". The extractor is configured to read the Library from this directory.

Run the extractor

There are two ways to run the extractor:

Run the extractor via command line

Start the JAR file by running the following command in the extractor installation directory:

 java -Xmx8g -Dspring.config.location=application-local.yml -jar extractor_file_name.jar

Running multiple extractors

If you have multiple on-premise extractors in the same folder, you must rename the application-default.yml files so that each is distinct. You can then start the respective extractor based on the following command:

 java -Dspring.config.location=configuration_file_name.yml -Xmx8g -jar extractor_file_name.jar

Run an on-premise extractor as a service

The major benefit of running an extractor as a service is that the extractor can be automatically started when the server is boots.

On Linux

To ensure the extractor starts automatically at system boot, configure it as a standard Linux service using systemd.

For this, you need to create a unit file and put it in the directory /etc/systemd/system/. You can use the following example unit file named celonis_extractor.service:

Note

If you have multiple extractors, and you are using the example below, each requires a unique .service file. If you are a more advanced user, you can also use a Systemd Unit Template to manage multiple services from a single master file.

[Unit]
Description=Celonis Extractor Service.

[Service]
Type=simple
User=username
WorkingDirectory=[path to installation folder]
ExecStart=/usr/bin/java -Xmx8g -jar extractor_file_name.jar
Restart=always

[Install]
WantedBy=multi-user.target

To enable and start the service, execute the following commands:

sudo systemctl enable celonis_extractor.service # registers the service so that it is started on boot
sudo systemctl start celonis_extractor.service # starts the service
On Windows

The extractor package contains four files that enable you to run the extractor as a Windows service:

  • Celonis<ConnectionType>Extractor.xml: The configuration file of the service. Normally, you do not need to make any changes to this file.

  • install.bat: The batch file to install the services on the service.

  • startup.bat: The batch file to start the service manually.

  • shutdown.bat: The batch file to stop the service manually.

To perform an install, a startup, or a shutdown, you need to run the batch file as an administrator. To do so, right-click on the respective file, and select Run as administrator.

Additional information

Accessing log files

You can access logs in two ways:

  • Locally: You can access the log files locally in the extractor’s installation directory. The default filename is <connection-type>_connector.log.

  • Within Celonis Platform: You can also access the log file via the connection context menu in the Celonis Platform:

    1. Navigate to your data connection list.

    2. Open the context menu (⋮) for the extractor connection, and select Extractor Logs.

    Tip

    You can also filter the logs by a specific date range before downloading.

Frequently Asked Questions about setting up an on-premise extractor

Can I copy my extractor to another machine?

No. To function properly, each extractor requires a unique ID.

Warning

Cloning or copying an existing extractor to multiple machines may cause the extractor to malfunction.

After starting the extractor, I see the message "Connection refused: connect; nested exception is java.net.ConnectException: Connection refused: connect". How can this be fixed?

Ensure your on-premise extractor server can reach the Celonis Platform. Specifically, verify:

  • You can reach https://[team].[realm].celonis.cloud in the browser of your server. If not, check your firewall settings.

  • Ensure no proxy is blocking the connection between the extractor and the Celonis Platform. For more information, see Proxy settings for on-premise extractors.

If I run the extractor as a Windows service, it always fails immediately, and I can only see the log files ending with the wrapper. What can I do?

Ensure you are using Java 17 by running the following command:

java -version

If it is the correct version, ensure the environment variables PATH and JAVA_HOME are set on a system level and not only on a user level. If they were only set at the user level, after updating them on the system level, restart your server so it is applied to the system user as well.

How do I override the temp folder used by extractor for storing the files temporarily?

You can do this by overriding the command that starts the extractor service.

For Windows

Windows service is installed according to the metadata that is defined in the file CelonisSapExtractor.xml, which is located in the extractor folder. You can modify the command line arguments there and override the directory which is by default %BASE%\temp.

For Linux

You need to modify the command that launches the extractor service. When running the command, you must explicitly override the temp folder. For example:

java -Djava.io.tmpdir="/define/your/directory" -Xmx8g -jar extractor_file_name.jar