Skip to main content

Celonis Product Documentation

Record flattening

As EMS is designed to work with relational data, the EMS connector supports a mechanism to automatically flatten object fields nested within the source record. This can be enabled by setting connect.ems.flattener.enable to True. When enabling the built-in flattener, repeated values such as arrays and maps will be JSON encoded and persisted as nullable VARCHAR columns in the target EMS table.

Chunked JSON encoding

As part of the built-in support for flattening nested structures, we provide the connect.ems.flattener.jsonblob.chunks setting which allows us to JSON encode record payloads into a configurable amount of target table fields (chunks).

You can set the byte size of such chunks using the Notice that this value must be between 1 and 65000 bytes, as that is the maximum length of a VARCHAR field in EMS. This is intended as an escape hatch to allow ingesting unstructured JSON data into EMS and then implement parsing and normalization as a separate process (forexample, a custom script run at regular intervals).