Extractor Builder FAQ
To use (delta) filters from the Extraction Configuration, it is required to configure a dynamic request parameter within the Extractor Builder.
You can define a new Request Parameter within the Configure Request section in your endpoint. Select the Get the values as a filter in the extraction configuration option to fetch the value dynamically. This will allow you to dynamically define the parameter value via the Extraction configuration. The correct Data Type and Date Format can be found in the source system's API documentation.
Note
Request Parameters are always added to the request as follows: {EndpointURL}/?{Parameter}={ParameterValue}. It is not possible to define nested parameters in a dynamic way.
In the following example, our API endpoint supports a request parameter lastModifiedFromDate. It is expecting a certain Date format which you can retrieve from the API documentation.
Note
If the request parameter is mandatory, we recommend configuring the endpoint response using a static value for the parameter and switching the parameter to a dynamic one in the end.
After you have configured the endpoint correctly, you can define a filter in the extraction configuration of the respective table. In the filter, you can define the value of your parameter (in our example lastModifiedFromDate). In the case of Datetime datatypes, we recommend using the Time Filters for creation date or change date filters. Alternatively, you can use a Data Job parameter with the respective value.
For our example, we can use the filter in a Data Job as follows:
as a filter statement (for example in combination with a dynamic parameter)
Note
Please note that you can only assign a specific value to the parameter with =. Operations like > or < are not supported because we need to assign a specific value to the request parameter.
This means delta filters are by design only possible if the API supports a request parameter that allows filtering for these (e.g., lastModifiedFromDate)
Note
The filter configurations will only become visible in the Extraction Configuration of a Data Job after you have defined a dynamic request parameter on the endpoint (as described above).
Yes, you can activate a debug mode on Data Job level. On the Extraction Task, go to Extraction Settings and activate the Enable debug mode toggle.
Note
Debug mode will only stay enabled for three days. After three days it will automatically get disabled.
 |
After enabling, you will see an additional debug log for every single request that the Extractor Builder is making.
 |
Yes, you can change the column Datatype to "TEXT" to save more characters in the field.
