Enabling parallel table extraction [performance]
While parallel table extractions are usually the default, double-check whether that feature is actually used.
You can enable parallel Table extraction for existing connections or when adding a new cloud connection. To do that, go to the Advanced settings of your connection and set the number of parallel requests. You can extract a maximum of ten tables in parallel.
There is a significant difference in how we handle SAP extractions and JDBC extractions, however.
SAP Extractions
For SAP extractions, we check the number of parallel extractions per hostname. Even if you define multiple connections or multiple extraction jobs, the number of parallel table extractions per hostname globally limits the parallelism. The number of parallel running jobs does not increase the limit of parallel running table extractions but only generates a queue. To achieve a higher throughput, you can slowly increase the value of parallel extractions up to a maximum of 30. Always check the extraction times, CPU load and memory usage on the source system and on the extractor server. CPU load and memory usage will increase on both systems due to higher parallelism and overload will make the extraction more inefficient.
JDBC extractions
For JDBC extractions, we check the number of parallel extractions per job. By running multiple jobs at the same time, we can achieve many times the maximum parallel table extractions defined in the JDBC connection. This makes load planning more difficult. It is therefore advisable to use clever time management to avoid configuring overlapping JDBC extractions per source system and Extractor server and instead slowly increase the number of parallel table extractions without overloading CPU and memory on the source system and on the extractor server.