Transfer data between Data Pools
New data integration content available
You're currently viewing a data integration topic that has been replaced and will eventually be removed from our documentation.
For the latest data integration content, see: Data Integration.
Purpose
In certain cases, you might want to separate related data into multiple Data Pools and then share it partially. This is a non-exhaustive list of reasons for doing that:
separating permissions between Extractions and Transformations
sharing common data across multiple Data Pools which need to be separated for reasons of differing permissions
In this case, Data Integration offers the possibility to share data based on Data Connections by exporting them from a source Data Pool and importing them into a target Data Pool within one team. This allows users of the source Data Pool to set up the Data Connection in a way that only the desired data is shared. Moreover, users of the target Data Pool do not need to have access to the source Data Pool at all.
Usage
Exporting Data Connections
 |
By exporting a Data Connection you share the data from your Data Connection with other Data Pools. Views will be created on the tables so that data is not duplicated and won't impact APC.
In the context menu of the Data Connection, you can select "Share Data" with another Data Pool. When sharing the Data Connection or when editing an existing export of a Data Connection, you can specify which Data Pools should be able to access the data from the Data Connection.
By selecting all Data Pools, the data will be made available to all existing and depending on the configuration also to all future Data Pools of your team
The list of all other Data Pools gives you the option to specify exactly which Data Pools are allowed to import the Data Connection. Data Pools that are not checked will not see the Data Connection in their available Data Connections to import.
Our Data Connection feature allows you to share data across multiple target Data Pools. To do this, you can define multiple target Data Pools that your Data Connection will be shared with. After defining your target Data Pools, you have two options to configure which tables specifically should be shared with each target Data Pool:
Default Configuration for all Target Data Pools: You can set a default configuration for all target Data Pools. This means that the table configuration you define will apply to all target Data Pools. This option is ideal if you want to share the same set of tables across all target Data Pools.
Dedicated Configuration per Target Data Pool: You can also set a dedicated configuration per target Data Pool. This allows you to define which specific tables should be shared with each target Data Pool. This option is ideal if you want to share different sets of tables with different target Data Pools.
To stop sharing the Data Connection with other Data Pools, you can simply deselect them from the list again. However, the imported Data Connection in the target Data Pool is not deleted, and thus also e.g. associated Data Jobs are preserved, but all data is removed immediately.
Importing Data Connection
 |
To import a Data Connection from another Data Pool, you need to click on "+ Add a Data Connection" and "Import Data from another Data Pool". When importing a Data Connection, you can choose among all available exported Data Connections from other Data Pools. Once imported, the Data Connection appears under the list of all Data Connections and is tagged respectively. To remove the imported Data Connection from the Data Pool you can delete it via the context menu. The deletion of an imported Data Connection will result in the deletion of all Data Jobs, Tasks, and Data Model tables associated with the previously imported Data Connection.
Note
The export step always has to be done first. If you miss a Data Connection, please make sure that it is already exported in the source Data Pool.
After the import is done, the Data Connection will appear in the list of Data Connections and views will be created on all tables and views of the source Data Connection.
Synchronizing Data Connections
 |
If new rows are added to the tables and views of the exported Data Connection, the data will be available automatically in the importing Data Pools, e.g. when executing a Data Job with those tables. However, if the structure of the tables change or tables are added/removed, the Data Connection needs to be synchronized.
This synchronization can be triggered in the "Data Connections" section by using the context menu of the imported Data Connection.
After starting the Synchronization procedure, you will be presented with an overview of the changes which will be made. After clicking Synchronize you can monitor the progress of the changes and close the window at the end after everything is done.