Foundry

The Foundry connector enables data sharing from one instance of Foundry to another. This workflow requires access to both Foundry instances, and designates one instance as the "source" and the other as the "destination." Throughout the data connection process, users will perform most functions on the destination instance.

For example, if a use case requires the transfer of data from red.palantirfoundry.com to blue.palantirfoundry.com, most of the setup and subsequent interactions will take place in the destination instance blue.palantirfoundry.com, which is where the transferred data will ultimately land. The workflows discussed below read data via ingest, rather than write data via export.

Supported capabilities

Setup

1. Open the Data Connection application and select + New Source in the upper-right corner of the screen.
2. Select Foundry from the available connector types.
3. Choose to use a direct connection over the Internet or to connect through an intermediary agent.
4. Input the hostname of your source Foundry instance. In this case, blue.palantirfoundry.com will pull data from red.palantirfoundry.com, so red.palantirfoundry.com is the source instance.
5. Choose a means of authentication.
6. Create an egress policy for the source instance. To ingest data from red.palantirfoundry.com to blue.palantirfoundry.com, create an egress policy for the URL https://red.palantirfoundry.com on port 443. Unlike traditional data connections, you must whitelist all IP addresses within the source instance. This is done through Control Panel by selecting the option to Configure network ingress.

Learn more about setting up a connector in Foundry.

Authentication

The Foundry connector supports the following authentication methods:

Client credentials (production): For long-lived connections, we only allow client credentials. To create a client credential follow the below steps:

1. Navigate to Developer Console with the following link:

Copied!
1
https://<SOURCE_FOUNDRY_INSTANCE>.palantirfoundry.com/workspace/developer-console/

2. Select + New application and provide a name.
3. Select No, I will not use an Ontology SDK and be sure to enable your application after selecting the organization it belongs to.
4. Select Backend service.
5. Provide your application with appropriate permissions. You can choose Application permissions or User permissions but, you should leave application permissions checked by default.
6. You will be shown a client secret that you must copy to your clipboard then paste into the destination Foundry instance.
7. After saving, navigate to Oauth & permissions in the left menu.
8. Copy your client ID.

This process will create a Service user for which you can provide or deny access to assets in Foundry. To check if this service user has access to a dataset or a project, you can use the Check access feature for the given asset.

Personal access token (temporary): For security purposes, we don't allow tokens to be used in production use cases. Ingests will fail if a sync is run while relying on a token with a life span greater than 36 hours.

Authentication credentials are input in the destination instance. In the source instance, you must create a token that will afford the destination instance the ability to read data. To do so, navigate to the following URL:

Copied!
1
https://<SOURCE_FOUNDRY_INSTANCE>.palantirfoundry.com/workspace/settings/tokens

Then, select + Create token in the upper-right corner. At this step you can name your token and choose its lifespan. Then, copy your token and navigate to the destination Foundry instance.

The provided credentials must have the following necessary privileges:

• Browse and read datasets in the source Foundry instance
• Read from specific projects and datasets being synced

Networking

The Foundry connector requires network access to the destination Foundry instance on port 443 (HTTPS). The destination instance needs an egress policy that corresponds to the URL of the source instance.

To enable direct connections from a Foundry instance to another Foundry instance, the appropriate egress policies must be added when setting up the source in the Data Connection application.

Egress policies are not needed for connection using an agent.

Sync data from Foundry

To set up a Foundry-to-Foundry sync, select Explore and create syncs in the upper-right of the source Overview screen. Browse the available projects and datasets in the source Foundry instance, then select the datasets you want to sync. When ready, select Create sync for x datasets.

Incremental syncs

Incremental syncs maintain state about the most recent sync and only ingest new or changed data from the target dataset. This is useful for large datasets that are frequently updated.