Google Drive

The Google Drive connector is a Palantir-provided driver for Google Drive.

To create a new Google Drive source, follow the standard setup flow for Palantir-provided drivers, then use the sections below for Google Drive-specific configuration and networking. For the complete property reference, see the official Google Drive driver documentation ↗.

Choose a Google Drive operation

Choose the source and operation based on the result you need:

The Files table's FileData property ↗ does not turn a file-content download into a dataset column. Use the metadata sync to discover file IDs, then pass an ID to DownloadFile or an on-demand Function. For a Function, the calling user must have access to the file, and the outbound application's authorized scope must permit the operation. See the Drive API access boundaries.

Supported capabilities

Configuration

The properties below are mandatory or recommended.

Networking

The table below lists the domains that the source must be able to access to run.

For each domain, add a corresponding egress policy. If the source is hosted on-premises and not directly reachable from Foundry, use an agent proxy egress policy instead. The agent host itself must also be able to reach the listed domains. See using an agent as a proxy for details.

OAuth 2.0 authentication

This connector supports OAuth 2.0 authentication. Follow the OAuth 2.0 guidance for Palantir-provided drivers to configure and authorize the connection.

Use a service account for metadata sync

To use a service account for metadata sync, configure these source properties:

Preserve the JSON document's double-quoted keys and escaped newlines in its private key. Keep this credential out of code repositories. Remove unused OAuthClientId and OAuthClientSecret rows from the source form; they are not part of this service-account setup.

Table exports

This connector supports table exports. Learn how to set up a table export.

Extract a file with a batch sync

To extract one file, run the Google Drive DownloadFile stored procedure ↗ in a sync definition:

Copied!
1
EXECUTE DownloadFile Id = '<FILE_ID>', LocalFile = ''

With LocalFile empty, the output dataset contains the file content in FileData, encoded as Base64 by default. Decode it to binary in a downstream data transformation. Use a source with Readonly unset or set to false: Readonly=true rejects stored procedures ↗.

Read and write blob files with the Drive API

For on-demand file-content operations, configure a separate REST API source backed by an OAuth 2.0 outbound application. Enable exports and source imports, then import the source into a Python or TypeScript v2 Functions repository. Replace <SOURCE_RID> in each example with the RID of that imported REST API source. Run the Function on behalf of a user who has authorized the application. This source uses the calling user's delegated access; it does not reuse the service-account JSON from the metadata source. See OAuth 2.0 in Functions for runtime and authorization requirements.

Allow the REST source to reach only the API host www.googleapis.com:443 and the OAuth hosts accounts.google.com:443 and oauth2.googleapis.com:443.

Choose the Google Drive scopes ↗ for the files each operation needs:

Public applications requesting drive.readonly must meet Google's restricted-scope verification requirements ↗.

A direct Function call in Workshop cannot start interactive authorization. To offer first-time authorization there, use a separate function-backed action; otherwise, the user must authorize the same outbound application before calling these Functions. Use serverless mode so the user's OAuth credential can refresh.

Read one blob file

Use Google Drive files.get with alt=media ↗ to read blob bytes. Check the file's metadata and download capability before using this with an arbitrary file ID. Native Google Workspace documents require files.export ↗ in a supported format. These examples return blob bytes as a base64 string.

Copied!
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
import base64

import requests
from functions.api import function
from functions.sources import get_source


SOURCE_RID = "<SOURCE_RID>"


@function(sources=[SOURCE_RID])
def read_google_drive_blob(file_id: str) -> str:
    source = get_source(rid=SOURCE_RID)
    session = requests.Session()
    proxy = source.get_https_proxy_uri()
    if proxy:
        session.proxies.update({"http": proxy, "https": proxy})
    session.headers["Authorization"] = (
        f"Bearer {source.get_session_credentials().get().access_token}"
    )

    response = session.get(
        f"https://www.googleapis.com/drive/v3/files/{file_id}",
        params={"alt": "media", "supportsAllDrives": "true"},
        allow_redirects=False,
    )
    if not 200 <= response.status_code < 300:
        raise RuntimeError(f"Google Drive returned {response.status_code}")
    return base64.b64encode(response.content).decode("ascii")

Copied!
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
import { getFetch, getSource } from "@palantir/functions-sources";

const SOURCE_RID = "<SOURCE_RID>";
export const config = { sources: [SOURCE_RID] };

export default async function readGoogleDriveBlob(fileId: string): Promise<string> {
  const source = await getSource({ rid: SOURCE_RID });
  const credentials = await source.sessionCredentials?.get();
  if (credentials?.type !== "oauth") throw new Error("Authorize the source first");
  const sourceFetch = await getFetch(source);

  const response = await sourceFetch(
    `https://www.googleapis.com/drive/v3/files/${fileId}?alt=media&supportsAllDrives=true`,
    {
      headers: { Authorization: `Bearer ${credentials.accessToken}` },
      redirect: "error",
    },
  );
  if (!response.ok) throw new Error(`Google Drive returned ${response.status}`);
  return Buffer.from(await response.arrayBuffer()).toString("base64");
}

Create a blob file

The WRITE examples use a simple media upload ↗ for content up to 5 MB. They do not retry an interrupted upload or read back the final bytes; add those controls before writing files you cannot easily restore.

This example creates metadata in the selected folder, then uploads the bytes. If the upload fails, the new file can remain empty. It returns only the new file ID.

Copied!
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
import base64

import requests
from functions.api import function
from functions.sources import get_source


SOURCE_RID = "<SOURCE_RID>"


@function(sources=[SOURCE_RID])
def create_google_drive_blob(
    parent_folder_id: str, file_name: str, mime_type: str, content_base64: str
) -> str:
    source = get_source(rid=SOURCE_RID)
    session = requests.Session()
    proxy = source.get_https_proxy_uri()
    if proxy:
        session.proxies.update({"http": proxy, "https": proxy})
    session.headers["Authorization"] = (
        f"Bearer {source.get_session_credentials().get().access_token}"
    )

    created = session.post(
        "https://www.googleapis.com/drive/v3/files",
        json={"name": file_name, "mimeType": mime_type, "parents": [parent_folder_id]},
        params={"fields": "id", "supportsAllDrives": "true"},
        allow_redirects=False,
    )
    if not 200 <= created.status_code < 300:
        raise RuntimeError(f"Google Drive returned {created.status_code}")
    file_id = created.json()["id"]

    uploaded = session.patch(
        f"https://www.googleapis.com/upload/drive/v3/files/{file_id}",
        params={"uploadType": "media", "supportsAllDrives": "true"},
        data=base64.b64decode(content_base64),
        headers={"Content-Type": mime_type},
        allow_redirects=False,
    )
    if not 200 <= uploaded.status_code < 300:
        raise RuntimeError(f"Google Drive returned {uploaded.status_code}")
    return file_id

Copied!
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
import { getFetch, getSource } from "@palantir/functions-sources";

const SOURCE_RID = "<SOURCE_RID>";
export const config = { sources: [SOURCE_RID] };

export default async function createGoogleDriveBlob(
  parentFolderId: string, fileName: string, mimeType: string, contentBase64: string,
): Promise<string> {
  const source = await getSource({ rid: SOURCE_RID });
  const credentials = await source.sessionCredentials?.get();
  if (credentials?.type !== "oauth") throw new Error("Authorize the source first");
  const sourceFetch = await getFetch(source);
  const created = await sourceFetch(
    "https://www.googleapis.com/drive/v3/files?fields=id&supportsAllDrives=true",
    {
      method: "POST",
      headers: { Authorization: `Bearer ${credentials.accessToken}`, "Content-Type": "application/json" },
      body: JSON.stringify({ name: fileName, mimeType, parents: [parentFolderId] }),
      redirect: "error",
    },
  );
  if (!created.ok) throw new Error(`Google Drive returned ${created.status}`);
  const { id } = (await created.json()) as { id: string };

  const uploaded = await sourceFetch(
    `https://www.googleapis.com/upload/drive/v3/files/${id}?uploadType=media&supportsAllDrives=true`,
    {
      method: "PATCH",
      headers: { Authorization: `Bearer ${credentials.accessToken}`, "Content-Type": mimeType },
      body: Buffer.from(contentBase64, "base64"),
      redirect: "error",
    },
  );
  if (!uploaded.ok) throw new Error(`Google Drive returned ${uploaded.status}`);
  return id;
}

Replace an editable blob file

This example overwrites the supplied file ID if the application can edit it. Supply only a file you intend to replace: there is no confirmation, collision check, or read-back.

Copied!
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
import base64

import requests
from functions.api import function
from functions.sources import get_source


SOURCE_RID = "<SOURCE_RID>"


@function(sources=[SOURCE_RID])
def replace_google_drive_blob(
    file_id: str, mime_type: str, content_base64: str
) -> str:
    source = get_source(rid=SOURCE_RID)
    session = requests.Session()
    proxy = source.get_https_proxy_uri()
    if proxy:
        session.proxies.update({"http": proxy, "https": proxy})
    session.headers["Authorization"] = (
        f"Bearer {source.get_session_credentials().get().access_token}"
    )

    response = session.patch(
        f"https://www.googleapis.com/upload/drive/v3/files/{file_id}",
        params={"uploadType": "media", "supportsAllDrives": "true"},
        data=base64.b64decode(content_base64),
        headers={"Content-Type": mime_type},
        allow_redirects=False,
    )
    if not 200 <= response.status_code < 300:
        raise RuntimeError(f"Google Drive returned {response.status_code}")
    return file_id

Copied!
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
import { getFetch, getSource } from "@palantir/functions-sources";

const SOURCE_RID = "<SOURCE_RID>";
export const config = { sources: [SOURCE_RID] };

export default async function replaceGoogleDriveBlob(
  fileId: string, mimeType: string, contentBase64: string,
): Promise<string> {
  const source = await getSource({ rid: SOURCE_RID });
  const credentials = await source.sessionCredentials?.get();
  if (credentials?.type !== "oauth") throw new Error("Authorize the source first");
  const sourceFetch = await getFetch(source);

  const response = await sourceFetch(
    `https://www.googleapis.com/upload/drive/v3/files/${fileId}?uploadType=media&supportsAllDrives=true`,
    {
      method: "PATCH",
      headers: {
        Authorization: `Bearer ${credentials.accessToken}`,
        "Content-Type": mimeType,
      },
      body: Buffer.from(contentBase64, "base64"),
      redirect: "error",
    },
  );
  if (!response.ok) throw new Error(`Google Drive returned ${response.status}`);
  return fileId;
}