Configure logging

Applications across Foundry emit logs that provide visibility into their operation, including transform runs, build failures, and other system events.

Foundry logs contain information about:

• Application and function execution events
• Transform and pipeline runs
• Build and deployment activities
• System errors and warnings

Additionally, any user-added logs written in Code Repositories will also be emitted.

These logs can be exported to a Foundry streaming dataset for real-time monitoring and analysis. This feature is only available to Organization Administrators, who can configure a destination where logs from Foundry applications will flow into a Foundry streaming dataset.

Along with log export management, Ontology and AIP in platform log access is managed by Organization Administrators and Information Security Officers.

To start managing the log observability settings for your organization:

1. Navigate to the Control Panel application.
2. In the top banner, select the relevant organization from the dropdown.
3. In the left-hand sidebar, select Log observability settings under the Organization section. Alternatively, in the home page, search for Log observability settings and select the relevant page in the search results.

Export Foundry logs

Foundry logs can be exported, per-organization, directly into a Foundry streaming dataset.

After defining an export configuration, log entries will be continuously written to the streaming dataset from the time the configuration is created. You can process and analyze the data in real-time using Foundry's extensive suite of data transformation and visualization tools or export the logs to external monitoring systems.

Export permissions

To export Foundry logs, a user must be an Organization Administrator. This role provides the necessary permissions to configure log exporting destinations for the organization. See organization permissions for more details about organization-level permissions.

Additionally, sensitive details in logs are obfuscated and/or removed to protect user privacy and security.

Define an export configuration

Foundry logs can be exported from resources belonging to a set of projects, or from all resources in the organization. Some limitations apply when defining an export configuration:

• Each project can be included in at most three export configurations.
  • This limit includes configurations that export logs from all projects.
• When selecting a set of projects for a configuration, you can include up to 25 projects per configuration.

Follow the steps below to define an export configuration:

1. Select Create log export, which will open a dialog to define a configuration.

2. You will see an option to switch between Filter by projects or All projects across spaces. When selecting projects to export relevant logs from, you can filter the search results by project name and space name.

3. Specify the name and location of the streaming dataset where logs will be exported. The location must be a location within the Foundry file system and cannot be a user-created folder.

4. Select Create configuration after reviewing the access requirements.

5. After the configuration is created, it will appear in the list of existing configurations.

Note that when first configuring a log export, it may take some time for logs to begin streaming into the dataset at the export location.

Disable log exporting

To disable log exporting, delete any log storage configurations listed by selecting the trash icon on the right side of the configuration. The resulting dataset will continue to exist, but no new logs will be exported to it.

Analyze log data

Log storage datasets can contain high volumes of streaming data, so we recommend filtering the dataset appropriately before performing analysis. Consider time-based filtering to focus on relevant log entries for your monitoring and troubleshooting needs.

Foundry provides many powerful tools for performing log analysis, such as Transforms and Pipeline Builder.

In platform log access for Ontology and AIP workflows

Ontology and AIP in platform log access is managed at the project level. The log access controls for in platform viewing are distinct from the permissions configured for the export streaming dataset. These controls pertain specifically to functions, actions, Automation workflows, and language models called from enabled projects.

To manage the in platform log access controls by project:

1. Navigate to the Log access tab. This tab displays the projects with log reading enabled for your organization, if any. By default, log access is disabled for all projects within an organization.

2. To enable log access for a project, select Add project, which will open a dialog to configure telemetry log visibility for a project.

3. Once a project is selected, a second dialog will appear to enable log reading for the project and apply necessary markings.

4. After log access is enabled for the project, it will appear in the list of enabled projects.

5. You should then see the updated policy reflected throughout the platform, with log viewing enabled for all source executor resources residing in the project.

To view logs of a workflow execution (with seven day retention), you will need:

• Log reading enabled for the project the source executor resides in.
• Markings access to any markings configured with the project log access settings.
• Edit permission on the source executor the logs were emitted under.

To enable log access only for a specific source executor resource in a project with log access disabled, administrators can create a resource override in platform.

Log schema

Foundry application logs are structured logs that follow a consistent schema, making them suitable for programmatic analysis and monitoring.

Specifically, Foundry logs contain information about various application events, including transform executions, build processes, system errors, and other operational activities. The log structure varies depending on the type of application or service that generates the log entry.

Application-specific information is captured within structured fields that provide context about the operation being performed, including execution details, error information, and performance metrics.

Foundry provides two schema types:

• An internal Palantir format that provides the payload as a serialized JSON string
• The OpenTelemetry (OTel) protocol provides logs in OTel protocol ↗ format, with the payload serialized as a protocol buffer binary for ingestion downstream to an OTel collector.

Log attribution

Foundry application logs are generated by various services and applications within the platform. When logs are streamed to the configured dataset, they are filtered to include only logs relevant to the organization that configured the log storage configuration.

Log exports configured for a given organization will receive logs from applications and services used by that organization. This includes transform executions, builds, and other activities performed within the organization's context.

The RID of the resource that generated the log is included in the log entry and can be used to trace the log back to the resource.

Log guarantees

Foundry logs are not audit logs. There is no guarantee of 100% reliable log delivery.

Log contents produced by Foundry are subject to change without notice.