Applications across Foundry emit logs that provide visibility into their operation, including transform runs, build failures, and other system events.
Foundry logs contain information about:
Additionally, any user-added logs written in Code Repositories will also be emitted.
These logs can be exported to a Foundry streaming dataset for real-time monitoring and analysis. This feature is only available to Organization Administrators, who can configure a destination where logs from Foundry applications will flow into a Foundry streaming dataset.
Foundry logs can be exported, per-organization, directly into a Foundry streaming Dataset. As part of the configuration setup, an Organization Administrator chooses where within the Foundry file system this dataset will be generated.
Once log data is flowing into the streaming dataset, you can process and analyze the data in real-time using Foundry's extensive suite of data transformation and visualization tools or export the logs to external monitoring systems.
To export Foundry logs, a user must be an Organization Administrator. This role provides the necessary permissions to configure log exporting destinations for the organization. See Organization Permissions for more details about organization-level permissions.
Additionally, sensitive details in logs are obfuscated and/or removed to protect user privacy and security.
To set up log exporting in Foundry, follow these steps:
Note that when first configuring log exporting, it may take some time for logs to begin flowing into the streaming dataset.
To disable log exporting, delete any log storage configurations listed by selecting the trash icon on the right side of the configuration.
Log storage datasets can contain high volumes of streaming data, so we recommend filtering the dataset appropriately before performing analysis. Consider time-based filtering to focus on relevant log entries for your monitoring and troubleshooting needs.
Foundry provides many powerful tools for performing log analysis, such as Transforms and Pipeline Builder.
Foundry application logs are structured logs that follow a consistent schema, making them suitable for programmatic analysis and monitoring.
Specifically, Foundry logs contain information about various application events, including transform executions, build processes, system errors, and other operational activities. The log structure varies depending on the type of application or service that generates the log entry.
Application-specific information is captured within structured fields that provide context about the operation being performed, including execution details, error information, and performance metrics.
Foundry application logs are generated by various services and applications within the platform. When logs are streamed to the configured dataset, they are filtered to include only logs relevant to the organization that configured the log storage configuration.
Log exports configured for a given organization will receive logs from applications and services used by that organization. This includes transform executions, builds, and other activities performed within the organization's context.
The RID of the resource that generated the log is included in the log entry and can be used to trace the log back to the resource.