Announcements
REMINDER: Sign up for the Foundry Newsletter to receive a summary of new products, features, and improvements across the platform directly to your inbox. For more information on how to subscribe, see the Foundry Newsletter and Product Feedback channels announcement.
Share your thoughts about these announcements in our Developer Community Forum ↗.
Use AIP to write code, analyze data, and create visualizations in JupyterLab® and RStudio® Code Workspaces
Date published: 2025-11-11
JupyterLab® and RStudio® Code Workspaces now provide an AIP agent accessible from a workspace's sidebar, enabling access to any of AIP's supported large language models (LLMs) to help you develop and deploy code in Foundry based on your specific use case. This experimental feature is available for all Foundry enrollments with AIP enabled.
An AIP agent helps you write code and generate visualizations in JupyterLab® and RStudio® Code Workspaces.
Open and configure the agent in your workspace
To get started, open your workspace, select the </> icon at the bottom of the left sidebar, and enter a prompt in the Ask a question... text box to initiate the agent. The agent will provide coding guidance or generate complete files for you based on its available tools. To configure the tools an agent can access to help it perform essential operations in your workspace, select the wrench icon to render all available Tools and opt the agent out of those which are not relevant for your use case. Agents can perform a wide range of tasks through its tools, such as author files, write and run code snippets, search for and install libraries, or execute terminal commands.
Configure the tools available to the AIP agent in your workspace.
Use the agent's Settings menu to rename conversation threads and view the system prompt. The agent will not persist your chat history after you shut it down or restart the workspace, so make sure to sync any code or model outputs you want to save before ending your session.
You can rename chat threads using the agent's Settings menu.
You can alternate the LLM your agent uses by selecting the name of the current model from the bottom of the prompt text box. Model behavior may vary across providers, so you can experiment with different models to find the approach that works best for your specific use case. Learn more about prompt engineering best practices.
Common AIP agent use cases
You can use a workspace's AIP agent to:
- Generate starter code for data analysis, visualizations, and modeling tasks.
- Receive coding tips and explanations about your data.
- Automate repetitive tasks, such as library installation and data cleaning.
- Build interactive Dash and Streamlit applications that you can publish and share.
What's next?
We will continue to refine the agent's capabilities and expand its toolkit as we gather feedback during its initial experimental release. Additionally, support for writing Foundry models will be available in the coming weeks.
Jupyter®, JupyterLab®, and the Jupyter® logos are trademarks or registered trademarks of NumFOCUS. RStudio® is a trademark of Posit™. All third-party trademarks (including logos and icons) referenced remain the property of their respective owners. No affiliation or endorsement is implied.
Monitor and analyze operational workflows with the new Machinery widget
Date published: 2025-11-11
The new Machinery widget, an analysis and real-time monitoring tool that provides operational insights for your configured Machinery processes, is available on all enrollments the week of November 10. This new capability enables teams to visualize process flows, track key metrics, and identify performance issues without requiring additional configuration beyond your existing Machinery setup.
The new Machinery widget at a glance.
The new Machinery widget natively supports multi-process graphs, allowing you to track metrics across multi-object-type process implementations. The widget is available in Workshop modules or as a stand-alone view in the Machinery application with limited features.
Automatic subprocess derivation and customizable metric views
Configuration is streamlined through automatic derivation of subprocess object sets using search arounds from parent processes. This means that you only need to configure one object input for each root process. If you have an application process with many linked review subprocesses, you can provide 100 application objects, and all related child objects will be automatically identified through configured link types.
Four metric views are preconfigured and can be customized by application builders; historical count, current count, historical duration, and current duration. Application builders can also add custom metric views to suit specific analytical needs. Users can switch between these views, hover over nodes to reveal all available metrics, and pin specific nodes for continuous monitoring across the graph visualization.
The new Machinery widget optimizes space usage using contextual zoom. When zoomed out, it will show many graph elements, but only a single metric. When zoomed in, nodes reveal additional information and metric cards show up to three available metrics.
Contextual zoom reveals additional information and metrics.
Path explorer and the duration distribution for deeper insights
Two analysis modes enable process investigation beyond visualization. Path explorer analyzes individual process paths and their frequency distribution, allowing you to select specific paths to filter outputs and understand exactly how objects flow through your workflow.
Analyze individual process paths and their frequency distribution with path explorer mode.
Duration distribution identifies performance outliers through the visualization of time spent in selected states across all objects. This allows the isolation of individual buckets or ranges of objects with undesirable behavior, such as spending excessive time in particular transitions or states. Both analysis modes update output object sets dynamically, enabling iterative investigation of process performance issues.
Use the duration distribution mode to identify outliers through visualization of time spent in states across selected objects.
Transition nodes, deviations, and subprocess graph features for flexible analysis
Multiple graph features adapt visualizations to different use cases where bottleneck identification is critical. Transition nodes simplify complex graphs by replacing actions and automations with implicit state transitions, providing a cleaner state-transition perspective. Additionally, subprocesses can be replaced with their implicit state transitions for visibility into transition metrics on the currently focused process.
The Machinery v2 widget automatically detects and removes objects that are deviating from the process definition, helping to remove noise from the performance analysis. Non-conforming objects can be made visible and explicitly included or isolated in the output. When visible, deviating states and transitions are visually highlighted, with metrics computed across all input objects rather than just conforming ones. This is valuable when investigating why certain processes deviate from expected patterns.
Your feedback matters
We want to hear about your experiences using Machinery and welcome your feedback. Share your thoughts with Palantir Support channels or on our Developer Community ↗ using the machinery tag ↗.
Learn more about the Machinery widget.
Workflow Builder is now Workflow Lineage
Date published: 2025-11-11
Starting the week of November 10, Workflow Builder will be rebranded as Workflow Lineage, better reflecting its role as an interactive workspace for visualizing, understanding, and managing application dependencies and their underlying processes.
The newly renamed Workflow Lineage home page.
Impact on users
All existing features and functionalities remain unchanged, and you can continue to use Workflow Lineage as usual. You should see the new name reflected across Foundry and platform communications. If you have any questions about this change, share them with Palantir Support channels or on our Developer Community ↗ .
Learn more about Workflow Lineage.
Ontology and AIP Observability: Tracing-backed telemetry is now available
Date published: 2025-11-06
Tracing, logging, and run history views for functions, actions, automations, and language models are now available in Workflow Lineage for all users. Additionally, starting the week of November 10, all in-platform logs (including those from the Ontology and AIP workflows) can be exported to a real time streaming dataset, allowing for powerful custom analysis of your telemetry.
What's new?
Ontology and AIP workflows now come out of the box with first-class tracing, logging, and run history views for all functions, actions, automations, and language models:
- Run history: View the run history of your functions, actions, and automate effects over the past seven days with the caller, timestamp, duration, and success or failure of the execution included for every run.
- Trace view: Dive into each execution and view individual units of work across your workflow for functions, language models, actions, and automations.
- Log view: View custom function logs and a suite of first-class system logs across the Ontology and AIP ecosystem for a given execution.
Telemetry highlights include the following:
- TypeScript v1 functions: All console logs, inputs, and outputs are logged along with automatically generated spans for each Ontology load.
- TypeScript v2 functions: Powerful OpenTelemetry libraries are available for users to emit custom spans and logs on top of the default instrumentation of every outbound network call.
- Actions: View granular steps of the action lifecycle such as edit calculation, Object Storage v2 and Funnel writes, webhook calls, edited objects, and more.
- Language models: Observe each language model call in your workflow execution with prompts, responses, and token usage logged for every invocation.
- Foundry streaming datasets: Export all Foundry logs from selected projects to a streaming dataset and perform custom analysis on your telemetry or export to a third-party system.
Get started
To start observing your Ontology and AIP workflows, follow the steps below:
- Open the Workflow Lineage application.
- Navigate to a function, action, or automation from the panel on the left.
- Select the Run history tab from the bottom preview panel to view recent executions.
- Select View log details on any execution node to access traces and logs.
- Ensure proper log permissions are configured for the projects of your resources.
The trace view for a function workflow execution.
As stated in the log permissions and configure logging documentation, users with the Information security officer or Enrollment administrator role can manage the Log observability settings for an organization in Control Panel.
Share your feedback
Let us know what you think about our new observability capabilities for Ontology and AIP workflows. Contact our Palantir Support channels, or leave your feedback in our Developer Community ↗ .
Create, view, and monitor Foundry peering connections using Peer Manager
Date published: 2025-11-06
Peer Manager enables you to view and monitor jobs associated with an established peering connection that synchronizes objects and links between Foundry enrollments in real-time as well as mediates changes made across ontologies. The application will be generally available across all enrollments the third week of November.
What is peering?
Peering enables organizations to establish secure, real-time Ontology data synchronization across distinct Foundry enrollments. Peer Manager is the central home for administering peering in Foundry. From Peer Manager, space administrators can create peer connections, monitor peering jobs, and configure data to peer.
What is Peer Manager?
After you create a peer connection, you can use Peer Manager's home page to garner information about your new connection and all other connections configured between your enrollment and other enrollments. Peer connections support the import and export of Foundry objects and their links as well as object sets configured in Object Explorer.
The Peer Manager home page provides an overview of all configured Peer Connections.
Select a connection to launch its Overview window, where you can track the health of each peer connection by viewing the status of individual peering jobs.
Peer Manager's Overview window offers a unified view of the status and health of peering jobs within a connection.
Select Ontology from the top ribbon to peer objects across an established connection, where Peer Manager enables you to peer all or a selection of properties on the object.
Learn more about object peering in Peer Manager.
Peer Manager's Ontology window enables you to peer object types and their links across a peer connection.
What's next on the development roadmap?
The ability to configure Artifact peering will be available in Peer Manager by the end of 2025. Contact Palantir Support with questions about peering or Peer Manager on your enrollment.
Use Pipeline Builder external pipelines to push down compute to Databricks
Date published: 2025-11-06
Pipeline Builder now offers the ability to create external pipelines using third-party compute engines, with Databricks as the first supported provider. This capability is in beta.
External pipelines require virtual table inputs and outputs from the same source as your compute. When using external pipelines, compute is orchestrated by Foundry and pushed down to the source system for execution.
Foundry’s external compute orchestration provides you with the flexibility to choose the most appropriate technology for your workload, use case, and architecture requirements. Pipelines built with external compute can also be composed together with Foundry-native compute pipelines using Foundry’s scheduling tools, allowing you to easily orchestrate complex multi-technology pipelines using the exact right compute at every step along the way.
With this improvement, you can now push down compute to Databricks using either code-based Python transforms or point-and-click Pipeline Builder boards. Learn more about creating external pipelines in Pipeline Builder.
Enabling push down compute in Pipeline Builder.
External pipeline with pushdown compute in Pipeline Builder.
Your feedback matters
As we continue to add features to Pipeline Builder, we want to hear about your experiences and welcome your feedback. Share your thoughts with Palantir Support channels or our Developer Community ↗ using the pipeline-builder tag ↗.
Import Iceberg and Delta virtual tables into Code Workspaces
Date published: 2025-11-06
Iceberg ↗ and Delta ↗ tables can now be imported as virtual tables into JupyterLab® code workspaces, providing more flexibility when working with externally stored data at large scales. Delta and Iceberg tables are open source table formats that enable reliable, scalable, and efficient management of large datasets, including tables stored in Databricks.
JupyterLab® code workspaces now support read and write capabilities for Iceberg and Delta tables, and provide table-specific code snippets in the Data panel to facilitate development.
A highlighted code snippet in the Data panel.
This feature enables running interactive Python notebooks against data stored and cataloged externally to Foundry in Iceberg and Delta tables, supporting a wide range of data science, analytics, and machine learning workflows.
Learn more about virtual tables and Code Workspaces.
Jupyter®, JupyterLab®, and the Jupyter® logos are trademarks or registered trademarks of NumFOCUS. All third-party trademarks (including logos and icons) referenced remain the property of their respective owners. No affiliation or endorsement is implied.
Package widget sets automatically with Marketplace products
Date published: 2025-11-04
Widget sets created in Custom Widgets can now be included as content in Marketplace products.
When you add a Workshop module that uses a widget set to a Marketplace product, the widget set is automatically packaged. Widget sets can also be manually packaged independently, allowing you to build Workshop modules on top of them.
If a widget set had Ontology API access enabled in the source environment, it will be installed with access disabled by default. After installation, you must manually enable Ontology API access on the widget set if needed.
Published Marketplace product containing a Workshop module that uses a widget set.
We want to hear from you
As we continue to develop new features for custom widgets, we want to hear about your experiences and welcome your feedback. Share your thoughts with Palantir Support channels or our Developer Community ↗ and use the custom-widgets ↗ tag.
Roll back datasets and queue snapshots in Data Lineage
Date published: 2025-11-04
Dataset rollback is now available in Data Lineage, giving you greater control over your data pipelines. Whether you encounter an outage, errors in your pipeline logic, or unexpected upstream data, dataset rollback provides a fast, reliable way to revert your datasets to a stable state. In addition, you can now queue snapshots, allowing datasets to snapshot automatically on their next build.
Dataset rollbacks provide several key benefits:
- Quickly revert to an earlier version of your dataset.
- Preserve incremental build workflows.
- Recover from data issues or outages.
Roll back a dataset
To get started with dataset rollback, open your dataset in Data Lineage and select a previous successful transaction in the History tab. You can roll back your dataset to that transaction by selecting Roll back to transaction.
The Roll back to transaction option, listed in a selected transaction's Overview tab.
Snapshot on next build
To queue a snapshot on your dataset's next build, open a dataset in Data Lineage and select Force snapshot In the History tab in the bottom panel.
The Force snapshot option in the History tab.
Note that you will need to acknowledge that this action cannot be undone before proceeding.
Important considerations
- Only users with the
Editorrole can perform rollbacks to ensure secure operations. - Rollbacks are only supported on transactional datasets, and you can only roll back to successful transactions.
- It is not possible to roll back to a transaction that was deleted based on a retention policy. However, you can roll back to a transaction that was deleted by a dataset rollback in Data Lineage.
- If your dataset backs an object type in object storage v2, you will need to reindex after rolling back.
Dataset rollback allows you to build, experiment, and iterate on your pipelines with confidence; the ability to revert to a stable state is available whenever you need it.
Your feedback matters
We want to hear about your experience and welcome your feedback as we develop more features in Data Lineage. Share your thoughts with Palantir Support channels or on our Developer Community ↗ using the data-lineage tag ↗.
Learn more about dataset rollback.
Flexible rebasing and conflict resolution in Ontology Manager
Date published: 2025-11-04
Ontology Manager now offers an improved rebasing and conflict resolution experience that gives you greater flexibility and control when managing branch changes. You can now rebase at any point without creating a proposal, view changes from both Main and your branch simultaneously, and resolve merge conflicts using multiple approaches—either through the Conflicts tab in the Save dialog or directly in the Ontology Manager interface for conflict resolution. This enhanced workflow prevents situations where unresolvable errors block your progress. This feature is available the week of November 3 across all enrollments.
Visit the documentation on testing changes in the ontology.
What is rebasing?
While you introduce changes on your branch, Main can also update with new changes made by others. Rebasing incorporates the latest changes from Main into your current branch to keep it up to date.
Resolve merge conflicts by choosing between changes from Main or your current branch directly in Ontology Manager.
During a rebase, Ontology Manager enters a new state where you can view and access changes from both Main and your branch. You may resolve merge conflicts by choosing between changes from Main or your current branch from the Conflicts tab in the save dialog. Alternatively, you can resolve conflicts by editing the ontology resource directly. This flexibility prevents situations where users become stuck due to unresolvable errors after conflict resolution.
Known limitations
Complex cases of schema migrations or datasource replacements are not yet handled by this rebasing experience. Refer to the known limitations section of the documentation for an alternative solution. We are actively working to resolve these limitations.
We want to hear from you
As we continue to develop new features for Foundry Branching, we want to hear about your experiences and welcome your feedback. Share your thoughts with Palantir Support channels or our Developer Community ↗ and use the foundry-branching ↗ tag.