AIP Observability

AIP Observability enables you to gain comprehensive insights into your AIP workflow executions, helping you understand the performance of your agents, functions, language models, automations, Actions, and Ontology.

By integrating with Workflow Builder, AIP Observability is part of Palantir's comprehensive observability strategy across the platform. The goal of AIP Observability is to enable cross-functional teams to monitor and optimize performance at every level of the applications, workflows, and products built with AIP.

Key capabilities of AIP Observability

• Execution history: Track Function, Action, and AIP Logic executions over the past seven days.
• Distributed tracing: Visualize the complete execution flow across Functions, Actions, Language Models, Automations, and Ontology loads.
• Logging and debugging: Access service logs, custom Function log messages, token usage, prompts, error details, and more.
• Performance monitoring: Identify bottlenecks and optimize execution times.

Getting started with AIP Observability

To use AIP Observability:

1. Navigate to a Function or Action in Workflow Builder.
2. Select the Run history tab to view recent executions.
3. Click View log details on any execution to access traces and logs.
4. Ensure proper log permissions are configured for your resources.

Observability across the platform

AIP Observability complements other monitoring capabilities in the Palantir platform. The following tools work together to provide comprehensive visibility into your AIP workflows, from individual function execution to platform-wide resource consumption.

Monitoring performance and optimization

• Function performance optimization: Best practices for optimizing TypeScript v1 functions, including live preview support.
• Action metrics: Monitor Action execution patterns and performance.

Monitoring resource usage and costs

• AIP usage metrics: Model usage coloring and charts for tracking token usage and model requests.
• Logic compute usage: Track compute consumption in AIP Logic applications.
• AIP compute usage: Understand compute allocation and usage across AIP features.

Monitoring model performance

• AIP Evals: Evaluate and monitor LLM performance systematically.

Upcoming AIP Observability features

• Bulk enablement: Configure log visibility on the project level or through a bulk select option in Workflow Builder.
• Enhanced data export: Stream your logs to a dataset and perform complex analysis on your telemetry.
• Deeper interoperability and customization: Export logs in OpenTelemetry format and emit custom telemetry using OTEL libraries with Typescript v2 and Python functions.
• Expanded log production: Produce expanded logs from LLMs, more function types, and other executable resources.