AI Platform (AIP)Agent SDKAdvanced customization

Advanced customization of agents in Agent SDK

Before proceeding with this advanced customization guide, ensure that you understand the following concepts:

Context items: The agent session, stateful strongly typed data.
Events: Something that happened, the way state changes.
Effects: Connects the agent to the outside world.

This page contains information on advanced customization options for:

System prompt
Model
Custom context items
Tools
State machine

System prompt

Dynamic system prompt

The systemPrompt function receives the current agent state. The SDK evaluates it for each model request, so a state update changes the instructions used for the next request. For example, an agent can adapt its explanation to the audience stored in the session:

Copied!1
2
3
4
5
6
7
import { SystemPromptRenderResult } from "@palantir/agents";

// Inside defineAgent({ ... }) with `audience` in the agent state:
systemPrompt: state =>
    SystemPromptRenderResult.ready([
        { type: "text", text: `Explain the answer for ${state.audience}.` },
    ]),

Model

Write a custom `LanguageModel`

The model option accepts any object implementing LanguageModel. For example, you can compose two models and use a backup if the first one returns a terminated response:

Copied!1
2
3
4
5
6
7
8
9
10
11
12
13
14
import { type LanguageModel } from "@palantir/agents";

// primaryModel and backupModel are LanguageModel values.
const languageModelWithBackup: LanguageModel = {
    async generateResponse(args) {
        const response = await primaryModel.generateResponse(args);
        return response.status.type === "terminated"
            ? backupModel.generateResponse(args)
            : response;
    },
};

// Inside defineAgent({ ... }):
model: languageModelWithBackup,

You can also implement generateResponse using your own model client. Convert the SDK's messages and tools into that client's request format, then convert its response into the SDK's assistant parts and a status. This allows calling a new open-source model or a custom fine-tuned model from your agent.

The built-in provider constructors handle this conversion for supported providers out of the box.

Select a model from agent state

Dynamically select the model at runtime by passing a function to model. If an event changes that state, subsequent requests can use a different model.

Copied!1
2
3
4
5
6
import { z } from "zod";

// fastModel and detailedModel are LanguageModel values.
// Inside defineAgent({ ... }):
agentStateSchema: z.object({ useDetailedModel: z.boolean() }),
model: state => state.useDetailedModel ? detailedModel : fastModel,

Configure provider-specific model settings

Each built-in provider constructor accepts modelSettings for settings supported by that provider. Common settings include reasoning, sampling temperature, and response length:

Copied!1
2
3
4
5
6
7
8
9
10
11
12
import { Aliases } from "@osdk/functions";
import { anthropic, openAi } from "@palantir/agents";

const anthropicModel = anthropic({
    modelRid: Aliases.model("anthropicModel").rid,
    modelSettings: { max_tokens: 4096, temperature: 0.2 },
});

const openAiModel = openAi({
    modelRid: Aliases.model("openAiModel").rid,
    modelSettings: { reasoning: { effort: "high" } },
});

Settings differ by provider and model.

Custom context items

Context items are typed pieces of session information. The SDK creates user messages, assistant messages, and tool error items automatically. Define a custom item with zodContextItem when you want to store structured data and control how it appears to the model. Register the definition in contextItemDefinitions:

Copied!1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
import { Message, RenderResult, UserMessagePart, zodContextItem } from "@palantir/agents";
import { z } from "zod";

const WeatherResult = zodContextItem({
    type: "weather-result",
    createArgsSchema: z.object({ city: z.string(), summary: z.string() }),
    dataSchema: z.object({ city: z.string(), summary: z.string() }),
    create: args => ({ data: args }),
    render: data =>
        RenderResult.ready([
            Message.user(UserMessagePart.text(`${data.city}: ${data.summary}`)),
        ]),
});

// Inside defineAgent({ ... }):
contextItemDefinitions: [WeatherResult],

The create function initializes stored data. The render function turns that data into model messages. Return RenderResult.pending() if the item is still waiting for data; generation resumes after the item becomes ready. Custom context items can also define events that update their data.

Tools

Dynamic tools

The tools option can be a function that receives the current agent state for each model request. Return tools based on that state; a state update changes the tools offered on later requests:

Copied!1
2
3
4
5
import { z } from "zod";

// Inside defineAgent({ ... }):
agentStateSchema: z.object({ weatherEnabled: z.boolean() }),
tools: state => state.weatherEnabled ? [WeatherTool] : [],

Tools that update session state

A tool can return a session event alongside its result. For example, if research notes include their context item IDs in their rendered text, the agent can hide a note when it is no longer useful:

Copied!1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
import { setContextItemOrder, UserMessagePart, zodSessionEvent, zodTool } from "@palantir/agents";
import { z } from "zod";

const HideContextItem = zodSessionEvent(
    "hide-context-item",
    z.object({ contextItemId: z.string() }),
);
const HideNote = zodTool({
    name: "hide_note",
    description: "Hide a research note that is no longer needed",
    argumentSchema: z.object({ contextItemId: z.string() }),
    emits: [HideContextItem],
    execute: async ({ contextItemId }) => ({
        contextItems: [{
            contextItemType: "@platform/user-message",
            createArgs: { parts: [UserMessagePart.text("Note hidden from context.")] },
        }],
        events: [HideContextItem.create({ contextItemId })],
    }),
});

// Inside defineAgent({ ... }):
tools: () => [HideNote],
eventDefinitions: [HideContextItem],
handleEvent: (state, event) => ({
    stateUpdates: [setContextItemOrder(
        state.contextItemOrder.filter(id => id !== event.payload.contextItemId),
    )],
}),

Removing a top-level item from contextItemOrder omits it from future model requests but keeps the item in session state. The agent can restore it by adding its ID back to the order.

Custom tool loading UI

For a long-running tool, the default behavior does not allow a custom UI showing the progress of the tool. Context items provide the framework to do this:

Have your tool return a context item immediately. The tool will not be marked complete until the context item renders Ready.
Do all long-running work through context item events and effects, returning RenderResult.ready() only when done.
In a custom frontend application, render your context item state however necessary to display rich intermediary states to users.

For example, the application could show “Searching flights to Tokyo…” while a pending context item loads results:

Copied!1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
import {
    Message,
    RenderResult,
    UserMessagePart,
    zodContextItem,
    zodContextItemEffect,
    zodContextItemEvent,
    zodTool,
} from "@palantir/agents";
import { z } from "zod";

const FlightsReady = zodContextItemEvent("flights-ready", z.string());

const SearchFlights = zodContextItemEffect(
    "search-flights",
    z.string(),
    [FlightsReady],
    async (destination, dispatchEvent) => {
        dispatchEvent(FlightsReady.create(await findFlights(destination)));
    },
);

const FlightSearchResult = zodContextItem({
    type: "flight-search",
    createArgsSchema: z.object({ destination: z.string() }),
    dataSchema: z.object({ destination: z.string(), result: z.string().nullable() }),
    eventDefinitions: [FlightsReady],
    create: ({ destination }) => ({
        data: { destination, result: null },
        effects: [SearchFlights.create(destination)],
    }),
    handleEvent: (data, event) => ({ data: { ...data, result: event.payload } }),
    render: data => data.result === null
        ? RenderResult.pending()
        : RenderResult.ready([Message.user(UserMessagePart.text(data.result))]),
});

const FindFlights = zodTool({
    name: "find_flights",
    description: "Find flights to a destination",
    argumentSchema: z.object({ destination: z.string() }),
    execute: async args => ({
        contextItems: [{ contextItemType: FlightSearchResult.type, createArgs: args }],
        events: [],
    }),
});

Human-in-the-loop tools

This uses the same pending context item pattern as custom tool loading UI. Instead of starting a data load in an effect, the tool returns a UserQuestion item for the application to display. A human responds asynchronously, and the application dispatches a context item event with the answer. The item then renders the answer, allowing the model to continue.

Copied!1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
const AnswerProvided = zodContextItemEvent("answer-provided", z.string());

const UserQuestion = zodContextItem({
    type: "user-question",
    createArgsSchema: z.object({ question: z.string() }),
    dataSchema: z.object({
        question: z.string(),
        answer: z.string().nullable(),
    }),
    create: ({ question }) => ({ data: { question, answer: null } }),
    eventDefinitions: [AnswerProvided],
    handleEvent: (data, event) => ({ data: { ...data, answer: event.payload } }),
    render: ({ answer }) =>
        answer === null
            ? RenderResult.pending()
            : RenderResult.ready([Message.user(UserMessagePart.text(answer))]),
});

const AskUser = zodTool({
    name: "ask_user",
    description: "Ask the user a question and wait for an answer",
    argumentSchema: z.object({ question: z.string() }),
    execute: async ({ question }) => ({
        contextItems: [
            { contextItemType: UserQuestion.type, createArgs: { question } },
        ],
        events: [],
    }),
});

State machine

In the Agent SDK, an agent is a state machine, processing queued events deterministically and updating its state. Effects can be used to do asynchronous work and report their results back to the agent by dispatching events.

Hooking into this custom state machine is a very powerful capability of the Agent SDK, as it allows you to embed deterministic logic within your agent.

Custom events

Custom events allow users to define typed, deterministic steps that their agent will take. Events can be dispatched by:

An external user manually sending one
An external automation that is triggered to send events on a schedule
Your agent internally dispatching an event from a custom effect

Events are queued and processed in order. An event handler can:

Update the AgentState
Create, delete, or reorder context items
Dispatch events to context items to update their data
Start custom effects for asynchronous work

Custom effects

Custom effects perform asynchronous work outside the event handler. An event handler can start a registered effect, which dispatches another registered event when its work finishes.

Example: Schedule a meeting

A scheduling agent can suggest a meeting time, then send a calendar invitation after the user confirms it. The confirmation event marks the session as scheduling and starts an effect that calls a calendar service. When the invitation is sent, the effect dispatches an event that updates the agent state to indicate that the meeting has been scheduled and adds all the necessary information.

Copied!1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
import { setAgentState, zodSessionEffect, zodSessionEvent } from "@palantir/agents";
import { z } from "zod";

const MeetingDetails = z.object({ title: z.string(), startTime: z.string() });
// Event triggered when user confirms meeting
const MeetingConfirmed = zodSessionEvent(
    "meeting-confirmed",
    MeetingDetails,
);
// Event triggered when the invitation is sent
const InvitationSent = zodSessionEvent(
    "invitation-sent",
    z.object({ meetingUrl: z.string() }),
    { type: "private" },
);
const SendInvitation = zodSessionEffect(
    "send-invitation",
    MeetingDetails,
    [InvitationSent],
    async (meeting, dispatchEvent) => {
        const meetingUrl = await createCalendarInvitation(meeting);
        dispatchEvent(InvitationSent.create({ meetingUrl }));
    },
);

// Inside defineAgent({ ... }):
agentStateSchema: z.object({ status: z.string(), meetingUrl: z.string().nullable() }),
eventDefinitions: [MeetingConfirmed, InvitationSent],
effectDefinitions: [SendInvitation],
handleEvent: (state, event) => {
    switch (event.type) {
        case MeetingConfirmed.type:
            return {
                stateUpdates: [setAgentState({
                    ...state.agentState,
                    status: "scheduling",
                    meetingUrl: null,
                })],
                // kick off the send invitation effect
                effects: [SendInvitation.create(event.payload)],
            };
        case InvitationSent.type:
            return {
                stateUpdates: [setAgentState({
                    ...state.agentState,
                    status: "scheduled",
                    meetingUrl: event.payload.meetingUrl,
                })],
            };
    }
},

Example: Load patient data at session start

A patient-facing agent can load a patient's complete record when a session starts. createSession starts the load, the effect fetches the record, and the resulting event stores it in agent state for later tools or model requests.

Copied!1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
import { getPatientData, PatientDataSchema } from "./patient-data";
import { setAgentState, zodSessionEffect, zodSessionEvent } from "@palantir/agents";
import { z } from "zod";

const PatientDataLoaded = zodSessionEvent(
    "patient-data-loaded",
    PatientDataSchema,
    { type: "private" },
);
const LoadPatientData = zodSessionEffect(
    "load-patient-data",
    z.object({ patientId: z.string() }),
    [PatientDataLoaded],
    async ({ patientId }, dispatchEvent) => {
        const patientData = await getPatientData(patientId);
        dispatchEvent(PatientDataLoaded.create(patientData));
    },
);

// Inside defineAgent({ ... }):
agentStateSchema: z.object({ patientData: PatientDataSchema.nullable() }),
createSessionArgsSchema: z.object({ patientId: z.string() }),
createSession: ({ patientId }) => ({
    agentState: { patientData: null },
    // automatically load the patient data into your agent state
    effects: [LoadPatientData.create({ patientId })],
}),
eventDefinitions: [PatientDataLoaded],
effectDefinitions: [LoadPatientData],
handleEvent: (state, event) => ({
    stateUpdates: [setAgentState({
        ...state.agentState,
        patientData: event.payload,
    })],
}),