Build a proxy or federation layer with AIP

A proxy or federation layer sits between AIP and one or more external model providers. Your compute module receives inference requests from AIP, applies custom logic, and forwards them to external providers. This allows you to control exactly what data leaves your environment and how requests are routed.

Use cases

• PII redaction: Strip or mask sensitive data from prompts before they reach an external provider.
• Prompt injection: Prepend system prompts, append instructions, or transform request content before forwarding.
• API format remapping: Translate between the provider API format that AIP expects and a provider that uses a different format.
• Multi-provider federation: Route requests across multiple providers behind a single registered model, based on cost, latency, or model availability.

Additional prerequisites

In addition to the general prerequisites for models backed by compute modules, using a compute module as a proxy or federation layer requires:

• Network egress: Your compute module must be able to reach external model providers. Configure egress through a source in Data Connection with the appropriate network policies.
• Secrets: API keys or other credentials for external providers must be attached to the source. Your container accesses these credentials at runtime through the SOURCE_CREDENTIALS environment variable. Review sources for details on configuring and reading secrets.

How to build a proxy or federation layer with AIP

1. Build a container that implements a supported provider API format. The container should receive requests from AIP, apply your custom logic (such as redaction, routing, or prompt injection), and forward them to one or more external providers.
2. Create a source in Data Connection with network policies that allow egress to your external providers. Attach API keys or other credentials as secrets.
3. Set the application.port label in your Dockerfile and publish the image to Artifacts. For general container guidance, review the compute module containers documentation.
4. Create a compute module with the published image and attach the source you created. Set minimum replicas to at least one.
5. Register the model in Control Panel and configure capabilities.
6. Select the model in any supported AIP application.