Self-hosting models with AIP is a Beta feature. Functionality may change during active development.
You can self-host models by backing a registered model with a compute module. Self-hosting allows you to run open-source or custom large language models on your own infrastructure and use them as first-class models in AIP. This is useful when you need:
Data sovereignty: Keep all inference traffic within your own network boundary.
On-premise or air-gapped deployments: Run models without any external connectivity.
Cost control: Use your own GPUs instead of paying per-token for hosted providers.
Early access to new models: Deploy newly released open-source models before they are available through a hosted provider.
Build a container image running an inference server (such as vLLM or Ollama) that serves your model weights and exposes a supported provider API format.
Set the application.port label in your Dockerfile and publish the image to Artifacts. For general container guidance, review the compute modules containers documentation.
Create a compute module with the published image. Set minimum replicas to at least one.