Add modeling resources to a Marketplace product
You can use Foundry DevOps to include your modeling resources in Marketplace products for other users to install and reuse. Learn how to create your first product.
Supported features
Models can be packaged as part of products in DevOps and deployed via Marketplace for release management purposes.
Package a model
Models can be packaged into a marketplace product as outputs (content that will be deployed), but not as inputs that can be mapped at installation time.
There are two ways to include a model in a product's outputs, Include with content, and Include with producer.
Include with content
The Include with content option installs the model as a static asset that cannot be rebuilt after it is deployed. The following are copied into the model resource when using this option:
- Saved model files and weights.
- The Python library containing the model adapter class.
- The container image that is used to back live deployments in the model application and in Modeling Objectives.
- The sidecar model inference in transforms.
Models can only be deployed as Include with content if the total model size, including files and underlying container images, is under two gibibytes. The DevOps application will validate this condition and display an error if it is not met.
Include with producer
The Include with producer option installs the logic used to build the model so it can be built after it is deployed. This enables users to deploy models that can be trained continuously by a schedule. This is available for models built in Code Repositories and in Jupyter® notebooks published as transforms, but not for models published interactively from a Jupyter® notebook. This is because models published interactively are considered static assets and do not have logic associated with them. Support for models created in Model Studio is currently under development. When using this option, it is recommended that you package the latest version of the model to avoid confusion.
Models deployed with a code repository producer do not have a defined container image by default, since this is produced when checks are run in the repository and used for subsequent builds that produce the model. Checks do not run when installing a product, so installed model builds are unable to produce a container image for the model. The lack of a container image on a model version will prevent the use of sidecar model inference in transforms and live deployments for the deployed version.
To solve this issue, manually trigger checks on the installed repository that produces the model to build the container image and make it available for subsequent model training builds. Manually triggering checks is required to ensure that the Python adapter logic and the environment in the container are the same as in the source model. This must be done whenever a new version of the model is deployed.
Live deployments
Models deployed via Marketplace can be used to serve as live deployments, subject to the caveat on model container images above. Live deployment needs to be configured manually only once after deployment, and cannot be automatically configured by Marketplace. If the model is rebuilt with a regular build or through an upgrade of the Marketplace product, the live deployment will automatically update to point to the latest version that was just created as described in the model deployment documentation.
Model functions
Set up a function on the deployed model
Like direct deployments, model functions need to be configured once on the model after deployment and will upgrade automatically upon new model version creation.
Use the model function in another function
Since model functions are not explicitly recognized by Marketplace, care needs to be taken if the model function is called from a Workshop application or from another function deployed via marketplace. Workshop and TypeScript v1 functions that call models will display the model function as an input to be provided during installation. If the model function is called from TypeScript v2 or a Python function via the Platform SDK, the resulting TypeScript v2 or Python function will not have an explicit dependency on the model function. Deploying such functions will require updating the model function's API name in the installed TypeScript v2 or Python repository, after a model function has been created on the model that was installed by Marketplace.
Given the above considerations, below is the recommended setup to package a model, its model function, and an outer function that calls the model function.
- The outer function that calls the model should be written in TypeScript v1.
- Create a product containing the model. After it is deployed for the first time, configure a live deployment and a function.
- Create a second product containing the outer function and any dependent applications, with the model function as an input. Use the function created in the previous step when installing this second product.
- The model function used as input needs to be updated to the latest version every time the model's API changes, since model functions will only accept typed objects fitting the model API definition.
Unsupported modeling features and limitations
- Modeling Objectives: Use direct model deployments for live objective deployments and model transforms for batch deployments.
- Model Studio models packaged with their producer rather than as static content are not supported.
- Models as inputs rather than as contents to the Marketplace product are not supported.
- Models packaged with content can only be packaged if their total size, including weights, Python environment and underlying container image, is under two gibibytes.
- External models are not supported in DevOps or in Marketplace.
- Container models can be included with content, but will not support sidecar model inference in transforms or direct deployments.