• Documentation
    • Search documentation
    karat

    +

    K

    API Reference ↗
    Data connectivity & integrationMedia sets (unstructured data)Importing media

    Importing media

    There are multiple ways that you can import media into a media set, including:

    • Direct upload
    • Connecting to an external source system
    • Transforms
    • Using actions
    • OSDK

    Path deduplication

    If a file is uploaded to a media set and has the same path (usually the filename, such as example.png) as an existing item in the media set, the new item will overwrite the existing one.

    Deduplication behavior

    You must ensure path uniqueness to avoid overwrites. No warning or confirmation will appear before an upload overwrites an existing media item.

    An overwritten media item will still be available when using a direct media reference to that media item. Overwritten media items can also be listed by setting deduplicate_by_path=False in transforms.

    For example:

    Copied!
    1
dataset = input_media.dataframe(deduplicate_by_path=False)

    Direct upload

    To import media files to a media set through a direct upload, drag and drop the files into your new media set. Files must match the expected file type specified upon creation of the media set to be uploaded to the media set.

    1. First, create a new media set by selecting New within a Project and selecting media set from the search bar as shown below.

    Create a media set from a Project

    1. Next, choose the desired media file type for the new media set and select Create media set.

    Choose a media file type

    1. Once you have created a media set, you can upload media via drag-and-drop onto the empty media set or by selecting the choose from your computer prompt.

    Upload from an empty media set

    Connecting to an external source system

    Data Connection

    Media sets can be imported using a sync to an external source through Data Connection. A detailed walk-through can be found in the media set sync documentation.

    To create a new media set sync, navigate to the Overview tab of the desired source.

    After you create the sync, trigger a build in the media set view for the media to appear in your media set.

    You can also connect an existing source to a new media set via the Select a source option.

    Add existing sources into a media set

    Virtual storage

    For supported source types, media sets can optionally be configured to read directly from the external source system so no data is copied into Foundry’s backing store. These are called virtual media sets.

    Currently, virtual media sets are only supported for certain source types. Contact Palantir Support if you are interested in virtual storage for other source types for your use case.

    storage policy

    External transforms

    For sources with REST APIs, you can import media to a media set through external transforms.

    Transforms

    Pipeline Builder

    Media sets can also be directly imported into Pipeline Builder. Learn more about available upload methods in Pipeline Builder.