• Documentation
    • Search documentation
    karat

    +

    K

    API Reference ↗
    Data connectivity & integrationAvailable connectorsSpark SQL

    Spark SQL

    The Spark SQL connector is a Palantir-provided driver for Spark SQL.

    To create a new Spark SQL source, follow the standard setup flow for Palantir-provided drivers, then use the sections below for Spark SQL-specific configuration and networking. For the complete property reference, see the official Spark SQL driver documentation ↗.

    Beta

    The Spark SQL connector is in the beta phase of development and may not be available on your enrollment. Functionality may change during active development.

    Supported capabilities

    CapabilityStatus
    Exploration🟡 Beta
    Batch syncs🟡 Beta
    Incremental🟡 Beta
    Table exports🟡 Beta

    Configuration

    The properties below are mandatory or recommended.

    PropertyRequired?DescriptionDefault
    AuthScheme ↗MandatoryThe authentication scheme used. Accepted entries are Plain, LDAP, NOSASL, and Kerberos.Plain
    Server ↗MandatoryThe host name or IP address of the server hosting the SparkSQL database.{serverAddress}
    UseSSL ↗MandatorySpecifies whether to use SSL Encryption when connecting to Hive.TRUE
    Database ↗RecommendedThe name of the SparkSQL database.—
    Password ↗RecommendedThe password used to authenticate with SparkSQL.—
    Port ↗RecommendedThe port for the SparkSQL database.10000
    TransportMode ↗RecommendedThe transport mode to use to communicate with the Hive server. Accepted entries are BINARY and HTTP.BINARY
    User ↗RecommendedThe username used to authenticate with SparkSQL.—

    Networking

    The table below lists the domains that the source must be able to access to run.

    For each domain, add a corresponding egress policy. If the source is hosted on-premises and not directly reachable from Foundry, use an agent proxy egress policy instead. The agent host itself must also be able to reach the listed domains. See using an agent as a proxy for details.

    DomainRequired
    <Server>:<Port>Always. Server and Port connection properties; default Port=10000
    <KerberosKDC>:88If AuthScheme=Kerberos
    <KerberosServiceKDC>:88If AuthScheme=Kerberos and Kerberos topology uses multiple realms

    Table exports

    This connector supports table exports. Learn how to set up a table export.