API Reference
Search documentation
karat

+

K

User Documentation ↗
Transformstransforms-pythontransforms.apiTableTransformInput

transforms.api.TableTransformInput

class transforms.api.TableTransformInput(rid, branch, table_dfreader)

The input object passed into transform objects at runtime for virtual table inputs. Mimics the TransformInput API.

property batch_incremental_configuration

The configuration for an incremental input that will be read in batches.

  • Type: BatchIncrementalConfiguration

property branch

The branch of the dataset.

property catalog

Returns the name of the table’s Spark catalog, intended for use in Spark procedures, if supported by the underlying table type.

  • Returns: The name of the table’s catalog.
  • Return type: str ↗

Throws: : ValueError: If the underlying table type does not expose a Spark catalog.

property column_descriptions

The column descriptions of the dataset.

  • Type: Dict[str, str]

property column_typeclasses

The column typeclasses of the dataset.

  • Type: Dict[str, str]

dataframe(options=None)

Return a pyspark.sql.DataFrame ↗ containing the full view of the table.

  • Parameters: options (dict ↗ , option) – Additional Spark read options to pass when reading the table.
  • Returns: The DataFrame for the table.
  • Return type: pyspark.sql.DataFrame ↗

property end_transaction_rid

The ending transaction of the input dataset.

filesystem()

Construct a FileSystem object for reading from FoundryFS.

property identifier

Returns the fully-qualified, catalog-prefixed, Spark V2 identifier of the table, if supported by the underlying table type.

  • Returns: The fully-qualified identifier of the table.
  • Return type: str ↗

Throws: : ValueError: If the underlying table type does not expose a Spark V2 identifier.

pandas()

pandas.DataFrame ↗: A pandas dataframe containing the full view of the dataset.

property path

The Compass path of the dataset.

property rid

The resource identifier of the dataset.

property start_transaction_rid

The starting transaction of the input dataset.