API Reference
Search documentation
karat

+

K

User Documentation ↗
Transformstransforms-pythontransforms.apiIncrementalTransformInput

transforms.api.IncrementalTransformInput

class transforms.api.IncrementalTransformInput(tinput, prev_txrid=None, batch_end_txrid=None)

TransformInput with added functionality for incremental computation.

property batch_incremental_configuration

The configuration for an incremental input that will be read in batches.

  • Type: BatchIncrementalConfiguration

History

  • Added in version 1.7.0.

property branch

The branch of the dataset.

property column_descriptions

The column descriptions of the dataset.

  • Type: Dict[str, str]

property column_typeclasses

The column typeclasses of the dataset.

  • Type: Dict[str, str]

dataframe(mode='added')

Return a pyspark.sql.DataFrame ↗ for the given read mode.

Only current, previous and added modes are supported.

  • Parameters: mode (str ↗ , optional) – The read mode, one of current, previous, added, modified, or removed. Defaults to added.
  • Returns: The DataFrame for the dataset.
  • Return type: DataFrame ↗

property end_transaction_rid

The ending transaction of the input dataset.

filesystem(mode='added')

Construct a FileSystem object for reading from FoundryFS for the given read mode.

Only current, previous and added modes are supported.

  • Parameters: mode (str ↗ , optional) – The read mode, one of current, previous, added, modified, or removed. Defaults to added.
  • Returns: A filesystem object for the given view.
  • Return type: FileSystem

pandas()

pandas.DataFrame ↗: A pandas dataframe containing the full view of the dataset.

property path

The Compass path of the dataset.

property rid

The resource identifier of the dataset.

property start_transaction_rid

The starting transaction of the input dataset.