The transforms API for lightweight compute has evolved over time to support more streamlined syntax options.
Reference this page for details on the evolution of the Lightweight API.
To get started writing Lightweight transforms, visit the Getting started page or any of the Polars or Pandas examples in the Python transforms documentation; these pages provide guidance on using Lightweight transforms as the default compute option.
The original syntax for lightweight compute is the @lightweight
decorator. This syntax option remains fully supported.
Copied!1 2 3 4 5 6 7 8 9 10 11
from transforms.api import transform, lightweight, Input, Output @lightweight @transform( output=Output("/path/data/output"), input=Input("/path/data/input"), ) def clean(output, input): df = input.pandas() output.write_table(df)
The new recommended syntax for accessing lightweight transforms is @transform.using
. This API removes the need for additional lightweight imports and streamlines the creation of lightweight transforms as the default.
The new API is available from transforms version 3.68.0 and higher. To make this available in Code Repositories, upgrade your repository with the repository upgrade guide. Ensure that the transformsLangPythonPluginVersion
is equal to 1.978.0 or higher.
To learn more about transforms versions, see the transforms versions overview.
Copied!1 2 3 4 5 6 7 8 9 10
from transforms.api import transform, Input, Output @transform.using( output=Output("/path/data/output"), input=Input("/path/data/input"), ) def clean(output, input): df = input.pandas() output.write_table(df)
To summarize, you can create a Lightweight transform using any of the below syntax options:
@transform.using(...)
@transform.lightweight(...)
@lightweight
@transform(...)
If you receive an error message similar to the following, you may be on an older version of the transforms library that does not support the updated syntax.
Usage of transform.using() on outdated repository
A function object does not have an attribute using
To resolve this issue, try the following options:
transforms
library and repository template following the repository upgrade guide.