Working with assets

Working with assets

Assets are the buildings blocks of data pipelines. By adhering to a stateful and declarative asset-based paradigm, Y42 ensures that your data pipelines are not only scalable but also easy to maintain, with built-in governance and observability guardrails.

Y42 assets

Y42 assets

Types of assets

Y42 offers a versatile range of assets, each designed to fulfill specific roles within your data pipeline architecture:

Asset typeCapability
Reference sourceRead or reference existing tables from your data warehouse.
CData connectorIngest data from hundreds of pre-built connectors, powered by CData.
Airbyte connectorSimilar to CData connector, powered by Airbyte (open source).
Python sourceIngest data using Python cloud functions.
Fivetran sourceTrigger syncs in Fivetran and reference the generated tables.
CSV seedStore CSV files and sync them with the data warehouse.
dbt modelTransform data with modular SQL models.
Python model (Closed beta)Transform data and run machine learning workloads on data platforms using Python.
SnapshotCapture snapshots of data for managing slowly changing dimensions (SCD Type 2).
ExposureDocument downstream data consumers or tools.
Python actionSync data pipelines with external systems or workflows using Python cloud functions.

Natively compatible assets

Despite their varied capabilities, all assets share a standard configuration schema, including metadata fields. This design principle guarantees natively compatible assets that can be easily managed and integrated. For instance, from dbt models, Airbyte connectors to Python cloud functions, all assets can be selected and executed using a common build command, streamlining the pipeline construction process.

Learn more about asset properties here.

Managing assets

Y42 lets you create, modify and delete assets through both a graphical interface and a code editor, powered by the open-source distribution of VS Code. At their core, assets are defined by SQL or YAML configuration files, which are stored and version-controlled within a Git repository. This dual-interface approach ensures that assets can be created and integrated into existing pipelines efficiently, catering to both GUI-oriented and code-savvy users.

Learn more about different development modes here.