Components
info
This feature is considered in a preview stage and is under active development. There may be API changes and feature gaps. Please go to the #dg-components channel in our Slack to report issues or give feedback.
Welcome to Dagster Components.
Dagster Components is a new way to structure your Dagster projects. It aims to provide:
- An opinionated project layout that supports ongoing scaffolding from "Hello world" to the most advanced projects.
- A class-based interface (
Component) for dynamically constructing Dagster definitions from arbitrary data (such as third-party integration configuration files). - A toolkit for building YAML DSLs for
Components, allowing instances of components to be defined with little to no Python code. - A Dagster-provided set of component types that provide a simplified user experience for common integrations.
Installation
To use Dagster Components, you'll need to install the dg command line tool. dg allows you to quickly scaffold Dagster projects and populate them with components.
dg API reference
For the full dg API reference, see the dg CLI documentation.
You can also run dg --help on the command line:
dg --help
Usage: dg [OPTIONS] COMMAND [ARGS]...
CLI for managing Dagster projects.
╭─ Options ────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ --clear-cache Clear the cache. │
│ --rebuild-component-registry Recompute and cache the set of available component types for the current │
│ environment. Note that this also happens automatically whenever the cache is │
│ detected to be stale. │
│ --install-completion Automatically detect your shell and install a completion script for the `dg` │
│ command. This will append to your shell startup file. │
│ --version -v Show the version and exit. │
│ --help -h Show this message and exit. │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Global options ─────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ --verbose Enable verbose output for debugging. │
│ --disable-cache Disable the cache.. │
│ --cache-dir TEXT Specify a directory to use for the cache. │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
╭─ Commands ───────────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ check Commands for checking the integrity of your Dagster code. │
│ dev Start a local instance of Dagster. │
│ docs Commands for generating docs from your Dagster code. │
│ init Initialize a new Dagster project, optionally inside a workspace. │
│ launch Launch a Dagster run. │
│ list Commands for listing Dagster entities. │
│ scaffold Commands for scaffolding Dagster code. │
│ utils Assorted utility commands. │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯