Metadata and tags provide two different ways for you to attach information to your jobs and the runs launched from those jobs. The main difference between metadata and tags is what "object" the information is attached to. Metadata is attached to JobDefinitions (specified using the @job decorator), and tags are attached to runs that are created by executing a job.
Metadata allows you to attach information to a job. This information can be whatever you want, but possible cases include:
keeping track of the team responsible for maintaining a job
linking to documentation or other resources
displaying the git hash corresponding to the current job definition
Note: If you are running Dagster using separate Dagit and user code installations (more info here), then your Dagit installation must be >=0.14.18 to use metadata on jobs.
When you attach metadata to a job, you do it as a dictionary of key value pairs. The keys must be a string, but the values can be any one of the MetadataValue classes we provide. You can also use primitive python types as values, and dagster will convert them to the appropriate MetadataValue.
@opdefmy_op():return"Hello World!"@job(
metadata={"owner":"data team",# will be converted to MetadataValue.text"docs": MetadataValue.url("https://docs.dagster.io"),})defmy_job_with_metadata():
my_op()
In addition to adding metadata on the @job decorator, you can also add metadata using the GraphDefinition.to_job method.
Tags allow you to attach information to the run created when you execute a job. Tags can contain any information you want, and dagster will also attach some tags to your runs (we'll cover these later).
You can specify tags you want attached to every run by adding them to a job. Tags are specified as a dictionary of key value pairs where the key must be a string and the value must be a string or json that is serializable to a string.