In Dagster, code is executed within an op. Sometimes this code can fail for transient reasons, and the desired behavior is to retry and run the function again.
Dagster provides both declarative RetryPolicys as well as manual RetryRequested exceptions to enable this behavior.
In addition to being able to set the policy directly on the op definition, it can also be set on specific invocations of an op, or a @job to apply to all ops contained within.
In certain more nuanced situations, we may need to evaluate code to determine if we want to retry or not. For this we can use a manual RetryRequested exception.
@opdefmanual():try:
fails_sometimes()except Exception as e:if should_retry(e):raise RetryRequested(max_retries=1, seconds_to_wait=1)from e
else:raise
Using raise from will ensure the original exceptions information is captured by Dagster.