jayhack / prefect

The easiest way to coordinate your dataflow

Home Page:https://prefect.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Ability to access image of flow run

jayhack opened this issue · comments

First check

  • I added a descriptive title to this issue.
  • I used the GitHub search to find a similar request and didn't find it.
  • I searched the Prefect documentation for this feature.

Prefect Version

2.x

Describe the current behavior

In prefect 1 the image of a flow run could be accessed via the context. This was very useful to create dask clusters using the same image.

Prefect 2 should enable access to information like the image by including the deployments infrastructure block in the runtime.

Describe the proposed behavior

prefect.runtime.deployment.infrastructure should contain the infrastructure information.

Example Use

When setting up a flow with a dask task runner like this:
@flow( name="mongodb_export.transfer_last_hour", task_runner=DaskTaskRunner( cluster_class=KubeCluster, adapt_kwargs={"minimum": 2, "maximum": 4}, cluster_kwargs={"pod_template": dask_pod_spec()}, ), )

the function dask_pod_spec would get the image used for the flow from the prefect.runtime

Additional context

No response