jayhack / prefect

The easiest way to coordinate your dataflow

Home Page:https://prefect.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Retrieving dict type parameter from flow_run context fails if the flow is run with default value [only in prefect cloud]

jayhack opened this issue · comments

First check

  • I added a descriptive title to this issue.
  • I used the GitHub search to find a similar issue and didn't find it.
  • I refreshed the page and this issue still occurred.
  • I checked if this issue was specific to the browser I was using by testing with a different browser.

Bug summary

Doing get_run_context().flow_run.parameters.get({parameter_name}) works as expected when run locally, but it fails if you run the flow with default parameters from the Prefect Cloud. {parameter_name} has to be of type dict in the flow definition. This also happens when the flow runs on its schedule (because it uses default values for parameters)

Reproduction

  1. Define a flow like this:
from prefect import flow
from prefect.context import get_run_context
from prefect.logging import get_run_logger

@flow()
def test_flow(parameter_1: dict = {"key_1": "value_1"}):
    print(f"this flow is running with {parameter_1}.")
    flow_run_context = get_run_context()
    my_param = flow_run_context.flow_run.parameters.get("parameter_1") # this line fails with AttributeError: 'NoneType' object has no attribute 'get'
    print("my parameter was: ", my_param)
    get_run_logger().info("Logger says my parameter was: ", my_param) # This line also fails with TypeError: not all arguments converted during string formatting
  1. Run the flow locally with test_flow(), it flow_run_context succeeds but logger fails.
  2. Deploy this flow to Prefect Cloud.
  3. Run the flow using Quick Run (or in Custom Run, leave the parameter empty), and flow_run_context line fails.
  4. Run the flow using Custom Run and provide {"key_1": "value_1"} as value to parameter_1 field, the flow run will succeed.

Error

Encountered exception during execution:
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/prefect/engine.py", line 650, in orchestrate_flow_run
    result = await run_sync(flow_call)
  File "/usr/local/lib/python3.9/site-packages/prefect/utilities/asyncutils.py", line 156, in run_sync_in_interruptible_worker_thread
    tg.start_soon(
  File "/usr/local/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 662, in __aexit__
    raise exceptions[0]
  File "/usr/local/lib/python3.9/site-packages/anyio/to_thread.py", line 31, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "/usr/local/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
    return await future
  File "/usr/local/lib/python3.9/site-packages/anyio/_backends/_asyncio.py", line 867, in run
    result = context.run(func, *args)
  File "/usr/local/lib/python3.9/site-packages/prefect/utilities/asyncutils.py", line 135, in capture_worker_thread_and_result
    result = __fn(*args, **kwargs)
  File "{my directory to the flow definition file}", line 9, in test_flow
    my_param = flow_run_context.flow_run.parameters.get("parameter_1").get("key_1")
AttributeError: 'NoneType' object has no attribute 'get'

Browers

  • Chrome
  • Firefox
  • Safari
  • Edge

Prefect version

No response

Additional context

No response