PrefectHQ / prefect-dbt

Collection of Prefect integrations for working with dbt with your Prefect flows.

Home Page:https://prefecthq.github.io/prefect-dbt/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Saving `DbtCliProfile` with `prefect_sqlalchemy.DatabaseCredentials` raises `TypeError: Type is not JSON serializable: URL`

markns opened this issue · comments

I'm trying to persist a DbtCliProfile block using a prefect_sqlalchemy.DatabaseCredentials, but running into a TypeError: Type is not JSON serializable: URL error.

Any ideas what's going wrong?

DEFAULT_BLOCK = "default"

credentials = DatabaseCredentials(
    driver=AsyncDriver.POSTGRESQL_ASYNCPG,
    username=os.environ.get("POSTGRES_USER", DEFAULT_BLOCK),
    password=os.environ.get("POSTGRES_PASSWORD", DEFAULT_BLOCK),
    database=os.environ.get("POSTGRES_DATABASE", DEFAULT_BLOCK),
    host=os.environ.get("POSTGRES_HOST", DEFAULT_BLOCK),
    port=os.environ.get("POSTGRES_PORT", DEFAULT_BLOCK),
)
credentials.save(DEFAULT_BLOCK)

dbt_cli_profile = DbtCliProfile(
    name="dbt_dwh",
    target=DEFAULT_BLOCK,
    target_configs=PostgresTargetConfigs(
        credentials=DatabaseCredentials.load(DEFAULT_BLOCK), 
        schema="public"
    ),
)
dbt_cli_profile.save(name="dbt_dwh")
  File "/Users/markns/Library/Caches/pypoetry/virtualenvs/alakazam-vR_Oc5bv-py3.10/lib/python3.10/site-packages/prefect/orion/utilities/schemas.py", line 124, in orjson_dumps
    return orjson.dumps(v, default=default).decode()
TypeError: Type is not JSON serializable: URL

Thanks for reporting this! The issue was that sqlalchemy.engine.url.URL is not JSON friendly. The PR linked above will fix this.

Also, I recommend renaming dbt_dwh to dbt-dwh since block names cannot contain underscores.

prefect-sqlalchemy==0.2.1 has been released which should fix this issue!