jayhack / prefect

The easiest way to coordinate your dataflow

Home Page:https://prefect.io

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Unexpected behavior with logging when migrated to prefect2

jayhack opened this issue · comments

First check

  • I added a descriptive title to this issue.
  • I used the GitHub search to find a similar issue and didn't find it.
  • I searched the Prefect documentation for this issue.
  • I checked that this issue is related to Prefect and not one of its dependencies.

Bug summary

When migrating from Prefect1 to Prefect2 I encountered a problem. Basically I call methods in Tasks and in those methods, I log my outputs to local .txt files.
When I run the flow with Prefect1 cloud, it works perfectly fine. But when I migrated to Prefect2 cloud, It doesn’t write logs to my local .txt files.
I tried so many things but none of them worked. Is there any possible solutions?

For example: I have some methods in task1.py in those methods I create some logs and I write those logs to a txt.

Reproduction

# task1.py

import logging

logging.basicConfig(handlers=[logging.FileHandler(filename="DailyBase.txt",
                                                  encoding='utf-8', mode='a+')],
                    format="%(asctime)s | %(name)s:%(levelname)s | %(message)s",
                    datefmt="%F %A %T",
                    level=logging.WARNING)
logger = logging.getLogger()

class log_test():
    def __init__(self) -> None:
        pass

    def push_log(self):
        logger.warning('testing log')



# prefectfile.py

from prefect import flow,task
from task1 import log_test

@task()
def task1():
    Log = log_test()
    Log.push_log()

@flow(name='test_flow')
def log_flow():
    tassk = task1(return_state=True)

Error

No response

Versions

Version:             2.8.5
API version:         0.8.4
Python version:      3.8.10
Git commit:          81a67202
Built:               Thu, Mar 9, 2023 4:27 PM
OS/Arch:             linux/x86_64
Profile:             default
Server type:         cloud

Additional context

image (2)