pytest.PytestUnraisableExceptionWarning : Error raised with program stop while testing the pipeline controller class using pytest.

[Kapil-23]

Created two separate python file and while testing the file using pytest it is raising exception with error : PytestUnraisableExceptionWarning

Details :
test1.py : Created a pipeline controller to execute a task on ClearML.
test2.py : Created the test cases to test the pipeline controller.

Run : $ pytest test2.py
Failed with error :
FAILED test2.py::test_main - pytest.PytestUnraisableExceptionWarning: Exception ignored in: <function Po...

To reproduce

file : test1.py

from clearml.automation import PipelineController

def dummy_function(rec_string):
    print(rec_string)

def main():
    pipe = PipelineController(
        name="dummy pipeline",
        project="dummy_project_name",
        version="0.0.1",
        add_pipeline_tags=False,
    )

    # set execution
    pipe.set_default_execution_queue("default")

    pipe.add_function_step(
        name="dummy 0",
        function=dummy_function,
        function_kwargs={
            "rec_string":"dummy"
        },
        function_return=[],
        project_name="dummy_project_name",
        task_name="task dummy 0",
    )

    pipe.start_locally(run_pipeline_steps_locally=True)

file : test2.py

from test1 import main
def test_main():
    """Test cases to validate the main function of test1.py"""
    main()    

Run command : $ pytest test2.py

Outputs :

def unraisable_exception_runtest_hook() -> Generator[None, None, None]:
        with catch_unraisable_exception() as cm:
            yield
            if cm.unraisable:
                if cm.unraisable.err_msg is not None:
                    err_msg = cm.unraisable.err_msg
                else:
                    err_msg = "Exception ignored in"
                msg = f"{err_msg}: {cm.unraisable.object!r}\n\n"
                msg += "".join(
                    traceback.format_exception(
                        cm.unraisable.exc_type,
                        cm.unraisable.exc_value,
                        cm.unraisable.exc_traceback,
                    )
                )
>               warnings.warn(pytest.PytestUnraisableExceptionWarning(msg))
E               pytest.PytestUnraisableExceptionWarning: Exception ignored in: <function Pool.__del__ at 0x7f3371481c60>
E
E               Traceback (most recent call last):
E                 File "/usr/lib/python3.11/multiprocessing/pool.py", line 268, in __del__
E                   _warn(f"unclosed running multiprocessing pool {self!r}",
E               ResourceWarning: unclosed running multiprocessing pool <multiprocessing.pool.ThreadPool state=RUN pool_size=16>

Environment

• ClearML installed Version : 1.13.2
• Pytest installed Version : 7.4.2

[Jake Henning]

Hi @Kapil-23,

Just to verify, this is not a regression from a previous version, correct?

[Kapil-23]

Hi @jkhenning

No, As mentioned we are using ClearML 1.13.2.

[Kapil-23]

Hi @jkhenning
Following up after the December holidays, any update here ? :)