allegroai / clearml

ClearML - Auto-Magical CI/CD to streamline your AI workload. Experiment Management, Data Management, Pipeline, Orchestration, Scheduling & Serving in one MLOps/LLMOps solution

Home Page:https://clear.ml/docs

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Parallel dataset verification bug in `master` fails on older versions of Python

charlienewey-odin opened this issue · comments

Describe the bug

A recent commit to dataset verification in dataset.py has introduced a Python version incompatibility.

This is the commit that introduced the bug: f964783

The problem is on this line

The cancel_futures argument was not introduced until Python 3.9 - see Python docs here: https://docs.python.org/3/library/concurrent.futures.html#concurrent.futures.Executor.shutdown. This renders clearml incompatible with many of the versions advertised in the README.

This bug should not have been able to find its way into master. Do ClearML have tox tests or similar?

To reproduce

Validate a dataset with Python <= 3.8.

Hi @charlienewey-odin ! We already have a fix for this which will be synced with Github soon. Note that this shouldn't have made it to an official release

Hey @charlienewey-odin! Just letting you know that this issue has been resolved in the recently released v1.13.2. Let us know if there are any issues :)