[BUG] Some checks from TrainTestValidationSuite don't show up in wandb
michaelwang1994-olo opened this issue · comments
Describe the bug
I ran a TrainTestValidationSuite on my data and when I do `.to_wandb(project='...'), I am missing the following checks in my wandb run charts:
- NewCategoryTrainTest
- StringMismatchComparison
- DatasetsSizeComparison --> This shows up under "Train Test Validation Suite/None"
See screenshot for all the Train Test Validation Suite checks that do show up in the run.
To Reproduce
Steps to reproduce the behavior:
train_ds = Dataset(
df_train,
label=label,
index_name=index_name,
datetime_name=datetime_name,
cat_features=cat_features
)
test_ds = Dataset(
df_test,
label=label,
index_name=index_name,
datetime_name=datetime_name,
cat_features=cat_features
)
validation_suite = train_test_validation(
timeout=timeout
)
validation_results = validation_suite.run(train_ds, test_ds)
validation_results.to_wandb(project=wandb.run.project)
Expected behavior
I would have expected all the checks from the suite to show up and be correctly named.
Environment (please complete the following information):
- OS: MacOS Ventura 13.4
- Python Version: 3.11.3
- Deepchecks Version: 0.17.3
Additional context
Thanks for reporting @michaelwang1994-olo!