tensorflow / data-validation

Library for exploring and validating machine learning data

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

AttributeError: 'StatsOptions' object has no attribute '_slice_sqls'

EdwardCuiPeacock opened this issue · comments

TFX version 1.6.1, TFDV version 1.6.0

When specifying the stats_options as the following

statistics_gen = StatisticsGen(
        examples=example_gen.outputs["examples"],
        stats_options=tfdv.StatsOptions(
            # Expecting to have no more than this many categorical features
            num_top_values=10000,
        ),
    )

the tfx pipeline StatisticsGen component throws the following error:

2022-02-17 17:41:28.914 ESTTraceback (most recent call last):
Error
2022-02-17 17:41:28.914 EST File "/opt/conda/lib/python3.7/runpy.py", line 193, in _run_module_as_main
Error
2022-02-17 17:41:28.914 EST "__main__", mod_spec)
Error
2022-02-17 17:41:28.914 EST File "/opt/conda/lib/python3.7/runpy.py", line 85, in _run_code
Error
2022-02-17 17:41:28.914 EST exec(code, run_globals)
Error
2022-02-17 17:41:28.914 EST File "/opt/conda/lib/python3.7/site-packages/tfx/orchestration/kubeflow/v2/container/kubeflow_v2_run_executor.py", line 171, in <module>
Error
2022-02-17 17:41:28.914 EST app.run(main, flags_parser=_parse_flags)
Error
2022-02-17 17:41:28.914 EST File "/opt/conda/lib/python3.7/site-packages/absl/app.py", line 312, in run
Error
2022-02-17 17:41:28.914 EST _run_main(main, args)
Error
2022-02-17 17:41:28.914 EST File "/opt/conda/lib/python3.7/site-packages/absl/app.py", line 258, in _run_main
2022-02-17 17:41:28.914 EST sys.exit(main(argv))
Error
2022-02-17 17:41:28.914 EST File "/opt/conda/lib/python3.7/site-packages/tfx/orchestration/kubeflow/v2/container/kubeflow_v2_run_executor.py", line 167, in main
Error
2022-02-17 17:41:28.914 EST _run_executor(args, beam_args)
Error
2022-02-17 17:41:28.914 EST File "/opt/conda/lib/python3.7/site-packages/tfx/orchestration/kubeflow/v2/container/kubeflow_v2_run_executor.py", line 97, in _run_executor
Error
2022-02-17 17:41:28.914 EST executor.Do(inputs, outputs, exec_properties)
Error
2022-02-17 17:41:28.914 EST File "/opt/conda/lib/python3.7/site-packages/tfx/components/statistics_gen/executor.py", line 139, in Do
Error
2022-02-17 17:41:28.914 EST | 'WriteStatsOutput[%s]' % split >>
Error
2022-02-17 17:41:28.914 EST File "/opt/conda/lib/python3.7/site-packages/apache_beam/pvalue.py", line 137, in __or__
Error
2022-02-17 17:41:28.914 EST return self.pipeline.apply(ptransform, self)
Error
2022-02-17 17:41:28.914 EST File "/opt/conda/lib/python3.7/site-packages/apache_beam/pipeline.py", line 652, in apply
Error
2022-02-17 17:41:28.914 EST transform.transform, pvalueish, label or transform.label)
2022-02-17 17:41:28.914 EST File "/opt/conda/lib/python3.7/site-packages/apache_beam/pipeline.py", line 662, in apply
Error
2022-02-17 17:41:28.914 EST return self.apply(transform, pvalueish)
Error
2022-02-17 17:41:28.914 EST File "/opt/conda/lib/python3.7/site-packages/apache_beam/pipeline.py", line 708, in apply
Error
2022-02-17 17:41:28.914 EST pvalueish_result = self.runner.apply(transform, pvalueish, self._options)
Error
2022-02-17 17:41:28.914 EST File "/opt/conda/lib/python3.7/site-packages/apache_beam/runners/runner.py", line 185, in apply
Error
2022-02-17 17:41:28.914 EST return m(transform, input, options)
Error
2022-02-17 17:41:28.914 EST File "/opt/conda/lib/python3.7/site-packages/apache_beam/runners/runner.py", line 215, in apply_PTransform
Error
2022-02-17 17:41:28.915 EST return transform.expand(input)
Error
2022-02-17 17:41:28.915 EST File "/opt/conda/lib/python3.7/site-packages/tensorflow_data_validation/api/stats_api.py", line 98, in expand
Error
2022-02-17 17:41:28.915 EST stats_impl.GenerateStatisticsImpl(self._options))
Error
2022-02-17 17:41:28.915 EST File "/opt/conda/lib/python3.7/site-packages/apache_beam/pvalue.py", line 137, in __or__
Error
2022-02-17 17:41:28.915 EST return self.pipeline.apply(ptransform, self)
Error
2022-02-17 17:41:28.915 EST File "/opt/conda/lib/python3.7/site-packages/apache_beam/pipeline.py", line 652, in apply
Error
2022-02-17 17:41:28.915 EST transform.transform, pvalueish, label or transform.label)
2022-02-17 17:41:28.915 EST File "/opt/conda/lib/python3.7/site-packages/apache_beam/pipeline.py", line 662, in apply
Error
2022-02-17 17:41:28.915 EST return self.apply(transform, pvalueish)
Error
2022-02-17 17:41:28.915 EST File "/opt/conda/lib/python3.7/site-packages/apache_beam/pipeline.py", line 708, in apply
Error
2022-02-17 17:41:28.915 EST pvalueish_result = self.runner.apply(transform, pvalueish, self._options)
Error
2022-02-17 17:41:28.915 EST File "/opt/conda/lib/python3.7/site-packages/apache_beam/runners/runner.py", line 185, in apply
Error
2022-02-17 17:41:28.915 EST return m(transform, input, options)
Error
2022-02-17 17:41:28.915 EST File "/opt/conda/lib/python3.7/site-packages/apache_beam/runners/runner.py", line 215, in apply_PTransform
Error
2022-02-17 17:41:28.915 EST return transform.expand(input)
Error
2022-02-17 17:41:28.915 EST File "/opt/conda/lib/python3.7/site-packages/tensorflow_data_validation/statistics/stats_impl.py", line 77, in expand
Error
2022-02-17 17:41:28.915 EST elif self._options.experimental_slice_sqls:
Error
2022-02-17 17:41:28.915 EST File "/opt/conda/lib/python3.7/site-packages/tensorflow_data_validation/statistics/stats_options.py", line 353, in experimental_slice_sqls
Error
2022-02-17 17:41:28.915 EST return self._slice_sqls
Error
2022-02-17 17:41:28.915 ESTAttributeError: 'StatsOptions' object has no attribute '_slice_sqls'

Problem resolved. Need to update tfdv dependencies.