Hive ingestion fails with UI Scheduler
sachinwadhwa opened this issue · comments
Hive ingestion fails with UI Ingestion Scheduler, work if I use python from my desktop.
Command failed with Can't load plugin: "
'sqlalchemy.dialects:databricks.pyhive. Run with --debug to get full trace\n'
** We are using v0.0.5
RUN_INGEST - {'errors': [],
'exec_id': 'f5e3929a-a73e-45c6-87ff-57477b8ae3ea',
'infos': ['2022-08-17 05:14:24.196974 [exec_id=f5e3929a-a73e-45c6-87ff-57477b8ae3ea] INFO: Starting execution for task with name=RUN_INGEST',
"2022-08-17 05:15:34.366608 [exec_id=f5e3929a-a73e-45c6-87ff-57477b8ae3ea] INFO: stdout=venv doesn't exist.. minting..\n"
'Requirement already satisfied: pip in /tmp/datahub/ingest/venv-hive-0.8.43/lib/python3.9/site-packages (21.2.4)\n'
'Collecting pip\n'
' Using cached pip-22.2.2-py3-none-any.whl (2.0 MB)\n'
'Collecting wheel\n'
' Using cached wheel-0.37.1-py2.py3-none-any.whl (35 kB)\n'
'Requirement already satisfied: setuptools in /tmp/datahub/ingest/venv-hive-0.8.43/lib/python3.9/site-packages (58.1.0)\n'
'Collecting setuptools\n'
' Downloading setuptools-65.0.2-py3-none-any.whl (1.2 MB)\n'
'Installing collected packages: wheel, setuptools, pip\n'
' Attempting uninstall: setuptools\n'
' Found existing installation: setuptools 58.1.0\n'
' Uninstalling setuptools-58.1.0:\n'
' Successfully uninstalled setuptools-58.1.0\n'
' Attempting uninstall: pip\n'
' Found existing installation: pip 21.2.4\n'
' Uninstalling pip-21.2.4:\n'
' Successfully uninstalled pip-21.2.4\n'
'Successfully installed pip-22.2.2 setuptools-65.0.2 wheel-0.37.1\n'
'Collecting acryl-datahub[datahub-kafka,datahub-rest,hive]==0.8.43\n'
' Using cached acryl_datahub-0.8.43-py3-none-any.whl (959 kB)\n'
'Collecting mypy-extensions>=0.4.3\n'
' Using cached mypy_extensions-0.4.3-py2.py3-none-any.whl (4.5 kB)\n'
'Collecting click>=6.0.0\n'
' Using cached click-8.1.3-py3-none-any.whl (96 kB)\n'
'Collecting types-Deprecated\n'
' Using cached types_Deprecated-1.2.9-py3-none-any.whl (3.2 kB)\n'
'Collecting pydantic>=1.5.1\n'
' Using cached pydantic-1.9.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (12.4 MB)\n'
'Collecting markupsafe<=2.0.1,>=1.1.1\n'
' Using cached MarkupSafe-2.0.1-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (30 kB)\n'
'Collecting python-dateutil>=2.8.0\n'
' Using cached python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)\n'
'Collecting PyYAML\n'
' Using cached PyYAML-6.0-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (661 kB)\n'
'Collecting click-default-group\n'
' Using cached click_default_group-1.2.2-py3-none-any.whl\n'
'Collecting psutil>=5.8.0\n'
' Using cached psutil-5.9.1-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (281 '
'kB)\n'
'Collecting toml>=0.10.0\n'
' Using cached toml-0.10.2-py2.py3-none-any.whl (16 kB)\n'
'Collecting typing-extensions>=3.10.0.2\n'
' Using cached typing_extensions-4.3.0-py3-none-any.whl (25 kB)\n'
'Collecting packaging\n'
' Using cached packaging-21.3-py3-none-any.whl (40 kB)\n'
'Collecting ratelimiter\n'
' Using cached ratelimiter-1.2.0.post0-py3-none-any.whl (6.6 kB)\n'
'Collecting stackprinter\n'
' Using cached stackprinter-0.2.7-py3-none-any.whl (29 kB)\n'
'Collecting avro-gen3==0.7.4\n'
' Using cached avro_gen3-0.7.4-py3-none-any.whl (26 kB)\n'
'Collecting humanfriendly\n'
' Using cached humanfriendly-10.0-py2.py3-none-any.whl (86 kB)\n'
'Collecting progressbar2\n'
' Using cached progressbar2-4.0.0-py2.py3-none-any.whl (26 kB)\n'
'Collecting Deprecated\n'
' Using cached Deprecated-1.2.13-py2.py3-none-any.whl (9.6 kB)\n'
'Collecting termcolor>=1.0.0\n'
' Using cached termcolor-1.1.0-py3-none-any.whl\n'
'Collecting avro<1.11,>=1.10.2\n'
' Using cached avro-1.10.2-py3-none-any.whl\n'
'Collecting mixpanel>=4.9.0\n'
' Using cached mixpanel-4.10.0-py2.py3-none-any.whl (8.9 kB)\n'
'Collecting expandvars>=0.6.5\n'
' Using cached expandvars-0.9.0-py3-none-any.whl (6.6 kB)\n'
'Collecting types-termcolor>=1.0.0\n'
' Using cached types_termcolor-1.1.5-py3-none-any.whl (2.2 kB)\n'
'Collecting tabulate\n'
' Using cached tabulate-0.8.10-py3-none-any.whl (29 kB)\n'
'Collecting entrypoints\n'
' Using cached entrypoints-0.4-py3-none-any.whl (5.3 kB)\n'
'Collecting aiohttp<4\n'
' Using cached aiohttp-3.8.1-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.2 MB)\n'
'Collecting typing-inspect\n'
' Using cached typing_inspect-0.7.1-py3-none-any.whl (8.4 kB)\n'
'Collecting docker\n'
' Using cached docker-5.0.3-py2.py3-none-any.whl (146 kB)\n'
'Collecting requests\n'
' Using cached requests-2.28.1-py3-none-any.whl (62 kB)\n'
'Collecting Jinja2<3.1.0\n'
' Using cached Jinja2-3.0.3-py3-none-any.whl (133 kB)\n'
'Collecting traitlets<5.2.2\n'
' Using cached traitlets-5.2.1.post0-py3-none-any.whl (106 kB)\n'
'Collecting acryl-pyhive[hive]>=0.6.13\n'
' Downloading acryl_PyHive-0.6.13-py3-none-any.whl (53 kB)\n'
' ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 53.8/53.8 kB 2.2 MB/s eta 0:00:00\n'
'Collecting greenlet\n'
' Using cached greenlet-1.1.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (153 kB)\n'
'Collecting sqlalchemy==1.3.24\n'
' Using cached SQLAlchemy-1.3.24-cp39-cp39-manylinux2010_x86_64.whl (1.3 MB)\n'
'Collecting great-expectations<0.15.3,>=0.14.11\n'
' Using cached great_expectations-0.15.2-py3-none-any.whl (5.0 MB)\n'
'Collecting fastavro>=1.2.0\n'
' Using cached fastavro-1.6.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.6 MB)\n'
'Collecting confluent-kafka>=1.5.0\n'
' Using cached confluent_kafka-1.9.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)\n'
'Collecting six\n'
' Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)\n'
'Collecting tzlocal\n'
' Using cached tzlocal-4.2-py3-none-any.whl (19 kB)\n'
'Collecting pytz\n'
' Using cached pytz-2022.2.1-py2.py3-none-any.whl (500 kB)\n'
'Collecting future\n'
' Downloading future-0.18.2.tar.gz (829 kB)\n'
' ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 829.2/829.2 kB 17.5 MB/s eta 0:00:00\n'
' Preparing metadata (setup.py): started\n'
" Preparing metadata (setup.py): finished with status 'done'\n"
'Collecting thrift>=0.10.0\n'
' Downloading thrift-0.16.0.tar.gz (59 kB)\n'
' ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 59.6/59.6 kB 3.5 MB/s eta 0:00:00\n'
' Preparing metadata (setup.py): started\n'
" Preparing metadata (setup.py): finished with status 'done'\n"
'Collecting sasl3>=0.2.11\n'
' Downloading sasl3-0.2.11.tar.gz (44 kB)\n'
' ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 44.7/44.7 kB 6.3 MB/s eta 0:00:00\n'
' Preparing metadata (setup.py): started\n'
" Preparing metadata (setup.py): finished with status 'done'\n"
'Collecting thrift-sasl>=0.1.0\n'
' Downloading thrift_sasl-0.4.3-py2.py3-none-any.whl (8.3 kB)\n'
'Collecting frozenlist>=1.1.1\n'
' Using cached frozenlist-1.3.1-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (158 '
'kB)\n'
'Collecting attrs>=17.3.0\n'
' Using cached attrs-22.1.0-py2.py3-none-any.whl (58 kB)\n'
'Collecting yarl<2.0,>=1.0\n'
' Using cached yarl-1.8.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (264 kB)\n'
'Collecting async-timeout<5.0,>=4.0.0a3\n'
' Using cached async_timeout-4.0.2-py3-none-any.whl (5.8 kB)\n'
'Collecting multidict<7.0,>=4.5\n'
' Using cached multidict-6.0.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (114 kB)\n'
'Collecting aiosignal>=1.1.2\n'
' Using cached aiosignal-1.2.0-py3-none-any.whl (8.2 kB)\n'
'Collecting charset-normalizer<3.0,>=2.0\n'
' Using cached charset_normalizer-2.1.0-py3-none-any.whl (39 kB)\n'
'Collecting nbformat>=5.0\n'
' Using cached nbformat-5.4.0-py3-none-any.whl (73 kB)\n'
'Collecting importlib-metadata>=1.7.0\n'
' Using cached importlib_metadata-4.12.0-py3-none-any.whl (21 kB)\n'
'Collecting jsonpatch>=1.22\n'
' Using cached jsonpatch-1.32-py2.py3-none-any.whl (12 kB)\n'
'Collecting ruamel.yaml<0.17.18,>=0.16\n'
' Using cached ruamel.yaml-0.17.17-py3-none-any.whl (109 kB)\n'
'Collecting tqdm>=4.59.0\n'
' Using cached tqdm-4.64.0-py2.py3-none-any.whl (78 kB)\n'
'Collecting colorama>=0.4.3\n'
' Using cached colorama-0.4.5-py2.py3-none-any.whl (16 kB)\n'
'Collecting numpy>=1.14.1\n'
' Using cached numpy-1.23.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.1 MB)\n'
'Collecting urllib3<1.27,>=1.25.4\n'
' Using cached urllib3-1.26.11-py2.py3-none-any.whl (139 kB)\n'
'Collecting cryptography>=3.2\n'
' Using cached cryptography-37.0.4-cp36-abi3-manylinux_2_24_x86_64.whl (4.1 MB)\n'
'Collecting mistune>=0.8.4\n'
' Using cached mistune-2.0.4-py2.py3-none-any.whl (24 kB)\n'
'Collecting scipy>=0.19.0\n'
' Using cached scipy-1.9.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (43.9 MB)\n'
'Collecting pandas>=0.23.0\n'
' Using cached pandas-1.4.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.7 MB)\n'
'Collecting notebook>=6.4.10\n'
' Using cached notebook-6.4.12-py3-none-any.whl (9.9 MB)\n'
'Collecting pyparsing<3,>=2.4\n'
' Using cached pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)\n'
'Collecting altair<5,>=4.0.0\n'
' Using cached altair-4.2.0-py3-none-any.whl (812 kB)\n'
'Collecting Ipython>=7.16.3\n'
' Using cached ipython-8.4.0-py3-none-any.whl (750 kB)\n'
'Collecting jsonschema>=2.5.1\n'
' Downloading jsonschema-4.10.0-py3-none-any.whl (80 kB)\n'
' ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 80.8/80.8 kB 2.6 MB/s eta 0:00:00\n'
'Collecting certifi>=2017.4.17\n'
' Using cached certifi-2022.6.15-py3-none-any.whl (160 kB)\n'
'Collecting idna<4,>=2.5\n'
' Using cached idna-3.3-py3-none-any.whl (61 kB)\n'
'Collecting wrapt<2,>=1.10\n'
' Using cached wrapt-1.14.1-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (77 kB)\n'
'Collecting websocket-client>=0.32.0\n'
' Using cached websocket_client-1.3.3-py3-none-any.whl (54 kB)\n'
'Collecting python-utils>=3.0.0\n'
' Using cached python_utils-3.3.3-py2.py3-none-any.whl (23 kB)\n'
'Collecting toolz\n'
' Using cached toolz-0.12.0-py3-none-any.whl (55 kB)\n'
'Collecting cffi>=1.12\n'
' Using cached cffi-1.15.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (441 kB)\n'
'Collecting zipp>=0.5\n'
' Using cached zipp-3.8.1-py3-none-any.whl (5.6 kB)\n'
'Collecting pygments>=2.4.0\n'
' Using cached Pygments-2.13.0-py3-none-any.whl (1.1 MB)\n'
'Collecting prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0\n'
' Using cached prompt_toolkit-3.0.30-py3-none-any.whl (381 kB)\n'
'Collecting jedi>=0.16\n'
' Using cached jedi-0.18.1-py2.py3-none-any.whl (1.6 MB)\n'
'Requirement already satisfied: setuptools>=18.5 in /tmp/datahub/ingest/venv-hive-0.8.43/lib/python3.9/site-packages (from '
'Ipython>=7.16.3->great-expectations<0.15.3,>=0.14.11->acryl-datahub[datahub-kafka,datahub-rest,hive]==0.8.43->-r '
'/tmp/datahub/ingest/requirements-f5e3929a-a73e-45c6-87ff-57477b8ae3ea.txt (line 1)) (65.0.2)\n'
'Collecting stack-data\n'
' Using cached stack_data-0.4.0-py3-none-any.whl (23 kB)\n'
'Collecting pickleshare\n'
' Using cached pickleshare-0.7.5-py2.py3-none-any.whl (6.9 kB)\n'
'Collecting pexpect>4.3\n'
' Using cached pexpect-4.8.0-py2.py3-none-any.whl (59 kB)\n'
'Collecting backcall\n'
' Using cached backcall-0.2.0-py2.py3-none-any.whl (11 kB)\n'
'Collecting matplotlib-inline\n'
' Downloading matplotlib_inline-0.1.5-py3-none-any.whl (9.4 kB)\n'
'Collecting decorator\n'
' Using cached decorator-5.1.1-py3-none-any.whl (9.1 kB)\n'
'Collecting jsonpointer>=1.9\n'
' Using cached jsonpointer-2.3-py2.py3-none-any.whl (7.8 kB)\n'
'Collecting pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0\n'
' Using cached pyrsistent-0.18.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (115 kB)\n'
'Collecting jupyter-core\n'
' Using cached jupyter_core-4.11.1-py3-none-any.whl (88 kB)\n'
'Collecting fastjsonschema\n'
' Using cached fastjsonschema-2.16.1-py3-none-any.whl (22 kB)\n'
'Collecting ipykernel\n'
' Using cached ipykernel-6.15.1-py3-none-any.whl (132 kB)\n'
'Collecting nbconvert>=5\n'
' Using cached nbconvert-6.5.3-py3-none-any.whl (563 kB)\n'
'Collecting nest-asyncio>=1.5\n'
' Using cached nest_asyncio-1.5.5-py3-none-any.whl (5.2 kB)\n'
'Collecting prometheus-client\n'
' Using cached prometheus_client-0.14.1-py3-none-any.whl (59 kB)\n'
'Collecting Send2Trash>=1.8.0\n'
' Using cached Send2Trash-1.8.0-py3-none-any.whl (18 kB)\n'
'Collecting tornado>=6.1\n'
' Using cached tornado-6.2-cp37-abi3-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (423 kB)\n'
'Collecting terminado>=0.8.3\n'
' Using cached terminado-0.15.0-py3-none-any.whl (16 kB)\n'
'Collecting ipython-genutils\n'
' Using cached ipython_genutils-0.2.0-py2.py3-none-any.whl (26 kB)\n'
'Collecting argon2-cffi\n'
' Using cached argon2_cffi-21.3.0-py3-none-any.whl (14 kB)\n'
'Collecting jupyter-client>=5.3.4\n'
' Using cached jupyter_client-7.3.4-py3-none-any.whl (132 kB)\n'
'Collecting pyzmq>=17\n'
' Using cached pyzmq-23.2.1-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.1 MB)\n'
'Collecting ruamel.yaml.clib>=0.1.2\n'
' Using cached ruamel.yaml.clib-0.2.6-cp39-cp39-manylinux1_x86_64.whl (539 kB)\n'
'Collecting pure-sasl>=0.6.2\n'
' Downloading pure-sasl-0.6.2.tar.gz (11 kB)\n'
' Preparing metadata (setup.py): started\n'
" Preparing metadata (setup.py): finished with status 'done'\n"
'Collecting pytz-deprecation-shim\n'
' Using cached pytz_deprecation_shim-0.1.0.post0-py2.py3-none-any.whl (15 kB)\n'
'Collecting pycparser\n'
' Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)\n'
'Collecting parso<0.9.0,>=0.8.0\n'
' Using cached parso-0.8.3-py2.py3-none-any.whl (100 kB)\n'
'Collecting lxml\n'
' Using cached lxml-4.9.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (7.0 MB)\n'
'Collecting mistune>=0.8.4\n'
' Using cached mistune-0.8.4-py2.py3-none-any.whl (16 kB)\n'
'Collecting beautifulsoup4\n'
' Using cached beautifulsoup4-4.11.1-py3-none-any.whl (128 kB)\n'
'Collecting pandocfilters>=1.4.1\n'
' Using cached pandocfilters-1.5.0-py2.py3-none-any.whl (8.7 kB)\n'
'Collecting tinycss2\n'
' Using cached tinycss2-1.1.1-py3-none-any.whl (21 kB)\n'
'Collecting bleach\n'
' Using cached bleach-5.0.1-py3-none-any.whl (160 kB)\n'
'Collecting defusedxml\n'
' Using cached defusedxml-0.7.1-py2.py3-none-any.whl (25 kB)\n'
'Collecting jupyterlab-pygments\n'
' Using cached jupyterlab_pygments-0.2.2-py2.py3-none-any.whl (21 kB)\n'
'Collecting nbclient>=0.5.0\n'
' Using cached nbclient-0.6.6-py3-none-any.whl (71 kB)\n'
'Collecting ptyprocess>=0.5\n'
' Using cached ptyprocess-0.7.0-py2.py3-none-any.whl (13 kB)\n'
'Collecting wcwidth\n'
' Using cached wcwidth-0.2.5-py2.py3-none-any.whl (30 kB)\n'
'Collecting argon2-cffi-bindings\n'
' Using cached argon2_cffi_bindings-21.2.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (86 kB)\n'
'Collecting debugpy>=1.0\n'
' Downloading debugpy-1.6.3-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.8 MB)\n'
' ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.8/1.8 MB 49.6 MB/s eta 0:00:00\n'
'Collecting tzdata\n'
' Using cached tzdata-2022.2-py2.py3-none-any.whl (336 kB)\n'
'Collecting asttokens\n'
' Using cached asttokens-2.0.8-py2.py3-none-any.whl (23 kB)\n'
'Collecting executing\n'
' Using cached executing-0.10.0-py2.py3-none-any.whl (17 kB)\n'
'Collecting pure-eval\n'
' Using cached pure_eval-0.2.2-py3-none-any.whl (11 kB)\n'
'Collecting nbclient>=0.5.0\n'
' Using cached nbclient-0.6.5-py3-none-any.whl (71 kB)\n'
' Using cached nbclient-0.6.4-py3-none-any.whl (71 kB)\n'
' Using cached nbclient-0.6.3-py3-none-any.whl (71 kB)\n'
'Collecting soupsieve>1.2\n'
' Using cached soupsieve-2.3.2.post1-py3-none-any.whl (37 kB)\n'
'Collecting webencodings\n'
' Using cached webencodings-0.5.1-py2.py3-none-any.whl (11 kB)\n'
'Building wheels for collected packages: sasl3, thrift, future, pure-sasl\n'
' Building wheel for sasl3 (setup.py): started\n'
" Building wheel for sasl3 (setup.py): finished with status 'done'\n"
' Created wheel for sasl3: filename=sasl3-0.2.11-cp39-cp39-linux_x86_64.whl size=302959 '
'sha256=ed12b04f601a164ceb5d8fb3f60662c58b7b99e8f27ca9fd2dc3b022ea92a285\n'
' Stored in directory: /home/datahub/.cache/pip/wheels/07/68/44/ed8ff2ab434972ea0b9508b5ef33edc428c68a0a0e62868a0c\n'
' Building wheel for thrift (setup.py): started\n'
" Building wheel for thrift (setup.py): finished with status 'done'\n"
' Created wheel for thrift: filename=thrift-0.16.0-cp39-cp39-linux_x86_64.whl size=462793 '
'sha256=33c8b1adfa9e737d3761ee7cb6f98825a0223032a44b8352e2ad99bb8f601b3b\n'
' Stored in directory: /home/datahub/.cache/pip/wheels/67/b1/a4/cf8b927fdbee8e55234f1f421b531685b960e5f23b6273cfcf\n'
' Building wheel for future (setup.py): started\n'
" Building wheel for future (setup.py): finished with status 'done'\n"
' Created wheel for future: filename=future-0.18.2-py3-none-any.whl size=491058 '
'sha256=bae4cc8ee67bb6594025c39546c55bc57a11c293c9210673999776d7aabb77a5\n'
' Stored in directory: /home/datahub/.cache/pip/wheels/2f/a0/d3/4030d9f80e6b3be787f19fc911b8e7aa462986a40ab1e4bb94\n'
' Building wheel for pure-sasl (setup.py): started\n'
" Building wheel for pure-sasl (setup.py): finished with status 'done'\n"
' Created wheel for pure-sasl: filename=pure_sasl-0.6.2-py3-none-any.whl size=11427 '
'sha256=a2bd378dd25f6b618a17b3e6248cebcf87f594c784f7bd31e12a6fcee8627f76\n'
' Stored in directory: /home/datahub/.cache/pip/wheels/46/87/60/7fb7069c8135d081326017206a8481795191246b680e2ff67a\n'
'Successfully built sasl3 thrift future pure-sasl\n'
'Installing collected packages: webencodings, wcwidth, types-termcolor, types-Deprecated, termcolor, Send2Trash, ratelimiter, pytz, '
'pure-sasl, pure-eval, ptyprocess, pickleshare, mypy-extensions, mistune, ipython-genutils, fastjsonschema, executing, confluent-kafka, '
'backcall, zipp, wrapt, websocket-client, urllib3, tzdata, typing-extensions, traitlets, tqdm, tornado, toolz, toml, tinycss2, tabulate, '
'stackprinter, sqlalchemy, soupsieve, six, ruamel.yaml.clib, pyzmq, PyYAML, python-utils, pyrsistent, pyparsing, pygments, pycparser, '
'psutil, prompt-toolkit, prometheus-client, pexpect, parso, pandocfilters, numpy, nest-asyncio, multidict, markupsafe, lxml, '
'jupyterlab-pygments, jsonpointer, idna, humanfriendly, greenlet, future, frozenlist, fastavro, expandvars, entrypoints, defusedxml, '
'decorator, debugpy, colorama, click, charset-normalizer, certifi, avro, attrs, async-timeout, yarl, typing-inspect, thrift, terminado, '
'scipy, sasl3, ruamel.yaml, requests, pytz-deprecation-shim, python-dateutil, pydantic, progressbar2, packaging, matplotlib-inline, '
'jupyter-core, jsonschema, jsonpatch, Jinja2, jedi, importlib-metadata, Deprecated, click-default-group, cffi, bleach, beautifulsoup4, '
'asttokens, aiosignal, tzlocal, thrift-sasl, stack-data, pandas, nbformat, mixpanel, jupyter-client, docker, cryptography, '
'argon2-cffi-bindings, aiohttp, acryl-pyhive, nbclient, Ipython, avro-gen3, argon2-cffi, altair, nbconvert, ipykernel, acryl-datahub, '
'notebook, great-expectations\n'
'Successfully installed Deprecated-1.2.13 Ipython-8.4.0 Jinja2-3.0.3 PyYAML-6.0 Send2Trash-1.8.0 acryl-datahub-0.8.43 acryl-pyhive-0.6.13 '
'aiohttp-3.8.1 aiosignal-1.2.0 altair-4.2.0 argon2-cffi-21.3.0 argon2-cffi-bindings-21.2.0 asttokens-2.0.8 async-timeout-4.0.2 '
'attrs-22.1.0 avro-1.10.2 avro-gen3-0.7.4 backcall-0.2.0 beautifulsoup4-4.11.1 bleach-5.0.1 certifi-2022.6.15 cffi-1.15.1 '
'charset-normalizer-2.1.0 click-8.1.3 click-default-group-1.2.2 colorama-0.4.5 confluent-kafka-1.9.2 cryptography-37.0.4 debugpy-1.6.3 '
'decorator-5.1.1 defusedxml-0.7.1 docker-5.0.3 entrypoints-0.4 executing-0.10.0 expandvars-0.9.0 fastavro-1.6.0 fastjsonschema-2.16.1 '
'frozenlist-1.3.1 future-0.18.2 great-expectations-0.15.2 greenlet-1.1.2 humanfriendly-10.0 idna-3.3 importlib-metadata-4.12.0 '
'ipykernel-6.15.1 ipython-genutils-0.2.0 jedi-0.18.1 jsonpatch-1.32 jsonpointer-2.3 jsonschema-4.10.0 jupyter-client-7.3.4 '
'jupyter-core-4.11.1 jupyterlab-pygments-0.2.2 lxml-4.9.1 markupsafe-2.0.1 matplotlib-inline-0.1.5 mistune-0.8.4 mixpanel-4.10.0 '
'multidict-6.0.2 mypy-extensions-0.4.3 nbclient-0.6.3 nbconvert-6.5.3 nbformat-5.4.0 nest-asyncio-1.5.5 notebook-6.4.12 numpy-1.23.2 '
'packaging-21.3 pandas-1.4.3 pandocfilters-1.5.0 parso-0.8.3 pexpect-4.8.0 pickleshare-0.7.5 progressbar2-4.0.0 prometheus-client-0.14.1 '
'prompt-toolkit-3.0.30 psutil-5.9.1 ptyprocess-0.7.0 pure-eval-0.2.2 pure-sasl-0.6.2 pycparser-2.21 pydantic-1.9.2 pygments-2.13.0 '
'pyparsing-2.4.7 pyrsistent-0.18.1 python-dateutil-2.8.2 python-utils-3.3.3 pytz-2022.2.1 pytz-deprecation-shim-0.1.0.post0 pyzmq-23.2.1 '
'ratelimiter-1.2.0.post0 requests-2.28.1 ruamel.yaml-0.17.17 ruamel.yaml.clib-0.2.6 sasl3-0.2.11 scipy-1.9.0 six-1.16.0 '
'soupsieve-2.3.2.post1 sqlalchemy-1.3.24 stack-data-0.4.0 stackprinter-0.2.7 tabulate-0.8.10 termcolor-1.1.0 terminado-0.15.0 '
'thrift-0.16.0 thrift-sasl-0.4.3 tinycss2-1.1.1 toml-0.10.2 toolz-0.12.0 tornado-6.2 tqdm-4.64.0 traitlets-5.2.1.post0 '
'types-Deprecated-1.2.9 types-termcolor-1.1.5 typing-extensions-4.3.0 typing-inspect-0.7.1 tzdata-2022.2 tzlocal-4.2 urllib3-1.26.11 '
'wcwidth-0.2.5 webencodings-0.5.1 websocket-client-1.3.3 wrapt-1.14.1 yarl-1.8.1 zipp-3.8.1\n'
'Elapsed seconds = 65\n'
' --report-to TEXT Provide an output file to produce a\n'
'This version of datahub supports report-to functionality\n'
'datahub ingest run -c /tmp/datahub/ingest/f5e3929a-a73e-45c6-87ff-57477b8ae3ea/recipe.yml --report-to '
'/tmp/datahub/ingest/f5e3929a-a73e-45c6-87ff-57477b8ae3ea/ingestion_report.json\n'
'[2022-08-17 05:15:31,407] INFO {datahub.cli.ingest_cli:170} - DataHub CLI version: 0.8.43\n'
'[2022-08-17 05:15:31,432] INFO {datahub.ingestion.run.pipeline:163} - Sink configured successfully. DataHubRestEmitter: configured '
'to talk to http://rank-datahub-gms:8080\n'
'[2022-08-17 05:15:33,495] INFO {datahub.cli.ingest_cli:119} - Starting metadata ingestion\n'
'[2022-08-17 05:15:33,508] INFO {datahub.cli.ingest_cli:123} - Source (hive) report:\n'
"{'workunits_produced': '0',\n"
" 'workunit_ids': [],\n"
" 'warnings': {},\n"
" 'failures': {},\n"
" 'cli_version': '0.8.43',\n"
" 'cli_entry_location': '/tmp/datahub/ingest/venv-hive-0.8.43/lib/python3.9/site-packages/datahub/__init__.py',\n"
" 'py_version': '3.9.9 (main, Dec 21 2021, 10:03:34) \\n[GCC 10.2.1 20210110]',\n"
" 'py_exec_path': '/tmp/datahub/ingest/venv-hive-0.8.43/bin/python3',\n"
" 'os_details': 'Linux-5.4.0-1085-azure-x86_64-with-glibc2.31',\n"
" 'tables_scanned': '0',\n"
" 'views_scanned': '0',\n"
" 'entities_profiled': '0',\n"
" 'filtered': [],\n"
" 'soft_deleted_stale_entities': []}\n"
'[2022-08-17 05:15:33,508] INFO {datahub.cli.ingest_cli:126} - Sink (datahub-rest) report:\n'
"{'records_written': '0', 'warnings': [], 'failures': [], 'gms_version': 'v0.8.43'}\n"
"[2022-08-17 05:15:33,938] ERROR {datahub.entrypoints:188} - Command failed with Can't load plugin: "
'sqlalchemy.dialects:databricks.pyhive. Run with --debug to get full trace\n'
'[2022-08-17 05:15:33,938] INFO {datahub.entrypoints:191} - DataHub CLI version: 0.8.43 at '
'/tmp/datahub/ingest/venv-hive-0.8.43/lib/python3.9/site-packages/datahub/__init__.py\n',
"2022-08-17 05:15:34.367584 [exec_id=f5e3929a-a73e-45c6-87ff-57477b8ae3ea] INFO: Failed to execute 'datahub ingest'",
'2022-08-17 05:15:34.369828 [exec_id=f5e3929a-a73e-45c6-87ff-57477b8ae3ea] INFO: Caught exception EXECUTING '
'task_id=f5e3929a-a73e-45c6-87ff-57477b8ae3ea, name=RUN_INGEST, stacktrace=Traceback (most recent call last):\n'
' File "/usr/local/lib/python3.9/site-packages/acryl/executor/execution/default_executor.py", line 122, in execute_task\n'
' self.event_loop.run_until_complete(task_future)\n'
' File "/usr/local/lib/python3.9/site-packages/nest_asyncio.py", line 89, in run_until_complete\n'
' return f.result()\n'
' File "/usr/local/lib/python3.9/asyncio/futures.py", line 201, in result\n'
' raise self._exception\n'
' File "/usr/local/lib/python3.9/asyncio/tasks.py", line 256, in __step\n'
' result = coro.send(None)\n'
' File "/usr/local/lib/python3.9/site-packages/acryl/executor/execution/sub_process_ingestion_task.py", line 112, in execute\n'
' raise TaskError("Failed to execute \'datahub ingest\'")\n'
"acryl.executor.execution.task.TaskError: Failed to execute 'datahub ingest'\n"]}
Execution finished with errors.
This issue is stale because it has been open for 30 days with no activity. If you believe this is still an issue on the latest DataHub release please leave a comment with the version that you tested it with. If this is a question/discussion please head to https://slack.datahubproject.io. For feature requests please use https://feature-requests.datahubproject.io
Hey @sachinwadhwa, this should've been fixed now. Feel free to ping or re-open if this happens again :)