PlotChanges error
swethas112 opened this issue · comments
Hi,
I was trying to plot the differentially binding of TFs using PlotChanges using the below command
TOBIAS PlotChanges --bindetect bindetect_results.txt --TFS TFs.txt --output plotChanges.pdf
TFs.txt contains only GATA.
I was getting following error:
File "/home/ssubramanian/.conda/envs/tobias/lib/python3.10/site-packages/pandas/core/groupby/groupby.py", line 1942, in _agg_py_fallback
res_values = self._grouper.agg_series(ser, alt, preserve_dtype=True)
File "/home/ssubramanian/.conda/envs/tobias/lib/python3.10/site-packages/pandas/core/groupby/ops.py", line 863, in agg_series
result = self._aggregate_series_pure_python(obj, func)
File "/home/ssubramanian/.conda/envs/tobias/lib/python3.10/site-packages/pandas/core/groupby/ops.py", line 884, in _aggregate_series_pure_python
res = func(group)
File "/home/ssubramanian/.conda/envs/tobias/lib/python3.10/site-packages/pandas/core/groupby/groupby.py", line 2454, in <lambda>
alt=lambda x: Series(x, copy=False).mean(numeric_only=numeric_only),
File "/home/ssubramanian/.conda/envs/tobias/lib/python3.10/site-packages/pandas/core/series.py", line 6529, in mean
return NDFrame.mean(self, axis, skipna, numeric_only, **kwargs)
File "/home/ssubramanian/.conda/envs/tobias/lib/python3.10/site-packages/pandas/core/generic.py", line 12413, in mean
return self._stat_function(
File "/home/ssubramanian/.conda/envs/tobias/lib/python3.10/site-packages/pandas/core/generic.py", line 12370, in _stat_function
return self._reduce(
File "/home/ssubramanian/.conda/envs/tobias/lib/python3.10/site-packages/pandas/core/series.py", line 6437, in _reduce
return op(delegate, skipna=skipna, **kwds)
File "/home/ssubramanian/.conda/envs/tobias/lib/python3.10/site-packages/pandas/core/nanops.py", line 147, in f
result = alt(values, axis=axis, skipna=skipna, **kwds)
File "/home/ssubramanian/.conda/envs/tobias/lib/python3.10/site-packages/pandas/core/nanops.py", line 404, in new_func
result = func(values, axis=axis, skipna=skipna, mask=mask, **kwargs)
File "/home/ssubramanian/.conda/envs/tobias/lib/python3.10/site-packages/pandas/core/nanops.py", line 720, in nanmean
the_sum = _ensure_numeric(the_sum)
File "/home/ssubramanian/.conda/envs/tobias/lib/python3.10/site-packages/pandas/core/nanops.py", line 1701, in _ensure_numeric
raise TypeError(f"Could not convert string '{x}' to numeric")
TypeError: Could not convert string 'GATA1TAL1_MA0140.3' to numeric
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/ssubramanian/.conda/envs/tobias/bin/TOBIAS", line 11, in <module>
sys.exit(main())
File "/home/ssubramanian/.conda/envs/tobias/lib/python3.10/site-packages/tobias/TOBIAS.py", line 162, in main
args.func(args)
File "/home/ssubramanian/.conda/envs/tobias/lib/python3.10/site-packages/tobias/tools/plot_changes.py", line 94, in run_plotchanges
table = bindetect.loc[chosen_TFS,].groupby("cluster").mean() #mean of each column
File "/home/ssubramanian/.conda/envs/tobias/lib/python3.10/site-packages/pandas/core/groupby/groupby.py", line 2452, in mean
result = self._cython_agg_general(
File "/home/ssubramanian/.conda/envs/tobias/lib/python3.10/site-packages/pandas/core/groupby/groupby.py", line 1998, in _cython_agg_general
new_mgr = data.grouped_reduce(array_func)
File "/home/ssubramanian/.conda/envs/tobias/lib/python3.10/site-packages/pandas/core/internals/managers.py", line 1470, in grouped_reduce
applied = sb.apply(func)
File "/home/ssubramanian/.conda/envs/tobias/lib/python3.10/site-packages/pandas/core/internals/blocks.py", line 393, in apply
result = func(self.values, **kwargs)
File "/home/ssubramanian/.conda/envs/tobias/lib/python3.10/site-packages/pandas/core/groupby/groupby.py", line 1995, in array_func
result = self._agg_py_fallback(how, values, ndim=data.ndim, alt=alt)
File "/home/ssubramanian/.conda/envs/tobias/lib/python3.10/site-packages/pandas/core/groupby/groupby.py", line 1946, in _agg_py_fallback
raise type(err)(msg) from err
TypeError: agg function failed [how->mean,dtype->object]```
Could you please help in identifying the issue? Is there something I've missed? Thanks in adavnce!
Hey @swethas112,
thank you for your issue. You did everything right, the error was caused by a pandas update that changed a default value in a function that is used during PlotChanges. A fix for this was already committed to the dev branch (1bf54db), but it is not yet included in the stable release. To fix your problem immediately, you can install the dev branch via pip install git+https://github.com/loosolab/TOBIAS@dev. Otherwise, the fix will be included in the next release.
I hope this resolves your problem. If you have further questions, feel free to ask!
Best regards,
Moritz
Thanks for sharing the code.
While installing the dev branch using pip install git+https://github.com/loosolab/TOBIAS@dev, I am getting the below error
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [57 lines of output]
<string>:11: _DeprecatedInstaller: setuptools.installer and fetch_build_eggs are deprecated.
!!
********************************************************************************
Requirements should be satisfied by a PEP 517 installer.
If you are using pip, you can try `pip install --use-pep517`.
********************************************************************************
!!
WARNING: The wheel package is not available.
/home/ssubramanian/.conda/envs/tobiasDev/bin/python3.10: No module named pip
Traceback (most recent call last):
File "<string>", line 8, in <module>
ModuleNotFoundError: No module named 'numpy'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/tmp/pip-build-env-dh9_vj7y/overlay/lib/python3.10/site-packages/setuptools/installer.py", line 101, in _fetch_build_egg_no_warn
subprocess.check_call(cmd)
File "/home/ssubramanian/.conda/envs/tobiasDev/lib/python3.10/subprocess.py", line 369, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/home/ssubramanian/.conda/envs/tobiasDev/bin/python3.10', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmp58ewoovl', '--quiet', 'numpy']' returned non-zero exit status 1.
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/ssubramanian/.conda/envs/tobiasDev/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
main()
File "/home/ssubramanian/.conda/envs/tobiasDev/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/home/ssubramanian/.conda/envs/tobiasDev/lib/python3.10/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
return hook(config_settings)
File "/tmp/pip-build-env-dh9_vj7y/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 325, in get_requires_for_build_wheel
return self._get_build_requires(config_settings, requirements=['wheel'])
File "/tmp/pip-build-env-dh9_vj7y/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 295, in _get_build_requires
self.run_setup()
File "/tmp/pip-build-env-dh9_vj7y/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 487, in run_setup
super().run_setup(setup_script=setup_script)
File "/tmp/pip-build-env-dh9_vj7y/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 311, in run_setup
exec(code, locals())
File "<string>", line 11, in <module>
File "/tmp/pip-build-env-dh9_vj7y/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 636, in fetch_build_eggs
return _fetch_build_eggs(self, requires)
File "/tmp/pip-build-env-dh9_vj7y/overlay/lib/python3.10/site-packages/setuptools/installer.py", line 38, in _fetch_build_eggs
resolved_dists = pkg_resources.working_set.resolve(
File "/tmp/pip-build-env-dh9_vj7y/overlay/lib/python3.10/site-packages/pkg_resources/__init__.py", line 813, in resolve
dist = self._resolve_dist(
File "/tmp/pip-build-env-dh9_vj7y/overlay/lib/python3.10/site-packages/pkg_resources/__init__.py", line 849, in _resolve_dist
dist = best[req.key] = env.best_match(
File "/tmp/pip-build-env-dh9_vj7y/overlay/lib/python3.10/site-packages/pkg_resources/__init__.py", line 1119, in best_match
return self.obtain(req, installer)
File "/tmp/pip-build-env-dh9_vj7y/overlay/lib/python3.10/site-packages/pkg_resources/__init__.py", line 1130, in obtain
return installer(requirement) if installer else None
File "/tmp/pip-build-env-dh9_vj7y/overlay/lib/python3.10/site-packages/setuptools/installer.py", line 103, in _fetch_build_egg_no_warn
raise DistutilsError(str(e)) from e
distutils.errors.DistutilsError: Command '['/home/ssubramanian/.conda/envs/tobiasDev/bin/python3.10', '-m', 'pip', '--disable-pip-version-check', 'wheel', '--no-deps', '-w', '/tmp/tmp58ewoovl', '--quiet', 'numpy']' returned non-zero exit status 1.
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.```
Seems like the installation instructions are missing some required packages on the dev branch for some reason. I was able to reproduce the error and solve it on my machine.
I needed to install numpy and cython via pip install numpy cython before using the command pip install git+https://github.com/loosolab/TOBIAS@dev to install the TOBIAS dev branch.
Sorry for the inconvenience, we will fix it with the next update. In the meantime, I hope this solution also works for you. If not, we'll investigate further.
Thanks, it worked!
May I know how I can get the volcano plot of differential binding from the bindetect results, with labels for selected TFs?
Great!
Volcano plots depicting differential binding scores should be auto-generated during the BINDetect analysis. They can be found as part of the bindetect_figures.pdf file. An interactive version of such a plot for each condition-comparison can also be found in the BINDetect output directory. Look for bindetect_*.html files.
I was unable to find it in the pdf. I'm attaching it bindetect_figures.pdf for your reference. Do you think it got truncated in the last page?
For me, the volcano plot shows up on page 4 in the file you attached.
It might take a little while to load because there are a lot of elements in the plot and there is a dendrogram below the plot on the same page that is also very large. Other than that, the tool you are using to open the pdf might struggle because the page is very long. Perhaps a different tool handles it better. I was able to view it using Firefox, Edge, and Chrome.
Does it show up for you if you open the file in a browser?
Otherwise, the interactive html plot should also depict the same thing.
Yes, it was the issue with the pdf viewer.
Thanks a lot!
Perfect, happy to help!
Also thank you for reporting this problem, we will try to adjust the output so it does not cause trouble with some pdf viewers in the future.