Failed to detect AST nodes in Windows ipython for verb calls
GitHunter0 opened this issue · comments
Feature Type
-
Adding new functionality to datar
-
Changing existing functionality in datar
-
Removing existing functionality in datar
Problem Description
The code below
import datar.all as d
import pandas as pd
new_name = "y"
pd.DataFrame(dict(x=[2])) >> d.rename(**{new_name: 'x'})
generates the following warning message:
c:\Users\me\miniconda3\envs\cenv\lib\site-packages\pipda\utils.py:89: PipeableCallCheckWarning:
Failed to detect AST node calling `rename`, assuming a piping call.
Feature Description
Being able to suppress the warning message above.
Additional Context
No response
Could you also indicate the environment where you were running the code? i.e. python script, jupyter, python REPL, etc
For sure @pwwang , I'm using in a .py file in VScode (via its Interactive Window, which uses jupyter kernel).
I can run it with no problem:
here is the version info:
Version: 1.74.3 (system setup)
Commit: 97dec172d3256f8ca4bfb2143f3f76b503ca0534
Date: 2023-01-09T16:59:02.252Z
Electron: 19.1.8
Chromium: 102.0.5005.167
Node.js: 16.14.2
V8: 10.2.154.15-electron.0
OS: Windows_NT x64 10.0.25276
Sandboxed: No
vscode-jupyter: v2022.11.1003412109
The problem was indeed with VScode, I updated it to the last version and that warning message disappeared, thank you @pwwang .
@pwwang , I checked that the issue was solved only for python 3.8 (after I updated datar and restarted VScode).
All conda environments I tested with python 3.9
or 3.10
that message appears, even directly in the command prompt (without VScode and ipython), it's very weird...
Steps to reproduce the issue.
Go to Windows OS powershell and type:
conda create --name py3.9 python=3.9
conda activate py3.9
pip install datar[pandas]
python
Then run:
import datar.all as d
import pandas as pd
new_name = "y"
pd.DataFrame(dict(x=[2])) >> d.rename(**{new_name: 'x'})
It will display this message:
C:\Users\me\miniconda3\envs\py3.9\lib\site-packages\pipda\utils.py:89: PipeableCallCheckWarning: Failed to detect AST node calling `rename`, assuming a piping call.
warnings.warn(
This is expected.
There is some description here: https://pwwang.github.io/pipda/piping/#fallbacks-when-ast-node-detection-fails
Tracing executing node is not available in the raw python REPL.
How about ipython or bpython?
I could not install bpython
to test, but with ipython
the same issue occurs and since VScode relies on ipython
for its Interactive Window, it inherits the same behavior.
Maybe an option to temporarily silent that message would suffice as a workaround (in the context of a function, we could turn off in the beginning and turn on in the end).
You can silence it by:
pd.DataFrame(dict(x=[2])) >> d.rename(**{new_name: 'x'}, __ast_fallback="piping")
It's kinda wired that ipython shows the same issue.
Could you provide datar.get_versions()
and also the version of your python?
Awesome, __ast_fallback="piping"
works.
Could you provide
datar.get_versions()
and also the version of your python?
For sure:
python : 3.9.16 (main, Jan 11 2023, 16:16:36) [MSC v.1916 64 bit (AMD64)]
datar : 0.11.1
simplug : 0.2.2
executing : 1.2.0
pipda : 0.11.0
datar-numpy : 0.1.0
numpy : 1.24.1
datar-pandas: 0.2.0
pandas : 1.5.2
The python version installed on my Windows OS is 3.9.4.
The python version installed on my Windows OS is 3.9.4.
Sorry, I meant ipython.
pipda
v0.11.1 was just released. With it, you can change the default fallback of a verb:
# importing ...
d.rename.ast_fallback = "piping"
# So you don't need to pass "__ast_fallback" to every call
new_name1 = 'y'
new_name2 = 'z'
pd.DataFrame(dict(x=[2])) >> d.rename(**{new_name1: 'x'})
pd.DataFrame(dict(x=[2])) >> d.rename(**{new_name2: 'x'})
pipda
v0.11.1 was just released. With it, you can change the default fallback of a verb:
Very nice to know.
Sorry, I meant ipython.
The ipython version is 8.8.0
Looks like the only difference is the platform. I can reproduce the issue on Windows. The snapshot I took was on Linux (WSL).
I'll dig deeper to see if there is a fix.
Looks like it is due to a python bug on Windows. AST nodes cannot be detected when we are trying to deconstruct dicts as keyword-arguments.
new_name = 'y'
# Warns
pd.DataFrame(dict(x=[2])) >> d.rename(**{new_name: 'x'})
# But this is Okay
pd.DataFrame(dict(x=[2])) >> d.rename(y='x')
Related:
alexmojaki/executing#27
samuelcolvin/python-devtools#93
samuelcolvin/python-devtools#93 (comment)
python/cpython#89271
Great, at least now we know the root cause.
Close it for now since this problem can not be solved in this repo.
A temporarily solution is to pass __ast_fallback
to bypass the AST node detection:
pd.DataFrame(dict(x=[2])) >> d.rename(**{new_name: 'x'}, __ast_fallback="piping")
See more details here: https://pwwang.github.io/pipda/piping/#fallbacks-when-ast-node-detection-fails