Tempfile of shuffled dataset should be better in /tmp dir of the host.
zhengzx-nlp opened this issue · comments
Currently, the temporary shuffled dataset is saved in the dir of original dataset (in /user_data/weihr/ for default), which leads to frequently writing the remote mounted dir /home/user_data. This will perhaps make the remote filesystem failed to mount, which happened today in 1080ti-1.
Maybe we should use os.getcwd() at this place. I'll submit a pr.