torch-tb-profiler doesn't support HDFS file system

Question

yundai424 opened this issue a year ago · comments

Hi team, we notice that torch TB profiler plugin doesn't currently support HDFS https://github.com/pytorch/kineto/blob/main/tb_plugin/torch_tb_profiler/io/file.py#L66 . Right now only local FS, S3, GS and azure blob are allowed.

We would like to have this support added (iiuc the only thing needed is to implement it and register) as our profiling result is stored on HDFS.

Aaron Shi · Answer 1 · Fri Jun 23 2023 22:29:56 GMT+0800 (China Standard Time)

Feel free to add the changes needed to support HDFS. We can help review the changes.

Yun Dai · Answer 2 · Tue Aug 01 2023 06:51:14 GMT+0800 (China Standard Time)

@aaronenyeshi sorry for the late response. I've added the changes. Would appreciate it if you could help to review #793 Thanks!

Aaron Shi · Answer 3 · Wed Aug 09 2023 06:12:45 GMT+0800 (China Standard Time)

Thanks for landing the fix, closing this now.