Giters

MaybeShewill-CV / attentive-gan-derainnet

Unofficial tensorflow implemention of "Attentive Generative Adversarial Network for Raindrop Removal from A Single Image (CVPR 2018) " model https://maybeshewill-cv.github.io/attentive-gan-derainnet/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Performance issues in /data_provider (by P3)

DLPerf opened this issue · comments

commented

Hello! I've found a performance issue in /data_provider/data_feed_pipline.py: batch() should be called before map(), which could make your program more efficient. Here is the tensorflow document to support it.

Detailed description is listed below:

  • dataset.batch(batch_size, drop_remainder=True)(here) should be called before dataset.map(map_func=tf_io_pipline_tools.decode,num_parallel_calls=CFG.TRAIN.CPU_MULTI_PROCESS_NUMS)(here), dataset.map(map_func=tf_io_pipline_tools.augment_for_train,num_parallel_calls=CFG.TRAIN.CPU_MULTI_PROCESS_NUMS)(here), dataset.map(map_func=tf_io_pipline_tools.augment_for_test,num_parallel_calls=CFG.TRAIN.CPU_MULTI_PROCESS_NUMS)(here) and dataset.map(map_func=tf_io_pipline_tools.normalize,num_parallel_calls=CFG.TRAIN.CPU_MULTI_PROCESS_NUMS)(here).

Besides, you need to check the function called in map()(e.g., tf_io_pipline_tools.normalize called in dataset.map(map_func=tf_io_pipline_tools.normalize,num_parallel_calls=CFG.TRAIN.CPU_MULTI_PROCESS_NUMS)) whether to be affected or not to make the changed code work properly. For example, if tf_io_pipline_tools.normalize needs data with shape (x, y, z) as its input before fix, it would require data with shape (batch_size, x, y, z).

Looking forward to your reply. Btw, I am very glad to create a PR to fix it if you are too busy.

commented

@DLPerf I have tested in my local env but no obvious performance boost was found here. Have you got a good test result yourself? :)