Get Weird Structure in Reproducing the Result of GDAS-FRC

[Wu Yuhang]

Hello .
​Using script NASNet-space-search-by-GDAS-FRC.sh, I got a weird structure as below, which is totally different from cell reported from GDAS Paper(figure 4).
The command I use is:

CUDA_VISIBLE_DEVICES=0 bash ./scripts-search/NASNet-space-search-by-GDAS-FRC.sh cifar10 1 -1

Structure in paper:

Weird Structure I got:

{'normal': [
    (('skip_connect', 1, 0.1485183) , ('dua_sepc_5x5', 0, 0.13894983)), 
    (('skip_connect', 1, 0.15187271), ('avg_pool_3x3', 1, 0.14358662)), 
    (('skip_connect', 1, 0.14807169), ('max_pool_3x3', 1, 0.14803562)),
    (('max_pool_3x3', 1, 0.15954553), ('skip_connect', 1, 0.14276329))], 
'normal_concat': [2, 3, 4, 5]}

​To find out whether it is only a outlier, I repeat searching procedure several times using different seeds. All result is similar to this weird structure. In these found structures, only one edge containing conv operator and others edges are all 'skip_connect' or pooling operator.

MY Environment is :

• NVIDIA-SMI 415.27
• CUDA 10.0
• Python 3.7.7
• cudatoolkit 10.0.130
• cudnn 7.6.5
• Pytorch 1.4.0
• torchvision 0.5.0

[Xuanyi Dong]

Thanks for sharing the results. Sorry that you get weird results... I reimplemented GDAS FRC based on my memory with the sacrifice of being compatible with the NATS-Bench/NAS-Bench-201 codebase, so I haven't carefully checked the performance. I will re-run this part of the codes to figure out the reasons.

[Wu Yuhang]

Thanks for sharing the results. Sorry that you get weird results... I reimplemented GDAS FRC based on my memory with the sacrifice of being compatible with the NATS-Bench/NAS-Bench-201 codebase, so I haven't carefully checked the performance. I will re-run this part of the codes to figure out the reasons.

Thanks for your reply and re-run, could you tell me whether you meet the same problem?

[Xuanyi Dong]

{'normal': [(('skip_connect', 1, 0.15519889), ('dil_sepc_5x5', 0, 0.1416643)), (('skip_connect', 1, 0.15950418), ('max_pool_3x3', 1, 0.1456287)), (('skip_connect', 1, 0.15625753), ('max_pool_3x3', 1, 0.1543942)), (('max_pool_3x3', 1, 0.15630342), ('skip_connect', 1, 0.15141682))], 'normal_concat': [2, 3, 4, 5]}

Here are my searched results.. I also get the same problem. I will try to figure out.

[Wu Yuhang]

{'normal': [(('skip_connect', 1, 0.15519889), ('dil_sepc_5x5', 0, 0.1416643)), (('skip_connect', 1, 0.15950418), ('max_pool_3x3', 1, 0.1456287)), (('skip_connect', 1, 0.15625753), ('max_pool_3x3', 1, 0.1543942)), (('max_pool_3x3', 1, 0.15630342), ('skip_connect', 1, 0.15141682))], 'normal_concat': [2, 3, 4, 5]}

Here are my searched results.. I also get the same problem. I will try to figure out.

Hello, I'm sorry to reply late. And have you found a solution?

[Kadir Altuğ]

Hello .​Using script NASNet-space-search-by-GDAS-FRC.sh, I got a weird structure as below, which is totally different from cell reported from GDAS Paper(figure 4). The command I use is:

CUDA_VISIBLE_DEVICES=0 bash ./scripts-search/NASNet-space-search-by-GDAS-FRC.sh cifar10 1 -1

Structure in paper: Weird Structure I got:

{'normal': [
    (('skip_connect', 1, 0.1485183) , ('dua_sepc_5x5', 0, 0.13894983)), 
    (('skip_connect', 1, 0.15187271), ('avg_pool_3x3', 1, 0.14358662)), 
    (('skip_connect', 1, 0.14807169), ('max_pool_3x3', 1, 0.14803562)),
    (('max_pool_3x3', 1, 0.15954553), ('skip_connect', 1, 0.14276329))], 
'normal_concat': [2, 3, 4, 5]}

​To find out whether it is only a outlier, I repeat searching procedure several times using different seeds. All result is similar to this weird structure. In these found structures, only one edge containing conv operator and others edges are all 'skip_connect' or pooling operator.

MY Environment is :

• NVIDIA-SMI 415.27
• CUDA 10.0
• Python 3.7.7
• cudatoolkit 10.0.130
• cudnn 7.6.5
• Pytorch 1.4.0
• torchvision 0.5.0

How can u use this codes in your code?