TypeError DARTS-V2.py

[FarahSaeed]

Hi

I was running darts for NAS bench. It was running fine initally. It showed error on this Line436

logger.log("{:}".format(api.query_by_arch(genotypes[total_epoch - 1]), "200"))

Is it because of a typo as I also compared with Line 423 that is

logger.log("{:}".format(api.query_by_arch(genotypes[epoch], "200")))

Here is output that was showing.
Traceback (most recent call last): File "./exps/NAS-Bench-201-algos/DARTS-V2.py", line 496, in <module> main(args) File "./exps/NAS-Bench-201-algos/DARTS-V2.py", line 436, in main logger.log("{:}".format(api.query_by_arch(genotypes[total_epoch - 1]), "200")) TypeError: query_by_arch() missing 1 required positional argument: 'hp'

Thanks

[Xuanyi Dong]

I will look at this issue by this week.

[FarahSaeed]

I ran it with
logger.log("{:}".format(api.query_by_arch(genotypes[total_epoch - 1], "200")))
It is not showing error after changing.

The searched architecture is
|skip_connect~0|+|skip_connect~0|none~1|+|skip_connect~0|skip_connect~1|skip_connect~2|

I was wondering as it does not have any conv operation, should I set number of nodes to 7. The searched architecture in darts paper have conv operations and max poolings.

I was also wondering if normal and reduction cells are searched separately.

Thank you so much! Looking forward to your response.

[Xuanyi Dong]

Thanks for pointing out this bug, I just fixed it in the last comment.

For your questions, you can set nodes to 7 for DARTS, while as our benchmark file does not contain the information of such architecture, you need to disable API. The separate search of normal and reduction cells is also supported. See (https://github.com/D-X-Y/AutoDL-Projects/blob/main/docs/ICLR-2019-DARTS.md#run-the-first-order-darts-on-the-nasnetdarts-search-space)

In our updating work of NAS-Bench-201 -- NATS-Bench, we updated some hps such that DARTS's performance is improved: See here (https://github.com/D-X-Y/AutoDL-Projects/tree/main/exps/NATS-algos) for more details.

[FarahSaeed]

Hi Thank you for the answer.

I was wondering do we have to set max_nodes parameter for searching a cell with 7 nodes in both NAS bench and NATS bench? Do I also have to disable NATS bench API when using NATS bench while training NAS?
I wanted to know if you can share the file location where I could see the reduction and normal cell.

Thank you very much.

[FarahSaeed]

Hi
As the searched architectures are only showing skipconnect, I was wondering that I might be mistaken somewhere.

Here is output on running darts v2.

[099-100] - [get_best_arch] : Structure(4 nodes with |skip_connect~0|+|skip_connect~0|none~1|+|skip_connect~0|skip_connect~1|skip_connect~2|) -> 14.2578125

Thanks.

[Xuanyi Dong]

If you are using the algorithms in NAS-Bench-201, that result is expected. You can have a try https://github.com/D-X-Y/AutoDL-Projects/blob/main/exps/NATS-algos/search-cell.py#L8

[FarahSaeed]

Thanks 👍