Giters

automl / NASLib

NASLib is a Neural Architecture Search (NAS) library for facilitating NAS research for the community by providing interfaces to several state-of-the-art NAS search spaces and optimizers.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Zero-Cost Predictor - Run on all 15,625 architecture for NasBench201

alec-flowers opened this issue · comments

commented

Hi, I wanted to test a Zero-Cost Predictor on all 15,625 architectures for NASBench201. In the NAS-Bench-Suite-Zero paper you state that you did this for all the architectures -

For all NAS-Bench-201 and TransNAS-Bench-101 tasks, we evaluate all ZC proxy values and the respective runtimes, for all architectures

But I am running across some issues. In the predictor_config.yaml it only has an option for random sampling of architectures with the parameter uniform_random.

I wrote my own code to exhaustively sample all architectures using naslib.search_spaces.nasbench201.graph.get_arch_iterator(). However, PredictorEvaluator.load_dataset() then loads 15,625 graphs into a list, which around 6,000 architectures in, requires too much memory for my computer. I wanted to see if I was going about this the best way or if there was a better way.

Thanks.

commented

Hi @alec-flowers

Thank you for your interest in our project! You'll find that all the code relating to zero cost proxies are in the zerocost branch. You can edit this runner file to run your proxies on all the nb201 models.

Best,
Arjun