Giters

automl / NASLib

NASLib is a Neural Architecture Search (NAS) library for facilitating NAS research for the community by providing interfaces to several state-of-the-art NAS search spaces and optimizers.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Zero-Cost Metrics

Zhenhan-Huang opened this issue · comments

commented

I downloaded zero-cost metrics using the script download_nbs_zero.sh. Can you explain how can I relate architecture encoding such as (4, 0, 3, 1, 4, 3) in nasbench201 to architecture string such as |nor_conv_1x1~0|+|avg_pool_3x3~0|avg_pool_3x3~1|+|avg_pool_3x3~0|nor_conv_3x3~1|avg_pool_3x3~2|? In addition, can you explain how you extract validation accuracy from NASBench201 api? Thank you!

commented

Hi @CNSaber,

Thank you for you interest in our project!

You can convert the architecture encoding to the string representation using convert_op_indices_to_str found in naslib\search_spaces\nasbench201\conversions.

Here's a complete snippet to convert the arch to string, and query it's validation accuracy.

from naslib.search_spaces import NasBench201SearchSpace
from naslib.search_spaces.nasbench201.conversions import convert_op_indices_to_str
from naslib.search_spaces.core.query_metrics import Metric
from naslib.utils import get_dataset_api

graph = NasBench201SearchSpace()
graph.set_spec((4, 0, 3, 1, 4, 3))

dataset_api = get_dataset_api(search_space="nasbench201", dataset="cifar10")
val_acc = graph.query(metric=Metric.VAL_ACCURACY, dataset="cifar10", dataset_api=dataset_api)
print(convert_op_indices_to_str(graph.get_hash()), val_acc)

Hope this helps!

Best,
Arjun

commented

Thank you very much for the clarification!