IMS-LOMONAS: Parameter-less Pareto Local Search for Multi-objective Neural Architecture Search with the Interleaved Multi-start Scheme
Quan Minh Phan, Ngoc Hoang Luong
- Clone this repo
- Install necessary packages and databases.
$ cd IMS-LOMONAS
$ bash install.sh
- Download data and database for the set of problems in the CEC'2023 Competition.
- Run the below cell to load the data
from evoxbench.database.init import config
config(<database_path>, <data_path>)
For example:
from evoxbench.database.init import config
config('/content/drive/database', '/content/drive/data')
This repo have already implemented following NAS algorithms:
- (IMS-)NSGA-II
- (IMS-)NSGA-III
- (IMS-)LOMONAS (ours)
To experiment on CEC'2023 problems, run the below script:
$ python main.py --optimizer <algo_name>[lomonas, ims-lomonas, nsga2, nsga3, ims-nsga2, ims-nsga3]
--test_suite <problem_name>[cec-c10, cec-in1k]
--pid <problem_id>[from 1 to 9]
--max_eval 10000 --n_run 31
--database_path <CEC_database_path> --data_path <CEC_data_path>
--using_archive --check_limited_neighbors --neighborhood_check_on_potential_sols --log_results
To experiment on other problems, run the below script:
$ python main.py --optimizer <algo_name>[lomonas, ims-lomonas]
--test_suite <problem_name>[gecco]
--pid <problem_id>[from 1 to 8]
--max_eval 3000 --n_run 31
--database_path <CEC_database_path> --data_path <CEC_data_path>
--using_archive --check_limited_neighbors --neighborhood_check_on_potential_sols --log_results
where:
Problem ID | Search Space | Dataset | Target Objectives | Search Objectives |
---|---|---|---|---|
1 | MacroNAS | CIFAR-10 | test_err & params | val_err & params |
2 | NAS-Bench-101 | CIFAR-10 | test_err & params | val_err_12 & params |
3 | NAS-Bench-201 | CIFAR-10 | test_err & params | val_err_12 & params |
4 | NAS-Bench-201 | ImageNet16-120 | test_err & params | val_err_12 & params |
5 | NAS-Bench-201 | CIFAR-10 | test_err & params | val_err_12 & params |
6 | NAS-Bench-201 | ImageNet16-120 | test_err & params | val_err_12 & params |
7 | NAS-Bench-201 | CIFAR-10 | test_err & params | synflow & jacov & params |
8 | NAS-Bench-101 | CIFAR-10 | test_err & params | synflow & jacov & params |
Set pid
to 7 and 8 to experiment TF-(IMS-)LOMONAS.
$ python evaluate.py --res_path <result_path>
--problem_id <problem_id>
--dataset [cifar10, cifar100, ImageNet16-120]
--algo_name [lomonas, ims-lomonas]
We want to give our thanks to the authors of NAS-Bench-101, NAS-Bench-201, and NAS-Bench-ASR for their search spaces; to the authors of Zero-cost Lightweight NAS and NAS-Bench-Zero-Suite for their zero-cost metric databases.