Neural Architecture Search

An extension of GATES@ECCV'20. We incorporate multifaceted information about NN’s operation-level and architecture-level computing semantics into the construction and training of the encoder.

A talk on model compression towards efficient DL inference

A talk on NAS researches at Renmin University.

To improve one-shot NAS, we apply curriculum learning to the sharing extent in one-shot supernet to achieve better ranking correlation.

We study one-shot performance estimators and 8 types of zero-shot estimators on 5 different benchmarks (NB101, 201, 301, NDS ResNet, ResNeXt-A).

TA-GATES is an encoding scheme specially designed for neural architectures, considering their distinguishing properties as DAGs with trainable operations. TA-GATES encodes an architecture by mimicking its training process, and thereby provides more discriminative architecture-level and operation-level encodings.

In order to improve the sample efficiency of NAS, we follow the line of predictor-based NAS and improve the encoder design and training of the predictor. (1) We design a Generic Graph-based neural ArchiTecture Encoding Scheme (GATES) to better encode NN architectures. (2) We propose to use ranking loss to train the predictor.

To mitigate the cold-start problem of predictor-based NAS, we design an ensemble method to fuse the knowledge from multiple experts trained with low-fidelity architectural information (e.g., complexities, zero-shot metrics).