Google’s Model Search automatically optimizes and identifies AI models

Google today announced the release of Model Search, an open source platform designed to help researchers develop machine learning models efficiently and automatically. Instead of focusing on a specific domain, Google says that Model Search is domain-agnostic, making it capable of finding a model architecture that fits a dataset and problem while minimizing coding time and compute resources.

The success of an AI model often depends on how well it can perform across various workloads. But designing a model that can generalize well can be extremely challenging. In recent years, AutoML algorithms have emerged to help researchers find the right model without the need for manual experimentation. However, more often than not, these algorithms are compute-heavy and need thousands of models to train.

Model Search — which is built on Google’s TensorFlow machine learning framework and can run either on a single machine or several — consists of multiple trainers, a search algorithm, a transfer learning algorithm, and a database to store evaluated models. Model Search runs training and evaluation experiments for AI models in an adaptive and asynchronous fashion, such that all trainers share the knowledge gained from their experiments while conducting each experiment independently. At the beginning of every cycle, the search algorithm looks up all the completed trials and decides what to try next, after which it “mutates” over one of the best architectures found up to that point and assigns the resulting model back to a trainer.

To further improve efficiency and accuracy, Model Search employs transfer learning during experiments. For example, it uses knowledge distillation and weight sharing, which bootstraps some of the variables in models from previously trained models. This enables faster training and by extension opportunities to discover more and ostensibly better architectures.

After a Model Search run, users can compare the many models found during the search. In addition, they can create their own search space to customize the architectural elements in their models.

Above: An example of an evolution of a model over many experiments. Each color represents a different type of architecture component.

Image Credit: Google

Google says that in an internal experiment, Model Search improved upon production models with minimal iterations, particularly in the areas of keyword spotting and language identification. It also managed to find an architecture suitable for image classification on the heavily explored CIFAR-10 open source imaging dataset.

“We hope the Model Search code will provide researchers with a flexible, domain-agnostic framework for machine learning model discovery,” Google research engineer Hanna Mazzawi and research scientist Xavi Gonzalvo wrote in a blog post. “By building upon previous knowledge for a given domain, we believe that this framework is powerful enough to build models with the state-of-the-art performance on well studied problems when provided with a search space composed of standard building blocks.”