AutoAI

---

AutoAI is a variation of the automated machine learning, or AutoML, technology. It applies intelligent automation to the task of building predictive machine learning models by preparing data for training, identifying the best type of model for the given data, then choosing the features, or columns of data, that best support the problem the model is solving. Finally, automation tests a variety of tuning options to reach the best result as it generates, then ranks, model-candidate pipelines. The best performing pipelines can be put into production to process new data, and deliver predictions based on the model training.

The Automated Machine Learning and Data Science Team (AMLDS), a small team within IBM Research, which was formed to “apply techniques from Artificial Intelligence (AI), Machine Learning (ML), and data management to accelerate and optimize the creation of machine learning and data science workflows,” is credited with developing AutoAI.

Use Case

A typical use case for AutoAI would be training a model to predict how customers might respond to a sales incentive. The model is first trained with actual data on how customers responded to the promotion. Presented with new data, the model can provide a prediction of how a new customer might respond, with a confidence score for the prediction. Prior to AutoML, data scientists had to build these predictive models by hand, testing various combinations of algorithms, then testing to see how predictions compared to actual results.Where AutoML automated some of the process of preparing the data for training, applying algorithms to process the data and then further optimizing the results, AutoAI provides greater intelligent automation that allows for testing significantly more combinations of factors to generate model candidate pipelines that more accurately reflect and address the problem being solved.

The AutoAI process

AutoIA process flow

The user initiates the process by providing a set of training data and identifying the prediction column, which sets up the problem to solve. For example, the prediction column might contain possible values of yes or no in response to an offered incentive. In the data pre-processing stage, AutoAI applies various algorithms, or estimators, to analyze, clean (for example, remove redundant information or impute missing data), and prepare structured raw data for machine learning.

The next step is automated model selection that matches the data with a model type, such as classification or regression.For example, if there are only two types of data in a prediction column, AutoAI prepares to build a binary classification model. If there is an unknowable set of possible answers, AutoAI prepares a regression model, which employs a different set of algorithms, or problem-solving transformations. AutoAI tests and ranks candidate algorithms against small subsets of the data, gradually increasing the size of the subset for the most promising algorithms to arrive at the best match. This process of iterative and incremental machine learning is what sets AutoAI apart from earlier versions of AutoML.

Feature engineering transforms the raw data into the combination of features that best represents the problem to achieve the most accurate prediction. Part of this process is to evaluate how data in the training data source can best support an accurate prediction. Using algorithms, it weights some data as more important than others to achieve the desired result. AutoAI automates the consideration of various feature construction choices in a structured, non-exhaustive manner, while progressively maximizing model accuracy using reinforcement learning. This results in an optimized sequence of data transformations that best match the algorithms of the model selection step.

Finally, AutoAI applies the hyperparameter optimization step to refine and advance the best performing model pipelines. Pipelines are model candidates that are evaluated and ranked by metrics such as accuracy and precision. At the end of the process, the user can review the pipelines and choose the pipeline or pipelines to put into production to deliver predictions on new data.

Drawbacks of automating model building

Although the race to automate machine learning is fierce, it is not without detractors. The chief concern is the loss of visibility into the model creation if it is done by machine rather than by data scientists. Writing for Dice magazine, Nick Kolakowski, cautions: "When you begin to automate these processes, you risk obfuscating at least a portion of the data and logic behind dashboards—which might lead some to question the output of the work." In response to this concern, the AutoAI team developed the capability to automatically generate and publish the code that creates the model pipelines. This allows data scientists to review and audit the model creation process without having to construct the model from the ground up.

History

In August 2017, AMLDS announced that they were researching the use of automated feature engineering to eliminate guesswork in data science. AMDLS members Udayan Khurana, Horst Samulowitz, Gregory Bramble, Deepak Toraga, and Peter Kirchner, along with Fatemeh Nargesian of the University of Toronto and Elias Khalil of Georgia Tech, presented their preliminary research at Brigham Young University (BYU) that same year.

Called “Learning-based Feature Engineering,” their method learned the correlations between feature distributions, target distributions, and transformations, built meta-models that used past observations to predict viable transformations, and generalized thousands of data sets spanning different domains. To address feature vectors of different sizes, it used Quantile Sketch Array to capture the essential character of a feature.

In 2018, IBM Research announced Deep Learning as a Service, which opened popular deep learning libraries such as Caffe, Torch and TensorFlow, to developers in the cloud. AMLDS continued their work and used it in a top Kaggle competition. It finished in the top 10 percent. Jean-Francois Puget, PhD, a distinguished engineer for machine learning and optimization at IBM, who entered the competition, decided it was ready to be a capability for IBM AI and data science platforms like Watson Machine Learning. In December 2018, IBM Research announced NeuNetS, a new capability that automated neural network model synthesis as part of automated AI model development and deployment.

In “A Formal Method for AutoML via ADMM,” a May 2019 Cornell research paper (updated in June 2019), authors Sijia Liu, Parikshit Ram, Djallel Bouneffouf, Deepak Vijaykeerthy, Gregory Bramble, Horst Samulowitz, Dakuo Wang, Andrew R Conn, and Alexander Gray proposed a method for AutoML that used  the alternating direction method of multipliers (ADMM) to configure multiple stages of an ML pipeline, such as transformations, feature engineering and selection, and predictive modeling. This was the first recorded time that IBM Research publicly applied the term “Auto” to machine-learning.

AutoAI: The evolution of AutoML

2019 was the year that AutoML became more widely discussed as a concept. “The Forrester New Wave™: Automation-Focused Machine Learning Solutions, Q2 2019,” evaluated AutoML solutions and found that the more powerful versions offered feature engineering. A Gartner Technical Professional Advice report from August 2019 reported that, based on their research, AutoML could augment data science and machine learning. They described AutoML as the automation of data preparation, feature engineering and model engineering tasks.

AutoAI is the evolution of AutoML. One of AutoAI's principal inventors, Jean-Francois Puget, PhD, describes it as automatically performing data preparation, feature engineering, machine learning algorithm selection, and hyper-parameter optimization to find the best possible machine learning model. The hyper-parameter optimization algorithm used in AutoAI differs from the hyper-parameter tuning of AutoML. The novel algorithm is optimized for costly function evaluations such as model training and scoring that are typical in machine learning, enabling rapid convergence to a good solution despite long evaluation times of each iteration.

Awards for AutoAI

In September 2019, AutoAI won the won the Best Innovation in Intelligent Automation Award at the AIconics AI Summit in San Francisco.

Winner, iF Design Guide award for Communication in a Software Application.

References

This article "AutoAI" is from Wikipedia. The list of its authors can be seen in its historical and/or the page Edithistory:AutoAI. Articles copied from Draft Namespace on Wikipedia could be seen on the Draft Namespace of Wikipedia and not main one.

This page exists already on Wikipedia.

---