Magging

In statistics, Magging is a method of aggregating estimators. It's a machine learning ensemble algorithm designed to improve the stability and accuracy of machine learning algorithms used in statistical classification and regression. In general, large-scale data analysis poses problems which need to be addressed simultaneously, and the solution is often straightforward if the data are homogeneous: one can use classical ideas of subsampling and mean aggregation to get it. However, we believe that many large-scale data are inherently inhomogeneous: that is, they are neither i.i.d. nor stationary observations from a distribution. If the data exhibit homogeneities, the same approach above will be inadequate.

Subsample and Aggregation

Construct groups $G_1,G_2,...G_g$ with $G_g\in \{1,2,...,n\}$, where n denotes the sample size and $\{1,2,...,n\}$ is the index set for the samples. For every group $G_g$, we compute an estimator ${\hat{\theta }}_g$ and these estimates are then aggregated to a single “overall” estimate ${\hat{\theta }}_{aggr}$, which can be achieved in different ways.

Magging

Magging stands for Maximin aggregating, it’s a kind of aggregation for heterogeneous data. It has been proposed by Meinshausen and Bühlmann. Magging is choosing the weight as a convex combination to minimize the $l_2$-norm of the fitted values.

Magging:${\hat{\theta }}_{aggr}:=\sum _{g=1}^G{w}_g{\hat{\theta }}_g,$

where

$w:=argmi{n}_{w\in C_G}\parallel \sum _g\hat{Y}(g)w_g{\parallel }_2$

and

$C_G:=\{w:mi{n}_g{w}_g\ge 0and\sum _g{w}_g=1\}$.

If the solution is not unique, we take the solution with lowest $l_2$-norm of the weight vector among all solutions. The optimization and computation can be implemented in a very efficient way.

References

This article "Magging" is from Wikipedia. The list of its authors can be seen in its historical. Articles copied from Draft Namespace on Wikipedia could be seen on the Draft Namespace of Wikipedia and not main one.

---