Alternating optimization is a technique for solving optimization functions with many parameters, where no explicit solution exists to solve for all simultaneously. For many such functions there are methods to solve for one or a subset of parameters if the others are considered fixed, so it is possible to alternate among these subproblems to iteratively determine a solution to the larger optimization.
Algorithm[edit | edit source]
- Assign values to all parameters. These may be chosen randomly or set carefully.
- Solve for one or a set of parameters via a known method, holding others constant.
- Replace the parameter values with those found in (2)
- Focus attention on the next parameter or set of parameters and repeat from (2)
- Break when the process has converged.
Convergence[edit | edit source]
It is possible an alternating optimization procedure gets stuck in a local minimum, but "Under reasonable assumptions, the general AO approach is shown to be locally, q-linearly convergent, and to also exhibit a type of global convergence."
References[edit | edit source]
- Jerome H. Friedman (1985). "Classification and Multiple Regression through Projection Pursuit" (PDF). Journal of the American Statistical Association. Retrieved 2018-01-31.
- James C. Bezdek (2003). "Convergence of alternating optimization". Neural, Parallel & Scientific Computations. Retrieved 2018-01-31.
This article "Alternating Optimization" is from Wikipedia. The list of its authors can be seen in its historical and/or its subpage Alternating Optimization/edithistory. Articles copied from Draft Namespace on Wikipedia could be seen on the Draft Namespace of Wikipedia and not main one.