Alternating Optimization
Alternating optimization is a technique for solving optimization functions with many parameters, where no explicit solution exists to solve for all simultaneously. For many such functions there are methods to solve for one or a subset of parameters if the others are considered fixed[1], so it is possible to alternate among these subproblems to iteratively determine a solution to the larger optimization.
Algorithm[edit]
- Assign values to all parameters. These may be chosen randomly or set carefully.
- Solve for one or a set of parameters via a known method, holding others constant.
- Replace the parameter values with those found in (2)
- Focus attention on the next parameter or set of parameters and repeat from (2)
- Break when the process has converged.
Convergence[edit]
It is possible an alternating optimization procedure gets stuck in a local minimum[1], but "Under reasonable assumptions, the general AO approach is shown to be locally, q-linearly convergent, and to also exhibit a type of global convergence."[2]
References[edit]
- ↑ 1.0 1.1 Jerome H. Friedman (1985). "Classification and Multiple Regression through Projection Pursuit" (PDF). Journal of the American Statistical Association. Retrieved 2018-01-31.
- ↑ James C. Bezdek (2003). "Convergence of alternating optimization". Neural, Parallel & Scientific Computations. Retrieved 2018-01-31.
This article "Alternating Optimization" is from Wikipedia. The list of its authors can be seen in its historical and/or the page Edithistory:Alternating Optimization. Articles copied from Draft Namespace on Wikipedia could be seen on the Draft Namespace of Wikipedia and not main one.
