Quadratic Programming (QP)

A quadratic programming (QP) problem is to minimize a quadratic function subject to some equality and/or inequality constraints:

#math1007#   #tex2html_wrap_indisplay20509# (224)
where #math1008##tex2html_wrap_inline9372# is an #tex2html_wrap_inline9374# positive definite matrix, #tex2html_wrap_inline9376# an #tex2html_wrap_inline9378# matrix, and #tex2html_wrap_inline9380# is an M-D vector. Also, #math1009##tex2html_wrap_inline9382#. Here the scalar constant #tex2html_wrap_inline9384# can be dropped as it does not play any role in the optimization. Note that #math1010##tex2html_wrap_inline9386# is a hyper elliptic paraboloid with the minimum at point #tex2html_wrap_inline9388# in the N-D space.

We first consider the special case where the QP problem is only subject to equality constraints and we assume #tex2html_wrap_inline9390#, i.e., the number of constraints is smaller than the number of unknowns in #tex2html_wrap_inline9392#. Then the solution #tex2html_wrap_inline9394# has to satisfy #math1011##tex2html_wrap_inline9396#, i.e., it has to be on #tex2html_wrap_inline9398# hyper planes in the N-D space.

The Lagrangian function of the QP problem is:

#math1012#   #tex2html_wrap_indisplay20525# (225)
To find the optimal solution, we first equate the derivatives of the Lagrangian with respect to both #tex2html_wrap_inline9400# and #math1013##tex2html_wrap_inline9402# to zero:
#math1014#
#tex2html_wrap_indisplay20529# #tex2html_wrap_indisplay20530# #tex2html_wrap_indisplay20531#  
#tex2html_wrap_indisplay20532# #tex2html_wrap_indisplay20533# #tex2html_wrap_indisplay20534# (226)
These two equations can be combined and expressed in matrix form as:
#math1015#   #tex2html_wrap_indisplay20536# (227)
We then solve this system of #tex2html_wrap_inline9404# equations to get the optimal solution #tex2html_wrap_inline9406# and the corresponding #math1016##tex2html_wrap_inline9408#.

<#5412#>Example<#5412#>

#math1017#   #tex2html_wrap_indisplay20541#  
where
#math1018#   #tex2html_wrap_indisplay20543#  

#math1019#   #tex2html_wrap_indisplay20545#  
Solving this equation we get the solution #math1020##tex2html_wrap_inline9410# and #tex2html_wrap_inline9412#, at which the function #math1021##tex2html_wrap_inline9414# is minimized subject to #tex2html_wrap_inline9416#.

If #tex2html_wrap_inline9418#, i.e., the number of equality constraints is the same as the number of variables, then the variable #tex2html_wrap_inline9420# is uniquely determined by the linear system #math1022##tex2html_wrap_inline9422#, as the intersect of #tex2html_wrap_inline9424# hyper planes, independent of the objective function #tex2html_wrap_inline9426#. Further if #tex2html_wrap_inline9428#, i.e., the system #math1023##tex2html_wrap_inline9430# is over constrained, and its solution does not exist in general. It is therefore more interesting to consider QP problems subject to both equality and inequality constraints:

#math1024#   #tex2html_wrap_indisplay20558#