Lec #24: SDPs I | 15-850: CMU Advanced Algorithms

Lec #24: SDPs I

Posted on March 30, 2017 by cmurandomized

Nicholas asked how, in the SoS example where we wanted to write
the polynomial

$\displaystyle P(x) = 2x_1^4 + 2x_1^3x_2 - x_1^2x_2^2 + 5x_2^4$

as a sum of square polynomials, we could quickly restrict ourselves to
${(x_1^2, x_2^2, x_1x_2)}$ .

Here’s one way. Of course, in general we have to consider the vector ${\bar{X} = (1, x_1, x_2, x_1^2, x_2^2, x_1x_2)^\intercal}$ , since the squares of these monomials span ${P(x)}$ . This means we must consider a much larger (i.e., ${6 \times 6}$ ) matrix ${Q}$ , and try to write ${P(x) = \bar{X}^\intercal Q \bar{X}}$ with ${Q \succeq 0}$ . But observe two simple facts:
- The first three diagonal terms of ${Q}$ are zero, by inspection.
- The bottom ${3 \times 3}$ principal minor is just our old matrix.
Now we claim that if some diagonal term in some PSD matrix is zero, then all entries in that row and column are zero too. Indeed, if ${Q_{ii} = 0}$ , then for any ${j}$ , the non-negativity of the determinants of the principal minors (see below) says: ${0 = Q_{ii}Q_{jj} \geq Q_{ij}^2}$ , so ${Q_{ij} = 0}$ . Hence, the problem just collapses to finding ${Q \succeq 0}$ such that

$\displaystyle P(x) = \begin{pmatrix} x_1^2 \\ x_2^2 \\ x_1x_2 \end{pmatrix}^\intercal \begin{pmatrix} & & \\ & Q & \\ & & \end{pmatrix} \begin{pmatrix} x_1^2 \\ x_2^2 \\ x_1x_2 \end{pmatrix} \ \ \ \ \ (1)$

which we saw a solution for. Indeed, if we set

$\displaystyle Q = \begin{pmatrix} 2 & -3 & 1\\ -2 & 5 & 0\\ 1 & 0 & 5 \end{pmatrix} \ \ \ \ \ (2)$

then the eigenvalues are ${\lambda_1 = 0, \lambda_2 = 5, \lambda_3 = 7}$ , with eigenvectors

$\displaystyle v_1 = \frac{1}{\sqrt{35}}(-5, -3, 1)^\intercal, v_2 = \frac{1}{\sqrt{10}}(0,1,3)^\intercal, v_3 = \frac{1}{\sqrt{14}}(2, -3, 1)^\intercal.$

Since ${Q = \sum_i \lambda_i v_i v_i^\intercal}$ , you get

$\displaystyle \begin{array}{rl} P(x) &= X^\intercal Q X = \sum_{i = 1}^3 \lambda_i X^\intercal (v_i v_i^\intercal) X = \sum_{i = 1}^3 \lambda_i \langle X, v_i \rangle^2\\ &= \frac12 (x_2^2 + 3x_1x_2)^2 + \frac12 (2x_1^2 - 3x_2^2 + x_1x_2)^2. \end{array}$

Which is the SoS representation we wanted.
Thanks, C.J.: I’d clean forgotten about the all-important equivalent definition of PSD-ness: ${A}$ is PSD if and only if all principal minors have non-negative determinants. Recall that a principal minor is obtained by restricting to some set ${S \subseteq [n]}$ of rows and columns. One direction of the proof is easy (and thankfully it’s the one we need): first, a matrix is PSD then all its principal minors are PSD. (Use the ${x^\intercal A x \geq 0}$ definition.) But the determinant of a principal minor is just the product of its eigenvalues, which is non-negative for PSD matrices. The other direction is a little more involved, I don’t know a one-line proof, please consult your favorite algebra text. (Or if you know one, catch me and tell me about it.)
Finally, the point that Goran asked a clarification for: if ${P(x) = \sum_i h_i(x)^2}$ has max-degree ${2d}$ , then the ${h_i(x)}$ need only be polynomials of degree at most ${d}$ . Suppose not. Let ${D > d}$ be the highest degree monomial in some ${h_i(x)}$ , and let ${\widehat{h}_i(x)}$ contain all the monomials in ${h_i(x)}$ of degree ${D}$ . Now ${H(x) = \sum_i \widehat{h}_i(x)^2}$ is a sum of square polynomials, and all its terms (of degree ${2D > 2d}$ ) also appear in ${\sum_i h_i(x)^2}$ .
Goran had asked: Can ${H(x)}$ be identically zero, with all its terms cancelling out? No, and here’s one easy way to see it: consider some ${x^*}$ for which some ${\widehat{h}_i(x^*) \neq 0}$ . Then ${H(x) \geq \widehat{h}_i(x^*)^2 > 0}$ . (I am sure there are another easy arguments? Please tell me about them.)

This entry was posted in Uncategorized. Bookmark the permalink.

Leave a comment Cancel reply