Description of this paper


Convex Optimization, Spring 2016 Homework 6 Due on 8:15a.m. May.-(Answered)


Step-by-step Instant Solution


The problem 3?Quantile regression in attached file CVXHW6.pdf

Convex Optimization, Spring 2016


Homework 6


Due on 8:15a.m. May. 26, 2016, before class


Note: Please also submit your Matlab code by printing it out (use the data provided in the xx.dat and


rename them as xx.mat).


1) Distributed lasso. Consider the `1 -regularized least-squares (?lasso?) problem














A1 0 B1 x1



x2 ? c1
+ ?


minimize f (z) = (1/2)




0 A2 B2


















with optimization variable z = (x1 , x2 , y) ? Rn1 ? Rn2 ? Rp . We can think of xi as the local variable for


system i, for i = 1, 2; y is the common or coupling variable.


(a) Primal decomposition. Explain how to solve this problem using primal decomposition, using (say)


the subgradient method for the master problem.(10 points)


(b) Dual decomposition. Explain how to solve this problem using dual decomposition, using (say) the


subgradient method for the master problem. Give a condition (on the problem data) that allows you




to guarantee that the primal variables xi converge to optimal values. (20 points)


2) Solving LPs via alternating projections. Consider an LP in standard form,





cT x



subject to



Ax = b


x  0,



with variable x ? Rn , and where A ? Rm?n . A tuple (x, ?, ?) ? R2n+m is primal-dual optimal if and


only if


Ax = b, x  0, ?AT ? + ? = c, ?  0, cT x + bT ? = 0.


These are the KKT optimality conditions of the LP. The last constraint, which states that the duality


gap is zero, can be replaced with an equivalent condition, ?T x = 0, which is complementary slackness.


(a) Let z = (x, ?, ?) denote the primal-dual variable. Express the optimality conditions as z ? A ? C,


where A is an affine set, and C is a simple cone. Give A as A = {z|F z = g}, for appropriate F and


g.(10 points)


(b) Explain how to compute the Euclidean projections onto A and also onto C.(10 points)


3) Quantile regression. For ? ? (0, 1), define h? : Rn ? R as


h? (x) = ?1T x+ + (1 ? ?)1T x? ,


where x+ = max{x, 0} and x? = max{?x, 0}, where the maximum is taken elementwise.


(a) Give a simple expression for the proximal operator of h? .(10 points)


(b) The quantile regression problem is





h? (Ax ? b),



with variable x ? Rn and parameters A ? Rm?n , b ? Rm , and ? ? (0, 1). Explain how to use ADMM


to solve this problem by introducing a new variable (and constraint) z = Ax ? b. Give the details of


each step in ADMM, including how one of the steps can be greatly speeded up after the first step.(20







(c) Implement your method on the given data [hw6 3c.mat] (i.e., A and b), for ? ? {0.2, 0.5, 0.8}. For


each of these three values of ?, give the optimal objective value, and plot a histogram of the residual


vector Ax ? b.(20 points)


Hint. You should develop, debug, and test your code on a smaller problem instance, so you can easily


(i.e., quickly) check the results against CVX.


4) Sum-of-norms heuristic for block sparsity. The basic `1 heuristic for finding a sparse vector x in a convex


set C is to minimize kxk1 over C. We are given a partition of x into subvectors: x = (x(1) , ? ? ? , x(k) ), with


x(i) ? Rni . Our goal is to find an x ? C with the fewest number of subvectors x(i) nonzero. Like the


basic sparsity problem, this problem is in general difficult to solve. But a variation on the `1 heuristic can


work well to give a block sparse, if not the block sparsest, vector. The heuristic is to minimize







kx(i) k






over x ? C.


(a) What happens if the norms above are `1 norms? Would you expect x to be sparse? Block sparse?


(Your answer can be very brief.)(10 points)


(b) Generate a nonempty polyhedron {y|Ay  b} using the given data[hw6 4b.mat]. We will divide


x into 20 subvectors, each of length 5. Use the heuristic above, with `1 , `2 , and `? norms on the


subvectors, to find block sparse x in your polyhedron.(10 points)


Hints. You may find the functions norms and reshape useful.


5) Rank minimization via the nuclear norm. A simple heuristic for finding a matrix of low (if not minimum)


rank, in a convex set, is to minimize its nuclear norm, i.e., the sum of its singular values, over the convex


set. Test this method on a numerical example, obtained by generating data matrices A0 , ? ? ? , An ? Rp?q


and attempting to minimize the rank of A0 + x1 A1 + ? ? ? + xn An . You can use n = 50, p = 10, q = 10


(data is given by [hw6 5.mat]).(10 points)


6) Robust geometric program with norm-based uncertainty. We consider a geometric program in variables


Rk+ , data matrix B ? Rm?k , and exponent vectors ?i ? Rn and ?ij ? Rn .


x ? Rn+ , with problem data a ? Q








For shorthand, we define x = j=1 xj j for vectors ? ? Rn . Consider the problem










a i x ?i






subject to








Bij x?ij ? 1 for i = 1, ? ? ? , m.






Now assume that we have norm-based data uncertainty in our multipliers ai and Bij , that is, there exist


constants ? ? 0 and  ? 0 where all we know is that the data could be ai ? ?i for a vectpr ? ? Rn such


that k?kp ? ?, and similarly for the rows of B. We would like to solve the robust formulation












(ai + ?i )x?i



k?kp ?? i=1



subject to








(Bij + ej )x?ij ? 1 for i = 1, ? ? ? , m and all e ? Rk s.t. kekp ? .






Here p ? [1, ?] generate the norms for the uncertainty set. Show how to formulate the robust problem


above as a geometric problem.(20 points)







Paper#9256611 | Written in 27-Jul-2016

Price : $16