Research Highlights

[Sensitivity Analysis in Nonlinear Semidefinite Programming] [Augmented Lagrangian Methods for Solving Composite Conic Programming] [Smoothing Newton Methods with Global and Local Superlinear Convergence] [Computations of the limiting (Mordukhovich) coderivative of the metric projection onto non-polyhedral convexs sets] [Second Order Methods for Solving Lasso Problems and Their Extensions] [A Restricted Wolfe Dual for Convex Quadratic Programming] [A Symmetric Gauss-Seidel Decomposition Theorem for Convex Composite Quadratic Functions] [Halpern Peaceman-Rachford (HPR) Acceleration Methods] [Extragradient Methods for Monotone Variational Inequalities under Continuity]

• Sensitivity Analysis in Nonlinear Semidefinite Programming: I have been conducting research on sensitivity analysis in nonlinear semidefinite programming (SDP) for over 25 years. My journey began in 1999 with Professor Jie Sun, when we established the strong semismoothness of the metric projector over the SDP cone. By taking advantage of this property, I solved the long-standing open question of characterizing Robinson’s strong regularity of nonlinear SDP problems. Consequently, Robinson’s strong regularity for linear SDP is proven to be true if and only if the primal nondegeneracy and the dual nondegeneracy hold simutaneously. Meanwhile, by using the strong semismoothness of the metric projector over the SDP cone, together with Professor Houduo Qi, we designed a highly efficient quadratically convergent semismooth Newton method for computing the nearest correlation matrix problem in “A quadratically convergent Newton method for computing the nearest correlation matrix” (the problem comes from finance and the “NCM” term was initially introduced by late Professor Nick Higham). The next milestone is the characterization of the robust isolated calmness for a class of conic programming problems. This line of inquiry culminated in achieving a long-standing goal: demonstrating that the Aubin property is equivalent to Robinson’s strong regularity at a local optimal solution for nonlinear SDP. What follows is a brief overview of my research in this area.

• Augmented Lagrangian Methods for Solving Composite Conic Programming: The paper by [Xinyuan Zhao, Defeng Sun, and Kim Chuan Toh, titled “A Newton-CG augmented Lagrangian method for semidefinite programming”, published in SIAM Journal on Optimization 20 (2010), pp. 1737–1765], initiated the research on using the semismooth Newton-CG augmented Lagrangian method (ALM) for solving semidefinite programming (SDP). SDPNAL+ is a MATLAB software for solving large scale SDP with bound constraints (click here for an introduction on how to use the package). This software was awarded [the triennial triennial Beale–Orchard-Hays Prize for Excellence in Computational Mathematical Programming by the Mathematical Optimization Society at Bordeaux, France, July 2-6, 2018. See Picture 1, Picture 2, and Picture 3.] For detailed information about the software, please refer to the papers by [Defeng Sun, Kim Chuan Toh, Yancheng Yuan, and Xinyuan Zhao, titled “SDPNAL+: A Matlab software for semidefinite programming with bound constraints (version 1.0)”, published in Optimization Methods and Software 35 (2020) 87–115] and by [Liuqin Yang, Defeng Sun, and Kim Chuan Toh, titled “SDPNAL+: a majorized semismooth Newton-CG augmented Lagrangian method for semidefinite programming with nonnegative constraints”, published in Mathematical Programming Computation 7 (2015), pp. 331-366.] For extensions to convex quadratic SDP, see the work by [Xudong Li, Defeng Sun, and Kim Chuan Toh, titled “QSDPNAL: A two-phase augmented Lagrangian method for convex quadratic semidefinite programming”, published in Mathematical Programming Computation 10 (2018) 703–743.] In the paper by [Ying Cui, Defeng Sun, and Kim Chuan Toh, titled “On the R-superlinear convergence of the KKT residuals generated by the augmented Lagrangian method for convex composite conic programming”, published in Mathematical Programming 178 (2019) 381—415], we provide a fairly comprehensive treatment of the theoretical convergence rates as well as practical implementations of the ALM for solving linear SDP and convex quadratic SDP.

• Smoothing Newton Methods with Global and Local Superlinear Convergence: In a series of papers—specifically, Xiaojun Chen, Liqun Qi and Defeng Sun, “Global and superlinear convergence of the smoothing Newton method and its application to general box constrained variational inequalities”, Mathematics of Computation, 67 (1998), pp. 519-540, Liqun Qi, Defeng Sun and Guanglu Zhou, “A new look at smoothing Newton methods for nonlinear complementarity problems and box constrained variational inequalities”, Mathematical Programming, 87 (2000), 1–35, and Defeng Sun, “A regularization Newton method for solving nonlinear complementarity problems”, Applied Mathematics and Optimization, 40 (1999), 315-339—we have developed globally convergent smoothing Newton methods that achieve local superlinear (or quadratic) convergence for solving semismooth equations under mild conditions. These methods extend the classical Newton methods for smooth equations to a broader class of problems.

• Computations of the limiting (Mordukhovich) coderivative of the metric projection onto non-polyhedral convexs sets: The limiting (Mordukhovich) coderivative of the metric projection onto a convex set S has played a central role in variational analysis, particularly in the study of the Aubin property. However, for non-polyhedral sets S, it was not known whether explicit formulas for these coderivatives could be computed. This situation began to change in 2008, when, together with Professor Jiri Outrata, we successfully derived explicit coderivative formulas for the metric projection onto the second-order cone. Our results were published in Jiri Outrata and Defeng Sun, “On the coderivative of the projection operator onto the second order cone”, Set-Valued Analysis 16 (2008) 999–1014. These coderivative formulas have since found important applications. For instance, in 2025, Liang Chen, Ruoning Chen, Defeng Sun, and Junyuan Zhu used them in their paper, “Aubin property and strong regularity are equivalent for nonlinear second-order cone programming”, published in SIAM Journal on Optimization 35:2 (2025) 712–738. In this work, they established the equivalence between the Aubin property and Robinson’s strong regularity for nonlinear second-order cone programming. See the Flowchart of the Proof. Further progress was made in 2014, when Chao Ding, Defeng Sun, and Jane Ye derived explicit formulas for the metric projection onto the cone of symmetric and positive semidefinite matrices—an important non-polyhedral cone in semidefinite programming. Their results appeared in “First order optimality conditions for mathematical programs with semidefinite cone complementarity constraints”, Mathematical Programming 147 (2014) 539-579.

• Second Order Methods for Solving Lasso Problems and Their Extensions: In statistics and machine learning, lasso (least absolute shrinkage and selection operator; also Lasso, LASSO or L1 regularization), a term coined by Professor Robert Tibshirani in 1996, is a regression analysis method that performs both variable selection and regularization in order to enhance the prediction accuracy and interpretability of the resulting statistical model. Most of the popular existing packages for solving lasso problems are based on first-order methods such as the block coordinate descent methods and proximal gradient methods. In order to get fast convergent algorithms with accurate solutions, from 2018 we started to develop second-order based semismooth Newton methods for solving lasso problems and their extensions. See Xudong Li, Defeng Sun, and Kim Chuan Toh, “A highly efficient semismooth Newton augmented Lagrangian method for solving Lasso problems’’, SIAM Journal on Optimization 28 (2018) 433–458. [This paper brought Xudong Li the Best Paper Prize for Young Researchers in Continuous Optimization announced in the ICCOPT 2019 held in Berlin, August 3-8, 2019. This is the only prize given in the flagship international conference on continuous optimization held every three years]; Xudong Li, Defeng Sun, and Kim Chuan Toh, “On efficiently solving the subproblems of a level-set method for fused lasso problems”, SIAM Journal on Optimization 28 (2018) 1842–1862; Yancheng Yuan, Defeng Sun, and Kim Chuan Toh, “An efficient semismooth Newton based algorithm for convex clustering”, Proceedings of the 35-th International Conference on Machine Learning (ICML), Stockholm, Sweden, PMLR 80, 2018; Meixia Lin, Yong-Jin Liu, Defeng Sun, and Kim Chuan Toh, “Efficient sparse semismooth Newton methods for the clustered lasso problem”, SIAM Journal on Optimization 29 (2019) 2026–2052; Ziyan Luo, Defeng Sun, Kim Chuan Toh, and Naihua Xiu, “Solving the OSCAR and SLOPE models using a semismooth Newton-based augmented Lagrangian method”, Journal of Machine Learning Research 20(106):1–25, 2019; Yangjing Zhang, Ning Zhang, Defeng Sun, and Kim Chuan Toh, “An efficient Hessian based algorithm for solving large-scale sparse group Lasso problems”, Mathematical Programming 179 (2020) 223–263; Yangjing Zhang, Ning Zhang, Defeng Sun, and Kim Chuan Toh, “A proximal point dual Newton algorithm for solving group graphical Lasso problems”, SIAM Journal on Optimization 30 (2020) 2197–2220; Peipei Tang, Chengjing Wang, Defeng Sun, and Kim Chuan Toh, “A sparse semismooth Newton based proximal majorization-minimization algorithm for nonconvex square-root-loss regression problems”, Journal of Machine Learning Research 21(226):1–38, 2020; Ning Zhang, Yangjing Zhang, Defeng Sun, and Kim Chuan Toh, “An efficient linearly convergent regularized proximal point algorithm for fused multiple graphical Lasso problems”, SIAM Journal on Mathematics of Data Science 3:2 (2021) 524–543; Defeng Sun, Kim Chuan Toh, and Yancheng Yuan, “Convex clustering: Model, theoretical guarantee and efficient algorithm”, Journal of Machine Learning Research 22(9):1−32, 2021; Yancheng Yuan, T.-H. Chang, Defeng Sun, and Kim-Chuan Toh, “A dimension reduction technique for structured sparse optimization problems with application to convex clustering”, SIAM Journal on Optimization 32 (2022) 2294–2318; Qian Li, Binyan Jiang, and Defeng Sun, “MARS: a second-order reduction algorithm for high-dimensional sparse precision matrices estimation”, Journal of Machine Learning Research 24 (134):1−44, 2023; Meixia Lin, Yancheng Yuan, Defeng Sun, and Kim-Chuan Toh, “A highly efficient algorithm for solving exclusive Lasso problems”, Optimization Methods and Software 39: 3 (2024) 489–518; Qian Li, Defeng Sun, and Yancheng Yuan, “An efficient sieving based secant method for sparse optimization problems with least-squares constraints”, SIAM Journal on Optimization 34:2 (2024) 2038–-2066; and Yancheng Yuan, Meixia Lin, Defeng Sun, and Kim-Chuan Toh, “Adaptive sieving: A dimension reduction technique for sparse optimization problems”, Mathematical Programming Computation (2025), in print. arXiv:2306.17369 (2023; Revised September 2024).

• A Restricted Wolfe Dual for Convex Quadratic Programming: It has long been established that a linear program (LP) has a precise dual form, which is another LP, enabling the derivation of a perfect duality theory. However, the situation is more complex for convex quadratic programming (QP) when the Hessian of the quadratic objective function is neither zero nor positive definite. In 1961, Philip Wolfe introduced a dual form, known as the Wolfe dual problem, for nonlinear programming. For a convex QP, the Wolfe dual problem is another convex QP whose solution set, if nonempty, is always unbounded unless the Hessian of the quadratic objective function is positive definite. This presents significant challenges in designing efficient algorithms for solving large-scale convex QPs. In 2018, together with Xudong Li and Kim Chuan Toh, we successfully addressed this issue by introducing a restricted Wolfe dual form for convex composite QPs. This restricted Wolfe dual eliminates the ambiguity caused by the rank deficiency of the Hessian of the objective function. It possesses many desirable theoretical properties that resemble those of linear conic programming and facilitates the design of efficient dual-based methods, such as the augmented Lagrangian methods, with guaranteed convergence, for solving convex composite QPs. See “my talk slides on the Restricted Wolfe Dual and the Symmetric Gauss-Seidel Decomposition Theorem”.

• A Symmetric Gauss-Seidel Decomposition Theorem for Convex Composite Quadratic Functions: In the paper by Xudong Li, Defeng Sun, and Kim Chuan Toh, titled “A block symmetric Gauss-Seidel decomposition theorem for convex composite quadratic programming and its applications”, published in Mathematical Programming 175 (2019) 395–418, we established a symmetric Gauss-Seidel decomposition theorem. This theorem plays a critical role in the successful design of alternating direction methods of multipliers (ADMMs) for multi-block convex optimization problems. It is particularly effective when combined with the convergence analysis of the semi-proximal ADMMs for solving linearly constrained convex optimization problems, as developed in Appendix B of the paper by [Maryam Fazel, Ting Kei Pong, Defeng Sun, and Paul Tseng, titled “Hankel matrix rank minimization with applications to system identification and realization”, Hankel-Matrix-semi-Proximal-ADMM published in SIAM Journal on Matrix Analysis and Applications 34 (2013) 946-977.] Moreover, this decompsoition theorem was used by Liang Chen, Xudong Li, Defeng Sun, and Kim Chuan Toh to prove the equivalence of inexact proximal augmented Lagrangian methods and ADMMs for a class of convex composite programming. Their results were published in “On the equivalence of inexact proximal ALM and ADMM for a class of convex composite programming”, Mathematical Programming 185 (2021) 111—161 [Correction to the Proof of Lemma 3.3].

• Halpern Peaceman-Rachford (HPR) Acceleration Methods: See the implementation of the HPR method for solving linear programming [Kaihuang Chen, Defeng Sun, Yancheng Yuan, Guojun Zhang, and Xinyuan Zhao, “HPR-LP: An implementation of an HPR method for solving linear programming”, arXiv:2408.12179 (August 2024)] and the theoretical foundation of the HPR method [Defeng Sun, Yancheng Yuan, Guojun Zhang, and Xinyuan Zhao, “Accelerating preconditioned ADMM via degenerate proximal point mappings”, SIAM Journal on Optimization 35:2 (2025) 1165–1193]. For solving optimal transport problems, please refer to [Guojun Zhang, Zhexuan Gu, Yancheng Yuan, and Defeng Sun, “HOT: An Efficient Halpern Accelerating Algorithm for Optimal Transport Problems”, IEEE Transactions on Pattern Analysis and Machine Intelligence (2025), in print. arXiv:2408.00598 (August 2024)] and [Guojun Zhang, Yancheng Yuan, and Defeng Sun, “An Efficient HPR Algorithm for the Wasserstein Barycenter Problem with $ O ({Dim (P)}/\varepsilon) $ Computational Complexity”. arXiv:2211.14881 (2022).]

• Extragradient Methods for Monotone Variational Inequalities under Continuity: In the paper, Defeng Sun, “Projected extragradient method for finding saddle points of general convex programming”, Qufu Shifan Daxue Xuebao Ziran Kexue Ban19:4 (1993) 10–17, we designed a Korpelevich-type extragradient method with a rigorous proof under continuity only. See the version translated in English. For extensions, please refer to Defeng Sun, “A new step-size skill for solving a class of nonlinear projection equations”, Journal of Computational Mathematics 13:4 (1995), 357–368 and Defeng Sun, “A class of iterative methods for solving nonlinear projection equations”, Journal of Optimization Theory and Applications, Vol. 91, No.1, 1996, pp. 123–140.