We use cookies to improve your experience with our site.

Indexed in:

SCIE, EI, Scopus, INSPEC, DBLP, CSCD, etc.

Submission System
(Author / Reviewer / Editor)
Advanced Search
Volume 32 Issue 1
January  2017
Turn off MathJax
Article Contents
Abstract
References
Related Articles
Lan Yao, Feng Zeng, Dong-Hui Li, Zhi-Gang Chen. Sparse Support Vector Machine with Lp Penalty for Feature Selection[J]. Journal of Computer Science and Technology, 2017, 32(1): 68-77. DOI: 10.1007/s11390-017-1706-2
Citation: Lan Yao, Feng Zeng, Dong-Hui Li, Zhi-Gang Chen. Sparse Support Vector Machine with Lp Penalty for Feature Selection[J]. Journal of Computer Science and Technology, 2017, 32(1): 68-77. DOI: 10.1007/s11390-017-1706-2
shu

Sparse Support Vector Machine with Lp Penalty for Feature Selection

  • 1 College of Mathematics and Econometrics, Hunan University, Changsha 410082, China;
    2 School of Software, Central South University, Changsha 410083, China;
    3 School of Mathematical Sciences, South China Normal University, Guangzhou 510631, China

Funds: This work is supported in part by the National Natural Science Foundation of China under Grant Nos. 61502159, 61379057, 11101081, and 11271069, and the Research Foundation of Central South University of China under Grant No. 2014JSJJ019.
More Information
  • Author Bio:

    Lan Yao is an assistant professor of the College of Mathematics and Econometrics, Hunan University, Changsha. She got her B.S. degree in computer science, M.S. and Ph.D. degrees in applied mathematics from Hunan University, Changsha, in 2000, 2006 and 2014 respectively. Her research interests include data mining, numerical methods in optimization and network optimization.

  • Corresponding author:

    Feng Zeng E-mail: fengzeng@csu.edu.cn

  • Received Date: February 27, 2016
  • Revised Date: September 06, 2016
  • Published Date: January 04, 2017
    Abstract
  • We study the strategies in feature selection with sparse support vector machine (SVM). Recently, the socalled L p-SVM (0< p< 1) has attracted much attention because it can encourage better sparsity than the widely used L1-SVM. However, Lp-SVM is a non-convex and non-Lipschitz optimization problem. Solving this problem numerically is challenging. In this paper, we reformulate the Lp-SVM into an optimization model with linear objective function and smooth constraints (LOSC-SVM) so that it can be solved by numerical methods for smooth constrained optimization. Our numerical experiments on artificial datasets show that LOSC-SVM (0< p< 1) can improve the classification performance in both feature selection and classification by choosing a suitable parameter p. We also apply it to some real-life datasets and experimental results show that it is superior to L1-SVM.
    • FullText(HTML)
    • References (30)
  • [1]
    Vapnik V N. The Nature of Statistical Learning Theory (2nd edition). Springer, 2000.
    [2]
    Guyon I, Gunn S, Nikravesh M, Zadeh L A. Feature Extraction:Foundations and Applications (1st edition). Springer, 2006.
    [3]
    Saeys Y, Inza I, Larranagal P. A review of feature selection techniques in bioinformatics. Bioinformatics, 2007, 23(19):2507-2517.
    [4]
    Guyon I, Weston J, Barnhill S, Vapnik V. Gene selection for cancer classification using support vector machines. Machine Learning, 2002, 46(1/2/3):389-422.
    [5]
    Rakotomamonjy A. Variable selection using SVM based criteria. The Journal of Machine Learning Research, 2003, 3:1357-1370.
    [6]
    Weston J, Mukherjee S, Chapelle O, Pontil M, Poggio T, Vapnik V. Feature selection for SVMs. In Advances in Neural Information Processing Systems 13, Leen T K, Dietterich T G, Tresp V (eds.), Massachusetts Institute of Technology, 2001, pp.668-674.
    [7]
    Peleg D, Meir R. A feature selection algorithm based on the global minimization of a generalization error bound. In Advances in Neural Information Processing Systems 17, Saul L K, Weiss Y, Bottou L (eds.), Massachusetts Institute of Technology, 2005, pp.1065-1072.
    [8]
    Bradley P S, Mangasarian O L. Feature selection via concave minimization and support vector machines. In Proc. the 5th International Conference on Machine Learning, July 1998, pp.82-90.
    [9]
    Weston J, Elisseeff A, Schölkopf B, Tipping M. Use of the zero norm with linear models and kernel methods. The Journal of Machine Learning Research, 2003, 3:1439-1461.
    [10]
    Amaldi E, Kann V. On the approximability of minimizing nonzero variables or unsatisfied relations in linear systems. Theoretical Computer Science, 1998, 209(1/2):237-260.
    [11]
    Chan A B, Vasconcelos N, Lanckriet G R G. Direct convex relaxations of sparse SVM. In Proc. the 24th International Conference on Machine Learning, June 2007, pp.145-153.
    [12]
    Fung G M, Mangasarian O L. A feature selection newton method for support vector machine classification. Computational Optimization and Applications, 2004, 28(2):185-202.
    [13]
    Bi J B, Bennett K, Embrechts M, Breneman C, Song M H. Dimensionality reduction via sparse support vector machines. The Journal of Machine Learning Research, 2003, 3:1229-1243.
    [14]
    Tibshirani R. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society Series B (Methodological), 1996, 58(1):267-288.
    [15]
    Neumann J, Schnörr C, Steidl G. Combined SVM-based feature selection and classification. Machine Learning, 2005, 61(1/2/3):129-150.
    [16]
    Chartrand R. Exact reconstruction of sparse signals via nonconvex minimization. IEEE Signal Processing Letters, 2007, 14(10):707-710.
    [17]
    Chartrand R. Nonconvex regularization for shape preservation. In Proc. the IEEE International Conference on Image Processing, September 16-October 19, 2007, pp.293-296.
    [18]
    Xu Z B, Zhang H, Wang Y, Chang X Y, Liang Y. L1/2 regularization. Science China Information Sciences, 2010, 53(6):1159-1169.
    [19]
    Liu J L, Li J P, Xu W X, Shi Y. A weighted Lq adaptive least squares support vector machine classifiers-Robust and sparse approximation. Expert Systems with Applications, 2011, 38(3):2253-2259.
    [20]
    Chen W J, Tian Y J. Lp-norm proximal support vector machine and its applications. Procedia Computer Science, 2010, 1(1):2417-2423.
    [21]
    Rakotomamonjy A, Flamary R, Gasso G, Canu S. lp-lq penalty for sparse linear and sparse multiple kernel multitask learning. IEEE Transactions on Neural Networks, 2011, 22(8):1307-1320.
    [22]
    Liu Y F, Zhang H H, Park C, Ahn J. Support vector machines with adaptive Lq penalty. Computational Statistics and Data Analysis, 2007, 51(12):6380-6394.
    [23]
    Liu Z Q, Lin S L, Tan M. Sparse support vector machines with L p penalty for biomarker identification. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 2010, 7(1):100-107.
    [24]
    Tan J Y, Zhang Z Q, Zhen L, Zhang C H, Deng N Y. Adaptive feature selection via a new version of support vector machine. Neural Computing and Applications, 2013, 23(3/4):937-945.
    [25]
    Tian Y J, Yu J, Chen W J. lp-norm support vector machine with CCCP. In Proc. the 7th International Conference on Fuzzy Systems and Knowledge Discovery, August 2010, pp.1560-1564.
    [26]
    Liu J W, Liu Y. Non-integer norm regularization SVM via Legendre-Fenchel duality. Neurocomputing, 2014, 144:537-545.
    [27]
    Chen X J, Xu F M, Ye Y Y. Lower bound theory of nonzero entries in solutions of l2-lp minimization. SIAM J. Sci. Comput., 2010, 32(5):2832-2852.
    [28]
    Zhang C H, Shao Y H, Tan J Y, Deng N Y. Mixed-norm linear support vector machine. Neural Computing and Applications, 2013, 23(7):2159-2166.
    [29]
    Li D H, Wu L, Sun Z, Zhang X J. A constrained optimization reformulation and a feasible descent direction method for L1/2 regularization. Computational Optimization and Applications, 2014, 59(1/2):263-284.
    [30]
    Newman D J, Hettich S, Blake C L, Merz C J. UCI repository of machine learning databases. Technical Report 9702, Department of Information and Computer Science, University of California, Irvine, 1998. http://archive.ics.uci.edu/ml/, Nov. 2016
    • Relative Articles

  • Related Articles

    [1]Wei-Dong Wang, Zhi Li, Li Zhang. Combining Innovative CVTNet and Regularization Loss for Robust Adversarial Defense[J]. Journal of Computer Science and Technology, 2024, 39(5): 1078-1093. DOI: 10.1007/s11390-024-3515-8
    [2]Li-Gang Gao, Meng-Yun Yang, Jian-Xin Wang. Collaborative Matrix Factorization with Soft Regularization for Drug-Target Interaction Prediction[J]. Journal of Computer Science and Technology, 2021, 36(2): 310-322. DOI: 10.1007/s11390-021-0844-8
    [3]Xiang-Jun Lu, Chi Zhang, Da-Wu Gu, Jun-Rong Liu, Qian Peng, Hai-Feng Zhang. Evaluating and Improving Linear Regression Based Profiling: On the Selection of Its Regularization[J]. Journal of Computer Science and Technology, 2020, 35(5): 1175-1197. DOI: 10.1007/s11390-020-9669-0
    [4]Shu-Zheng Zhang, Zhen-Yu Zhao, Chao-Chao Feng, Lei Wang. A Machine Learning Framework with Feature Selection for Floorplan Acceleration in IC Physical Design[J]. Journal of Computer Science and Technology, 2020, 35(2): 468-474. DOI: 10.1007/s11390-020-9688-x
    [5]Cheng-Zhang Zhu, Rong Hu, Bei-Ji Zou, Rong-Chang Zhao, Chang-Long Chen, Ya-Long Xiao. Automatic Diabetic Retinopathy Screening via Cascaded Framework Based on Image- and Lesion-Level Features Fusion[J]. Journal of Computer Science and Technology, 2019, 34(6): 1307-1318. DOI: 10.1007/s11390-019-1977-x
    [6]Tie-Ke He, Hao Lian, Ze-Min Qin, Zhen-Yu Chen, Bin Luo. PTM: A Topic Model for the Inferring of the Penalty[J]. Journal of Computer Science and Technology, 2018, 33(4): 756-767. DOI: 10.1007/s11390-018-1854-z
    [7]Zhong-Gui Sun, Song-Can Chen, Li-Shan Qiao. A Two-Step Regularization Framework for Non-Local Means[J]. Journal of Computer Science and Technology, 2014, 29(6): 1026-1037. DOI: 10.1007/s11390-014-1487-9
    [8]Tapio Pahikkala, Antti Airola, Fabian Gieseke, Oliver Kramer. On Unsupervised Training of Multi-Class Regularized Least-Squares Classifiers[J]. Journal of Computer Science and Technology, 2014, 29(1): 90-104. DOI: 10.1007/s11390-013-1414-5
    [9]Wei Wu, Hang Li, Yun-Hua Hu, Rong Jin. A Kernel Approach to Multi-Task Learning with Task-Specific Kernels[J]. Journal of Computer Science and Technology, 2012, 27(6): 1289-1301. DOI: 10.1007/s11390-012-1305-1
    [10]Bin Gu, Jian-Dong Wang, Tao Li. Ordinal-Class Core Vector Machine[J]. Journal of Computer Science and Technology, 2010, 25(4): 699-708. DOI: 10.1007/s11390-010-1054-y
    • Supplements (0)

Catalog

    Get Citation
    PDF
    XML
    Article views (45) PDF downloads (1673) Cited by()
    Related

    Copyright ©2025 JCST, All Rights Reserved     京ICP备05002829号-1

    Institute of Computing Technology, Chinese Academy of Sciences
    P.O. Box 2704, Beijing 100190, P.R. China

    Tel.: 86-10-62610746 E-mail: jcst@ict.ac.cn

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return