F-SVM: Combination of Feature Transformation and SVM Learning via Convex Relaxation

by   Xiaohe Wu, et al.

The generalization error bound of support vector machine (SVM) depends on the ratio of radius and margin, while standard SVM only considers the maximization of the margin but ignores the minimization of the radius. Several approaches have been proposed to integrate radius and margin for joint learning of feature transformation and SVM classifier. However, most of them either require the form of the transformation matrix to be diagonal, or are non-convex and computationally expensive. In this paper, we suggest a novel approximation for the radius of minimum enclosing ball (MEB) in feature space, and then propose a convex radius-margin based SVM model for joint learning of feature transformation and SVM classifier, i.e., F-SVM. An alternating minimization method is adopted to solve the F-SVM model, where the feature transformation is updatedvia gradient descent and the classifier is updated by employing the existing SVM solver. By incorporating with kernel principal component analysis, F-SVM is further extended for joint learning of nonlinear transformation and classifier. Experimental results on the UCI machine learning datasets and the LFW face datasets show that F-SVM outperforms the standard SVM and the existing radius-margin based SVMs, e.g., RMM, R-SVM+ and R-SVM+μ.


page 1

page 2

page 3

page 4

page 5

page 6

page 7

page 8


Fast SVM-based Feature Elimination Utilizing Data Radius, Hard-Margin, Soft-Margin

Margin maximization in the hard-margin sense, proposed as feature elimin...

∝SVM for learning with label proportions

We study the problem of learning with label proportions in which the tra...

Maximing the Margin in the Input Space

We propose a novel criterion for support vector machine learning: maximi...

Radius-margin bounds for deep neural networks

Explaining the unreasonable effectiveness of deep learning has eluded re...

Feature Learning Viewpoint of AdaBoost and a New Algorithm

The AdaBoost algorithm has the superiority of resisting overfitting. Und...

Neutralized Empirical Risk Minimization with Generalization Neutrality Bound

Currently, machine learning plays an important role in the lives and ind...

Deep Learning for Signal Demodulation in Physical Layer Wireless Communications: Prototype Platform, Open Dataset, and Analytics

In this paper, we investigate deep learning (DL)-enabled signal demodula...

Please sign up or login with your details

Forgot password? Click here to reset