A generalized quadratic loss for SVM and Deep Neural Networks

by   Filippo Portera, et al.

We consider some supervised binary classification tasks and a regression task, whereas SVM and Deep Learning, at present, exhibit the best generalization performances. We extend the work [3] on a generalized quadratic loss for learning problems that examines pattern correlations in order to concentrate the learning problem into input space regions where patterns are more densely distributed. From a shallow methods point of view (e.g.: SVM), since the following mathematical derivation of problem (9) in [3] is incorrect, we restart from problem (8) in [3] and we try to solve it with one procedure that iterates over the dual variables until the primal and dual objective functions converge. In addition we propose another algorithm that tries to solve the classification problem directly from the primal problem formulation. We make also use of Multiple Kernel Learning to improve generalization performances. Moreover, we introduce for the first time a custom loss that takes in consideration pattern correlation for a shallow and a Deep Learning task. We propose some pattern selection criteria and the results on 4 UCI data-sets for the SVM method. We also report the results on a larger binary classification data-set based on Twitter, again drawn from UCI, combined with shallow Learning Neural Networks, with and without the generalized quadratic loss. At last, we test our loss with a Deep Neural Network within a larger regression task taken from UCI. We compare the results of our optimizers with the well known solver SVMlight and with Keras Multi-Layers Neural Networks with standard losses and with a parameterized generalized quadratic loss, and we obtain comparable results.


page 1

page 2

page 3

page 4


Generalized version of the support vector machine for binary classification problems: supporting hyperplane machine

In this paper there is proposed a generalized version of the SVM for bin...

Towards Searching Efficient and Accurate Neural Network Architectures in Binary Classification Problems

In recent years, deep neural networks have had great success in machine ...

Learning rates for classification with Gaussian kernels

This paper aims at refined error analysis for binary classification usin...

GADGET SVM: A Gossip-bAseD sub-GradiEnT Solver for Linear SVMs

In the era of big data, an important weapon in a machine learning resear...

Quadratic models for understanding neural network dynamics

In this work, we propose using a quadratic model as a tool for understan...

Quadratically constrained quadratic programming for classification using particle swarms and applications

Particle swarm optimization is used in several combinatorial optimizatio...

SVM-based Deep Stacking Networks

The deep network model, with the majority built on neural networks, has ...

Please sign up or login with your details

Forgot password? Click here to reset