Where academic tradition
meets the exciting future

You are here: TUCS > PUBLICATIONS > Publication Search > Feature Selection for Regulari...

Feature Selection for Regularized Least-Squares: New Computational Short-Cuts and Fast Algorithmic Implementations

Tapio Pahikkala, Antti Airola, Tapio Salakoski, Feature Selection for Regularized Least-Squares: New Computational Short-Cuts and Fast Algorithmic Implementations. In: Samuel Kaski, David J. Miller, Erkki Oja, Antti Honkela (Eds.), Proceedings of the IEEE International Workshop on Machine Learning for Signal Processing (MLSP 2010), 295–300, IEEE, 2010.

Abstract:

We propose novel computational short-cuts for constructing sparse linear predictors with regularized least-squares (RLS), also known as the least-squares support vector machine or ridge regression. The short-cuts make it possible to accelerate the search in the power set of features with leave-one-out criterion as a search heuristic. Our first short-cut finds the optimal search direction in the power set. The direction means either adding a new feature into the set of selected features or removing one of the previously added features. The second short-cut updates the set of selected features and the corresponding RLS solution according to a given direction. The computational complexities of both short-cuts are O(mn), where m and n are the numbers of training examples and features, respectively. The short-cuts can be used with various different feature selection strategies. As case studies, we present efficient implementations of greedy and floating forward feature selection algorithm for RLS.

Files:

Full publication in PDF-format

BibTeX entry:

@INPROCEEDINGS{inpPaAiSa10a,
  title = {Feature Selection for Regularized Least-Squares: New Computational Short-Cuts and Fast Algorithmic Implementations},
  booktitle = {Proceedings of the IEEE International Workshop on Machine Learning for Signal Processing (MLSP 2010)},
  author = {Pahikkala, Tapio and Airola, Antti and Salakoski, Tapio},
  editor = {Kaski, Samuel and Miller, David J. and Oja, Erkki and Honkela, Antti},
  publisher = {IEEE},
  pages = {295–300},
  year = {2010},
  keywords = {Regularized least-squares, feature selection, computational complexity, least-squares support vector machine, ridge regression, greedy algorithm},
}

Belongs to TUCS Research Unit(s): Algorithmics and Computational Intelligence Group (ACI), Turku BioNLP Group

Publication Forum rating of this publication: level 1

Edit publication