[Pkg-exppsy-maintainers] SVM+RFE vs. SMLR

Per B. Sederberg persed at princeton.edu
Wed Mar 5 20:30:24 UTC 2008

Howdy Yarik:

I've thought more about the RFE vs. SMLR test and I think it's more
clear to me why the SMLR starts to do worse as you pull out features
from what it already has pulled out itself.  Essentially, it has a set
of weights that it has learned to help it classify and then you start
pulling these "relevant" weights away.  So, unless you are redoing the
regression at each step, removing those features completely, you are
actually punishing the SMLR each time you pull out weights that it
thinks are valuable.

So, if this is the case, we will see equivalent performance between
the SVM+RFE and the SMLR when you get the parameters of SMLR set to
the values that will give rise to the same number of features as the
best SVM.  In fact, you may even see better performance because it
will be learning features that may have hidden correlations that might
help the classification.

Let me know :)


More information about the Pkg-exppsy-maintainers mailing list