[Pkg-exppsy-maintainers] SVM+RFE vs. SMLR

[Yaroslav Halchenko]

Plenty but just now found that I screwed up a bit: min error in the
labels in minimal not across means across runs - but global min, thus is
not what we see in the plots

Also damn legend covered some part but those are not interesting ones

Find all subjects plots for those SMLRs 
http://www.onerussian.com/Sci/analysis/pymvpa/smlrs1/

as I see ml=0.01 doesn't do good

going to try it with lm=1.5 and all 1e-5 (on all lms)

Another tiny new bit:
As you could see in git's yoh/master there is now
doc/examples/clfs_examples.py (could be renamed) the main purpose of
which is to serve an extended version of smlr_example benchmark

For now I just added only very dummy datasets (so avg train/test times are
kinda bogus and denominated ;-)) and basic classifiers, and here is
current output

$> doc/examples/clfs_examples.py
Dummy 2-class univariate with 2 useful features: <Dataset / float64 20 x 1000 uniq: 2 labels 5 chunks>
  Linear C-SVM (default)        : correct=65.0% train:0.0sec predict:0.0sec
  Linear nu-SVM (default)       : correct=65.0% train:0.0sec predict:0.0sec
  SMLR(default)                 : correct=90.0% train:0.1sec predict:0.0sec
  SMLR(Python)                  : correct=90.0% train:7.8sec predict:0.0sec
  RidgeReg(default)             : correct=50.0% train:6.8sec predict:0.0sec
  Rbf C-SVM (default)           : correct=60.0% train:0.0sec predict:0.0sec
  Rbf nu-SVM (default)          : correct=65.0% train:0.1sec predict:0.0sec
  kNN(default)                  : correct=55.0% train:0.0sec predict:0.0sec
Dummy XOR-pattern: <Dataset / float64 80 x 2 uniq: 2 labels 80 chunks>
  Linear C-SVM (default)        : correct=0.0% train:0.0sec predict:0.0sec
  Linear nu-SVM (default)       : correct=71.2% train:0.0sec predict:0.0sec
  SMLR(default)                 : correct=0.0% train:0.0sec predict:0.0sec
  SMLR(Python)                  : correct=0.0% train:0.0sec predict:0.0sec
  RidgeReg(default)             : correct=50.0% train:0.0sec predict:0.0sec
  Rbf C-SVM (default)           : correct=0.0% train:0.0sec predict:0.0sec
  Rbf nu-SVM (default)          : correct=97.5% train:0.0sec predict:0.0sec
  kNN(default)                  : correct=98.8% train:0.0sec predict:0.0sec


The goal is to extend with interesting data and evolved SVMs (ie SVM + RFE for
instance, or SVM + feature selection based on ANOVA/ SMLR's weights/SVM weights
but without RFE -- just plain non-0 or 1% of highest weights). That should
provide illustrative example of built-in ML techniques in hands we have here,
and provide easy assessment of efficiency in terms of computation time.

On Thu, 06 Mar 2008, Per B. Sederberg wrote:

> So, do you have the new results?
> P

> On Wed, Mar 5, 2008 at 3:59 PM, Yaroslav Halchenko
> <debian at onerussian.com> wrote:
> > > pulling these "relevant" weights away.  So, unless you are redoing the
> >  > regression at each step, removing those features completely, you are
> >  > actually punishing the SMLR each time you pull out weights that it
> >  > thinks are valuable.
> >  but we do retrain after each such feature removal,
> >  ie we don't simply prune the weight (set it to 0), we remove that
> >  feature from training data for a classifier, then retrain classifier.

> >  so we do fair job imho ;-) or have I misread your message?

> >  --
> >  Yaroslav Halchenko
> >  Research Assistant, Psychology Department, Rutgers-Newark
> >  Student  Ph.D. @ CS Dept. NJIT
> >  Office: (973) 353-5440x263 | FWD: 82823 | Fax: (973) 353-1171
> >         101 Warren Str, Smith Hall, Rm 4-105, Newark NJ 07102
> >  WWW:     http://www.linkedin.com/in/yarik

> >  _______________________________________________
> >  Pkg-exppsy-maintainers mailing list
> >  Pkg-exppsy-maintainers at lists.alioth.debian.org
> >  http://lists.alioth.debian.org/mailman/listinfo/pkg-exppsy-maintainers



-- 
Yaroslav Halchenko
Research Assistant, Psychology Department, Rutgers-Newark
Student  Ph.D. @ CS Dept. NJIT
Office: (973) 353-5440x263 | FWD: 82823 | Fax: (973) 353-1171
        101 Warren Str, Smith Hall, Rm 4-105, Newark NJ 07102
WWW:     http://www.linkedin.com/in/yarik