[Pkg-exppsy-pynifti] [Nipy-devel] Data packaging, a proposal for feedback

[John Hunter]

On Wed, Aug 5, 2009 at 2:37 PM, Fernando Perez<fperez.net at gmail.com> wrote:
> Hi all,
>
> just as an FYI, this came out on the mpl-dev list:
>
> http://sourceforge.net/mailarchive/forum.php?thread_name=88e473830907311110w5b0097aai313d2c4c5dd6b165%40mail.gmail.com&forum_name=matplotlib-devel

This was introduced to solve a problem in that we want our example
code to "run anywhere" even if the data is not installed locally,
mainly so gallery users can run examples that require data.  I hacked
up something to grab from a dedicated svn dir using urllib, and Jouni
made some very nice enhancements, writing a custom urllib2 BaseHandler
that checks the "ETag" HTTP header for the svn revision number
(apparently something viewvc provides) to make a conditional http
request (grab only if the revision number has changed).  The class
dumps the file to a local cachedir, and reloads from there going
forward.  If the revision number changes, the cache is updated with
the new file.  All you need is urllib2 -- no svn dependency.

Anticipating someone might want to use this outside mpl, I refactored
the code this morning so it can be used independently from mpl's svn
repo or cachedir.  demo.py::

    import matplotlib.cbook as cbook
    cachedir = '/tmp/mydata'
    # replace this with your viewvc server
    baseurl = 'http://matplotlib.svn.sourceforge.net/viewvc/matplotlib/trunk/sample_data/'
    myserver = cbook.ViewVCCachedServer(cachedir, baseurl)

    datafile = 'testdir/subdir/testsub.csv'
    localfile = myserver.get_sample_data(datafile, asfileobj=False)
    print localfile

w/ the resulting output::

    johnh at udesktop191:~> python demo.py
    /tmp/mydata/testdir/subdir/testsub.csv

By default file handles are returned.

JDH