[Teammetrics-discuss] Updating commitstats

[Andreas Tille]

Hi,

On Fri, Apr 19, 2013 at 12:21:25AM -0400, Sukhbir Singh wrote:
> 
> If you remember, we save the state of the commits that have been parsed,
> so that we don't have to parse them again. For the SVN commits, this
> data is stored on vasks.debian.org.

It might be me that I do not remember.  However, I do think that storing
status on several places is (obviously) the wrong approach.  If we build
a database on two different hosts (say for testing purpose or as we do
now an initial import) this will break import data.  Simply assume I
would create a test database instance at home and will run a daily
import.  The consequence would be that the status on vasks.d.o will be
updated daily and if our production import runs at the beginning of a
month it will see no new commits.  That's broken design.

The consequence needs to be that we do the housekeeping *inside* each
database because the housekeeping and the data belong together and it
actually needs to be done in a *transaction* to make sure that the
housekeeping will fit the data status exactly.

> Now, when you ran commistat from
> scratch, we didn't clear the state on vasks, so it was not parsing the
> commits that had already been parsed.
> 
> I cleared the file on vasks and ran commitstat. Everything looks fine
> now; this should make you happy:
> 
>             name             | count 
> -----------------------------+-------
>  Andreas Tille               |  4612
>  Charles Plessy              |  3254
>  Mathieu Malaterre           |  1614
>  Steffen Möller              |  1005
> 
> :)

Well, as I said, the numbers do fit *now* but the method is not safe and
needs to be fixed anyway.

> (Only the state for svnstat.py is saved on vasks, everything else is
> local.)

So the very quick hack to cure the situation above would be to also
store the svn data in /var/cache/teammetrics.

The "real" solution would probably as I mentioned briefly in my past
mail that we need to store also these data inside the database rather
than in /var/cache/teammetrics.  This would enable us to do clean
backups of the database.  OK, with some proper backup method we could
also keep the dir /var/cache/teammetrics - but hmmm, I'm somehow lacking
the motivation to keep one part of the data in files and the other part
inside the database.
 
> I have updated the graphs.

Thanks for this anyway.

BTW, in the debian-l10n team there need to be some names adjusted.
There is some Nicolas_F and Nicolas_F? as well as a user fzt.  In
debian-science there is sebastien-guest and sebastien and barbier-guest.

Kind regards

         Andreas.

-- 
http://fam-tille.de