[Collab-qa-commits] r2276 - archive-rebuilds

[Lucas Nussbaum]

Author: lucas
Date: 2012-05-07 09:50:54 +0000 (Mon, 07 May 2012)
New Revision: 2276

Added:
   archive-rebuilds/fetch-and-process-results-aws
Log:
fetch results from aws

Added: archive-rebuilds/fetch-and-process-results-aws
===================================================================
--- archive-rebuilds/fetch-and-process-results-aws	                        (rev 0)
+++ archive-rebuilds/fetch-and-process-results-aws	2012-05-07 09:50:54 UTC (rev 2276)
@@ -0,0 +1,50 @@
+#!/usr/bin/ruby -w
+# script to fetch the result from f38, copy them to gluck, and prepare the
+# local results list
+#
+# needs:
+# export DATE=2007/08/06
+#
+DATE=ENV['DATE']
+ARCH=ENV['ARCH']
+CHROOT=ENV['CHROOT']
+DDATE=DATE.gsub('/', '-')
+ID="#{CHROOT}-#{ARCH}.#{DDATE}"
+TARGETDIR="#{DDATE}-#{CHROOT}-#{ARCH}"
+
+puts "parsing logs..."
+system "ssh ec2 \"cd /tmp/logs && cqa-scanlogs > ../res.#{ID}\""
+puts "building log list..."
+system "ssh ec2 \"cd /tmp && cat res.#{ID} | grep -v ' OK ' | sed 's/ [0-9]*:/ /' | awk ' { print \\$1 \\\"_\\\" \\$2 \\\"_#{CHROOT}.log\\\" } ' > /tmp/loglist.#{ID} \""
+puts "transfering logs to ravel ..."
+system "ssh -A ec2 \"ssh lucas at ravel.debian.org mkdir -p public_html/logs/#{DATE}\""
+system "ssh -A ec2 \"cd /tmp/logs/ && rsync -avzP --files-from=/tmp/loglist.#{ID} . lucas at ravel.debian.org:public_html/logs/#{DATE}\""
+system "ssh -A ec2 \"ssh lucas at ravel.debian.org chmod -R a+rX public_html/logs/#{DATE}\""
+
+system "mkdir #{TARGETDIR}"
+Dir::chdir(TARGETDIR)
+puts "copying results file locally..."
+system "ssh ec2 \"cat /tmp/res.#{ID} | sort\" > fullresults.#{DDATE}.txt"
+system "grep -v ' OK ' fullresults.#{DDATE}.txt > failed.#{DDATE}.tmp"
+puts ""
+puts "now, merge old results in new file:"
+puts "  cd #{TARGETDIR}"
+puts "  ../merge-results.rb OLDRESULTFILE failed.#{DDATE}.tmp > failed.#{DDATE}.txt"
+puts ""
+puts "  mkdir /tmp/cqa.#{ARCH}.#{DDATE}"
+puts "  rsync -avzP lucas at ravel.debian.org:public_html/logs/#{DATE}/ /tmp/cqa.#{ARCH}.#{DDATE}/"
+
+Dir::chdir('..')
+File::open("#{TARGETDIR}/README", "w") do |f|
+  f.puts "Merging old results in new file:"
+  f.puts "  ../merge-results.rb OLDRESULTFILE failed.#{DDATE}.tmp > failed.#{DDATE}.txt"
+  f.puts "  e.g ../merge-results.rb ../2011-07-18-lsid64-amd64/failed.txt failed.#{DDATE}.tmp > failed.#{DDATE}.txt"
+  f.puts
+  f.puts "Retrieving all logs:"
+  f.puts "  mkdir /tmp/cqa.#{ARCH}.#{DDATE}"
+  f.puts "  rsync -avzP ravel.debian.org:~lucas/public_html/logs/#{DATE}/ /tmp/cqa.#{ARCH}.#{DDATE}/"
+  f.puts
+  f.puts "or: (non-DD version)"
+  f.puts "  cd /tmp/cqa.#{ARCH}.#{DDATE}/"
+  f.puts "  wget -r -np -nd http://people.debian.org/~lucas/logs/#{DATE}/"
+end


Property changes on: archive-rebuilds/fetch-and-process-results-aws
___________________________________________________________________
Added: svn:executable
   + *