[Bash-completion-devel] Running tests against multiple bash versions.

[Ville Skyttä]

On Saturday 16 January 2010, Crestez Dan Leonard wrote:
> I tried to run bash-completion tests against multiple versions of bash.

Cool!  Which platform was this on?

What I find surprising in your results is the difference between 3.1 and 3.2.  
Probably a direct result of people regularly testing with >= 3.2 only AFAIK.

> Did anyone else attempt this; perhaps with better results?

I'm running with:
- 4.0.23(1)-release on Linux (Fedora 11), regularly
- 4.0.33(0)-release on FreeBSD 8, sometimes
- 3.2.25(1)-release on Linux (CentOS 5), sometimes
- 3.00.16(1)-release on Solaris 10, sometimes

With better results, no.  Also my test coverage looks much smaller than yours, 
and the systems vary so they aren't directly comparable.  And as you noted, 
we're basically untestable with < 3.1 at the moment due to the printf -v 
issue.

bash-4.0.23(1)-release on Fedora:
# of expected passes            358
# of unexpected failures        2
# of expected failures          2
# of unresolved testcases       3
# of unsupported tests          165

Failures: dcop (will report a bug sometime, not necessarily an easy one), and 
finger (was discussed on this list in November, seems to have resurfaced or 
maybe it's a different issue, haven't looked more closely yet).

bash-4.0.33(0)-release on FreeBSD:
# of expected passes            316
# of unexpected failures        34
# of expected failures          2
# of unresolved testcases       5
# of unsupported tests          166

Failures: acroread, cdrecord -d, cpio -R, cvsps, env --, evince, expand, fmt 
-, fold --, gprof --, head --, kdvi, kldunload, kpdf, ls --, m4 --, 
pkg_deinstall, portinstall, ri, scp -F config, sed --, sftp -F config, sftp -F 
spaced\ \ conf, split --, sysctl fs, tail --, touch --, tr --, uname --, 
unexpand --, uniq --, units --, wc --, who --.  I'm slowly working though 
these.  Many of them are due to basic system command completions that use 
_longopt and the test suite expects option completions for them, but the 
commands on FreeBSD don't have long options (will post a separate thread on 
list soon about this).

bash-3.2.25(1)-release on CentOS:
# of expected passes            311
# of unexpected failures        5
# of expected failures          1
# of unresolved testcases       1
# of unsupported tests          185

Failures: cdrecord -d, finger, perldoc, ssh, yum.  Haven't looked into these 
yet at all.