<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
</head>
<body text="#000000" bgcolor="#ffffff">
Hi,<br>
<br>
(writing the same in French, you'll get it in a few minutes)<br>
<br>
Where I work now, we focus very much on tests and having the tests
covering all the source code, only it is in Ada, not Lisaac (though
they are interested in Lisaac).<br>
<br>
<br>
<br>
I discovered thanks to them Cucumber, it is a tool to make tests
easier.<br>
<a href="http://cukes.info/">http://cukes.info/</a><br>
<br>
My idea would be to create tests that would be a specification of the
Lisaac language, the behaviour reference if you want.<br>
<br>
The specificity of Cucumber is that the tests are written in English
and are called features. That makes them easy to understand for
everyone. That is also a great way to write specifications that two
people can agree with (even over a contract).<br>
Example of features: (files with .feature suffix)<br>
<a
href="http://git.debian.org/?p=lisaac/compiler.git;a=tree;f=features;hb=HEAD">http://git.debian.org/?p=lisaac/compiler.git;a=tree;f=features;hb=HEAD</a><br>
<br>
<b>Explanation of features, how to read them</b><br>
<br>
(look at how a feature file is structured before continuing)<br>
<ul>
<li>The first part of a feature (after "Feature:") is descriptive
only</li>
<li>Then, the scenarios are run one after another. The background
is run before each feature to set up the environment.</li>
<li>In scenarios (and in the background), actions to be done are
prefixed by one of the keywords "Given", "When", "Then". The special
keyword "And" is a way to repeat the keyword of the previous line.</li>
<li>"Given" lines set up the environment</li>
<li>"When" lines run actions</li>
<li>"Then" lines check the result of those actions</li>
</ul>
<br>
Each of the lines match some ruby code to be found in the
features/step_definitions directory. A regular expression is matched
against the lines and matches are extracted. Then ruby code is
executed.<br>
<br>
<b>Run the features</b><br>
<br>
To install cucumber, you would need ruby gems (I heard that on
Debian/Ububntu, the ruby gems weren't working properly, you might
have to install ruby-gems manually as my coworker had to). Then you
can simply install cucumber using gem:<br>
gem install cucumber<br>
gem install rake<br>
<br>
And you can start playing with cucumber, for example, in the compiler
directory, just run:<br>
<br>
cucumber features/*.feature<br>
<br>
And all the features in the feature directory will start (and you get
a report).<br>
<br>
<b>Code coverage</b><br>
<br>
Now, tests are useful, but they are even more so when the cover the
complete source code and do not left some part of the software
untested. That's why there are plenty of tools for plenty of
languages to check the code coverage.<br>
<br>
Unfortunately there is none for Lisaac.<br>
<br>
Thinking about it, I don't think that creating a code coverage tool
for Lisaac would be that hard. First, let's look at how code coverage
tool works:<br>
<br>
<ol>
<li>You compile the code to add instrumentation</li>
<li>You run the instrumented executable</li>
<li>The executable creates files containing the code coverage
information</li>
<ul>
<li>Which lines were executed</li>
<li>Which lines weren't executed</li>
<li>Which lines cannot be executed (comment, blank lines ...)<br>
</li>
</ul>
<li>You run a tool that would convert the files generated by the
executable into useful information, HTML reports...<br>
</li>
</ol>
So, that got me thinking, how can we generate coverage information.
Lisaac makes it easy because it strictly forbid branching within a
{BLOCK} or in a (list). So we only have to record once in a block
that the execution flow got here and we know the whole block or list
was executed.<br>
<br>
Now, trickier, how to differentiate between blocks and lists that are
not executed, and parts of the code that are not source code. This
involves the parser. Each time it finds a block or a list, it has to
record it somewhere so that we can match blocks lists executed with
the complete set of blocks and lists.<br>
<br>
<br>
<br>
The way I see it is like that: When the parser find a block (or list)
it record it in a list with its beginning position and end position
(to match with the source code afterwards). It also given them an id.
And the compiler will end up storing all that in a file for future
use.<br>
<br>
The parser would also add an instruction at the beginning of a block
(or list). That instruction would be a call to a function (either
using a C external, quick and easy, or using Lisaac code, more
flexible). The parameters to this function is an id of the block (or
list).<br>
<br>
This function will, when the executable runs, gather the list of all
executed blocks (or lists) retaining their id, and when the program
exit (or on the fly) write them on a file.<br>
<br>
Then, we run a tool to match the two files, and we are done.<br>
<br>
<b>Conclusion</b><br>
<br>
That way, we can have a very robust compiler, and make sure that
tests are effective.<br>
<br>
It can also be very useful to remove dead code that is no longer
used, I think Benoit will like this side of things.<br>
<br>
<br>
Mildred<br>
<br>
<pre class="moz-signature" cols="72">--
Mildred Ki'Lya
╭───────── ｍｉｌｄｒｅｄ５９３＠ｏｎｌｉｎｅ．ｆｒ ──────────
│ Jabber, GoogleTalk: <a class="moz-txt-link-rfc2396E" href="mailto:mildred@jabber.fr"><mildred@jabber.fr></a>
│ Website: <a class="moz-txt-link-rfc2396E" href="http://ki.lya.online.fr"><http://ki.lya.online.fr></a> GPG ID: 9A7D 2E2B
│ Fingerprint: 197C A7E6 645B 4299 6D37 684B 6F9D A8D6 9A7D 2E2B
</pre>
</body>
</html>