[Ltrace-devel] The ABI branch

[Petr Machata]

Hi there,

there's a couple deficiencies in ltrace that could be tagged as ABI
problems.  On the branch pmachata/abi, I'm working towards fixing them.
The areas where ltrace is lacking are following:

- we assume that all interesting data fit into long.  That's not true on
  32-bit machines, where long will typically be 4 bytes, and double is
  8.  It's even less true if we want to support types like long long,
  long double, or even obscure types like 128-bit integers, float128 and
  others.  It's a matter of debate whether we do, but I argue that we do
  want to support double on 32-bit machines.

- we assume that structs are always passed by reference.

- to be able to support passing small structures by value, we, again,
  need support for "large" values (see the first point).  On x86_64 at
  least, small structures are passed in registers, so they need to be
  copied out (you can't form a pointer to register), and long won't be
  enough to hold them.

- on x86_64 in particular, the support for function calls that mix
  integral and floating-point arguments is rather weak.  I suspect that
  also on some other arches we use arg_num to figure out which register
  to access.  Some arches implement the register allocation machine, but
  definitely not all of them.

Most of the above is currently implemented in raw form on the ABI
branch.  E.g. for the following config line (this test is now in
parameters.exp):

  struct(float,char,char) func_struct_2(int, struct(array(char, 6),float), double);

Where we used to get this:

  func_struct_2(17, <struct without address>, 0.000000) = <struct without address>

We now get this:

  func_struct_2(17, { "ABCDE\0", 0.250000 }, 0.500000) = { 0.250000, 'B', 'C' }

It's all tested only on x86_64 as of now, and even on that, there are
known bugs, and the test suite doesn't cleanly pass yet.

A quick summary of changes:

- Arbitrary-sized values are implemented in value.c/.h (retiring of
  single common.h is, IMHO, long overdue).

- value_dict.c/.h contains code for maintaining set of values.  We need
  to ask for all arguments right away, in case you, for example, want to
  encode array length in an argument that comes after the array itself.
  Earlier the length-evaluating code would just ask e.g. gimme_arg(6),
  but the back end doesn't know where to look for arg 6, unless it has
  seen types of args 1-5 before that.  So we just fetch it all right
  away.  We do dereferencing and formatting in two bunches, like before.

- x86_64 back end was rewritten from scratch and now pretty closely
  implements the type classification algorithm described in the x86_64
  ABI document.  The syscall ABI piggy-backs on this, although there's
  never anything but integers and pointers in system calls.

- expr.c/.h contains code for evaluating simple expressions with array
  lengths.  This used to be a single int, and painfully not enough for
  the job.  Admittedly, this is not directly relevant to ABI problems,
  so I can extract it and post separately, if there is interest.

Thanks,
PM