[Ltrace-devel] [PATCH 0/1] Attempt to fix libdl tracing when attaching to PIDs

[Joe Damato]

On Tue, Mar 19, 2013 at 6:25 PM, Petr Machata <pmachata at redhat.com> wrote:
> Joe Damato <ice799 at gmail.com> writes:
>
>> There is something odd happening that I need to investigate a bit
>> more. I've noticed when trying to use ltrace to trace symbols loaded
>> with dlsym, only -x seems to actually work. I think this is because of
>> where and how plt_filter and static_filter are examined in the
>> codebase. I tried to re-arrange them and some surrounding code, but I
>> ended up breaking existing tests for -e.
>
> This may be obvious, but note that -e really only matches PLT entries,
> so you wouldn't see any actual calls from libm (unless they were really
> done via PLT slots, like for malloc and free).

Right, I see. I misunderstood why/when -e should be used.

>> I wrote a test program that dlopen's libm, and then uses dlsym to find
>> "acos" and calls the function after a short sleep so I can attach
>> ltrace.
>
> [...]
>
>> joe at ubuntu:~/code/ltrace$ /tmp/test/usr/local/bin/ltrace -e* -p 2821
>> libm.so.6->(0x7fff07366f90, 0x7fff07366f90, 0, -1)                                                      = 0x7f0384fe06a0
>> libdl.so.2->__cxa_finalize(0x7f038582f078, 0, 0xffffffff, 0x7f038582ed50)                               = 0x7f0385627350
>> libm.so.6->(0x7f038526c0c0, 0, 0, 3)                                                                    = 0x7f0385627350
>> +++ exited (status 0) +++
>
> Suspicious.  This doesn't show up at my machine.  Valgrind is clean as
> well.  we used to have a hack in ltrace that produced "anonymous" calls
> like this, but that's been gone since before 0.7.
>
> They shouldn't be the acos calls.  dlsym doesn't return a PLT slot
> address, it returns the actual symbol, so the calls end up being direct.

Ah, right of course.

> I'm seeing the following:
>
> $ ./a.out & ~/proj/ltrace/master/64/ltrace -p $(jobs -p) -e*
> [1] 15103
> my pid: 15103
> libdl.so.2->__cxa_finalize(0x7fb12de47d78, 0, 0xffffffff, 0x7fb12de47d48)                        = 0x7fb12dc3f9d0
> libm.so.6->__cxa_finalize(0x7fb12d8a9de0, 0, 0, 3)                                               = 0x7fb12dc3f9d0
> +++ exited (status 0) +++
>
> You have two calls from libm, I only have one.  The latter might me "my"
> __cxa_finalize, but no clue what the former could be, or why the symbols
> are not initialized.  When I trace with -x*@libm*, I get:
>
> acos at libm.so.6(0x7fff59585930, 0x7fff59585930, 0, -1 <unfinished ...>
> __ieee754_acos at libm.so.6(0x7fff59585930, 0x7fff59585930, 0, -1)                                  = 0x7f3017209500
> isnan at libm.so.6(1059, 1066, 0x3fe00000, 1067)                                                    = 0
> <... acos resumed> )                                                                             = 0
>
> Is perhaps your libm somehow calling one of these via a PLT slot?  In my
> libm, both calls are direct:
>
> 00000000000242d0 <acos>:
> [...]
>    242dd:       e8 0e 3b fe ff          callq  7df0 <__ieee754_acos>
> [...]
>    24301:       e8 2a 42 00 00          callq  28530 <__GI___isnan>

Appears that my libm does go through the PLT for 2 things:

0000000000025ca0 <acos>:
[...]
   25cb6:       e9 f5 f7 fd ff          jmpq   54b0 <*ABS*+0x10590 at plt>
[...]
   25cd7:       e9 64 fc fd ff          jmpq   5940 <*ABS*+0x1fc30 at plt+0x3e0>

> Or, isn't your acos an IFUNC symbol?  readelf would tell you.  I'm not
> at all sure about the interaction between IFUNC and dlsym, but IFUNC
> symbols as such are not supported by ltrace at all.

Nope, not an IFUNC.

> You might also want to look into symbol name initialization,
> i.e. ltrace-elf.c.  Maybe they are mis-loaded to begin with.  That's
> where the filters are applied as well: clearly that part of the code
> thinks that those symbols are worth tracing, and adds them to symbol
> table, otherwise ltrace wouldn't even trace them.  But either it fails
> to initialize the names, or they get lost somewhere later.

Thanks for digging into this with me.

FWIW, I think the patch and test you wrote for fixing the dlsym/attach
bug look great and can be merged in whenever you feel it's ready.

Joe