[Ltrace-devel] ./ltrace: proc.c: 755: breakpoint_for_symbol: Assertion `bp->libsym == ((void *)0)' failed.

Thu Nov 8 17:21:27 UTC 2012

On Thu, Nov 8, 2012 at 5:12 PM, Edgar E. Iglesias
<edgar.iglesias at gmail.com> wrote:
> On Thu, Nov 08, 2012 at 04:27:54PM +0100, Sedat Dilek wrote:
>> On Thu, Nov 8, 2012 at 4:08 PM, Edgar E. Iglesias
>> <edgar.iglesias at gmail.com> wrote:
>> > On Thu, Nov 08, 2012 at 03:53:26PM +0100, Sedat Dilek wrote:
>> >> On Fri, Nov 2, 2012 at 11:58 PM, Petr Machata <pmachata at redhat.com> wrote:
>> >> > "Edgar E. Iglesias" <edgar.iglesias at gmail.com> writes:
>> >> >
>> >> >> On Fri, Nov 02, 2012 at 06:07:50PM +0100, Sedat Dilek wrote:
>> >> >>> This happens with latest ltrace-git...
>> >> >>>
>> >> >>> ./ltrace: proc.c: 755: breakpoint_for_symbol: Assertion `bp->libsym ==
>> >> >>> ((void *)0)' failed.
>> >> >>>
>> >> >> I've seen this bug, and I'm actually inclined to think that the assert
>> >> >> should go away and that we instead should somehow just not use that
>> >> >> break point. But not sure, maybe Petr has more input on this.
>> >> >
>> >> > If you assign more than one symbol to a breakpoint, and that breakpoint
>> >> > hits, there's no way to tell which of the symbols actually hit.  So
>> >> > there's no point storing more than one symbol to a breakpoint.  We could
>> >> > still drop the assert and just ignore the symbol, as you write.
>> >> >
>> >> > What might make sense is having several breakpoint handlers per
>> >> > breakpoint, so that ltrace allows several service breakpoints per
>> >> > address.  Software singlestepping in particular should work well even if
>> >> > it steps over another breakpoint, service or not.  (As should normal
>> >> > singlestepping, and I'm not positive that it does, but that's a
>> >> > different can of worms.)
>> >> >
>> >> >> The issue I've seen is with weak symbols that are made to point to
>> >> >> other symbols. I think, I saw it with strlen IIRC. In our glibc,
>> >> >> strlen points resolves to the same addr as __strlen but it's got
>> >> >> it's own PLT entry and symbol.
>> >> >
>> >> > Hmm, how about having two PLT slots, each for a different symbol, but
>> >> > those symbols are aliases in the target library.  Ltrace filters
>> >> > aliases, but only for purposes of -x.  For -e there's no way to tell
>> >> > that they end up being aliases.  And normally you don't care anyway, but
>> >> > on MIPS breakpoints are put not to PLT slots, but to destination
>> >> > addresses.  That's how you end up needing two breakpoints on the same
>> >> > address.  Could it be the cause?
>> >> >
>> >> > If yes, then the reproducer is as simple as -e func -x func.
>> >> >
>> >> >> I've got a local patch for it, but I unfortunately I haven't had much
>> >> >> time lately to look at it further. My plan was to create a test case
>> >> >> that hopefully can trig the issue on x86 aswell. Then we can work out
>> >> >> a solution from there.
>> >> >
>> >> > I guess on x86, one could trigger the same by IFUNC symbols, but those
>> >> > will need to be supported at all in the first place.
>> >> >
>> >>
>> >> Hi Edgar,
>> >>
>> >> Any news or even patch(es) around for this issue?
>> >>
>> >> I re-tested with ltrace-0.7.0-git1584cfc again to see if any new
>> >> changes did or did not break build on MIPSEL (own toolchain built with
>> >> Freetz build-system).
>> >> Building is fine, but the reported issue still remains.
>> >>
>> >> I can only help with more testing these days...
>> >
>> >
>> > Hi Sedak,
>> >
>> > This was the latest patch in my tree when I had to stop. I need to go
>> > through this before something can be submitted. But if you want to try it
>> > out, that would help.
>> >
>>
>> Test results attached.
>
> Thanks. Looks like it passed ok no?
>

Hmmm, not sure how to interpret all those debug-lines...

I see the output "blah".

$ grep -n -A1 -B1 'blah' debug/ltrace-L-x-main-debug-71-20121108-2.txt
1655-DEBUG: trace.c:164: continue_process: pid=2643
1656:blah
1657-DEBUG: events.c:144: next_event()

What's with this...?
"a.out" was generated via mipsel-cross-toolchain on my AMD64
Ubuntu/precise host.
IIRC the Freetz toolchain was built with no PLT relocs.
So, this seems to be OK.

949:DEBUG: ltrace-elf.c:476: /proc/2643/exe 4 PLT relocations
1553:DEBUG: ltrace-elf.c:476: ./a.out 4 PLT relocations
1560:DEBUG: ltrace-elf.c:433: /lib/libgcc_s.so.1 has no PLT relocations
1565:DEBUG: ltrace-elf.c:433: /lib/libc.so.0 has no PLT relocations
1570:DEBUG: ltrace-elf.c:433: /lib/ld-uClibc.so.0 has no PLT relocations

So, this line is the most interesting...

$ grep -n 'breakpoint_for_symbol' debug/ltrace-L-x-main-debug-71-20121108-2.txt
1557:breakpoint_for_symbol: bp_addr=0x400830 bp->libsym=0x493338
libsym=0x493198 main main at a.out

A bit more lines...

$ grep -A3 -B3 'breakpoint_for_symbol'
debug/ltrace-L-x-main-debug-71-20121108-2.txt
DEBUG: ltrace-elf.c:488: do_close_elf()
DEBUG: proc.c:836: added library a.out at 0x400000 (./a.out) to 2643
DEBUG: dict.c:160: dict_find_entry()
breakpoint_for_symbol: bp_addr=0x400830 bp->libsym=0x493338
libsym=0x493198 main main at a.out
DEBUG: ltrace-elf.c:342: do_init_elf(filename=/lib/libgcc_s.so.1)
DEBUG: ltrace-elf.c:343: Reading ELF from /lib/libgcc_s.so.1...
DEBUG: ltrace-elf.c:433: /lib/libgcc_s.so.1 has no PLT relocations

SingleStep-Tracing is expected?

1600:DEBUG: trace.c:679: all stopped, now singlestep/cont 2643

What's with those main() "no return" and "resumed" lines?

$ egrep -A3 -B3 'no return|resumed'
debug/ltrace-L-x-main-debug-71-20121108-2.txt
DEBUG: dict.c:124: new dict entry at 0x4b8590[730]: (0x2ab33320,0x493558)
DEBUG: breakpoint.c:85: enable_breakpoint: pid=2643, addr=0x2ab33320,
symbol=(null)
DEBUG: breakpoint.c:48: arch_enable_breakpoint: pid=2643,
addr=0x2ab33320, symbol=(null)
main(1, 0x7ff9f9f4, 0x7ff9f9fc, 0 <no return ...>
DEBUG: trace.c:901: continue_after_breakpoint: pid=2643, addr=0x400830
DEBUG: trace.c:856: process_install_stopping_handler: pid=2643
DEBUG: proc.c:707: install_event_handler(pid=2643, 0x492f38)
--
DEBUG: handle_event.c:90: handle_event(pid=2643, type=11)
DEBUG: handle_event.c:178: [2643] event: breakpoint 0x2ab33320
DEBUG: handle_event.c:600: handle_breakpoint(pid=2643, addr=0x2ab33320)
<... main resumed> )                             = 5
DEBUG: handle_event.c:742: callstack_pop(pid=2643)
DEBUG: breakpoints.c:248: delete_breakpoint(pid=2643, addr=0x2ab33320)
DEBUG: dict.c:160: dict_find_entry()

Just stepping in the dark :-).

Have a nice tea,
- Sedat -

> Cheers
>
>