Bug#378721: vim-lesstif: gvim always crashes during startup

[Ben Hutchings]

The function in which the crash occurs is:

Boolean
XmWidgetGetBaselines(Widget w, Dimension **baselines, int *line_count)
{
    if (XmIsLabel(w))
    {
	XmPrimitiveClassExt *extptr;

	extptr = _XmGetPrimitiveClassExtPtr(XtClass(w), NULLQUARK);

	if (extptr && *extptr && (*extptr)->widget_baseline)
	{
	    return ((*extptr)->widget_baseline) (w, baselines, line_count);
	}
    }
    else if (XmIsLabelGadget(w))
    {
	XmGadgetClassExt *extptr;

	extptr = _XmGetGadgetClassExtPtr(XtClass(w), NULLQUARK);

	if (extptr && *extptr && (*extptr)->widget_baseline)
	{
	    return ((*extptr)->widget_baseline) (w, baselines, line_count);
	}
    }

    return False;
}

In this case the widget class is XmEnhancedButton, which is
vim-lesstif's subclass of XmPushButton, which is a subclass of XmLabel.
extptr ends up pointing to a structure with garbage in the
widget_baseline and following fields; I suspect it's some other sort of
extension record and not an XmPrimitiveClassExtRec.  (The backtrace in
gdb shows that we're in the second conditional block, despite the fact
the widget meets the first condition.  This is because at optimisation
level 2 gcc combines the common code of the two blocks and the debug
information doesn't show that there are two possible line numbers.)

XmEnhancedButton doesn't initially have an XmPrimitiveClassExtRec of its
own (primitive_class.extension is NULL) so I'm not sure how it gets one.
Using a copy of the record from XmPushButton works around the bug in
vim:

--- vim70/src/gui_xmebw.c.orig	2006-08-27 01:28:52.000000000 +0000
+++ vim70/src/gui_xmebw.c	2006-08-27 01:30:04.000000000 +0000
@@ -138,6 +138,17 @@
     }
 };
 
+XmPrimitiveClassExtRec xmEnhancedButtonPrimClassExtRec =
+{
+    /* next_extension      */ NULL,
+    /* record_type         */ NULLQUARK,
+    /* version             */ XmPrimitiveClassExtVersion,
+    /* record_size         */ sizeof(XmPrimitiveClassExtRec),
+    /* widget_baseline     */ XmInheritBaselineProc,
+    /* widget_display_rect */ XmInheritDisplayRectProc,
+    /* widget_margins      */ NULL
+};
+
 XmEnhancedButtonClassRec xmEnhancedButtonClassRec =
 {
     {
@@ -184,7 +195,7 @@
 	/* arm and activate	 */ XmInheritArmAndActivate,
 	/* synthetic resources	 */ NULL,
 	/* number of syn res	 */ 0,
-	/* extension		 */ NULL,
+	/* extension		 */ (XtPointer)&xmEnhancedButtonPrimClassExtRec,
     },
 
     /* label_class fields */
-- END --

Ben.

-- 
Ben Hutchings -- ben at decadentplace.org.uk shortened to ben at decadent.org.uk
If you've signed my GPG key, please send a signature on and to the new uid.
I'm not a reverse psychological virus.  Please don't copy me into your sig.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://lists.alioth.debian.org/pipermail/pkg-vim-maintainers/attachments/20060827/8e41b901/attachment.pgp