[Pkg-haproxy-maintainers] Bug#767670: Segfault upon reloading haproxy

Vincent Bernat bernat at debian.org
Sat Nov 1 19:35:52 UTC 2014


Package: haproxy
Version: 1.5.6-1
Severity: important
Tags: patch

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hi!

Documenting an important bug in haproxy 1.5.6 which may segfault on
reload:

commit 44cf5450005c3bb822fce3c0971f515c470b678b
Author: Willy Tarreau <w at 1wt.eu>
Date:   Wed Oct 22 19:06:31 2014 +0200

    BUG/MAJOR: cli: explicitly call cli_release_handler() upon error
    
    Dmitry Sivachenko reported an embarrassing problem where haproxy
    would sometimes segfault upon reload. After careful analysis and
    code inspection, what happens is related to the "show sess" command
    on the CLI, and it is not limited to reload operations only.
    
    When a "show sess" is running, once the output buffer is full, the
    stats applet grabs a reference to the session being dumped in order
    for the current pointer to be able to advance by itself should this
    session disappear while the buffer is full. The applet also uses a
    release handler that is called when the applet terminates to release
    such references.
    
    The problem is that upon error, the command line parser sets the
    applet state to STAT_CLI_O_END indicating it wants to terminate the
    processing. Unfortunately, the release handler which is called later
    to clean everything up relies on the applet's state to know what
    operations were in progress, and as such it does not release the
    reference. A later "show sess" or the completion of the task being
    watched lead to a LIST_DEL() on the task's list which point to a
    location that does not match the applet's reference list anymore
    and the process dies.
    
    One solution to this would be to add a flag to the current applet's
    state mentionning it must leave, without affecting the state indicating
    the current operation. It's a bit invasive but could be the long term
    solution. The short term solution simply consists in calling the
    release handler just before changing the state to STAT_CLI_O_END.
    That way everything that must be released is released in time.
    
    Note that the probability to encounter this issue is very low.
    It requires a lot of "show sess" or "show sess all" calls, and
    that one of them dies before being completed. That can happen
    if "show sess" is run in scripts which truncate the output (eg:
    "echo show sess|socat|head"). This could be the worst case as it
    almost ensures that haproxy fills a buffer, grabs a reference and
    detects the error on the socket.
    
    There's no config-based workaround to this issue, except refraining
    from issuing "show sess" on large connection counts or "show sess all".
    If that's not possible to block everyone, restricting permissions on
    the stats socket ensures only authorized tools can connect.
    
    This fix must be backported to 1.5 and to 1.4 (with some changes in
    1.4 since the release function does not exist so the LIST_DEL sequence
    must be open-coded).
    
    Special thanks to Dmitry for the fairly complete report.

diff --git a/src/dumpstats.c b/src/dumpstats.c
index ebf66ecac55b..26b0a9f1d7a7 100644
- --- a/src/dumpstats.c
+++ b/src/dumpstats.c
@@ -131,6 +131,7 @@ static int stats_dump_stat_to_buffer(struct stream_interface *si, struct uri_aut
 static int stats_pats_list(struct stream_interface *si);
 static int stats_pat_list(struct stream_interface *si);
 static int stats_map_lookup(struct stream_interface *si);
+static void cli_release_handler(struct stream_interface *si);
 
 /*
  * cli_io_handler()
@@ -2336,6 +2337,7 @@ static void cli_io_handler(struct stream_interface *si)
 		}
 		else {	/* output functions: first check if the output buffer is closed then abort */
 			if (res->flags & (CF_SHUTR_NOW|CF_SHUTR)) {
+				cli_release_handler(si);
 				appctx->st0 = STAT_CLI_END;
 				continue;
 			}
@@ -2389,6 +2391,7 @@ static void cli_io_handler(struct stream_interface *si)
 					appctx->st0 = STAT_CLI_PROMPT;
 				break;
 			default: /* abnormal state */
+				cli_release_handler(si);
 				appctx->st0 = STAT_CLI_PROMPT;
 				break;
 			}


- -- System Information:
Debian Release: jessie/sid
  APT prefers unstable
  APT policy: (500, 'unstable'), (101, 'experimental')
Architecture: amd64 (x86_64)
Foreign Architectures: i386

Kernel: Linux 3.16-3-amd64 (SMP w/4 CPU cores)
Locale: LANG=fr_FR.utf8, LC_CTYPE=fr_FR.utf8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash

Versions of packages haproxy depends on:
ii  adduser              3.113+nmu3
ii  init-system-helpers  1.21
ii  libc6                2.19-12
ii  libpcre3             1:8.35-3.1
ii  libssl1.0.0          1.0.1j-1
ii  zlib1g               1:1.2.8.dfsg-2

haproxy recommends no packages.

Versions of packages haproxy suggests:
ii  haproxy-doc  1.5.6-1
pn  vim-haproxy  <none>

- -- no debconf information

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBCAAGBQJUVTYUAAoJEJWkL+g1NSX5CRQP/Ri21ETlvlV7JECHCcFhUibe
7IA5kDSd0MeDrBqw5a0G8ijtrP5uXJkGsB7PAi5QE80bG9jRPVd/4BJvcqM59LhU
+Uw3ESXG2mtDpN7ZQ3bWCocbMxv6X4yGKW99NCTFMMwGa5PirCAKGT2Rasp/Fy94
Ztoh6m99BtGZN//nYgtzoIydsYIozOlVn+7T/Gv3N7nzUeUrs1R4q7Wp5K+iIC/L
V3lB8GcZm+Vo5hoHeMZrp7uNrI8JSbHblUQGw08Vxxd3rNZ3fvhEtL+g6MIjYOoX
1lIlnJDNKIgE8A+0XTArO35QYdCv/tk9erCVihCR5FFc3goM/659CZLmjoYqBqY3
Khn/qmk0dawPHVdYW6dUtIXsQwfD85dXQjayecK7zV1lCH8yrVTJTbLn0SjgXoOQ
7lWXQjOXYeHDWpCKgROASTvk+5be5+5MdQf2AqqDcRoHup8sr9zpf9QQr6hMi5IB
LU1xsjh/H0SZsqj19gT0uTzKsXFUpaCMeWQSThZKXOsk+mOD/LX6l76pvkUOTX+o
IahOpHer0XiW6Wreg2aX7jcdK7lA09H/0w7XzvnwZVjPLjrQv3I/TE3IsbnK7Eop
OW9sO3jtn9KdU8hHn1ZnEgsrYetND+aqQDxLhEdFdoR6pX+MyiXrEBnF37Ani3DQ
FuHqoOjTC4xNQ8mon7TK
=ANYB
-----END PGP SIGNATURE-----



More information about the Pkg-haproxy-maintainers mailing list