[linux] 01/01: splice: sendfile() at once fails for big files (Closes: #785189)
debian-kernel at lists.debian.org
debian-kernel at lists.debian.org
Mon Nov 23 01:32:36 UTC 2015
This is an automated email from the git hooks/post-receive script.
benh pushed a commit to branch wheezy
in repository linux.
commit a6eccd0a55128909299f5d3610d034427031c1e8
Author: Ben Hutchings <ben at decadent.org.uk>
Date: Sun Nov 22 23:43:52 2015 +0000
splice: sendfile() at once fails for big files (Closes: #785189)
---
debian/changelog | 6 +
...lice-sendfile-at-once-fails-for-big-files.patch | 132 +++++++++++++++++++++
debian/patches/series | 1 +
3 files changed, 139 insertions(+)
diff --git a/debian/changelog b/debian/changelog
index 28cb393..59156d6 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -1,3 +1,9 @@
+linux (3.2.73-2) UNRELEASED; urgency=medium
+
+ * splice: sendfile() at once fails for big files (Closes: #785189)
+
+ -- Ben Hutchings <ben at decadent.org.uk> Sun, 22 Nov 2015 23:43:42 +0000
+
linux (3.2.73-1) wheezy; urgency=medium
* New upstream stable update:
diff --git a/debian/patches/bugfix/all/splice-sendfile-at-once-fails-for-big-files.patch b/debian/patches/bugfix/all/splice-sendfile-at-once-fails-for-big-files.patch
new file mode 100644
index 0000000..eaa9596
--- /dev/null
+++ b/debian/patches/bugfix/all/splice-sendfile-at-once-fails-for-big-files.patch
@@ -0,0 +1,132 @@
+From: Christophe Leroy <christophe.leroy at c-s.fr>
+Date: Wed, 6 May 2015 17:26:47 +0200
+Subject: splice: sendfile() at once fails for big files
+Bug-Debian: https://bugs.debian.org/785189
+Origin: https://git.kernel.org/linus/0ff28d9f4674d781e492bcff6f32f0fe48cf0fed
+
+Using sendfile with below small program to get MD5 sums of some files,
+it appear that big files (over 64kbytes with 4k pages system) get a
+wrong MD5 sum while small files get the correct sum.
+This program uses sendfile() to send a file to an AF_ALG socket
+for hashing.
+
+/* md5sum2.c */
+#include <stdio.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <string.h>
+#include <fcntl.h>
+#include <sys/socket.h>
+#include <sys/stat.h>
+#include <sys/types.h>
+#include <linux/if_alg.h>
+
+int main(int argc, char **argv)
+{
+ int sk = socket(AF_ALG, SOCK_SEQPACKET, 0);
+ struct stat st;
+ struct sockaddr_alg sa = {
+ .salg_family = AF_ALG,
+ .salg_type = "hash",
+ .salg_name = "md5",
+ };
+ int n;
+
+ bind(sk, (struct sockaddr*)&sa, sizeof(sa));
+
+ for (n = 1; n < argc; n++) {
+ int size;
+ int offset = 0;
+ char buf[4096];
+ int fd;
+ int sko;
+ int i;
+
+ fd = open(argv[n], O_RDONLY);
+ sko = accept(sk, NULL, 0);
+ fstat(fd, &st);
+ size = st.st_size;
+ sendfile(sko, fd, &offset, size);
+ size = read(sko, buf, sizeof(buf));
+ for (i = 0; i < size; i++)
+ printf("%2.2x", buf[i]);
+ printf(" %s\n", argv[n]);
+ close(fd);
+ close(sko);
+ }
+ exit(0);
+}
+
+Test below is done using official linux patch files. First result is
+with a software based md5sum. Second result is with the program above.
+
+root at vgoip:~# ls -l patch-3.6.*
+-rw-r--r-- 1 root root 64011 Aug 24 12:01 patch-3.6.2.gz
+-rw-r--r-- 1 root root 94131 Aug 24 12:01 patch-3.6.3.gz
+
+root at vgoip:~# md5sum patch-3.6.*
+b3ffb9848196846f31b2ff133d2d6443 patch-3.6.2.gz
+c5e8f687878457db77cb7158c38a7e43 patch-3.6.3.gz
+
+root at vgoip:~# ./md5sum2 patch-3.6.*
+b3ffb9848196846f31b2ff133d2d6443 patch-3.6.2.gz
+5fd77b24e68bb24dcc72d6e57c64790e patch-3.6.3.gz
+
+After investivation, it appears that sendfile() sends the files by blocks
+of 64kbytes (16 times PAGE_SIZE). The problem is that at the end of each
+block, the SPLICE_F_MORE flag is missing, therefore the hashing operation
+is reset as if it was the end of the file.
+
+This patch adds SPLICE_F_MORE to the flags when more data is pending.
+
+With the patch applied, we get the correct sums:
+
+root at vgoip:~# md5sum patch-3.6.*
+b3ffb9848196846f31b2ff133d2d6443 patch-3.6.2.gz
+c5e8f687878457db77cb7158c38a7e43 patch-3.6.3.gz
+
+root at vgoip:~# ./md5sum2 patch-3.6.*
+b3ffb9848196846f31b2ff133d2d6443 patch-3.6.2.gz
+c5e8f687878457db77cb7158c38a7e43 patch-3.6.3.gz
+
+Signed-off-by: Christophe Leroy <christophe.leroy at c-s.fr>
+Signed-off-by: Jens Axboe <axboe at fb.com>
+---
+ fs/splice.c | 12 +++++++++++-
+ 1 file changed, 11 insertions(+), 1 deletion(-)
+
+--- a/fs/splice.c
++++ b/fs/splice.c
+@@ -1189,7 +1189,7 @@ ssize_t splice_direct_to_actor(struct fi
+ long ret, bytes;
+ umode_t i_mode;
+ size_t len;
+- int i, flags;
++ int i, flags, more;
+
+ /*
+ * We require the input being a regular file, as we don't want to
+@@ -1232,6 +1232,7 @@ ssize_t splice_direct_to_actor(struct fi
+ * Don't block on output, we have to drain the direct pipe.
+ */
+ sd->flags &= ~SPLICE_F_NONBLOCK;
++ more = sd->flags & SPLICE_F_MORE;
+
+ while (len) {
+ size_t read_len;
+@@ -1245,6 +1246,15 @@ ssize_t splice_direct_to_actor(struct fi
+ sd->total_len = read_len;
+
+ /*
++ * If more data is pending, set SPLICE_F_MORE
++ * If this is the last data and SPLICE_F_MORE was not set
++ * initially, clears it.
++ */
++ if (read_len < len)
++ sd->flags |= SPLICE_F_MORE;
++ else if (!more)
++ sd->flags &= ~SPLICE_F_MORE;
++ /*
+ * NOTE: nonblocking mode only applies to the input. We
+ * must not do the output in nonblocking mode as then we
+ * could get stuck data in the internal pipe:
diff --git a/debian/patches/series b/debian/patches/series
index dcd2af9..7e98551 100644
--- a/debian/patches/series
+++ b/debian/patches/series
@@ -1165,3 +1165,4 @@ bugfix/all/rds-fix-race-condition-when-sending-a-message-on-unbound-socket.patch
bugfix/all/isdn_ppp-add-checks-for-allocation-failure-in-isdn_p.patch
bugfix/all/ppp-slip-validate-vj-compression-slot-parameters-com.patch
bugfix/x86/kvm-svm-unconditionally-intercept-db.patch
+bugfix/all/splice-sendfile-at-once-fails-for-big-files.patch
--
Alioth's /usr/local/bin/git-commit-notice on /srv/git.debian.org/git/kernel/linux.git
More information about the Kernel-svn-changes
mailing list