[Pkg-ceph-commits] [ceph] 60/85: Add patch osd-limit-omap-data-in-push-op

[Gaudenz Steinlin]

This is an automated email from the git hooks/post-receive script.

gaudenz pushed a commit to branch jewel
in repository ceph.

commit 07aef441739e991b74054c5fcf702ffa32f5bb8f
Author: Gaudenz Steinlin <gaudenz at debian.org>
Date:   Wed Dec 14 16:53:46 2016 +0100

    Add patch osd-limit-omap-data-in-push-op
    
    Limit the amount of data per chunk in omap push operations to 64k,
    ensuring that OSD threads don't hit timeouts during recovery
    operations (LP: #1628750). Cherry picked from upstream master branch.
---
 .../patches/osd-limit-omap-data-in-push-op.patch   | 40 ++++++++++++++++++++++
 debian/patches/series                              |  1 +
 2 files changed, 41 insertions(+)

diff --git a/debian/patches/osd-limit-omap-data-in-push-op.patch b/debian/patches/osd-limit-omap-data-in-push-op.patch
new file mode 100644
index 0000000..1754ffe
--- /dev/null
+++ b/debian/patches/osd-limit-omap-data-in-push-op.patch
@@ -0,0 +1,40 @@
+From 56064c5cb645254574dbad00e6c16b783115bb93 Mon Sep 17 00:00:00 2001
+From: Wanlong Gao <wanlong.gao at easystack.cn>
+Date: Thu, 23 Jun 2016 20:04:25 +0800
+Subject: [PATCH] osd: limit omap data in push op
+
+We already have the config osd_recovery_max_chunk to limit the total
+size of omap entries and omap data. But we need an individual config
+to limit the number of omap entries independently. We call this config
+osd_recovery_max_omap_entries_per_chunk here with the default number
+of 64000.
+
+Signed-off-by: Wanlong Gao <wanlong.gao at easystack.cn>
+---
+ src/common/config_opts.h     | 1 +
+ src/osd/ReplicatedBackend.cc | 4 +++-
+ 2 files changed, 4 insertions(+), 1 deletion(-)
+
+--- a/src/common/config_opts.h
++++ b/src/common/config_opts.h
+@@ -749,6 +749,7 @@ OPTION(osd_recovery_delay_start, OPT_FLO
+ OPTION(osd_recovery_max_active, OPT_INT, 3)
+ OPTION(osd_recovery_max_single_start, OPT_INT, 1)
+ OPTION(osd_recovery_max_chunk, OPT_U64, 8<<20)  // max size of push chunk
++OPTION(osd_recovery_max_omap_entries_per_chunk, OPT_U64, 64000) // max number of omap entries per chunk; 0 to disable limit
+ OPTION(osd_copyfrom_max_chunk, OPT_U64, 8<<20)   // max size of a COPYFROM chunk
+ OPTION(osd_push_per_object_cost, OPT_U64, 1000)  // push cost per object
+ OPTION(osd_max_push_cost, OPT_U64, 8<<20)  // max size of push message
+--- a/src/osd/ReplicatedBackend.cc
++++ b/src/osd/ReplicatedBackend.cc
+@@ -1985,7 +1985,9 @@ int ReplicatedBackend::build_push_op(con
+ 	 iter->valid();
+ 	 iter->next(false)) {
+       if (!out_op->omap_entries.empty() &&
+-	  available <= (iter->key().size() + iter->value().length()))
++	  ((cct->_conf->osd_recovery_max_omap_entries_per_chunk > 0 &&
++	    out_op->omap_entries.size() >= cct->_conf->osd_recovery_max_omap_entries_per_chunk) ||
++	   available <= iter->key().size() + iter->value().length()))
+ 	break;
+       out_op->omap_entries.insert(make_pair(iter->key(), iter->value()));
+ 
diff --git a/debian/patches/series b/debian/patches/series
index 32564b1..226d788 100644
--- a/debian/patches/series
+++ b/debian/patches/series
@@ -15,3 +15,4 @@ tests-disable.patch
 fix-cycles-arch.patch
 skip-setup.py-makefiles.patch
 disable-openssl-linking.patch
+osd-limit-omap-data-in-push-op.patch

-- 
Alioth's /usr/local/bin/git-commit-notice on /srv/git.debian.org/git/pkg-ceph/ceph.git