[Pkg-ceph-commits] [ceph] 04/05: Limit the amount of data per chunk in omap push operations to 64k, ensuring that OSD threads don't hit timeouts during recovery operations (LP: #1628750):

[James Downing Page]

This is an automated email from the git hooks/post-receive script.

jamespage pushed a commit to branch ubuntu-xenial
in repository ceph.

commit 4ae0d3d3d831fe0bc8103b442b3b38cfc8ccb735
Author: James Page <james.page at ubuntu.com>
Date:   Thu Sep 29 09:34:25 2016 +0100

    Limit the amount of data per chunk in omap push operations to 64k, ensuring that OSD threads don't hit timeouts during recovery operations (LP: #1628750):
    
    * Limit the amount of data per chunk in omap push operations to 64k,
      ensuring that OSD threads don't hit timeouts during recovery
      operations (LP: #1628750):
      - d/p/osd-limit-omap-data-in-push-op.patch: Cherry pick fix from
        upstream master branch.
---
 debian/changelog                                   |  5 +++
 .../patches/osd-limit-omap-data-in-push-op.patch   | 40 ++++++++++++++++++++++
 debian/patches/series                              |  1 +
 3 files changed, 46 insertions(+)

diff --git a/debian/changelog b/debian/changelog
index d55bb13..a0e6175 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -3,6 +3,11 @@ ceph (10.2.3-0ubuntu0.16.04.1) UNRELEASED; urgency=medium
   * New upstream stable release (LP: #1628809).
     - d/p/*: Refresh.
     - d/p/rocksdb-flags.patch: Dropped, accepted upstream.
+  * Limit the amount of data per chunk in omap push operations to 64k,
+    ensuring that OSD threads don't hit timeouts during recovery
+    operations (LP: #1628750):
+    - d/p/osd-limit-omap-data-in-push-op.patch: Cherry pick fix from
+      upstream master branch.
 
  -- James Page <james.page at ubuntu.com>  Thu, 29 Sep 2016 09:28:57 +0100
 
diff --git a/debian/patches/osd-limit-omap-data-in-push-op.patch b/debian/patches/osd-limit-omap-data-in-push-op.patch
new file mode 100644
index 0000000..1754ffe
--- /dev/null
+++ b/debian/patches/osd-limit-omap-data-in-push-op.patch
@@ -0,0 +1,40 @@
+From 56064c5cb645254574dbad00e6c16b783115bb93 Mon Sep 17 00:00:00 2001
+From: Wanlong Gao <wanlong.gao at easystack.cn>
+Date: Thu, 23 Jun 2016 20:04:25 +0800
+Subject: [PATCH] osd: limit omap data in push op
+
+We already have the config osd_recovery_max_chunk to limit the total
+size of omap entries and omap data. But we need an individual config
+to limit the number of omap entries independently. We call this config
+osd_recovery_max_omap_entries_per_chunk here with the default number
+of 64000.
+
+Signed-off-by: Wanlong Gao <wanlong.gao at easystack.cn>
+---
+ src/common/config_opts.h     | 1 +
+ src/osd/ReplicatedBackend.cc | 4 +++-
+ 2 files changed, 4 insertions(+), 1 deletion(-)
+
+--- a/src/common/config_opts.h
++++ b/src/common/config_opts.h
+@@ -749,6 +749,7 @@ OPTION(osd_recovery_delay_start, OPT_FLO
+ OPTION(osd_recovery_max_active, OPT_INT, 3)
+ OPTION(osd_recovery_max_single_start, OPT_INT, 1)
+ OPTION(osd_recovery_max_chunk, OPT_U64, 8<<20)  // max size of push chunk
++OPTION(osd_recovery_max_omap_entries_per_chunk, OPT_U64, 64000) // max number of omap entries per chunk; 0 to disable limit
+ OPTION(osd_copyfrom_max_chunk, OPT_U64, 8<<20)   // max size of a COPYFROM chunk
+ OPTION(osd_push_per_object_cost, OPT_U64, 1000)  // push cost per object
+ OPTION(osd_max_push_cost, OPT_U64, 8<<20)  // max size of push message
+--- a/src/osd/ReplicatedBackend.cc
++++ b/src/osd/ReplicatedBackend.cc
+@@ -1985,7 +1985,9 @@ int ReplicatedBackend::build_push_op(con
+ 	 iter->valid();
+ 	 iter->next(false)) {
+       if (!out_op->omap_entries.empty() &&
+-	  available <= (iter->key().size() + iter->value().length()))
++	  ((cct->_conf->osd_recovery_max_omap_entries_per_chunk > 0 &&
++	    out_op->omap_entries.size() >= cct->_conf->osd_recovery_max_omap_entries_per_chunk) ||
++	   available <= iter->key().size() + iter->value().length()))
+ 	break;
+       out_op->omap_entries.insert(make_pair(iter->key(), iter->value()));
+ 
diff --git a/debian/patches/series b/debian/patches/series
index 5533f3f..b04f5ea 100644
--- a/debian/patches/series
+++ b/debian/patches/series
@@ -16,3 +16,4 @@ fix-argparse-defaults.patch
 fix-cycles-arch.patch
 skip-setup.py-makefiles.patch
 disable-openssl-linking.patch
+osd-limit-omap-data-in-push-op.patch

-- 
Alioth's /usr/local/bin/git-commit-notice on /srv/git.debian.org/git/pkg-ceph/ceph.git