[Pkg-ganeti-devel] Bug#875710: ganeti: initialization times out because services don't start up

Martin Weinelt martin at darmstadt.freifunk.net
Wed Sep 13 20:27:08 UTC 2017 

Package: ganeti
Version: 2.15.2-10
Severity: important

Dear Maintainer,

I have ganeti 2.15.2-10 from sid installed on a stretch system.

Initialization fails reliably, because the masterd isn't coming up,
making the process timeout.

Retrying the init says the cluster is already initialized. Restarting the ganeti service no daemon comes up.

```
root at sleepy:/home/hexa# gnt-cluster init --debug --no-etc-hosts --master-netdev=br-vlan30 --vg-name=system --enabled-hypervisors=kvm --hypervisor-parameters kvm:initrd_path=,kernel_path= --enabled-disk-templates=plain,drbd --ssh-key-type=rsa snowwhite.darmstadt.freifunk.net
2017-09-13 22:02:29,956: gnt-cluster init pid=35793 cli:1218 DEBUG Command line: gnt-cluster init --debug --no-etc-hosts --master-netdev=br-vlan30 --vg-name=system --enabled-hypervisors=kvm --hypervisor-parameters kvm:initrd_path=,kernel_path= --enabled-disk-templates=plain,drbd --ssh-key-type=rsa snowwhite.darmstadt.freifunk.net
2017-09-13 22:02:29,956: gnt-cluster init pid=35793 node:93 INFO Using PycURL PycURL/7.43.0 libcurl/7.52.1 GnuTLS/3.5.8 zlib/1.2.8 libidn2/0.16 libpsl/0.17.0 (+libidn2/0.16) libssh2/1.7.0 nghttp2/1.18.1 librtmp/2.3
2017-09-13 22:02:29,959: gnt-cluster init pid=35793 netutils:326 DEBUG Attempting to reach TCP port 1811 on target 192.168.30.254 with a timeout of 5 seconds
2017-09-13 22:02:33,023: gnt-cluster init pid=35793 process:221 INFO RunCmd vgs --noheadings --units m --nosuffix -o name,size
2017-09-13 22:02:33,042: gnt-cluster init pid=35793 bootstrap:607 DEBUG Stopping daemons (if any are running)
2017-09-13 22:02:33,043: gnt-cluster init pid=35793 process:221 INFO RunCmd /usr/lib/ganeti/daemon-util stop-all
2017-09-13 22:02:33,150: gnt-cluster init pid=35793 __init__:833 WARNING Specified a file storage directory, although file storage is not enabled.
2017-09-13 22:02:33,150: gnt-cluster init pid=35793 __init__:833 WARNING Specified a sharedfile storage directory, although sharedfile storage is not enabled.
2017-09-13 22:02:33,151: gnt-cluster init pid=35793 __init__:833 WARNING Specified a gluster storage directory, although gluster storage is not enabled.
2017-09-13 22:02:33,152: gnt-cluster init pid=35793 process:221 INFO RunCmd ip link show dev br-vlan30
2017-09-13 22:02:33,161: gnt-cluster init pid=35793 process:221 INFO RunCmd ssh-keygen -b 2048 -t rsa -f /root/.ssh/id_rsa -q -N ''
2017-09-13 22:02:33,713: gnt-cluster init pid=35793 __init__:2891 DEBUG Writing configuration and unlocking it
2017-09-13 22:02:33,944: gnt-cluster init pid=35793 security:85 DEBUG Generating new cluster certificate at /var/lib/ganeti/server.pem
2017-09-13 22:02:34,298: gnt-cluster init pid=35793 bootstrap:144 DEBUG Writing new confd HMAC key to /var/lib/ganeti/hmac.key
2017-09-13 22:02:34,300: gnt-cluster init pid=35793 security:85 DEBUG Generating new RAPI certificate at /var/lib/ganeti/rapi.pem
2017-09-13 22:02:34,777: gnt-cluster init pid=35793 bootstrap:173 DEBUG Generating new self-signed SPICE certificate at /var/lib/ganeti/spice.pem
2017-09-13 22:02:34,933: gnt-cluster init pid=35793 bootstrap:179 DEBUG Writing the public certificate to /var/lib/ganeti/spice.pem
2017-09-13 22:02:34,934: gnt-cluster init pid=35793 bootstrap:188 DEBUG Generating new cluster domain secret at /var/lib/ganeti/cluster-domain-secret
2017-09-13 22:02:34,941: gnt-cluster init pid=35793 __init__:2891 DEBUG Writing configuration and unlocking it
2017-09-13 22:02:34,949: gnt-cluster init pid=35793 __init__:2891 DEBUG Writing configuration and unlocking it
2017-09-13 22:02:35,184: gnt-cluster init pid=35793 process:221 INFO RunCmd /usr/lib/ganeti/daemon-util start ganeti-noded
2017-09-13 22:02:38,776: gnt-cluster init pid=35793 client:142 DEBUG Starting request <ganeti.http.client.HttpClientRequest 192.168.30.2:1811 POST /version at 0x7f9d77a53fd0>
2017-09-13 22:02:38,881: gnt-cluster init pid=35793 client:228 DEBUG Request <ganeti.http.client.HttpClientRequest 192.168.30.2:1811 POST /version at 0x7f9d77a53fd0> finished, errmsg=None
2017-09-13 22:02:38,882: gnt-cluster init pid=35793 bootstrap:824 DEBUG Starting daemons
2017-09-13 22:02:38,882: gnt-cluster init pid=35793 process:221 INFO RunCmd /usr/lib/ganeti/daemon-util start-all
2017-09-13 22:02:55,513: gnt-cluster init pid=35793 cli:1225 ERROR Error during command processing
Traceback (most recent call last):
  File "/usr/share/ganeti/2.15/ganeti/cli.py", line 1221, in GenericMain
    result = func(options, args)
  File "/usr/share/ganeti/2.15/ganeti/rpc/node.py", line 141, in wrapper
    return fn(*args, **kwargs)
  File "/usr/share/ganeti/2.15/ganeti/client/gnt_cluster.py", line 345, in InitCluster
    ssh_key_bits=ssh_key_bits,
  File "/usr/share/ganeti/2.15/ganeti/bootstrap.py", line 831, in InitCluster
    _WaitForMasterDaemon()
  File "/usr/share/ganeti/2.15/ganeti/bootstrap.py", line 273, in _WaitForMasterDaemon
    " %s seconds" % _DAEMON_READY_TIMEOUT)
OpExecError: Master daemon didn't answer queries within 10.0 seconds
Failure: command execution error:
Master daemon didn't answer queries within 10.0 seconds
```

-- Package-specific info:
Version symlinks:
  /etc/ganeti/share -> /usr/share/ganeti/2.15
  /etc/ganeti/lib -> /usr/lib/ganeti/2.15
Cluster config version: 2.15.2
Address family: IPv4
Enabled hypervisors: kvm
kvm hypervisor parameters:
  spice_ip_version=0
  nic_type=paravirtual
  cpu_threads=0
  use_chroot=False
  vnet_hdr=True
  cpu_sockets=0
  security_model=none
  cpu_cores=0
  vnc_tls=False
  boot_order=disk
  serial_console=True
  spice_playback_compression=True
  user_shutdown=False
  virtio_net_queues=1
  disk_aio=threads
  reboot_behavior=reboot
  spice_use_tls=False
  spice_use_vdagent=True
  migration_port=8102
  spice_tls_ciphers=HIGH:-DES:-3DES:-EXPORT:-DH
  migration_downtime=30
  use_localtime=False
  vhost_net=False
  cpu_mask=all
  disk_cache=default
  kvm_path=/usr/bin/kvm
  acpi=True
  vnc_x509_verify=False
  migration_bandwidth=32
  disk_type=paravirtual
  migration_mode=live
  kernel_args=ro
  serial_speed=38400
  root_path=/dev/vda1

-- System Information:
Debian Release: 9.1
  APT prefers stable-updates
  APT policy: (500, 'stable-updates'), (500, 'stable')
Architecture: amd64 (x86_64)

Kernel: Linux 4.9.0-3-amd64 (SMP w/24 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE=en_US:en (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Init: systemd (via /run/systemd/system)

Versions of packages ganeti depends on:
ii  adduser              3.115
ii  ganeti-2.15          2.15.2-10
ii  ganeti-haskell-2.15  2.15.2-10
ii  ganeti-htools-2.15   2.15.2-10
ii  lsb-base             9.20161125
ii  python               2.7.13-2

Versions of packages ganeti recommends:
ii  drbd-utils                   8.9.10-2
ii  ganeti-instance-debootstrap  0.16-3
ii  ndisc6                       1.0.3-3
ii  qemu-kvm                     1:2.8+dfsg-7

Versions of packages ganeti suggests:
pn  blktap-dkms  <none>
pn  ganeti-doc   <none>
pn  molly-guard  <none>

-- no debconf information

More information about the Pkg-ganeti-devel mailing list