[Virtual-pkg-base-maintainers] Bug#721316: base: NETDEV WATCHDOG: eth0 (igb): transmit queue 0 timed out

Fri Aug 30 07:56:47 UTC 2013

Package: base
Severity: critical
Justification: breaks the whole system

   * What led up to the situation?
Nothing special, unexpected fail network subsystem

   * What exactly did you do (or not do) that was effective (or
     ineffective)?
I did nothing, my server works as a virtual hypervisor server.

   * What was the outcome of this action?
One of two network interface has stopped working. Second one continues to work.
servicve networking restart, ifconfig down/up, ip link, ifdown/ifup - nothing helped.

   * What outcome did you expect instead?
I expected my server continues to run.

-- System Information:
Debian Release: 7.1
  APT prefers stable
  APT policy: (500, 'stable')
Architecture: amd64 (x86_64)

Kernel: Linux 2.6.32-23-pve (SMP w/4 CPU cores)
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8) (ignored: LC_ALL set to en_US.UTF-8)
Shell: /bin/sh linked to /bin/dash

error log:
# grep 16:07 /var/log/messages
Aug 29 16:07:33 proxve kernel: ------------[ cut here ]------------
Aug 29 16:07:33 proxve kernel: WARNING: at net/sched/sch_generic.c:267 dev_watchdog+0x28a/0x2a0() (Not tainted)
Aug 29 16:07:33 proxve kernel: Hardware name: X9SRE/X9SRE-3F/X9SRi/X9SRi-3F
Aug 29 16:07:33 proxve kernel: NETDEV WATCHDOG: eth0 (igb): transmit queue 0 timed out
Aug 29 16:07:33 proxve kernel: Modules linked in: vzethdev vznetdev pio_nfs pio_direct pfmt_raw pfmt_ploop1 ploop simfs vzrst nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 vzcpt nf_conntrack vzdquota vzmon vzdev ip6t_REJECT ip6table_mangle ip6table_filter ip6_tables xt_length xt_hl xt_tcpmss xt_TCPMSS iptable_mangle iptable_filter xt_multiport xt_limit vhost_net tun macvtap xt_dscp macvlan ipt_REJECT kvm_intel ip_tables kvm fuse vzevent ib_iser rdma_cm ib_addr iw_cm ib_cm ib_sa ib_mad ib_core iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi nfsd nfs nfs_acl auth_rpcgss fscache lockd sunrpc ipv6 snd_pcsp snd_pcm snd_page_alloc sb_edac snd_timer snd serio_raw ioatdma iTCO_wdt i2c_i801 edac_core shpchp soundcore iTCO_vendor_support wmi acpi_pad ext3 mbcache jbd sg isci igb i2c_algo_bit i2c_core dca libsas scsi_transport_sas ahci [last unloaded: scsi_wait_scan]
Aug 29 16:07:33 proxve kernel: Pid: 3785, comm: kvm veid: 0 Not tainted 2.6.32-22-pve #1
Aug 29 16:07:33 proxve kernel: Call Trace:
Aug 29 16:07:33 proxve kernel: <IRQ>  [<ffffffff8106f5a7>] ? warn_slowpath_common+0x87/0xe0
Aug 29 16:07:33 proxve kernel: [<ffffffff8106f6b6>] ? warn_slowpath_fmt+0x46/0x50
Aug 29 16:07:33 proxve kernel: [<ffffffff81489b8a>] ? dev_watchdog+0x28a/0x2a0
Aug 29 16:07:33 proxve kernel: [<ffffffff8105aa4a>] ? update_curr+0xba/0x160
Aug 29 16:07:33 proxve kernel: [<ffffffff81489900>] ? dev_watchdog+0x0/0x2a0
Aug 29 16:07:33 proxve kernel: [<ffffffff81083b26>] ? run_timer_softirq+0x176/0x370
Aug 29 16:07:33 proxve kernel: [<ffffffff81033655>] ? native_apic_msr_write+0x35/0x40
Aug 29 16:07:33 proxve kernel: [<ffffffff8107930b>] ? __do_softirq+0x11b/0x260
Aug 29 16:07:33 proxve kernel: [<ffffffff810abfb5>] ? tick_dev_program_event+0x65/0xc0
Aug 29 16:07:33 proxve kernel: [<ffffffff810ac03a>] ? tick_program_event+0x2a/0x30
Aug 29 16:07:33 proxve kernel: [<ffffffff8100c32c>] ? call_softirq+0x1c/0x30
Aug 29 16:07:33 proxve kernel: [<ffffffff8100de95>] ? do_softirq+0x75/0xb0
Aug 29 16:07:33 proxve kernel: [<ffffffff810795e5>] ? irq_exit+0xc5/0xd0
Aug 29 16:07:33 proxve kernel: [<ffffffff815492c0>] ? smp_apic_timer_interrupt+0x70/0x9b
Aug 29 16:07:33 proxve kernel: [<ffffffff8100bcd3>] ? apic_timer_interrupt+0x13/0x20
Aug 29 16:07:33 proxve kernel: <EOI>  [<ffffffffa04783d7>] ? kvm_arch_vcpu_ioctl_run+0x387/0xfd0 [kvm]
Aug 29 16:07:33 proxve kernel: [<ffffffffa04783cb>] ? kvm_arch_vcpu_ioctl_run+0x37b/0xfd0 [kvm]
Aug 29 16:07:33 proxve kernel: [<ffffffff81064430>] ? wake_up_state+0x10/0x20
Aug 29 16:07:33 proxve kernel: [<ffffffff810b9e6a>] ? wake_futex+0x6a/0x90
Aug 29 16:07:33 proxve kernel: [<ffffffff810ba08a>] ? futex_wake+0x10a/0x130
Aug 29 16:07:33 proxve kernel: [<ffffffffa045f8e3>] ? kvm_vcpu_ioctl+0x2e3/0x580 [kvm]
Aug 29 16:07:33 proxve kernel: [<ffffffff810bc469>] ? do_futex+0x159/0xb10
Aug 29 16:07:33 proxve kernel: [<ffffffff8104f02f>] ? __dequeue_entity+0x2f/0x50
Aug 29 16:07:33 proxve kernel: [<ffffffff811b352a>] ? vfs_ioctl+0x2a/0xa0
Aug 29 16:07:33 proxve kernel: [<ffffffff811b3b5e>] ? do_vfs_ioctl+0x7e/0x570
Aug 29 16:07:33 proxve kernel: [<ffffffff81012899>] ? read_tsc+0x9/0x20
Aug 29 16:07:33 proxve kernel: [<ffffffff810a5563>] ? ktime_get_ts+0xb3/0xf0
Aug 29 16:07:33 proxve kernel: [<ffffffff811b409f>] ? sys_ioctl+0x4f/0x80
Aug 29 16:07:33 proxve kernel: [<ffffffff8109a509>] ? sys_clock_gettime+0x69/0xa0
Aug 29 16:07:33 proxve kernel: [<ffffffff8100b182>] ? system_call_fastpath+0x16/0x1b
Aug 29 16:07:33 proxve kernel: ---[ end trace 3b2ef6752edd7cb1 ]---

# lspci
00:00.0 Host bridge: Intel Corporation Xeon E5/Core i7 DMI2 (rev 07)
00:01.0 PCI bridge: Intel Corporation Xeon E5/Core i7 IIO PCI Express Root Port 1a (rev 07)
00:02.0 PCI bridge: Intel Corporation Xeon E5/Core i7 IIO PCI Express Root Port 2a (rev 07)
00:03.0 PCI bridge: Intel Corporation Xeon E5/Core i7 IIO PCI Express Root Port 3a in PCI Express Mode (rev 07)
00:03.2 PCI bridge: Intel Corporation Xeon E5/Core i7 IIO PCI Express Root Port 3c (rev 07)
00:04.0 System peripheral: Intel Corporation Xeon E5/Core i7 DMA Channel 0 (rev 07)
00:04.1 System peripheral: Intel Corporation Xeon E5/Core i7 DMA Channel 1 (rev 07)
00:04.2 System peripheral: Intel Corporation Xeon E5/Core i7 DMA Channel 2 (rev 07)
00:04.3 System peripheral: Intel Corporation Xeon E5/Core i7 DMA Channel 3 (rev 07)
00:04.4 System peripheral: Intel Corporation Xeon E5/Core i7 DMA Channel 4 (rev 07)
00:04.5 System peripheral: Intel Corporation Xeon E5/Core i7 DMA Channel 5 (rev 07)
00:04.6 System peripheral: Intel Corporation Xeon E5/Core i7 DMA Channel 6 (rev 07)
00:04.7 System peripheral: Intel Corporation Xeon E5/Core i7 DMA Channel 7 (rev 07)
00:05.0 System peripheral: Intel Corporation Xeon E5/Core i7 Address Map, VTd_Misc, System Management (rev 07)
00:05.2 System peripheral: Intel Corporation Xeon E5/Core i7 Control Status and Global Errors (rev 07)
00:05.4 PIC: Intel Corporation Xeon E5/Core i7 I/O APIC (rev 07)
00:11.0 PCI bridge: Intel Corporation C600/X79 series chipset PCI Express Virtual Root Port (rev 06)
00:16.0 Communication controller: Intel Corporation C600/X79 series chipset MEI Controller #1 (rev 05)
00:16.1 Communication controller: Intel Corporation C600/X79 series chipset MEI Controller #2 (rev 05)
00:1a.0 USB controller: Intel Corporation C600/X79 series chipset USB2 Enhanced Host Controller #2 (rev 06)
00:1c.0 PCI bridge: Intel Corporation C600/X79 series chipset PCI Express Root Port 1 (rev b6)
00:1c.4 PCI bridge: Intel Corporation C600/X79 series chipset PCI Express Root Port 5 (rev b6)
00:1d.0 USB controller: Intel Corporation C600/X79 series chipset USB2 Enhanced Host Controller #1 (rev 06)
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev a6)
00:1f.0 ISA bridge: Intel Corporation C600/X79 series chipset LPC Controller (rev 06)
00:1f.2 SATA controller: Intel Corporation C600/X79 series chipset 6-Port SATA AHCI Controller (rev 06)
00:1f.3 SMBus: Intel Corporation C600/X79 series chipset SMBus Host Controller (rev 06)
00:1f.6 Signal processing controller: Intel Corporation C600/X79 series chipset Thermal Management Controller (rev 06)
03:00.0 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
03:00.1 Ethernet controller: Intel Corporation I350 Gigabit Network Connection (rev 01)
06:00.0 Serial Attached SCSI controller: Intel Corporation C602 chipset 4-Port SATA Storage Control Unit (rev 06)
08:00.0 PCI bridge: Intel Corporation 6700PXH PCI Express-to-PCI Bridge A (rev 09)
08:00.2 PCI bridge: Intel Corporation 6700PXH PCI Express-to-PCI Bridge B (rev 09)
0b:04.0 VGA compatible controller: Matrox Electronics Systems Ltd. MGA G200eW WPCM450 (rev 0a)
ff:08.0 System peripheral: Intel Corporation Xeon E5/Core i7 QPI Link 0 (rev 07)
ff:08.3 System peripheral: Intel Corporation Xeon E5/Core i7 QPI Link Reut 0 (rev 07)
ff:08.4 System peripheral: Intel Corporation Xeon E5/Core i7 QPI Link Reut 0 (rev 07)
ff:09.0 System peripheral: Intel Corporation Xeon E5/Core i7 QPI Link 1 (rev 07)
ff:09.3 System peripheral: Intel Corporation Xeon E5/Core i7 QPI Link Reut 1 (rev 07)
ff:09.4 System peripheral: Intel Corporation Xeon E5/Core i7 QPI Link Reut 1 (rev 07)
ff:0a.0 System peripheral: Intel Corporation Xeon E5/Core i7 Power Control Unit 0 (rev 07)
ff:0a.1 System peripheral: Intel Corporation Xeon E5/Core i7 Power Control Unit 1 (rev 07)
ff:0a.2 System peripheral: Intel Corporation Xeon E5/Core i7 Power Control Unit 2 (rev 07)
ff:0a.3 System peripheral: Intel Corporation Xeon E5/Core i7 Power Control Unit 3 (rev 07)
ff:0b.0 System peripheral: Intel Corporation Xeon E5/Core i7 Interrupt Control Registers (rev 07)
ff:0b.3 System peripheral: Intel Corporation Xeon E5/Core i7 Semaphore and Scratchpad Configuration Registers (rev 07)
ff:0c.0 System peripheral: Intel Corporation Xeon E5/Core i7 Unicast Register 0 (rev 07)
ff:0c.1 System peripheral: Intel Corporation Xeon E5/Core i7 Unicast Register 0 (rev 07)
ff:0c.6 System peripheral: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller System Address Decoder 0 (rev 07)
ff:0c.7 System peripheral: Intel Corporation Xeon E5/Core i7 System Address Decoder (rev 07)
ff:0d.0 System peripheral: Intel Corporation Xeon E5/Core i7 Unicast Register 0 (rev 07)
ff:0d.1 System peripheral: Intel Corporation Xeon E5/Core i7 Unicast Register 0 (rev 07)
ff:0d.6 System peripheral: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller System Address Decoder 1 (rev 07)
ff:0e.0 System peripheral: Intel Corporation Xeon E5/Core i7 Processor Home Agent (rev 07)
ff:0e.1 Performance counters: Intel Corporation Xeon E5/Core i7 Processor Home Agent Performance Monitoring (rev 07)
ff:0f.0 System peripheral: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller Registers (rev 07)
ff:0f.1 System peripheral: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller RAS Registers (rev 07)
ff:0f.2 System peripheral: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller Target Address Decoder 0 (rev 07)
ff:0f.3 System peripheral: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller Target Address Decoder 1 (rev 07)
ff:0f.4 System peripheral: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller Target Address Decoder 2 (rev 07)
ff:0f.5 System peripheral: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller Target Address Decoder 3 (rev 07)
ff:0f.6 System peripheral: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller Target Address Decoder 4 (rev 07)
ff:10.0 System peripheral: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller Channel 0-3 Thermal Control 0 (rev 07)
ff:10.1 System peripheral: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller Channel 0-3 Thermal Control 1 (rev 07)
ff:10.2 System peripheral: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller ERROR Registers 0 (rev 07)
ff:10.3 System peripheral: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller ERROR Registers 1 (rev 07)
ff:10.4 System peripheral: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller Channel 0-3 Thermal Control 2 (rev 07)
ff:10.5 System peripheral: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller Channel 0-3 Thermal Control 3 (rev 07)
ff:10.6 System peripheral: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller ERROR Registers 2 (rev 07)
ff:10.7 System peripheral: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller ERROR Registers 3 (rev 07)
ff:11.0 System peripheral: Intel Corporation Xeon E5/Core i7 DDRIO (rev 07)
ff:13.0 System peripheral: Intel Corporation Xeon E5/Core i7 R2PCIe (rev 07)
ff:13.1 Performance counters: Intel Corporation Xeon E5/Core i7 Ring to PCI Express Performance Monitor (rev 07)
ff:13.4 Performance counters: Intel Corporation Xeon E5/Core i7 QuickPath Interconnect Agent Ring Registers (rev 07)
ff:13.5 Performance counters: Intel Corporation Xeon E5/Core i7 Ring to QuickPath Interconnect Link 0 Performance Monitor (rev 07)
ff:13.6 System peripheral: Intel Corporation Xeon E5/Core i7 Ring to QuickPath Interconnect Link 1 Performance Monitor (rev 07)