[Pkg-ofed-devel] ofed + ubuntu
Mark J. Pearrow
mjp at mit.edu
Mon Aug 3 16:21:55 UTC 2009
Ah, many thanks for this info. The issue that prompted me to use the
diagnostics was that when trying to get GlusterFS to run over IB, I
got the same error (see below). I suspect they are using XRC in their
startup routine to check the connection, etc.
root at ba7:~# glusterfs --debug -f /etc/glusterfs/glusterfsd.vol
[2009-08-03 12:17:51] D [glusterfsd.c:354:_get_specfp] glusterfs:
loading volume file /etc/glusterfs/glusterfsd.vol
[2009-08-03 12:17:51] D [xlator.c:634:xlator_set_type] xlator:
dlsym(notify) on /usr/lib/glusterfs/2.0.4/xlator/features/locks.so:
undefined symbol: notify -- neglecting
[2009-08-03 12:17:51] D [xlator.c:634:xlator_set_type] xlator:
dlsym(notify) on /usr/lib/glusterfs/2.0.4/xlator/performance/io-
threads.so: undefined symbol: notify -- neglecting
=
=
=
=
=
=
=
=
========================================================================
Version : glusterfs 2.0.4 built on Aug 3 2009 08:33:26
TLA Revision : v2.0.4
Starting Time: 2009-08-03 12:17:51
Command line : glusterfs --debug -f /etc/glusterfs/glusterfsd.vol
PID : 9284
System name : Linux
Nodename : ba7
Kernel Release : 2.6.28-11-generic
Hardware Identifier: x86_64
Given volfile:
+
------------------------------------------------------------------------------+
1: # file: /etc/glusterfs/glusterfs-server.vol
2: volume posix
3: type storage/posix
4: option directory /d2/gluster
5: end-volume
6:
7: volume locks
8: type features/locks
9: option mandatory-locks on
10: subvolumes posix
11: end-volume
12:
13: volume brick
14: type performance/io-threads
15: option thread-count 8
16: subvolumes locks
17: end-volume
18:
19: volume server
20: type protocol/server
21: option transport-type ib-verbs/server
22: option auth.addr.brick.allow 192.168.66.*
23: subvolumes brick
24: end-volume
+
------------------------------------------------------------------------------+
[2009-08-03 12:17:51] D [glusterfsd.c:1205:main] glusterfs: running in
pid 9284
[2009-08-03 12:17:51] D [io-threads.c:2280:init] brick: io-threads:
Autoscaling: off, min_threads: 8, max_threads: 8
[2009-08-03 12:17:51] D [transport.c:141:transport_load] transport:
attempt to load file /usr/lib/glusterfs/2.0.4/transport/ib-verbs.so
mlx4: There is a mismatch between the kernel and the userspace
libraries: Kernel does not support XRC. Exiting.
[2009-08-03 12:17:51] E [ib-verbs.c:1382:ib_verbs_get_device]
transport/ib-verbs: cannot open device `mlx4_0'
[2009-08-03 12:17:51] E [ib-verbs.c:1544:ib_verbs_init] transport/ib-
verbs: could not create ib_verbs device for mlx4_0
[2009-08-03 12:17:51] E [ib-verbs.c:2369:init] server: Failed to
initialize IB Device
[2009-08-03 12:17:51] E [transport.c:200:transport_load] transport:
'ib-verbs' initialization failed
[2009-08-03 12:17:51] E [server-protocol.c:7643:init] server: failed
to load transport
[2009-08-03 12:17:51] E [xlator.c:736:xlator_init_rec] xlator:
Initialization of volume 'server' failed, review your volfile again
[2009-08-03 12:17:51] E [glusterfsd.c:513:_xlator_graph_init]
glusterfs: initializing translator failed
[2009-08-03 12:17:51] E [glusterfsd.c:1217:main] glusterfs: translator
initialization failed. exiting
On Aug 3, 2009, at 12:12 PM, Guy Coates wrote:
> Hi Mark,
>
> Thanks for the installation report.
>
> XRC support is currently not in the mainline kernel infiniband
> modules.
>
> You have two options:
>
> 1) Not worry about it. The only package that actually uses XRC are the
> benchmarking programs in the perftest package.
>
>
> 2) Build a set of OFED kernel modules.
>
> On debian you can use module-assistant to build the OFED kernel for
> you.
>
> http://pkg-ofed.alioth.debian.org/howto/infiniband-howto-3.html
>
> In theory, this should also work on ubuntu too. However, the ofa-
> kernel package
> only contains patches for kernels <= 2.6.26, so I suspect that it
> will not build
> correctly against the ubuntu 2.6.28. (I don't have an ubuntu system
> to test).
>
> If you want to try anyhow, you will need to dig into the ofa-kernel
> package and
> hack the ofed_scripts/ofed_patch.sh, which is responsible for
> guessing the
> kernel version and building the appropriate modules.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.alioth.debian.org/pipermail/pkg-ofed-devel/attachments/20090803/aaaf0191/attachment.htm>
More information about the Pkg-ofed-devel
mailing list