[Pkg-ofed-commits] r479 - /trunk/howto/infiniband-howto.sgml

gmpc-guest at alioth.debian.org gmpc-guest at alioth.debian.org
Tue Sep 15 15:46:07 UTC 2009


Author: gmpc-guest
Date: Tue Sep 15 15:46:07 2009
New Revision: 479

URL: http://svn.debian.org/wsvn/pkg-ofed/?sc=1&rev=479
Log:
Simplify MPI tests: Add instructions to install ofa-kernel deb.

Modified:
    trunk/howto/infiniband-howto.sgml

Modified: trunk/howto/infiniband-howto.sgml
URL: http://svn.debian.org/wsvn/pkg-ofed/trunk/howto/infiniband-howto.sgml?rev=479&op=diff
==============================================================================
--- trunk/howto/infiniband-howto.sgml (original)
+++ trunk/howto/infiniband-howto.sgml Tue Sep 15 15:46:07 2009
@@ -1,6 +1,6 @@
 <!doctype linuxdoc system>
 <article>
-<title>Infiniband Howto
+<title>Infiniband HOWTO
 <author>Guy Coates 
 
 <toc>
@@ -173,7 +173,16 @@
  module-assistant build ofa-kernel
 </verb>
 </tscreen>
-This will create a deb which you can then install. As the deb contains replacements for existing kernel modules you will need to either manually remove 
+
+This procedure will create an ofa-kernel-modules deb in /usr/src. You can the install the deb using dpkg or by running:
+<tscreen>
+<verb>
+ module-assistant install ofa-kernel
+</verb>
+</tscreen>
+The deb can also be copied to your other infiniband hosts and installed using dpkg.
+
+As the deb contains replacements for existing kernel modules you will need to either manually remove 
 any infiniband modules which have already been loaded, or reboot the machine, before you can use the new modules. 
 
 The new kernel modules will be installed into /usr/lib/&lt;kernel-version&gt/updates. They will not overwrite the original kernel modules, but the module
@@ -191,9 +200,8 @@
 </verb>
 </tscreen>
 
-
-Note that if you wish to rebuild the kernel modules (eg for a new kernel version) then you must issue
-the module-assistant clean command before trying a new build.
+Note that if you wish to rebuild the kernel modules for any reason, (eg for a new kernel version or to continue an interrupted build) then you must issue
+the "module-assistant clean" command before trying a new build.
 
 
 <sect>Setting up a basic infiniband network   
@@ -521,7 +529,7 @@
 
 <sect1>ARP and dual ported cards.
 <p>
-If you have a dual ported card with both ports on the same IB subnet but a  different IP subnet, you
+If you have a dual ported card with both ports on the same IB subnet, but different IP subnets, you
 will need to tweak the ARP settings for the IPoIB interfaces.  See  ipoib_release_notes.txt in the ofed-docs package for a full
 discussion of this issue.
 
@@ -580,30 +588,27 @@
 aptitude install mpitests
 </tscreen>
 
-<sect1>Configure Hosts
+<sect1>Configure Host Access
 <p>
 OpenMPI uses ssh to spawn jobs on remote hosts. You should configure a public/private keypair to ensure that  you 
 can ssh between hosts without entering a password. You should also ensure that your login process is silent.
 
-Choose two hosts on which to test the program and put their hostnames into a file called hostfile:
-
-<tscreen>
-<verb>
- hostA slots=1
- hostB slots=1
-</verb>
-</tscreen>
-
 <sect1>Run the MPI PingPong benchmark.
 <p>
 
-We will use the MPI PingPong benchmark for our testing.  By default, openmpi should use inifiniband networks in preference to any tcp networks it finds. However, we will force mpi to be extra-chatty during the test  to ensure  that we are really using the infiniband interfaces.
-
-(ADDME: Is there a better way to confirm which networks openmpi is using?)
-
-
-<verb>
-mpirun --mca btl_openib_verbose 1 --mca btl ^tcp -n 2 -hostfile /path/to/hostfile IMB-MPI1 PingPong
+We will use the MPI PingPong benchmark for our testing.  By default, openmpi should use inifiniband networks in preference to any tcp networks it finds. However, we will force mpi to ignore tcp networks to ensure that is using the infiniband network.
+
+
+<verb>
+#!/bin/bash
+#Infiniband MPI test program
+#Edit the hosts below to match your test hosts
+cat > /tmp/hostfile.$$.mpi <<EOF
+hostA slots=1
+HostB slots=1
+EOF
+
+mpirun --mca btl_openib_verbose 1 --mca btl ^tcp -n 2 -hostfile /tmp/hostfile.$$.mpi IMB-MPI1 PingPong
 </verb>
 
 If all goes well you should see openib debugging messages from both hosts, together with the job output.
@@ -655,9 +660,17 @@
 If you want to compare infiniband performance with your ethernet/TCP networks, you can re-run the tests using flags to tell openmpi to use your ethernet network. (The example below assumes that your test nodes are connected via eth0).
 
 <verb>
- mpirun --mca btl ^openib --mca btl_tcp_if_include eth0 --hostfile hostfile -n 2 IMB-MPI1 -benchmark PingPong
-</verb>
-
+#!/bin/bash
+#TCP MPI test program
+#Edit the hosts below to match your test hosts
+cat > /tmp/hostfile.$$.mpi <<EOF
+hostA slots=1
+HostB slots=1
+EOF
+mpirun --mca btl ^openib --mca btl_tcp_if_include eth0 --hostfile hostfile -n 2 IMB-MPI1 -benchmark PingPong
+</verb>
+
+You should notice signficantly higher latencies than for the infiniband test.
 
 <sect>SDP
 <p>




More information about the Pkg-ofed-commits mailing list