This README is for userspace RDMA cm library.
Building
========
To make this directory, run:
./autogen.sh && ./configure && make && make install
Typically the autogen and configure steps only need be done the first
time unless configure.in or Makefile.am changes.
Libraries are installed by default at /usr/local/lib.
Device files
============
The userspace CMA uses a single device file regardless of the number
of adapters or ports present.
To create the appropriate character device file automatically with
udev, a rule like
KERNEL="rdma_cm", NAME="infiniband/%k", MODE="0666"
can be used. This will create the device node named
/dev/infiniband/rdma_cm
or you can create it manually
mknod /dev/infiniband/rdma_cm c 231 255
Common issues
=============
Using multiple interfaces
The librdmacm does support multiple interfaces. To make use
of multiple interfaces, however, you need to instruct linux
to only send ARP reples on the interface targetted in the ARP
request. This can be done using a command similar to the
following:
sysctl -w net.ipv4.conf.all.arp_ignore=2
Without this change, it's possible for linux to resopnd to ARP
requests on a different interface (IP address) than the IP
address carried in the ARP request. This causes the RDMA stack
to incorrectly map the remote IP address to the wrong RDMA
device.
Using loopback
The librdmacm relies on ARP to resolve IP address to RDMA
addresses. To support loopback connections between different
ports on the same system, ARP must be enabled for local
resolution:
sysctl net.ipv4.conf.all.accept_local=1
Without this setting, loopback connections may timeout
during address resolution.