This README is for userspace RDMA cm library. Building ======== To make this directory, run: ./autogen.sh && ./configure && make && make install Typically the autogen and configure steps only need be done the first time unless configure.in or Makefile.am changes. Libraries are installed by default at /usr/local/lib. Device files ============ The userspace CMA uses a single device file regardless of the number of adapters or ports present. To create the appropriate character device file automatically with udev, a rule like KERNEL="rdma_cm", NAME="infiniband/%k", MODE="0666" can be used. This will create the device node named /dev/infiniband/rdma_cm or you can create it manually mknod /dev/infiniband/rdma_cm c 231 255 Common issues ============= Using multiple interfaces The librdmacm does support multiple interfaces. To make use of multiple interfaces, however, you need to instruct linux to only send ARP reples on the interface targetted in the ARP request. This can be done using a command similar to the following: sysctl -w net.ipv4.conf.all.arp_ignore=2 Without this change, it's possible for linux to resopnd to ARP requests on a different interface (IP address) than the IP address carried in the ARP request. This causes the RDMA stack to incorrectly map the remote IP address to the wrong RDMA device. Using loopback The librdmacm relies on ARP to resolve IP address to RDMA addresses. To support loopback connections between different ports on the same system, ARP must be enabled for local resolution: sysctl net.ipv4.conf.all.accept_local=1 Without this setting, loopback connections may timeout during address resolution.