DanieleParravicini / ACCL

MPI-like collectives for Xilinx Alveo FPGAs

ACCL: Accelerated Collective Communication Library

* Note: This project is under active development. We will tag a stable release in the coming weeks.*

ACCL is a Vitis kernel and associated Pynq and XRT drivers which together provide MPI-like collectives for Xilinx FPGAs. ACCL is designed to enable compute kernels resident in FPGA fabric to communicate directly under host supervision but without requiring data movement between the FPGA and host. Instead, ACCL uses Vitis-compatible TCP and UDP stacks to connect FPGAs directly over Ethernet at up to 100 Gbps on Alveo cards.

ACCL currently supports Send/Recv and the following collectives:

• Broadcast
• Scatter
• Gather
• All-gather
• Reduce
• All-reduce

Repository Structure

The repository is organized as follows:

• kernel: builds the ACCL Vitis kernel (called CCL Offload)
  • hls: FPGA IPs generated from C++ code
  • fw: firmware running on the embedded MicroBlaze
  • tcl: TCL build automation
• driver: drivers for the ACCL.
  • pynq: a PYNQ based python driver
  • xrt: an XRT based C++ driver
• demo: ACCL example systems on Alveo.
  • build: build bitstreams for Alveo cards
    • config: board-specific linker configuration files
    • hls: auxiliary modules written in HLS
    • tcl: various TCL automation
    • submodules for supported network stacks
  • host: host code which drives the demo
    • debug: host-side debug scripts for XSCT
• docs: documentation.

Build a demo with TCP stack.

More info at demo/build/readme.md

1. Source xrt
2. Go in demo/build
   1. Modify demo/build/Makefile to target the right shell (e.g. xilinx_u280_xdma_201920_3) and the right mode (e.g. tcp_cmac)
   2. Source Vitis 2020.1
   3. Build tcp network stack ips
   4. Source Vitis 2020.2
   5. run make to create a .xclbin