tkgw / swish-cuda

Swish Activation - PyTorch CUDA Implementation

Swish Activation - PyTorch CUDA Implementation

This is a PyTorch CUDA implementation of the Swish activation function (https://arxiv.org/abs/1710.05941).

Installation

It is currently distributed as a source only PyTorch extension. So you need a properly set up toolchain and CUDA compilers to install.

1. Toolchain - In conda the gxx_linux-64 package provides an appropriate toolchain. However there can still be compatbility issues with this depending on system. You can also try with the system toolchain.
2. CUDA Toolkit - The nVidia CUDA Toolkit is required in addition to drivers to provide needed headers and tools. Get the appropriate version for your Linux distro from nVidia or check for distro specific instructions otherwise.

It is important your CUDA Toolkit matches the version PyTorch is built for or errors can occur. Currently PyTorch builds for v10.0 and v9.2.