The GPU Virtualization Service (GVirtuS) presented in this work tries to fill the gap between in-house hosted computing clusters, equipped with GPGPUs devices, and pay-for-use high performance virtual clusters deployed via public or private computing clouds. gVirtuS allows an instanced virtual machine to access GPGPUs in a transparent and hypervisor independent way, with an overhead slightly greater than a real machine/GPGPU setup. The performance of the components of gVirtuS is assessed through a suite of tests in different deployment scenarios, such as providing GPGPU power to cloud computing based HPC clusters and sharing remotely hosted GPGPUs among HPC nodes.
https://link.springer.com/chapter/10.1007/978-3-642-15277-1_37
-
Giunta, G., Montella, R., Agrillo, G., Coviello, G. (2010) A GPGPU Transparent Virtualization Component for High Performance Computing Clouds. In: Euro-Par 2010 - Parallel Processing. Euro-Par 2010. Lecture Notes in Computer Science, vol 6271. Springer, Berlin, Heidelberg
-
Montella, R., Coviello, G., Giunta, G., Laccetti, G., Isaila, F., & Blas, J. G. (2011, September). A general-purpose virtualization service for HPC on cloud computing: an application to GPUs. In International Conference on Parallel Processing and Applied Mathematics (pp. 740-749). Springer, Berlin, Heidelberg.
-
Montella, R., Kosta, S., Oro, D., Vera, J., Fernández, C., Palmieri, C., ... & Laccetti, G. (2017). Accelerating Linux and Android applications on low‐power devices through remote GPGPU offloading. Concurrency and Computation: Practice and Experience, 29(24), e4286.
-
Montella, R., Ferraro, C., Kosta, S., Pelliccia, V., & Giunta, G. (2016, December). Enabling android-based devices to high-end gpgpus. In International Conference on Algorithms and Architectures for Parallel Processing (pp. 118-125). Springer, Cham.
-
Montella, R., Giunta, G., & Laccetti, G. (2014). Virtualizing high-end GPGPUs on ARM clusters for the next generation of high performance cloud computing. Cluster computing, 17(1), 139-152.
GCC, G++ with C++17 extension (minmum version: 7)
OS: CentOS 7.3, Ubuntu 18.04 (tested)
CUDA Toolkit: version 10.2
This package are required: build-essential autotools-dev automake git libtool libxmu-dev libxi-dev libgl-dev libosmesa-dev liblog4cplus-dev
Ubuntu: sudo apt-get install build-essential libxmu-dev libxi-dev libgl-dev libosmesa-dev git liblog4cplus-dev
CentOS:
sudo yum install centos-release-scl
sudo yum install devtoolset-8-gcc
scl enable devtoolset-8 bash
-
Clone the GVirtuS main repository
git clone https://github.com/gvirtus/GVirtuS.git
-
Compile and install GVirtuS using CMake:
cd GVirutS mkdir build cd build cmake .. make make install
By default GVirtuS will be installed in ${HOME}/GVirtuS; to override this behavior export the GVIRTUS_HOME variable before running cmake, i.e.:
export GVIRTUS_HOME=/opt/GVirtuS
On the remote machine where the cuda executables will be executed
Modify the GVirtuS configuration file backend if the default port 9999 is occuped or the machine is remote:
$GVIRTUS_HOME/etc/properties.json
{
"communicator": [
{
"endpoint": {
"suite": "tcp/ip",
"protocol": "oldtcp",
"server_address": "127.0.0.1",
"port": "9999"
},
"plugins": [
"cudart",
"cudadr",
"cufft",
"cublas",
"curand"
]
}
],
"secure\_application": false
}
Execute application server gvirtus-backend with follow command:
LD_LIBRARY_PATH=${GVIRTUS_HOME}/lib:${LD_LIBRARY_PATH} $GVIRTUS_HOME/bin/gvirtus-backend ${GVIRTUS_HOME}/etc/properties.json
Modify the Gvirtus configuration file frontend:
$GVIRTUS_HOME/etc/properties.json
{
"communicator": [
{
"endpoint": {
"suite": "tcp/ip",
"protocol": "oldtcp",
"server_address": "127.0.0.1",
"port": "9999"
},
"plugins": [
"cudart",
"cudadr",
"cufft",
"cublas",
"curand"
]
}
],
"secure\_application": false
}
NOTE: In the local configuration GVirtuS Backend and Frontend share the same configuration files.
Export the dynamic GVirtuS library:
export LD_LIBRARY_PATH=${GVIRTUS_HOME}/lib/frontend:${GVIRTUS_HOME}/lib/frontend:${LD_LIBRARY_PATH}
Optionally set a different configuration file
export CONFIG_FILE=$HOME/dev/properties.json
execute the cuda application compiled with cuda dynamic library (with -lcuda -lcudart)
./example
If you are using nvcc be sure you are compiling using shared libraries:
export EXTRA_NVCCFLAGS="--cudart=shared"
In order to change the loging level, define the GVIRTUS_LOGLEVEL environment variable:
export GVIRTUS_LOGLEVEL=<loglevel>
The value is defined as follows:
OFF_LOG_LEVEL = 60000
FATAL_LOG_LEVEL = 50000
ERROR_LOG_LEVEL = 40000
WARN_LOG_LEVEL = 30000
INFO_LOG_LEVEL = 20000
DEBUG_LOG_LEVEL = 10000
TRACE_LOG_LEVEL = 0