About
Comparison of learning and inference speed of different GPU with various CNN models in pytorch
- 1080TI
- TITAN XP
- TITAN V
- 2080TI
- Titan RTX
- RTX 2060
- RTX 3090
- A100-PCIE
- A100-SXM4
Specification
| Graphics Card Name | GTX 1080 Ti | TITAN XP | TITAN V | RTX 2060 | RTX 2080 Ti | TITAN RTX | A100-PCIE | RTX 3090 |
|---|---|---|---|---|---|---|---|---|
| Process | 16nm | 16nm | 12nm | 12nm | 12nm | 12nm | 7nm | 8 nm |
| Die Size | 471mm² | 471mm² | 815mm² | 445 mm² | 754mm² | 754mm² | 826mm² | 628 mm² |
| Transistors | 11,800 million | 11,800 million | 21,100 million | 10,800 million | 18,600 million | 18,600 million | 54,200 million | 28,300 million |
| CUDA Cores | 3584 Cores | 3840 Cores | 5120 Cores | 1920 Cores | 4352 Cores | 4608 Cores | 6912 Cores | 10496 Cores |
| Tensor Cores | None | None | 640 Cores | 240 | 544 Cores | 576 Cores | 432 Cores | 328 Cores |
| Clock(base) | 1481 MHz | 1405 Mhz | 1200 MHz | 1365 MHz | 1350 MHz | 1350 MHz | 765 MHz | 1395 MHz |
| FP16 (half) | 177.2 GFLOPS | 189.8GFLOPS | 29,798 GFLOPS | 12.90 TFLOPS | 26,895 GFLOPS | 32.62 TFLOPS | 77.97 TFLOPS | 35.58 TFLOPS |
| FP32 (float) | 11,340 GFLOPS | 12.15FLOPS | 14,899 GFLOPS | 6.451 TFLOPS | 13,448 GFLOPS | 16.31 TFLOPS | 19.49 TFLOPS | 35.58 TFLOPS |
| FP64 (double) | 354.4 GFLOPS | 379.7 GFLOPS | 7,450 GFLOPS | 201.6 GFLOPS | 420.2 GFLOPS | 509.8 GFLOPS | 9.746 TFLOPS | 556.0 GFLOPS |
| Memory | 11GB GDDR5X | GDDR5X | 12 GB HBM2 | 6GB GDDR6 | 11 GB GDDR6 | 24 GB GDDR6 | 40GB HBM2e | 24GB GDDR6X |
| Memory Interface | 352-bit | 384bit | 3072-bit | 192 bit | 352-bit | 384 bit | 5120 bit | 384 bit |
| Memory Bandwidth | 484 GB/s | 547.6GB/s | 653GB/s | 336.0 GB/s | 616 GB/s | 672.0 GB/s | 1,555 GB/s | 936.2 GB/s |
| Price | $699 US | $1,199 US | $2,999 US | $ 349 US | $1,199 US | $2,499 US | $ 1,499 USD | |
| Release Date | Mar 10th, 2017 | Apr 6th 2017 | Dec 7th, 2017 | Jan 7th, 2019 | Sep 20th, 2018 | Dec 18th, 2018 | Jun 22nd, 2020 | Sep 1st, 2020 |
-
Single & multi GPU with batch size 12: compare training and inference speed of **SequeezeNet, VGG-16, VGG-19, ResNet18, ResNet34, ResNet50, ResNet101, ResNet152, DenseNet121, DenseNet169, DenseNet201, DenseNet161 mobilenet mnasnet ... **
-
Experiments are performed on three types of the datatype. single-precision, double-precision, half-precision
-
making plot(plotly)
Usage
./test.sh
Results
requirement
- python>=3.6(for f-formatting)
- torchvision
- torch>=1.0.0
- pandas
- psutil
- plotly(for plot)
- cufflinks(for plot)
Environment
- Pytorch version
1.4 - Number of GPUs on current device
4 - CUDA version =
10.0 - CUDNN version=
7601 nvcr.io/nvidia/pytorch:20.10-py3(docker container in A100 and 3090)
Change Log
- 2021/02/27
- Addition result in RTX3090
- Addition result in RTX2060(thanks for gutama)
- 2021/01/07
- Addition result in TITANXP
- 2021/01/05
- Addition result in A100 A100-PCIE(PR#14)
- 2021/01/04
- Addition result in A100 SXM4
- Addition result in TitanRTX
- Edit coding style benchmark_model
- f-formatting
- save option for json
- Edit test.sh for bash shell
- Edit README.md
- 2020/09/01
- Addition result in windows10
- Edit README.md
- 2020/01/17
- Edit coding style and some bug
- Change plot method
- Add results of various model experiments(only 2080ti)
- 2019/01/09
- PR Update typo (thanks for johmathe)
- Add requirements.txt
- Add result figures
- Add ('TkAgg') for cli
- Addition Muilt GPUS (DGX-station)
RTX-3090
- 2021/02/27
RTX-2060
- 2021/02/27 thanks for gutama issue#16
TITANXP
- 2021/01/05 thanks for kirk86 pr#14
A100-PCIE(DGX A100)
- 2021/01/05 Thanks for kirk86 pr#14
A100-SXM4(DGX A100)
- 2021/01/04
TitanRTX
- 2021/01/04
2080ti result on (new_results) windows10 system
- thanks for olixu
2080ti result on ubuntu (new_results)
- based on 2020/01/17 update
Comparison between networks (single GPU)
Each network is fed with 12 images with 224x224x3 dimensions. For training, time durations of 20 passes of forwarding and backward are averaged. For inference, time durations of 20 passes of forwarding are averaged. 5 warm-up steps are performed that do not calculate towards the final result.
I conducted the experiment using two RTX 2080ti.
| Mode | gpu | precision | densenet121 | densenet161 | densenet169 | densenet201 | resnet101 | resnet152 | resnet18 | resnet34 | resnet50 | squeezenet1_0 | squeezenet1_1 | vgg16 | vgg16_bn | vgg19 | vgg19_bn |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Training | TITAN V | single | 56.17 ms | 120.7 ms | 72.59 ms | 93.35 ms | 84.59 ms | 119.5 ms | 16.69 ms | 28.27 ms | 50.54 ms | 15.30 ms | 9.857 ms | 72.85 ms | 80.95 ms | 85.55 ms | 94.42 ms |
| Inference | TITAN V | single | 17.49 ms | 39.33 ms | 23.63 ms | 30.93 ms | 23.96 ms | 34.22 ms | 4.827 ms | 8.428 ms | 14.27 ms | 4.565 ms | 2.765 ms | 22.94 ms | 25.41 ms | 27.55 ms | 30.28 ms |
| Training | TITAN V | double | 139.8 ms | 387.4 ms | 175.9 ms | 224.5 ms | 509.9 ms | 720.0 ms | 94.21 ms | 194.6 ms | 271.7 ms | 68.38 ms | 31.18 ms | 1463. ms | 1484. ms | 1993. ms | 2016. ms |
| Inference | TITAN V | double | 47.68 ms | 170.5 ms | 60.73 ms | 78.43 ms | 317.7 ms | 448.6 ms | 60.26 ms | 129.9 ms | 159.8 ms | 42.37 ms | 11.95 ms | 1261. ms | 1266. ms | 1745. ms | 1751. ms |
| Training | TITAN V | half | 43.79 ms | 75.16 ms | 57.53 ms | 70.88 ms | 47.82 ms | 67.43 ms | 10.48 ms | 17.19 ms | 29.08 ms | 13.15 ms | 9.390 ms | 36.03 ms | 46.84 ms | 41.16 ms | 52.65 ms |
| Inference | TITAN V | half | 11.87 ms | 22.88 ms | 16.04 ms | 20.70 ms | 12.80 ms | 18.11 ms | 3.085 ms | 5.116 ms | 7.608 ms | 3.694 ms | 2.329 ms | 10.96 ms | 13.26 ms | 12.72 ms | 15.17 ms |
| Training | 1080ti | single | 77.18 ms | 164.0 ms | 99.66 ms | 127.6 ms | 112.8 ms | 158.7 ms | 22.48 ms | 36.80 ms | 68.87 ms | 20.56 ms | 13.29 ms | 101.8 ms | 114.1 ms | 119.9 ms | 133.2 ms |
| Inference | 1080ti | single | 23.53 ms | 51.53 ms | 31.82 ms | 41.73 ms | 33.02 ms | 47.02 ms | 6.426 ms | 10.97 ms | 20.17 ms | 7.174 ms | 4.370 ms | 33.73 ms | 37.25 ms | 39.95 ms | 44.12 ms |
| Training | 1080ti | double | 779.5 ms | 2522. ms | 940.4 ms | 1196. ms | 2410. ms | 3546. ms | 463.3 ms | 969.9 ms | 1216. ms | 259.9 ms | 131.5 ms | 4227. ms | 4271. ms | 5475. ms | 5522. ms |
| Inference | 1080ti | double | 47.68 ms | 275.2 ms | 1157. ms | 328.6 ms | 414.9 ms | 1080. ms | 1589. ms | 181.1 ms | 390.8 ms | 529.6 ms | 110.9 ms | 49.96 ms | 2094. ms | 2103. ms | 2775. ms |
| Training | 1080ti | half | 43.79 ms | 70.00 ms | 148.4 ms | 89.43 ms | 113.6 ms | 151.0 ms | 219.5 ms | 21.00 ms | 34.84 ms | 76.24 ms | 19.60 ms | 13.18 ms | 91.60 ms | 105.9 ms | 108.1 ms |
| Inference | 1080ti | half | 18.62 ms | 42.26 ms | 25.27 ms | 33.01 ms | 27.49 ms | 38.88 ms | 5.645 ms | 9.765 ms | 16.26 ms | 5.869 ms | 3.576 ms | 30.69 ms | 33.22 ms | 36.71 ms | 39.51 ms |
| Mode | gpu | precision | resnet18 | resnet34 | resnet50 | resnet101 | resnet152 | densenet121 | densenet169 | densenet201 | densenet161 | squeezenet1_0 | squeezenet1_1 | vgg16 | vgg16_bn | vgg19_bn | vgg19 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Training | RTX 2080ti(1) | single | 16.36 ms | 28.44 ms | 49.63 ms | 81.40 ms | 115.1 ms | 57.69 ms | 75.18 ms | 91.69 ms | 112.7 ms | 14.49 ms | 9.108 ms | 75.86 ms | 85.42 ms | 98.43 ms | 88.05 ms |
| Inference | RTX 2080ti(1) | single | 4.894 ms | 8.624 ms | 14.65 ms | 24.57 ms | 35.15 ms | 16.70 ms | 21.94 ms | 28.89 ms | 34.64 ms | 4.704 ms | 2.765 ms | 23.70 ms | 26.25 ms | 30.82 ms | 28.03 ms |
| Training | RTX 2080ti(1) | double | 367.9 ms | 755.4 ms | 939.9 ms | 1844. ms | 2702. ms | 593.5 ms | 724.3 ms | 921.3 ms | 1916. ms | 187.8 ms | 94.99 ms | 3251. ms | 3277. ms | 4265. ms | 4238. ms |
| Inference | RTX 2080ti(1) | double | 165.0 ms | 328.5 ms | 436.4 ms | 831.0 ms | 1196. ms | 213.8 ms | 266.0 ms | 339.5 ms | 910.7 ms | 82.71 ms | 35.79 ms | 1702. ms | 1708. ms | 2280. ms | 2274. ms |
| Training | RTX 2080ti(1) | half | 13.17 ms | 22.25 ms | 35.46 ms | 57.50 ms | 81.38 ms | 51.11 ms | 66.88 ms | 80.20 ms | 88.37 ms | 17.87 ms | 35.75 ms | 53.16 ms | 63.06 ms | 72.75 ms | 61.95 ms |
| Inference | RTX 2080ti(1) | half | 3.423 ms | 5.662 ms | 9.035 ms | 14.51 ms | 20.52 ms | 13.47 ms | 17.54 ms | 22.51 ms | 27.10 ms | 4.280 ms | 2.397 ms | 16.14 ms | 18.14 ms | 19.76 ms | 17.89 ms |
| Training | RTX 2080ti(2) | single | 16.92 ms | 29.51 ms | 51.46 ms | 84.90 ms | 120.0 ms | 58.13 ms | 75.96 ms | 92.47 ms | 117.6 ms | 14.95 ms | 9.255 ms | 78.95 ms | 88.71 ms | 102.3 ms | 91.67 ms |
| Inference | RTX 2080ti(2) | single | 5.107 ms | 8.976 ms | 15.18 ms | 25.60 ms | 36.60 ms | 17.02 ms | 22.40 ms | 29.46 ms | 36.72 ms | 4.852 ms | 2.786 ms | 24.76 ms | 27.25 ms | 32.05 ms | 29.27 ms |
| Training | RTX 2080ti(2) | double | 381.9 ms | 781.5 ms | 971.6 ms | 1900. ms | 2777. ms | 610.6 ms | 744.7 ms | 948.1 ms | 1974. ms | 191.9 ms | 97.27 ms | 3317. ms | 3350. ms | 4357. ms | 4329. ms |
| Inference | RTX 2080ti(2) | double | 171.8 ms | 341.7 ms | 449.5 ms | 849.5 ms | 1231. ms | 221.1 ms | 275.2 ms | 352.5 ms | 938.9 ms | 83.66 ms | 36.48 ms | 1715. ms | 1721. ms | 2294. ms | 2289. ms |
| Training | RTX 2080ti(2) | half | 13.57 ms | 22.97 ms | 36.55 ms | 59.10 ms | 83.81 ms | 51.74 ms | 68.35 ms | 81.21 ms | 89.46 ms | 15.75 ms | 35.46 ms | 55.28 ms | 65.43 ms | 75.75 ms | 64.62 ms |
| Inference | RTX 2080ti(2) | half | 3.520 ms | 5.837 ms | 9.272 ms | 14.93 ms | 21.13 ms | 13.38 ms | 18.71 ms | 22.40 ms | 26.82 ms | 4.446 ms | 2.406 ms | 16.29 ms | 17.91 ms | 20.90 ms | 19.14 ms |
TitanV ,1080ti , 2080ti result(old_results)
- Results using codes prior to 2020/01/17
DGX
contribute
If you want to contribute to the experiment in an additional environment, please contribute to the result by subfolder in fig.