Memory differs due to the matrix alignment or invisible gradient buffer tensors

[David MacLeod]

Hi

I was just wondering what this message means in the MemReporter output

Total Tensors: 266979334        Used Memory: 924.71M
The allocated memory on cuda:0: 1.31G
Memory differs due to the matrix alignment or invisible gradient buffer tensors

Also what is the difference between Used Memory and allocated memory?

Many thanks

[Kaiyu Shi]

The gap between used memory and allocated memory comes from three aspects;

1. Memory layout alignment, e.g. you have a tensor with size 1024 x 125, but it takes 1024 x 128 for cache & speed optimization
2. Some tensor introduced in AutoGrad is not visible in python's garbage collect view, so I cannot count them w/o modifying pytorch source code. (usually this one produces most gap)

Hope it helps.