Random Block Algorithm seems like not definitely true

[Duc-Viet Hoang]

powersgd/gradient_reducers.py

Line 467 in d3b21a6

 size = self.rank * (m.shape[0] + m.shape[1]) 

what does it mean ? I think with rank > 1, it always be the number of tensor elements.

[Thijs Vogels]

Hey! This should be different from the number of tensor elements (note the +)

Say 'm' is a matrix of shape (100, 50), then size = (100 * rank) + (rank * 50), for the size of the two components that together make up a low-rank approximation of m.

Hope this clarifies it. Let me know if you have more questions!