rjagerman / mammoth

Mammoth: Web-scale topic modelling for the ClueWeb dataset

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Scale up the summing of local models

rjagerman opened this issue · comments

Currently models are summed by returning and adding the β matrices element-wise. This only really works for small β matrices. To scale this up we would want to emit all the elements (j,k) of β, and use a map-reduce style summation.