Giters

dig-team / amie

Mavenized AMIE+Typing

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Memory requirement

lzzhaha opened this issue · comments

commented

Hi,

I tried to use AMIE3 to mine rules from a subset of WikiData, where the subset has 124,121,392 triples and 96 types of predicates. How much memory do I need to set aside for AMIE3 in this case?

I tried to allocate 128G using the option "-XX:-UseGCOverheadLimit -Xmx128G", but still got out of memory error. Could you help me with this issue?

Thank you very much.

commented

Hello,

AMIE is memory-intensive. I am afraid we do not have an accurate way to predict how much memory will be needed to run AMIE on a dataset. You can however get a rough idea. On the AMIE 3 paper, a server of 500GB was enough to run AMIE on 386M Wikidata triples with 1188 predicates. Memory consumption depends both on the KG size and on the branching factor during rule exploration, with the later depending on the number of predicates and how interconnected they are. If 128G are still not enough, then my intuition is that a server with 256GB may do the job.

Best,
Luis

commented

Thanks for your suggestion. The program works now.