stingergraph / stinger

The STINGER in-memory graph store and dynamic graph analysis platform. Millions to billions of vertices and edges at thousands to millions of updates per second.

Home Page:http://www.stingergraph.com

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Stinger resizes even when config->memory_size should be sufficiently large

edward-kao opened this issue · comments

According to stinger_new_full() in stinger.c, it seems that I should be able to avoid a resize by setting config->memory_size to be large enough. However, my such attempt did not work. For example:

g = Stinger(StingerGraphs.StingerConfig(n, n*5, 1, 1, 64*10^7, 0, 0, 0)) # where n = 500

caused a resizing of the object. That surprised me because calculate_stinger_size(nv=500, nebs=2500, netypes=1, nvtypes=1) should return a size much smaller than 64*10^7. Thank you for your help!

^ is the XOR operator in Python. ** is the exponentiation operator.

In [1]: 64*10^7
Out[1]: 647
In [2]: 64*10e7
Out[2]: 6400000000.0
In [3]: 64*10**7
Out[3]: 640000000

I am doing this in Julia actually, and I verified that 64*10^7 is indeed 640000000.

Ah OK, I guessed wrong.

I just tested creating a stinger with those same parameters in C++, and it does not trigger a resize. I suspect there is an issue with the Julia bindings, can you file an issue against that repository?

    stinger_config = (struct stinger_config_t *)xcalloc(1,sizeof(struct stinger_config_t));
    stinger_config->nv = 500;
    stinger_config->nebs = 2500;
    stinger_config->netypes = 1;
    stinger_config->nvtypes = 1;
    stinger_config->memory_size = 64 * 10e7;
    stinger_config->no_resize = 1;
    S = stinger_new_full(stinger_config);

Ok, will do, thank you.

commented

@edward-kao can you run that c++ code on your machine to see if you get a resize?

commented

I tried to reproduce on a cluster here at GT and get the expected behavior:

julia> using StingerGraphs

julia> n=500; g = Stinger(StingerGraphs.StingerConfig(n, n*5, 1, 1, 64*10^7, 0, 0, 0))
StingerGraphs.Stinger(Ptr{Void} @0x0000000006732180)

julia> n=500; g = Stinger(StingerGraphs.StingerConfig(n, n*5, 1, 1, 64*10^3, 0, 0, 0))
stinger[82985]: stinger_new_full 827: Resizing stinger to fit into memory (detected as 64000)
StingerGraphs.Stinger(Ptr{Void} @0x0000000004d487d0)

julia> n=500; g = Stinger(StingerGraphs.StingerConfig(n, n*5, 1, 1, 64*10^8, 0, 0, 0))
StingerGraphs.Stinger(Ptr{Void} @0x0000000006899b70)

Does your machine have sufficient memory for 64e7?

Turns out the resizing was happening when the graph was returned and assigned to another graph object which was not initialized with the same config setting. I fixed that and am getting the expected behavior now. Sorry about the confusion!