Giters

jkudla / konect2networkx

Bridging the gap between the KONECT project and NetworkX

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Loading purely in memory

jkudla opened this issue · comments

commented

Currently, networks are downloaded from the KONECT.cc site, written to a ZIP archive file, unpacked and then loaded into memory. This requires writing to disk and leaves the ZIP archive along with the unpacked directory.

As a result of the HTTP request performed, the entire ZIP archive is already stored in memory before writing to a file. Hence the following

Suggestion: Allow unpacking and reading the network file without relying on any disk operations. Introduce an option to toggle this behaviour.

Edit: Regarding design, it could make sense to have three classes Loader, DiskLoader and MemoryLoader such that the last two inherit from Loader. In main.py, an object of type Loader is instantiated suitably to obtain the network from disk or the ZIP file stored in memory.