name_filter command fails for large queries
AyrA opened this issue · comments
I noticed that the name_filter command fails when there is a large number of results to return
namecoind name_filter "^d/" 150 This command succeeds
namecoind name_filter "^d/" This command fails
I am using namecoin 0.3.80 on WIndows 7 x64. There is about 5 GB of memory available.
The error message I get is always: error: no response from server
The server instance does not prints any errors.
It seems like the call times out because it takes too long to process. In any case, name_filter is not very efficient - particularly on the old client that you use. As a first step, try using the new Namecoin Core client. This may already solve the issue for you.
0.3.80 is the newest version advertised in the download section of the website
EDIT: and the forums
That's true, since the Namecoin Core client is still considered "experimental". Nevertheless, it runs without problems since half a year (and is also used by a major miner since then). I think that the issue in question is that the old client first loads the full name database into memory and then processes the regexp for name_filter. Namecoin Core applies the filter as it reads the database and needs much less memory. That's why I think that Namecoin Core should be able to resolve your issue.
Since the plan is to replace the old client with Namecoin Core "relatively soon" (basically as soon as a new UI is finished), we will not investigate more develoment effort into it unless absoluately needed (including improving the performance of name_filter). But if you have one, a pull request is, of course, gladly accepted. :)
@JeremyRand: Maybe we should change the website to "tentatively" recommend Namecoin Core if no "manage names" UI is needed?
Do you have a windows binary available?
@domob1812 I'm planning on proposing an "Implementations" page for namecoin.org, which will list both NamecoinQ and Namecoin Core (along with libcoind, NamecoinArmory, and a few others) along with brief descriptions of them. Would this satisfy your request?
@AyrA I don't believe we've posted official binaries. This is one reason it's not advertised on the website yet. I'm away at a conference this week, but dealing with posting official binaries of Namecoin Core is something I intend to tackle in the next month or so.
As an aside, if your use case is just grabbing all the names for a specific namespace, name_filter is way overkill, since it uses a full RegExp parser which is really slow. I think @domob1812 and I were discussing adding an RPC call for a prefix match... this might be a better solution to your problem, and would be much faster than name_filter, even on Namecoin Core. See namecoin/namecoin-core#16
@JeremyRand I would also be happy, if I can extract everything from the namecoin blockchain. I can then feed it into an SQL database and search it this way.
@AyrA You might be interested in libcoind. It's a full Bitcoin/Namecoin client (also supports a few other altcoins) that stores all the blockchain data (including Namecoin name data) in a SQLite database. https://github.com/libcoin/libcoin/ I believe it's used by Kraken for their audits.
Doing some spring cleaning here. Namecoin Core Windows binaries have been officially available for quite a while. @AyrA can you please try with Namecoin Core, and open an issue in the Namecoin Core repo if it's still a problem? Closing this issue since Namecoin Legacy is not maintained. Thanks.