Giters

ralyodio / dht_indexer

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

chore: Synergize hash finder module with indexing process

0xf333 opened this issue · comments

commented

Description

Please refer to PR #2 for more context

commented

can we move ./examples/initial_hashes.csv to ./database and then add ./database to .gitignore?

commented

@ralyodio

can we move ./examples/initial_hashes.csv to ./database and then add ./database to .gitignore?

Absolutely!
I've already planned to do that once I complete the scope in the above ticket.

commented

@ralyodio
Regarding the scope mentioned in this thread title, will today evening work for you?
Reason being I need to complete 2 sprints for an MVP that should be presented to a VC tomorrow.
Let me know...

commented

yeah that's fine.

commented

@ralyodio

yeah that's fine.

👍

commented 
===> Seeder/Leecher counts updated in the database

###########################################

/home/ubuntu/www/profullstack.com/dht_indexer/node_modules/lru/index.js:54
      this.cache[next].prev = prev
                            ^

TypeError: Cannot set properties of undefined (setting 'prev')
    at LRU._unlink (/home/ubuntu/www/profullstack.com/dht_indexer/node_modules/lru/index.js:54:29)
    at LRU.set (/home/ubuntu/www/profullstack.com/dht_indexer/node_modules/lru/index.js:80:10)
    at done (file:///home/ubuntu/www/profullstack.com/dht_indexer/node_modules/bittorrent-dht/client.js:677:20)
    at done (/home/ubuntu/www/profullstack.com/dht_indexer/node_modules/k-rpc/index.js:248:5)
    at process.processTicksAndRejections (node:internal/process/task_queues:77:11)

Node.js v20.11.0
commented

@ralyodio
Make sure that you have the latest commit and do not edit the csv file

commented

ok that worked

commented

its no longer populating the db

commented

@ralyodio

its no longer populating the db

  1. The hash finder module is not fully integrated with the indexer yet. Please read the top title and post on this GitHub ticket/chat thread.
  2. Alternatively you can stop the indexer for now and run npm run hash_hunter so it can collect more hashes and then stop it once you have additional hashes and run npm start again, it will carry on populating the db with more torrents.
commented

I'm running g both at the same time. It's still not populating the db

commented

I'm running g both at the same time. It's still not populating the db

@ralyodio

  1. Don't run both at the same time yet, this can cause data race conditions.

  2. If there are no new hashes discovered due to the state of the network ( there's not busy traffic constantly, there are downtimes too ), the indexer will carry on only tracking hashes:

    • That it has
    • That are active

  3. Make sure that you didn't decreased the time interval in Setttings.toml. In fact, set it with the value of 30 so it can update the database every 30 minutes.
    In production, this should be set to 12hrs-24hrs.

commented

ok let me know when you have them both working.

commented

@ralyodio
Good morning, I just completed a couple of minutes ago what I mentioned here : Link to previous comment being referred to
It took a bit longer than expected.

Regarding completing ticket #3

Sorry for the reschedule, please give me till around late afternoon today.

commented

Completed in PR #4

commented

its finding new hashes, but they aren't ending up in the db.

commented

its finding new hashes, but they aren't ending up in the db.

responded at : https://github.com/0xf333/dht_indexer/pull/4#issuecomment-1938554469