[Feature request]: Automated wordlists updater
molangning opened this issue · comments
Describe the feature request:
Some wordlists goes stale easily, especially if the update schedule is extremely inconsistent. A github action that checks for updates daily will greatly reduce the time and effort needed to keep a list up to date.
Next steps:
It might be possible for changes to be overwritten, so another automation should ideally warn and/or block a pull request that makes changes to a file that will get overwritten next update.
- I intend to open a pull request later
Let’s say a few wordlists is imported from other sources which may or may not be from github. The script should check those location and see if there is any remote changes. If there is, the script should pull those and commit those changes. Does that answer your question?
We have already made some Github Actions which periodically check for updates and update the files accordingly: https://github.com/danielmiessler/SecLists/actions
@molangning Do you have a wordlist in mind which is not covered by those CI flows?
@ItsIgnacioPortal There is a great wordlist from wallarm that updates irregularly.
There are a lot of wordlists here that are imported from from other sources as well, so I believe that it would be helpful if we can just define a remote source to pull/check daily
Oh, you mean https://github.com/danielmiessler/SecLists/Passwords/scraped-JWT-secrets.txt. One of the existing github actions could be used as a template for making an updater for that wordlist. If you find any other outdated wordlist, please let me know.
Features like crontab can be added