A collection of scrappers to gather data from the Dark Souls wiki.
As these scrappers use CSS selectors tailored to each page, changes on those pages may break the parsers.
- Scrapper for the Dark Souls wiki
Documentation sources are included with the project, and used to generate the documentation sites:
- The latest docs are always generated for the latest release, kept in the 'master' branch
- The development docs are generated from the latest code in the 'develop' branch
The source files for the docs, a small Sphinx project, are kept in the 'docs folder.
These can be built if needed:
python setup.py build_docs
Dependencies are indicated on the requirements.txt file.
These can be installed with:
pip install --upgrade -r requirements.txt
The runner file takes care of executing the scrappers:
python runner.py
The tests included with the project can be run with:
python setup.py test
This will delegate the execution to tox.
It is possible to run just one of the test profiles, in this case the py36 profile:
python setup.py test -p "py3.8"
Any kind of help with the project will be well received, and there are two main ways to give such help:
- Reporting errors and asking for extensions through the issues management
- or forking the repository and extending the project
Issues are managed at the GitHub project issues tracker, where any Github user may report bugs or ask for new features.
If you wish to fork or modify the code, visit the GitHub project page, where the latest versions are always kept. Check the 'master' branch for the latest release, and the 'develop' for the current, and stable, development version.
The project has been released under the MIT License.