WebHere is an Objective-C framework for web scraping, packaged for iOS 5+ and OSX 10.7+ platforms.
Briefly put, web scraping is parsing of a website and extraction of data from the HTML pages contained in it.
This work has been inspired by RestKit, but aimed at HTML data and working in a simpler form (no mapping upfront, model classes declare their own building strategy); it is mostly relying on:
- AFNetworking to perform all network operations.
- HTMLDocument to extract data using XPath.
Those two projects really deserve attention on their own, make sure to visit their page and understand their APIs, as WebHere will mostly provide a unified facade to their APIs.
- Downloads HTML pages and extracts data into user-defined classes.
- Allows the user to use XPath to query the HTML document.
- Pre-defined methods to extract links and forms.
- Tested.
- At this moment only GET and POST REST methods have been tested.
- Please pay attention to the legal issues when peforming web scraping.
- Authorization credentials (username/password and token based) have not been tested.
- Preferred way is by using CocoaPod (Work In Progress).
- You should be able to add WebHere to you source tree. If you are using git, consider using a
git submodule
Dependencies are automatically managed by Cocoapod. In case you have to add WebHere to your source tree and use it outside Cocoapod, you must add the following projects along with WebHere:
- AFNetworking for network operations
- OCLogTemplate for logging purpose
TBD - Having a look at the test cases provided should give you an overview of the API.
Project has been covered by unit tests using:
Please notice that all tests are performed locally, meaning that no actual network access is needed, all requests being stubed by Nocilla.
- Fork it
- Create your feature branch
- Commit your changes
- Push to the branch
- Create new Pull Request