Disclaimer: This is a POC(proof of concept) project improved from chenditic's original design for education purpose.
http://hackjutsu.com/pokemongo-map-poc-demo/
Temporarily disabled to save my budget on AWS ;)
All Pokemon data, which are essentially fake, are generated randomly by a mock API. In principle, there is no communction between the crawlers and the real Pokemon Go servers.
A distributed crawler system that achieves heuristic data crawling based on geographical location information, said system comprising:
- A web app sends network requests with said geographical locations information to an API gateway,
- An API gateway subsystem constructed by said API gateways, routes said requests to a query cluster,
- A query subsystem constructed by said query cluster, queries for Pokemons data based on said geographical locations information from a database on AWS, and returns said Pokemon data to said web app, and sends said geographical locations information to message queues,
- A message queue subsystem constructed from said message queues, stores said messages from said query subsystem, and provides said messages to a crawler cluster
- A crawler subsystem constructed by said crawler cluster, retrieves said geographical locations information from said message queue subsystem, and gets fake Pokemons data from a mock Pokemon go API, and saves said fake Pokemons data to a database,
- A database subsystem constructed by said database, stores said Pokemons data.
This repository composes of an implemenation for the query server and an implementation of the crawler server. An implementation for the web app can be found here.