Tools for putting Redis Geo search through it's paces, written in Go. I've been benchmarking and evaluating Redis performance for location based searches. Most of the script centers around the GEORADIUS command.
go get github.com/holmser/redisgeo
Database of Places for testing:
wget http://download.geonames.org/export/dump/US.zip- Begin to query them
- Play with goroutines and channels
TODO
Haversine formula is a standard way to calculate distance between 2 points on the surface of a sphere. Earth is not a perfect sphere, so this method may introduce errors of up to .5%. This is usually an acceptable error rate for social proximity searches.
High level process is:
-
Select point
-
retrieve all points within 8 boxes
-
apply haversine formula to results and drop from radius.
-
Redis uses Haversine formula, error rate may be up to 0.5%
-
Add via name, lat, lon Sample data:
4085315 Ragland Cemetery Ragland Cemetery 34.67398 -86.82695 S CMTY US AL 083 0 180 178 America/Chicago 2006
- Pipelining
geoadd places lon lat name
-
Get places near me with activities on specific date/range
-
Get places near me with activities
-
query for 10 closest races near me, get 10 raceids returned
- pipeline query for data associated with those 10 ids
- take all 10 queries and shove them into 1 request, get all results back at once. can be 100x more efficient or more.
-
Loading data:
- 1 record at a time: 15+ minutes for a 2.3 million records
- pipelined: 32 seconds
- dump geodata into Redis
- as events get added, push events
- refill every night?
Memory usage: initial empty container: 1.4MB with geodata: 275MB
time go run main.go load | redis-cli --pipe
All data transferred. Waiting for the last reply...
Last reply received from server.
errors: 0, replies: 2232904
real 0m32.357s
user 0m7.010s
sys 0m5.994s#Outside goroutine
real 0m2.443s
user 0m0.848s
sys 0m0.352s