Scala MaxMind IP Lookups
Introduction
This is a Scala wrapper for the MaxMind Java Geo-IP2 library. The main benefits of using this wrapper over directly calling the Java library from Scala are:
- Provides a common interface to four MaxMind databases - it works with MaxMind's databases for looking up geographic location, ISP, domain, and connection type from an IP address
- Better type safety - the MaxMind Java library is somewhat null-happy. This wrapper uses Option-boxing wherever possible
- Better performance - as well as or instead of using MaxMind's own caching (
CHMCache
), you can also configure an LRU (Least Recently Used) cache of variable size
Installation
The latest version of scala-maxmind-iplookups is 0.5.0 and is compatible with Scala 2.11 and 2.12.
Add this to your SBT config:
val maxmindIpLookups = "com.snowplowanalytics" %% "scala-maxmind-iplookups" % "0.5.0"
Retrieve the GeoLite2-City.mmdb
file from the MaxMind downloads page
(direct link).
MaxMind also has databases for looking up ISPs, domain names, and connection types from IP addresses. Scala MaxMind IP Lookups supports all of these.
Usage
Here is a simple usage example, performing just a geographic lookup and not the ISP, domain, or connection type lookups:
import cats.effect.IO
import com.snowplowanalytics.maxmind.iplookups.IpLookups
val result = (for {
ipLookups <- IpLookups.createFromFilenames[IO](
geoFile = Some("/opt/maxmind/GeoLite2-City.mmdb")
ispFile = None,
domainFile = None,
connectionTypeFile = None,
memCache = false,
lruCacheSize = 20000
)
lookup <- ipLookups.performLookups[IO]("175.16.199.0")
} yield lookup).unsafeRunSync()
result.ipLocation match {
case Some(Right(loc)) =>
println(loc.countryCode) // => "CN"
println(loc.countryName) // => "China"
case _ =>
println("Lookup failed")
}
Note that GeoLite2-City.mmdb
is updated by MaxMind each month..
For further usage examples for Scala MaxMind IP Lookups, please see the tests in
IpLookupsTest.scala
. The test suite uses test databases provided by
MaxMind.
Implementation details
IpLookups constructor
The signature is as follows:
case class IpLookups(
geoFile: Option[File],
ispFile: Option[File],
domainFile: Option[File],
connectionTypeFile: Option[File],
memCache: Boolean = true,
lruCache: Int = 10000
)
In the IpLookups
companion object there is an alternative constructor which takes Option[String]
as file paths to the databases instead:
def apply(
geoFile: Option[String],
ispFile: Option[String],
domainFile: Option[String],
connectionTypeFile: Option[String],
memCache: Boolean = true,
lruCache: Int = 10000
)
The first four arguments are the MaxMind databases from which the lookup should be performed.
geoFile
, ispFile
, domainFile
, and connectionTypeFile
refer respectively to MaxMind's
databases for looking up location, ISP, domain, and connection type based on an IP address. They are
all wrapped in Option
, so if you don't have access to all of them, just pass in None
as in the
example above.
In both signatures, the memCache
flag is set to true
by default. This flag enables MaxMind's own
caching (CHMCache
).
The lruCache
value defaults to 10000
- meaning Scala MaxMind IP Lookups will maintain an LRU
cache of 10,000 values, which it will check prior to making a MaxMind lookup. To disable the LRU
cache, set its size to zero, i.e. lruCache = 0
.
Returned value
The performLookups(ip)
method returns a:
case class IpLookupResult(
ipLocation: Option[Either[Throwable, IpLocation]],
isp: Option[Either[Throwable, String]],
organization: Option[Either[Throwable, String]],
domain: Option[Either[Throwable, String]],
connectionType: Option[Either[Throwable, String]]
)
The first element is the result of the geographic location lookup. It is either None
(if no
geographic lookup database was provided) or Some(ipLocation)
, where ipLocation
is an instance of
the IpLocation
case class described below. The other three elements in the tuple are Option
s
wrapping the results of the other four possible lookups: ISP, organization, domain, and connection
type.
Note that enabling providing an ISP database will return an organization
in addition to an isp
.
IpLocation case class
The geographic lookup returns an IpLocation
case class instance with the following structure:
case class IpLocation(
countryCode: String,
countryName: String,
region: Option[String],
city: Option[String],
latitude: Float,
longitude: Float,
timezone: Option[String],
postalCode: Option[String],
metroCode: Option[Int],
regionName: Option[String],
isInEuropeanUnion: Boolean
)
An example using multiple databases
This example shows how to do a lookup using all four databases.
import com.snowplowanalytics.maxmind.iplookups.IpLookups
val ipLookups = IpLookups(
geoFile = Some("/opt/maxmind/GeoLite2-City.mmdb"),
ispFile = Some("/opt/maxmind/GeoIP2-ISP.mmdb"),
domainFile = Some("/opt/maxmind/GeoIP2-Domain.mmdb"),
connectionType = Some("/opt/maxmind/GeoIP2-Connection-Type.mmdb"),
memCache = false,
lruCache = 10000
)
val lookupResult = ipLookups.performLookups("70.46.123.145")
// Geographic lookup
println(lookupResult.ipLocation).map(_.countryName) // => Some(Right("United States"))
println(lookupResult.ipLocation).map(_.regionName) // => Some(Right("Florida"))
// ISP lookup
println(lookupResult.isp) // => Some(Right("FDN Communications"))
// Organization lookup
println(lookupResult.organization) // => Some(Right("DSLAM WAN Allocation"))
// Domain lookup
println(lookupResult.domain) // => Some(Right("nuvox.net"))
// Connection type lookup
println(lookupResult.connectionType) // => Some(Right("Dialup"))
LRU cache
We recommend trying different LRU cache sizes to see what works best for you.
Please note that the LRU cache is not thread-safe (see this note). Switch
it off if you are thinking about performing ip lookups with the same IpLookups
instance across
threads.
Building etc
Assuming you already have SBT installed:
$ git clone git://github.com/snowplow/scala-maxmind-iplookups.git
$ cd scala-maxmind-iplookups
$ sbt test
<snip>
[info] Passed: : Total 276, Failed 0, Errors 0, Passed 276, Skipped 0
GeoLite Legacy discontinuation
Maxmind are discontinuing updates for the GeoLite Legacy databases in April 2018.
We will be discontinuing updates to the GeoLite Legacy databases as of April 1, 2018. You will still be able to download the April 2018 release until January 2, 2019. GeoLite Legacy users will need to update their integrations in order to switch to the free GeoLite2 or commercial GeoIP databases by April 2018.
For more information, please visit our Support Center.
In addition, in 2019, latitude and longitude coordinates in the GeoLite2 databases will be removed.* Latitude and longitude coordinates will continue to be provided in GeoIP2 databases. Please check back for updates.
As such we recommend upgrading to version 0.4.0 as soon as possible
Copyright and license
Copyright 2012-2018 Snowplow Analytics Ltd.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this software except in compliance with the License.
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.