Exception during parsing UAG after some uptime

Question

Exception during parsing UAG after some uptime

dkovalenko opened this issue 7 years ago · comments

I have a problem with a parser, it appears in production after some time, not just after start.
Exception is

Caused by: java.lang.IllegalStateException: Entry.next=null, data[removeIndex]=Mozilla/5.0 (Windows NT 6.1; rv:41.0) Gecko/20100101 Firefox/41.0=Client(UserAgent(Firefox,Some(41),Some(0),None),OS(Windows 7,None,None,None,None),Device(Other,None,None)) previous=Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.15.2924.87 Safari/537.36=Client(UserAgent(Chrome,Some(56),Some(15),Some(2924)),OS(Windows 10,None,None,None,None),Device(Other,None,None)) key=Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.95 Safari/537.36 MRCHROME value=Client(UserAgent(Mail.ru Chromium Browser,Some(27),Some(0),Some(1453)),OS(Windows 7,None,None,None,None),Device(Other,None,None)) size=1000 maxSize=1000 Please check that your keys are immutable, and that you have used synchronization properly. If so, then please report this to commons-dev@jakarta.apache.org as a bug.
	at org.apache.commons.collections.map.LRUMap.reuseMapping(LRUMap.java:301)
	at org.apache.commons.collections.map.LRUMap.addMapping(LRUMap.java:267)
	at org.apache.commons.collections.map.AbstractHashedMap.put(AbstractHashedMap.java:284)
	at scala.collection.convert.Wrappers$JMapWrapperLike$class.put(Wrappers.scala:269)
	at scala.collection.convert.Wrappers$JMapWrapper.put(Wrappers.scala:298)
	at com.twitter.util.JMapWrapperLike$class.put(LruMap.scala:20)
	at com.twitter.util.JMapWrapper.put(LruMap.scala:33)
	at org.uaparser.scala.CachingParser$$anonfun$parse$1.apply(CachingParser.scala:11)
	at org.uaparser.scala.CachingParser$$anonfun$parse$1.apply(CachingParser.scala:9)
	at scala.Option.getOrElse(Option.scala:121)
	at org.uaparser.scala.CachingParser.parse(CachingParser.scala:9)
	at service.UaService$Checkers$$anonfun$isMobile$1.apply(UaService.scala:18)

	...

	at play.api.mvc.ActionBuilder$$anon$2.apply(Action.scala:458)
	at play.api.mvc.Action$$anonfun$apply$2$$anonfun$apply$5$$anonfun$apply$6.apply(Action.scala:112)
	at play.api.mvc.Action$$anonfun$apply$2$$anonfun$apply$5$$anonfun$apply$6.apply(Action.scala:112)
	at play.utils.Threads$.withContextClassLoader(Threads.scala:21)
	at play.api.mvc.Action$$anonfun$apply$2$$anonfun$apply$5.apply(Action.scala:111)
	at play.api.mvc.Action$$anonfun$apply$2$$anonfun$apply$5.apply(Action.scala:110)
	at scala.Option.map(Option.scala:146)
	at play.api.mvc.Action$$anonfun$apply$2.apply(Action.scala:110)
	at play.api.mvc.Action$$anonfun$apply$2.apply(Action.scala:103)
	at scala.concurrent.Future$$anonfun$flatMap$1.apply(Future.scala:251)

UaService.scala:18 is just

val client = parser.parse(uag)

Other variant of exception is

play.api.UnexpectedException: Unexpected exception[IllegalStateException: Entry.next=null, data[removeIndex]=Mozilla/5.0 (Windows NT 6.1; rv:41.0) Gecko/20100101 Firefox/41.0=Client(UserAgent(Firefox,Some(41),Some(0),None),OS(Windows 7,None,None,None,None),Device(Other,None,None)) previous=Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.15.2924.87 Safari/537.36=Client(UserAgent(Chrome,Some(56),Some(15),Some(2924)),OS(Windows 10,None,None,None,None),Device(Other,None,None)) key=Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.95 Safari/537.36 MRCHROME value=Client(UserAgent(Mail.ru Chromium Browser,Some(27),Some(0),Some(1453)),OS(Windows 7,None,None,None,None),Device(Other,None,None)) size=1000 maxSize=1000 Please check that your keys are immutable, and that you have used synchronization properly. If so, then please report this to commons-dev@jakarta.apache.org as a bug.]
	at play.api.http.HttpErrorHandlerExceptions$.throwableToUsefulException(HttpErrorHandler.scala:276)
	at play.api.http.DefaultHttpErrorHandler.onServerError(HttpErrorHandler.scala:206)
	at play.core.server.netty.PlayRequestHandler$$anonfun$2$$anonfun$apply$1.applyOrElse(PlayRequestHandler.scala:100)
	at play.core.server.netty.PlayRequestHandler$$anonfun$2$$anonfun$apply$1.applyOrElse(PlayRequestHandler.scala:99)
	at scala.concurrent.Future$$anonfun$recoverWith$1.apply(Future.scala:344)
	at scala.concurrent.Future$$anonfun$recoverWith$1.apply(Future.scala:343)
	at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32)
	at play.api.libs.iteratee.Execution$trampoline$.execute(Execution.scala:70)
	at scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:40)
	at scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:248)
	at scala.concurrent.Promise$class.complete(Promise.scala:55)
	at scala.concurrent.impl.Promise$DefaultPromise.complete(Promise.scala:153)
	at scala.concurrent.Future$$anonfun$flatMap$1.apply(Future.scala:249)
	at scala.concurrent.Future$$anonfun$flatMap$1.apply(Future.scala:249)
	at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32)
	at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
	at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:91)
	at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(BatchingExecutor.scala:91)
	at akka.dispatch.BatchingExecutor$BlockableBatch$$anonfun$run$1.apply(BatchingExecutor.scala:91)
	at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72)
	at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:90)
	at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:39)
	at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:409)
	at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
	at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
	at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
	at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)

UAG samples

Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; MRSPUTNIK 2, 4, 1, 209; .NET CLR 1.1.4322; .NET CLR 2.0.50727)

Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.93 Safari/537.36

Mozilla/5.0 (Windows; U; Windows NT 6.0; ru; rv:1.9.2.6) Gecko/20100625 MRA 5.6 (build 03399) Firefox/3.6.6

Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 YaBrowser/17.6.1.771 Yowser/2.5 Safari/537.36

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36

I'm using parser in singleton context.

Travis Brown · Answer 1 · Tue Sep 12 2017 20:34:34 GMT+0800 (China Standard Time)

I'm not sure what's happening here, but is there any chance you can update to the 0.2.0 release, which replaces the Twitter Util LruMap with an equivalent built on the standard library (#12)?

Dmytro Kovalenko · Answer 2 · Wed Sep 13 2017 17:28:43 GMT+0800 (China Standard Time)

Thank you for your reply, i'll update the version and will report results.

Dmytro Kovalenko · Answer 3 · Tue Oct 03 2017 15:59:09 GMT+0800 (China Standard Time)

After update to 0.2.0 and two weeks of uptime i can confirm that problem is fixed, thank you.

Travis Brown · Answer 4 · Tue Oct 03 2017 23:10:51 GMT+0800 (China Standard Time)

Great, thanks for letting us know, @dkovalenko!