ahujaraman / live_log_analyzer_spark

Spark Application for analysis of Apache Access logs and detect anamolies! Along with Medium Article.

Home Page:https://towardsdatascience.com/build-log-analytics-application-using-apache-spark-b5eeca1e53ba

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

live_log_analyzer_spark

Spark Application for analysis of Apache Access logs and detect anamolies!

Link to My Medium Article in Towards Data Science Publication : Article

The Application has features to extract information from unstrucutred text logs, and analyze the trends the trends based on Response codes, Traffic, Frequent Visitors, Top End Points, Content Flow etc to detect the anamolies.

Frequent End Points

Frequent End Point

Response Code Analysis

Response Code Analysis

Traffic Flow Weekly Analysis

Traffic Flow weekly analysis

Traffic Flow Particular Day

Traffic Flow Particular Day

The Above plot, depicts the anamolies of spikes on certain times, which can be crucial starting point to detect the anamolies.

About

Spark Application for analysis of Apache Access logs and detect anamolies! Along with Medium Article.

https://towardsdatascience.com/build-log-analytics-application-using-apache-spark-b5eeca1e53ba


Languages

Language:Python 50.6%Language:Jupyter Notebook 49.4%