Crime Analysis Prediction
Linear Model to predict the crime rate of North Carolina. Detailed EDA done prior to building the model.
Data Description
The dataset contains the data for crime rate in the state of North Carolina aggregated by county.
Data source: https://github.com/AdroitAnandAI/Crime-Analysis-Prediction/blob/master/crime_v2.csv
Data Attributes
- county - county identifier
- year - 1987
- crmrte - crimes committed per person
- prbarr - 'probability' of arrest
- prbconv - 'probability' of conviction
- prbpris - 'probability' of prison sentence
- avgsen - avg. sentence, days
- polpc - police per capita
- density - people per sq. mile
- taxpc - tax revenue per capita
- west - =1 if in western N.C.
- central - =1 if in central N.C.
- urban - =1 if in SMSA
- pctmin80 - perc. minority, 1980
- wcon - weekly wage, construction
- wtuc - wkly wge, trns, util, commun
- wtrd - wkly wge, whlesle, retail trade
- wfir - wkly wge, fin, ins, real est
- wser - wkly wge, service industry
- wmfg - wkly wge, manufacturing
- wfed - wkly wge, fed employees
- wsta - wkly wge, state employees
- wloc - wkly wge, local gov emps
- mix - offense mix: face-to-face/other
- pctymle - percent young male
Objective
- To perform univariate and bivariate exploratory analysis of the dataset provided.
- To develop a suitable linear model with crmrte as the dependent variable based on the EDA findings.
- To explain the various aspects of the model used.
Data Analysis & Data Cleaning
-
The last column was getting read as 'object' data. It was found to be due to the special symbol at the last row, last column 0.074198931'. Removed the special symbol from input csv file, to fix it.
-
It is to be noted that the maximum value of probability features, prbarr & prbconv, are > 1 which is a data anomaly. prbpris & pctymle are found to be < 1.
-
There are 91 entries for all the 25 columns. Hence, there is no missing value in the input dataset. Thus, no need to do data imputation or to drop any feature.
-
The zeros for features, west, central and urban are expected, as the data is inherently boolean.
Observations
From above analysis, it is found, that some rows have to be dropped before doing regression analysis. The probability values of some rows are found to be > 1 and location of one row was found to be both ’west’ and ’central’ at the same time. We will drop these rows before building the model. The special character error in the input dataset is also fixed.
Univariate Analysis
Univariate visualization provides summary statistics for each field in the raw data set. It is conducted to find out how much a single feature in the dataset would be helpful to determine the target feature, here in this case, crime rate.
Observation:
The features density, mix, police per capita, probability of conviction and tax revenue per capita seems to have similar distribution as crime rate. But no definitive conclusion can be made from this observation. Lets examine further using bivariate analysis.
Probability/ Cumulative Distribution Function (CDF)
Observations:
a) One strange observation is in weekly wages of service industry (wser). More than 95% of wages lies below 400, but the maximum wage is around 2250. From the data, this is identified to be county 185. As the percentage of minorities in this county is high (nearly 65%) and wages in other sectors are comparatively less, the wages of service industry is mostly an error. We will remove "county 185" from the input data.
b) Though the maximum value of tax revenue per capita is 120, more than 50% of values lies below 40.
c) Though the maximum value of police per capita is 0.009, more than 60% of values lies below 0.001.
Bivariate Analysis
Bivariate visualization is performed to find the relationship between each variable in the dataset and the target variable of interest, i.e. crime rate. The plot of all features against crime rate is done as below.
Observations:
a) Based on the above pairplot, it can be noted that density is most positively correlated with crime rate. There is also some correlation with weekly wages under different domains but it needs further investigation, as they are not so pronounced.
b) Strangely, the weekly wage features and crime rate is found to be slightly positively correlated. This signifies unequal distribution of income or probably high unemployment rate. One of the most important features that is not in the given data may be unemployment rate.
Lets try to find if there is any correlation among features for each location: ’west’, ’central’ & ’urban’.
Correlation among features for each location: 'west', 'central' & 'urban'
Lets try to find if there is any correlation among features for each location: 'west', 'central' & 'urban'.
Number of data points in category: west is 23 Number of data points in category: central is 34 Number of data points in category: urban is 8
Location: west
Location: central
Location: urban
Observations:
1) Some of the correlation lines are showing upward or downward trends more than before.
2) Probability of conviction is found to have negative correlation with crime rate in both west and central, but not in urban areas.
3) Tax Per capita is found to have positive correlation with crime rate in both central and urban areas.
4) Percentage of minority is positively correlated with crime rate, both in west and in urban areas.
5) Thus, a combination of density and location (urban/ west/ central) might help aid crime rate prediction.
6) However, there seems to be not much data for ’urban areas’ to arrive at a conclusion.
Linear Regression Fit of Strongly Correlated Features
We have a lot of features to analyse in the input dataset. So let's take the strongly correlated quantitative features from this dataset and analyse them one by one.
Box Plots
Let's do the box plot & violin plot for the boolean features 'west', 'central', 'urban' to find impact on crime rate, if any.
Violin Plots
Observations:
a) The crime rate in urban areas is found to be significantly high. Thus, the feature ’urban’ is an useful variable for prediction.
b) The crime rate in west is found to be less and central moderate. But as there is significant overlap, such variations may not be very helpful for prediction.
Feature-Feature Correlation Analysis
Many times, more than one input could be dependent on each other. In Linear Regression, the requirement is that all the input variables are independent of each other.
When a feature is dependent on one or more of the other input features, it leads to a phenomenon known as multi-collinearity. Multi-collinearity among features can be identified by doing Feature-Feature correlation analysis.
Observations from the Feature HeatMap:
a) The density and urban variable seems to be highly correlated, which is obvious, because urban areas are densely populated.
b) Some of the "wage features" are positively correlated, as the wage increase/ decrease in one domain would certainly influence the other. For example, wtrd & wfir are positively correlated to wfed & wloc. Also, wfir and wtrd have moderate correlation.
Observations from Zoomed Feature HeatMap:
a) Density and crime rate have a correlation of 0.73. But density has high correlation with ’urban’ feature. Hence, whether both features, density and urban, are useful to predict crime rate needs further investigation. We wll use linear regression to sort out this question.
b) The feature, ’urban’ has a correlation of 0.62 with crime rate, but whether the correlation is because ’urban’ has very high correlation with ’density’ is yet to be known.
c) Wage columns, wfed & wtrd are positively correlated to ’density’ feature. This can be intuitively understood as the weekly wages would be higher in urban areas.
Conclusions
Data Analysis and Cleaning
It is found, that some rows have to be dropped before doing regression analysis. The probability values of some rows are found to be > 1 and location of one row was found to be both ’west’ and ’central’, at the same time. We will drop these rows before building the model. The special character error in the input dataset is also fixed.
Univariate Analysis
-
The features density, mix, police per capita, probability of conviction and tax revenue per capita seems to have similar distribution as crime rate. But no definitive conclusion can be made from this observation.
-
One strange observation is in weekly wages of service industry (wser). More than 95% of wages lies below 400, but the maximum wage is around 2250.
From the data, this is identified to be county 185. As the percentage of minorities in this county is high (nearly 65%) and wages in other sectors are comparatively less, the wages of service industry is mostly an error. We will remove "county 185" from the input data. -
Though the maximum value of tax revenue per capita is 120, more than 50% of values lies below 40.
-
Though the maximum value of police per capita is 0.009, more than 60% of values lies below 0.001.
Bivariate Analysis
-
Based on the above pairplot, it can be noted that density is most positively correlated with crime rate. There is also some correlation with weekly wages under different domains but it needs further investigation, as they are not so pronounced.
-
Strangely, the weekly wage features and crime rate is found to be slightly positively correlated. This signifies unequal distribution of income or probability of high unemployment rate. One of the most important features that is not in the given data is unemployment rate.
Correlation among features for each boolean feature
-
Some of the correlation lines are showing upward or downward trends more than before.
-
Probability of conviction is found to have negative correlation with crime rate in both west and central, but not in urban areas.
-
Tax Per capita is found to have positive correlation with crime rate in both central and urban areas.
-
Percentage of minority is positively correlated with crime rate, both in west and in urban areas.
-
Thus, a combination of density and urban (or west or central) can help aid crime rate prediction.
-
However, there seems to be not much data for 'urban areas' to arrive at a conclusion.
Linear Fit of Top Correlated Features
-
The crime rate in urban areas is found to be significantly high. Thus, the feature 'urban' is an useful variable for prediction.
-
The crime rate in west is found to be less and central moderate. But as there is significant overlap, such variations may not to be very helpful for prediction.
Feature-Feature Correlation Analysis
-
Many times, more than one input could be dependent on each other. It leads to a phenomenon known as multi-collinearity, which can be identified by doing Feature-Feature correlation analysis. In Linear Regression, the requirement is that all the input variables are independent of each other.
-
The density and urban variable seems to be highly correlated, which is obvious, because urban areas are densely populated.
-
Some of the "wage features" are positively correlated, as the wage increase/ decrease in one domain would certainly influence the other. For example, wtrd & wfir are positively correlated to wfed & wloc. Also, wfir and wtrd have moderate correlation.
-
Density and crime rate have a correlation of 0.73. But density has high correlation with 'urban' feature. Hence, whether both features, density and urban, are useful to predict crime rate needs further investigation. We wll use linear regression to sort out this question.
-
The feature, 'urban' has a correlation of 0.62 with crime rate, but whether the correlation is because 'urban' has very high correlation with 'density' is yet to be known.
-
Wage columns, wfed & wtrd are positively correlated to 'density' feature. This can be intuitively understood as the weekly wages would be higher in urban areas.
The above observations from EDA would be carried forward to help Linear Regression (Part II).
==============================================================================
Linear Model on North Carolina Crime Rate Dataset (Part II)
Objective
To use insights from EDA to develop a suitable linear model with crmrte as the dependent variable and explain the various aspects of the model.
Actionable Observations from EDA
-
The density and urban variable has highest correlation with crime rate.
-
But, density and urban variable seems to be highly correlated, which is obvious, because urban areas are densely populated. Hence, there is a high chance of multicollinearity between density and urban features. We wll use linear regression to sort out this question.
-
The feature, 'urban' has a correlation of 0.62 with crime rate, but whether the correlation is because 'urban' has very high correlation with 'density' is yet to be known.
-
A combination of density and urban (or west or central) can help aid crime rate prediction.
-
Wage columns, wfed & wtrd are positively correlated to 'density' feature. This can be intuitively understood as the weekly wages would be higher in urban areas.
-
Some of the "wage features" are positively correlated, as the wage increase/ decrease in one domain would certainly influence the other.
-
wtrd & wfir are positively correlated to wfed & wloc. Also, wfir and wtrd have moderate correlation with each other.
-
There are 6 strongly correlated values with Crime Rate: crmrte, density, urban, wfed, taxpc, wtrd.
Evaluate Observations using Linear Regression Model
Lets evaluate the above observations by building Linear Regression Models, as it helps to understand the relation between variables better.
Creating Model with Most Correlated Feature
Interim Observations:
a) As the p-value of density is 0 (small), the changes in crime rate has got close relation with changes in density.
b) R-squared value is found to be 0.525 with only density as predictor variable. This means that 52.5% variability of crime rate is explained by density feature.
c) Co-effient estimate of 0.0086 indicates one value increase of density would cause 0.0086 value increase in crime rate.
Creating Model with Top 2 Correlated Features
Interim Observations:
a) R-squared value is found to be slightly higher (0.527) when the variable, 'urban' is coupled with density as predictor variables. But, R Squared always goes up when you add more variables regardless of whether the added variable help in prediction or not.
b) Adjusted R Squared, penalizes for adding more variables. Thus, it can go down when you add variables that don’t contribute. Here note that, Adjusted R-squared value has gone down from 0.519 to 0.514. Also, the AIC value is increased from -470 to -469 (the smaller the AIC value, the better the model is).
c) It has been noticed that the p-value of 'density' feature has been increased slightly.
Thus, the model has become more less reliable to explain crime rate, because the feature 'urban' doesnt contribute to prediction. The confusion about the correlation between 'urban' and 'density' variable during EDA step, has been sorted out.
Note: If we add variables that are not meaningful as predictor, then it would cause 'Overfitting'. Then, prediction model would perform great with the training data but not with the real world data.
Multiple Linear Regression: Model with all Features
Removing features from all-feature Model
Interim Observations:
a) Adj. R-squared improved from 0.825 in all-feature model to 0.830, after removal of 2 features 'urban', 'county'.
b) AIC value decreased from -591.3 in all-feature model to -595.2, after removal of 2 features 'urban', 'county'.
Thus, we have a better model than the all-feature model. We will try to remove more features and analyze the model indicators.
Interim Observations:
a) Adj. R-squared of the above model with 8 features dropped is better than the all-feature model.
b) AIC value of the above model is better than the all-feature model.
Thus, we have a better model than the all-feature model by removing more features such as 'wmfg', 'prbpris', 'wloc', 'west', 'wtuc'. We will try to remove even more features with p > 0.05 and evaluate using RMSE.
Model Evaluation Using Cross Validation & RMSE
We will test the change in RMSE value when the features with p > 0.05 are removed. The features with p > 0.05 are prbconv, mix, wfed, wtrd, wcon & avgsen. We will also check the RMSE values for the features removed in the previous model.
RMSE with None removed = 0.011598193103785095
RMSE with wmfg removed = 0.01136661900197904
RMSE with prbpris removed = 0.011775293806203662
RMSE with wloc removed = 0.011190102338212366
RMSE with west removed = 0.011516684556469887
RMSE with wtuc removed = 0.010807252151383978
RMSE with prbconv removed = 0.011772727414832405
RMSE with mix removed = 0.012039673462990959
RMSE with wfed removed = 0.011975451306453819
RMSE with wtrd removed = 0.010915720261174582
RMSE with wcon removed = 0.011764476830686559
RMSE with avgsen removed = 0.011416545616172611
From the bar chart, the RMSE values performs better than 'None' when wtuc, wtrd, wloc, west, wmfg and avgsen are removed. Thus, in addition to the previous model, wtrd & avgsen features are removed. But the R-squared and AIC figures degrade when both the features are removed. Since wtrd has a higher p value, we will remove wtrd in our model.
The lowest cross validation MSE is for wloc, wtrd, prbconv and wcon. Thus, in addition to the previous exclusions, prbconv & wcon also can be dropped. But removal of either feature would increase the RMSE value as per the above plot. Thus, we will remove only wtrd in our model.
OLS Regression Characteristic of Final Model
Testing the Model on Input Data
Conclusion
-
The Actual vs Predicted plot is linear. This signifies the prediction is working fine. The input data set is limited. With more data, the plot could be more linear.
-
As an improvement, we can combine the boolean features: west, central and urban into a single feature with categorical values 1, 2 & 3.. Such a feature can help aid the prediction.
-
If there is a chance to add features, then it might be helpful to get 'unemployment rate'.