By Shashank Agarwal (agarwal.202@osu.edu) Anurag Kalra (kalra.25@osu.edu)
all code is in the code6 folder and reports are in the report6 folder in pdf and docx format
##Instructions
Part 1:
Procedure:
-
Create assosiation rules for all (or split 80-20) documents in files "docs.txt" and labels.txt using apriori command: ./apriori -s5 -c80 -tr -R labels.txt docs_80.txt output.txt Final file-> output.txt
-
Iterate over various values of support (-s) and confidence (-c)
-
Use classification rules to predict files and check accuracy commnd: sudo python test.py output.txt docs_20.txt res.txt Final File->res.txt
-
Do above for 60-40 split
===========================================================================================================================
Part 2:
Procedure
-
Create feature_vector for the docs file "feature_matrix.pytext"
-
Use the feature vector to cluster (k=8,k=16) command: kmeans.py 8
-
Create separate document & labels files for each cluster output -> files from range 0.txt to k.txt labels from 0_labels to k_labels.txt
-
Split and run apriori on each file command: ./apriori -s3 -c50 -m2 -tr -R labels.txt docs.txt output.txt final_file-> output.txt
-
Use classification rules to predict files and check accuracy commnd: python test.py output.txt 0.txt res.txt Final File->res.txt
A simple way to do all above is to use the below comman command: python shell.py docs.txt labels.txt 60 3 50 here 60 is the split (60-40) 3 is the support 50 is the confidence
The files from 8 means where (k=8) are provided in file 0 to k.txt and labels from 0 to k_labels.txt
Executing the Code:
To run apriori on docs.txt & labels.txt enter:
make apr
#deafault values: -s3 -c50
To get the accuracy run: make accuracy
To split and run tests run
make shell
The results are saved in results.txt file
This runs the following command:
python shell.py docs.txt labels.txt 60 3 50
here 60 is the split
3 is the support
50 is the confidence
=========================================================================================================================== Contributions: Shashank Agarwal & Anurag Kalra both implemented the "apriori" classification rules generation and testing scripts. The work overlaps and is hard to be segregated.
P.S: The kmeans file have not been included as the feature matrix was very large and kmeans required a lot of files that have been submitted in previous projects.