USCDataScience / autoextractor

A toolkit for clustering web pages based on various similarity measures.

Auto Extractor

An intelligent extractor library which learns the structures of the input web pages and then figures out a strategy for scraping the structured content.

Links

• Build Instructions
• Clustering Web Pages using Apache Spark

Developers:

• Thamme Gowda, USC
• Chris Mattmann, USC & NASA JPL

Citation:

If you use this work, please cite: https://ieeexplore.ieee.org/abstract/document/7785739

@inproceedings{gowda2016clustering,
  title={Clustering Web Pages Based on Structure and Style Similarity (Application Paper)},
  author={Gowda, Thamme and Mattmann, Chris A},
  booktitle={Information Reuse and Integration (IRI), 2016 IEEE 17th International Conference on},
  pages={175--180},
  year={2016},
  organization={IEEE}
}

References :

• K. Zhang and D. Shasha. 1989. "Simple fast algorithms for the editing distance between trees and related problems". SIAM J. Comput. 18, 6 (December 1989), 1245-1262.
• Jarvis, R.A.; Patrick, Edward A., "Clustering Using a Similarity Measure Based on Shared Near Neighbors," in Computers, IEEE Transactions on , vol.C-22, no.11, pp.1025-1034, Nov. 1973