A collection of Python scripts that leverage Selenium and/or Scrapy to scrape content from Question Answering sites, such as and other than Stack Overflow.
So far, three scrapers are available:
Please, refer to the README.md files within each subfolder for more.
Pleace, cite the following paper if you decide to use these scripts for your own research purposes.
F. Calefato, F. Lanubile, N. Novielli. “Moving to Stack Overflow: Best-Answer Prediction in Legacy Developer Forums.” In Proc. 10th Int’l Symposium on Empirical Softw. Eng. and Measurement (ESEM’16), Ciudad Real, Spain, Sept. 8-9, 2016, DOI:10.1145/2961111.2962585.
@inproceedings{calefato_2016_esem,
author = {Calefato, Fabio and Lanubile, Filippo and Novielli, Nicole},
title = {Moving to Stack Overflow: Best-Answer Prediction in Legacy Developer Forums},
booktitle = {Proc. of the 10th ACM/IEEE Int'l Symposium on Empirical Software Engineering and Measurement},
series = {ESEM '16},
year = {2016},
isbn = {978-1-4503-4427-2},
location = {Ciudad Real, Spain},
pages = {13:1--13:10},
articleno = {13},
numpages = {10},
url = {http://doi.acm.org/10.1145/2961111.2962585},
doi = {10.1145/2961111.2962585},
acmid = {2962585},
publisher = {ACM},
address = {New York, NY, USA},
keywords = {Best-answer prediction, Developer forums, Q\&A sites, Stack Overflow},
}