Verba FGPS - Facebook Group Posts Scraper

This tool was responsible as synchronization mechanism between FB group and site storage Written in Java and Selenide Web Driver. Simple jar file scrapes facebook group posts and push them to database

Important info

The tool has been developed to store data into Firebase. Currently, export data into .xsl file and postgres is not supported

What does it scrape

post text
post images
post link
post date
author name
author link
author avatar

How it scrapes

tool reads last N posts from specified facebook group
store them into database (needs to be intalled on computer)
wait Z minutes
go to step 1

N, Z and database parameters are configurable

Requirements:

java 8;
maven 3.6.0
FireFox 60.3.0esr (64-bit);

Build project:

Adjust project settings in PostDataToFirebase.properties and application.properties files
Navigate to project dir from cmd
Run mvn clean install command

Run project:

Run JAR: java -jar fbreaper-MILESTONE-2.1.jar --scheduling.enabled=false
Run JAR with scheduling: java -jar target\fbreaper-MILESTONE-2.1.jar --scheduling.enabled=true --cron.expression="* */5 * * * *"

Availabel parameters:

PostDataToFirebase.properties:

fb.login fb.pass   
fb.group.url
posts.to.fetch

application.properties:

firebase.jsonfile.path
firebase.storage.bucket
cron.expression
fb.big.images.limit
fb.big.images.load.timeout
selenide.timeout

If the codebase helped you somehow and/or you just want to say "thank you" then you may share link to my Java blog stacktrace.biz at your social networks and/or follow me at Facebook or Twitter

Kriztoper / facebook-group-scrape-tool