Social networks crawler

A program to download photo and video from any site (e.g. Reddit, Twitter, Instagram, TikTok, RedGifs, PornHub, XHamster, XVIDEOS, LPSG).

If you like SCrawler, please like the program on this site

Do you like this program? Consider adding to my coffee fund by making a donation to show your support. 😊

Bitcoin: BC1Q0NH839FT5TA44DD7L7RLR97XDQAG9V8D6N7XET

What can program do:

Download pictures and videos from users' profiles and subreddits:
- Reddit images, galleries of images, videos (downloading Reddit hosted video is going through ffmpeg (ffmpeg only works with the x64 program)), saved posts;
- Redgifs videos (https://www.redgifs.com/);
- Twitter images and videos, saved (bookmarked) posts;
- Instagram images and videos, tagged posts, stories, saved posts;
- TikTok videos (limited);
- Imgur images, galleries and videos;
- Gfycat videos;
- PornHub images, videos, save (liked) posts;
- XHamster images, videos, saved posts;
- XVIDEOS videos;
- Other supported sites
Parse channel and view data
Download saved Reddit, Twitter and Instagram posts
Add users from parsed channel
Advanced user management
Automation (downloading data automatically every X minutes)
Feed (feed of downloaded media files)
Labeling users
Create download groups
Adding users to favorites and temporary
Filter exists users by label or group
Selection of media types you want to download (images only, videos only, both)
Download a special video, image or gallery
Making collections (grouping users into collections)
Specifying a user folder (for downloading data to another location)
Changing user icons
Changing view modes
...and many others...

SITES REQUIREMENTS

How it works

First, the program downloads the full profile. After the program downloads only new posts. The program remembers downloaded posts.

The program parses all user posts, obtain MD5 images hash and compares them with existing ones to remove duplicates. Then the media will be downloaded.

Other sites

The program parses all user posts and compares file names with existing ones to remove duplicates. Then the media will be downloaded.

How to request a new site

Read here about

Requirements

Windows 10, 11 with NET Framework 4.6.1 or higher (v4.6.1 must be installed). You can check version compatibility with this tool.
SITES REQUIREMENTS

Guide

Main window
SETTINGS
- SITES REQUIREMENTS
- Reddit
- Twitter
- Instagram
- TikTok
- RedGifs
- PornHub
- XHamster
- XVIDEOS
- LPSG

Full guide you can find here

Installation

Just download the latest release, unzip the program archive to any folder, copy the file ffmpeg.exe into it and enjoy. 😊

Don't put program in the Program Files system folder (this is portable program and program settings are stored in the program folder)

Updating

Just download latest version and unpack it into the program folder. Before starting a new version, I recommend making a backup copy of the program settings folder.

How to build from source

Read about how to build from source here

How to make a plugin

Read about how to make plugin here.

How to support

Read more about how to support the program here.

Settings and usage

The program has an intuitive interface.

SITES REQUIREMENTS

Just add a user profile and click the Download button.

Read more about adding users and subreddits here

Using program as just video downloader

Create a shortcut for the program. Open shortcut properties. In the Shortcut tab, in the Target field, just add the letter v at the end across the space.

Example: D:\Programs\SCrawler\SCrawler.exe v

halalgami / SCrawler