vkscraper

Download posts, photos, and videos along with their captions and other metadata from ВКонтакте.

Features

Download posts, photos and videos from users profiles and communities.
Built with archivists in mind: incremental sync.
JSON metadata is stored alongside the media.
Batch file, cronjob friendly.

Usage

Authentication

$ vkscraper --login=MyUsername [--password=MyPassword] [...]

When logging in, vkscraper stores the access token in a file called ./MyUsername.vksession, which will be reused later the next time --login is given. Do not delete the session file, logging in is an expensive operation.

Downloading users and communities

$ vkscraper --login=MyUsername insidevk

By default, all available content will be downloaded.

Archive options

--no-photos
--no-posts
--no-stories
--no-tagged-photos
--no-videos

Directory structure

.
├── photos
│   ├── 2022-03-02T10:20:28Z (34894549853).jpg
│   └── 2022-03-02T10:20:28Z (34894549853).json
├── posts
│   ├── 2022-03-02T10:20:15Z (14312312311).json
│   └── 2022-03-02T10:20:15Z (14312312311).txt
├── stories
│   ├── 2022-01-01T10:20:15Z (54894549852).jpg
│   ├── 2022-01-01T10:20:15Z (54894549852).json
│   ├── 2022-01-01T10:20:28Z (54894549853).mp4
│   └── 2022-01-01T10:20:28Z (54894549853).json
├── tagged_photos
│   ├── 2022-03-02T10:20:28Z (84893123153).jpg
│   └── 2022-03-02T10:20:28Z (84893123153).json
├── videos
│   ├── 2022-01-01T10:20:28Z (64812111853).mp4
│   └── 2022-01-01T10:20:28Z (64812111853).json
├── meta.json
└── id

Batch file

vkscraper can read user profiles and communities from a file. Lines starting with an # or empty lines are considered as comments and ignored. Inline comments are also ignored.

Given DataHoarder.txt:

# Official VK community
insidevk

klavdiacoca # Inline comment 1; Клава Кока; profile
klavacoca   # Inline comment 2; Клава Кока; community

Download stories, photos, videos and posts:

$ vkscraper --login=MyUsername --batch-file=DataHoarder.txt

Fast update

For each target, stop when encountering the first already-downloaded resource. This option is recommended when you use vkscraper to update your personal archive.

This option was taken from Instaloader.

$ vkscraper --login=MyUsername --fast-update insidevk

API

vkscraper is not meant to be a comprehensive API client for VK. The functions and structures used internally may be imported as a library.

import "github.com/kandayo/vkscraper/pkg/vk"

vk := vk.NewClient()

// Login with a username and password.
vk.Login("username", "password")
// Or set an access token.
vk.SetAccessToken("token")

// Find the user or community id.
user, err := vk.Utils.ResolveScreenName("klavacoca")

// Retrieve the user stories feed.
stories, err := vk.Stories.Get(user.ID)

perPage := 100
initialOffset := 0

// Retrieve the user photos.
stories, err := vk.Photos.GetAll(user.ID, perPage, initialOffset)

// Retrieve the user videos.
stories, err := vk.Videos.Get(user.ID, perPage, initialOffset)

lucasintel / vkscraper