vkontakte-user scrapes get redirected to badbrowser.php
4b5urdX opened this issue · comments
Describe the bug
Scraping for a VKontakte User returns 0 results, while searched accounts do exist.
How to reproduce
snscrape vkontakte-user navalny
Expected behaviour
Get the desired data on the account.
Screenshots and recordings
No response
Operating system
Ubuntu 20.04
Python version: output of python3 --version
python 3.10.12
snscrape version: output of snscrape --version
snscrape 0.7.0.20230622
Scraper
vkontakte-user
How are you using snscrape?
CLI (snscrape ... as a command, e.g. in a terminal)
Backtrace
No response
Log output
2023-09-23 21:42:16.797 INFO snscrape.modules.vkontakte Retrieving initial data
2023-09-23 21:42:16.831 INFO snscrape.base Retrieving https://vk.com/navalny
2023-09-23 21:42:17.599 INFO snscrape.base Retrieved https://vk.com/navalny (redirected to https://vk.com/badbrowser.php): 200
2023-09-23 21:42:17.604 INFO snscrape.modules.vkontakte Wall has no posts
2023-09-23 21:42:17.604 INFO snscrape._cli Done, found 0 results
Dump of locals
No response
Additional context
If you open https://vk.com/badbrowser.php it states the browser in use is outdated. While visiting the site with my default browser it works fine, however snscrape requests always get redirected, which is probably why the 0 results bug occurs. So there seems to be problem with the request itself.
Thanks. This is likely user agent detection on VK's side.
Relatedly, the scraper was already broken before: #737