automatically get new UserAgents
MadAim123 opened this issue · comments
MadAim123 commented
Is your feature request related to a problem? Please describe.
The UserAgents will be old after some days. An automatically way to update them could be good
Describe the solution you'd like
Crawl the newest 4 needed useragents without manually work
Describe alternatives you've considered
Something like this could be used in "utils":
try:
# JSON URL containing user agents
json_url = "https://jnrbsn.github.io/user-agents/user-agents.json"
# Fetch the JSON data from the URL
response = requests.get(json_url)
user_agents_json = response.json()
# Extract the first 4 user agents
first_4_user_agents = user_agents_json[:4]
# Save the user agents into the USERAGENTS list
USERAGENTS = first_4_user_agents
except Exception as e:
USERAGENTS = [
"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36",
"Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36",
"Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36",
"Mozilla/5.0 (Macintosh; Intel Mac OS X 14_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/117.0.0.0 Safari/537.36",
]
deedy5 commented
Hello!
Thanks for the suggestion, but there's no point in doing it.
- Api requests will work with headers of previous browser versions as well.
- User can specify any headers he wants.
At least for the moment.