SSL connect error
tonybyng opened this issue · comments
Hi
Im getting an
SSL connect error for "https://bbc.in/3cNMnkw".
when connecting to some websites. I'm writing a tweetdeck style system and so will scrape the site that the tweet links to to pick up the details so that a nicer link can be generated. Any ideas on how to solve this ? Most work fine, its just a few that fail.
Also Im sometimes getting a "This browser is no longer supported" coming through as a description from the page. I notice you are passing through 'Mozilla/5.0 (compatible; PHP Scraper/0.x; +https://phpscraper.de)' as the user agent. Is it worth altering the user agent ? I dont think I can pass it in as a parameter anywhere from what I can see?
Thanks.
Hello @tonybyng,
sorry for the late response. I've been flat out busy. Sounds interesting what you are building. Sometimes changing the agency can help solving the issue. Did you try it ?
Cheers,
Peter
Hello @spekulatius,
Something similar happens to me, I am trying to scrape the elcorteingles.es website, but I got only: Access Denied. I tried to change 'HTTP_USER_AGENT' but it didn't work for me, is there a way to pass other parameters to the header?
Thanks
Hello @gnunez-dev
currently there aren't any other options to pass in parameters. Definitely something worthy to consider. I'm pretty busy atm, but I'm open to reviewing a PR if you are keen to give it a try.
Cheers,
Peter
Closing this for now. Please open a new issue when it still is an issue.