Giters

ruippeixotog / scala-scraper

A Scala library for scraping content from HTML pages

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

set browser cookies

KiaraGrouwstra opened this issue · comments

commented

I'm fetching from a site that requires users to login in. This works fine using this lib.
However, if I restart the program, it will have lost the cookies, and needs to redo the login procedure. I'd like to be able to persist the cookies to resolve this.
Currently cookieMap is private however, preventing me from being able to load in persisted cookies. It would be nice if there were a public setter for it.

commented

Hi @tycho01, thanks for submitting the issue! Yes, having setCookie and setCookies in the Browser trait seems very reasonable to me.

I see that you already added it in 1536930. Do you want to do a pull request with it? I'll only ask for two things:

  • I'd prefer if setCookies imitated the Set-Cookie HTTP header and just replaced the cookie that you passed specifically (in other words, setCookies would merge the map passed as argment with the internal one, instead of completely replacing it). The behavior you have right now could still be achieved with a clearCookies followed by a setCookies, of course;
  • Can you make the signature like setCookies(url: String, m: Map[String, String]), with the URL being the first argument? It's a purely personal preference and just a suggestion.
commented

Sure! I've tried incorporating your feedback now. Let me know if there's anything else I can improve about it!