ruippeixotog / scala-scraper

A Scala library for scraping content from HTML pages

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

>> and >?> operators produce empty string in case of a missing attribute

sengo4hd opened this issue · comments

Hi, thanks for your work on this project.
The title says it all. I'm scraping web pages to get informations like :
doc >?> attr("content")("meta[property=og:description]")
If the meta[property=og:description] is missing I get Some("") instead of None, or simply "" with the >> operator.
I didn't really dig deep into the code as a simple wrapper around >> was enough to get the expected result. But the current behavior is a bit deceptive.

Hi! I agree, in your case doc >> attr("content")("meta[property=og:description]") should throw an exception and doc >?> attr("content")("meta[property=og:description]") should return None. I'll fix this promptly.