literotica.com: Get Story URLs from Web Page with 'Show More' author list
CountZero85 opened this issue · comments
I noticed on literotica that when authors have a lot of individual stories (enough that the indicator shows load more ) or very long series it will only scrape a portion of the authors works
ex https://www.literotica.com/authors/BlaQQuill/works/stories , Blaquil it scrapes till volume 64 not 65
or a more extreme example
JukeboxEMCSA ; https://www.literotica.com/authors/JukeboxEMCSA/works/stories
Has over 700 stories only scrapes 50 or so
The site limits the author story list now and uses javascript to fetch more data when you click 'Show More'. FFF doesn't have javascript.
As a work around, after you've clicked 'Show More', you can select the text on the page containing the story links you want, and then if you drag/drop the selected text into FFF's download URLs edit box, FFF will pick out the story URLs.
If someone wants to put in the effort, it should be possible to override the before_get_urls_from_page() and/or get_urls_from_page() methods for the literotica adapter.