extractus / feed-extractor

Simplest way to read & normalize RSS/ATOM/JSON feed data

Home Page:https://extractor-demos.pages.dev/feed-extractor

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Some items being ignore due to hardcoded limits

dsl101 opened this issue · comments

Can I ask what the purpose behind the length checks at the beginning of normalize() are? Specifically this:

  if (!link || !title ||
    !isString(link) || !isString(title) ||
→   link.length < 10 || title.length < 10) {
    return false;
  }

It's taken me ages to track down why some items in a feed are returning as undefined, and it's because the title was short.

I have a service that uses this package to parse content for some websites (TechPush, FOMO, etc). When I get an article with title < 10 letters, if there is no error, that article itself is often not interesting to readers.
Related: #1

Ah yes - I didn't spot #1 but I really think that should be under control of the application not the library...