acalejos / Reed

Streaming RSS reader for Elixir

Reed

Streaming RSS parser with a built-in Req plugin for network-enabled chunked streaming.

Installation

def deps do
  [
    {:reed, "~> 0.1.0"}
  ]
end

Reed implements a Sax-based parser for RSS feeds using the Saxy library.

You can manually use the Reed.Handler (which implements the Saxy.Handler behaviour) with Saxy to parse strings or from Streams, but the killer feature of Reed is the Reed.ReqPlugin module, which powers the top-level Reed.get / Reed.get! API.

Reed.ReqPlugin takes advantage of Req's chunking capability to parse RSS feeds directly from over the network, applying transformation functions to each RSS item lazily.

This means you do not have to store the entire RSS feed in memory or on disk to convert to a traditional Elixir Stream (as is required to use Saxy.parse_stream/4), but instead directly uses Saxy.Partial to parse chunk-by-chunk directly over the wire.

The Reed.Transformers module provides some convenient transformation functions to be used during the parsing.

The transformation pipeline is invoked whenever a new RSS item is read, and works with an accumulating state that persists during the entire RSS read.

Examples

Get the feed metadata

import Reed.Transformers
Reed.get!(rss_url, transform: transform(halt()))

Get all items in a list

import Reed.Transformers
Reed.get!(rss_url, transform: transform(collect()))

Get the first 5 items in a list

import Reed.Transformers
Reed.get!(rss_url, transform: collect() |> limit(5) |> transform())

Get all itunes: namespaced elements from the first 2 items as a list

import Reed.Transformers

Reed.get!(rss_url,
  transform:
    transform_item(
      &Map.filter(&1, fn
        {<<"itunes:", _rest::binary>>, _v} -> true
        _ -> false
      end)
    )
    |> collect()
    |> limit(2)
    |> transform()
)