bladerunnersde / blog-aggregator

A RSS feed aggregator in Go

Blog Aggregation

Introduction

It's a web server that allows clients to:

• Add RSS feeds to be collected
• Follow and unfollow RSS feeds that other users have added
• Fetch all of the latest posts from the RSS feeds they follow
• RSS feeds are a way for websites to publish updates to their content.

You can use this project to keep up with your favorite blogs, news sites, podcasts, and more!

Steps

I published each step of my design process in Medium, and it'll be helpful to follow along the details of this project with the articles if you want to learn to scrape:

1. Build a Content Aggregator in Go(1-Servers)
2. Build a Content Aggregator in Go(2-PostgreSQL)
3. Build a Content Aggregator in Go(3-Create Users)
4. Build a Content Aggregator in Go(4-API Key)
5. Build a Content Aggregator in Go(5-Create a Feed)
6. Build a Content Aggregator in Go(6-Get all feeds)
7. Build a Content Aggregator in Go(7-Feed Follows)
8. Build a Content Aggregator in Go(8-Scraper)
9. Build a Content Aggregator in Go(9-Database POSTs Table)

Improvements

The codebase needs refactoring and more optimizations, please feel free to contribute.

Some ideas:

• Support pagination of the endpoints that can return many items
• Support different options for sorting and filtering posts using query parameters
• Classify different types of feeds and posts (e.g. blog, podcast, video, etc.)
• Add a CLI client that uses the API to fetch and display posts, maybe it even allows you to read them in your terminal
• Scrape lists of feeds themselves from a third-party site that aggregates feed URLs
• Add support for other types of feeds (e.g. Atom, JSON, etc.)
• Add integration tests that use the API to create, read, update, and delete feeds and posts
• Add bookmarking or "liking" to posts
• Create a simple web UI that uses the backend API