xarantolus / Collect

A server to collect & archive websites that also supports video downloads

Home Page:https://010.one/Collect/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Modernize

xarantolus opened this issue · comments

This software is

  1. quite old (especially regarding TypeScript/JS coding conventions)
  2. not well suited for larger collections (no search, no tagging etc.)
  3. barely updated (dependencies are out of date, some of them no longer supported etc)
    • I think this server doesn't work with newer versions of the website-scraper node module, so it's still on an older version

While it still works (I've had it running for 4+ years), it feels like it's time for an overhaul. The following could be good steps:

  1. Creating a better frontend (the code in browser.js works, but is a mess)
  2. Creating a docker (compose) configuration that just works to simplify installation
  3. Maybe switch to a database instead of just storing everything in a JSON file (not sure I'll do this, as it complicates setup etc.)
  4. Updating the downloader code to use newer website-scraper versions

2 & 4 should be possible without major rewrites, the others are somewhat more involved.

Maybe it also makes sense to rewrite the server in Go because I like it better, but I'm not sure if there's a good website-scraper-like module for Go. I actually tried writing something like that that started a chromium browser and used SingleFile to download pages, but it didn't work that well.

I'm not sure when (or if) I'll work on this.