ibiBgOR / lodestone

Personal Document Archiving (DMS, EDMS for Personal/Home Office use)

Home Page:https://forms.gle/u1RXnbocbFWqfxGb9

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

lodestone_view

Lodestone - Personal Document Search & Archive

GitHub license Docker Pulls Gitter chat Get Notified

NOTE: Lodestone is a Work-in-Progress and is not production ready.

If you want to be a tester/developer, please fill out this Google Form with your contact information: https://forms.gle/u1RXnbocbFWqfxGb9

Lodestone is designed to be the modern and digital equivalent of a home filing cabinet. If you've gone searching for something similar in the past, you might be familiar with terms like Electronic Document Management System (EDMS), Document Management System (DMS) or Personal Archival.

Lodestone is designed around a handful of core features:

  • Full text document search - It doesn't matter what format you're document is in, we should be able to parse it (using OCR) and let you search for the text.
  • Rich tagging - Unlike a physical file cabinet where a document can only exist in one place, digital documents support tags, allowing you to create a flexible organizational structure that works for you.
  • Automated - Document collection & OCR processing should be automatic. Just saving a file to your network drive should be enough to start document processing.
  • Non-destructive - When Lodestone processes a document, the original file will be left untouched, exactly where you left it.
  • Web Accessible - Lodestone is designed to run on a trusted home server and be accessible 24x7.
  • Filesystem/Cloud Sync - Optionally synchronize your tagged documents via a cloud storage provider of your choice (Dropbox, GDrive, etc) or access via a FUSE filesystem mount.

Screenshot

Dashboard

More screenshots available in the docs/screenshots directory.

Installation

Lodestone is made up of a handful of open-source components, and as such its easiest to deploy using Docker/Docker Compose

docker-compose up

# then open the following url in your browser

http://localhost/web

Place your documents in the /data/storage/documents directory, and the Filesystem Collector should automatically start processing them.

If you would like some test documents to play with safely, you can take a look at the AnalogJ/lodestone-test-docs repository.

Configuration

Lodestone follows a [Convention over Configuration](https://en.wikipedia.org/wiki/Convention_over_configuration design, which means that it works out of the box with sane defaults, but you can customize them to match your needs.

Most of the configuration files are stored in the webappimage, and requested by various components when they start up.

  • filetypes.json contains lists of includes and excludes that are used by the processor container to decide which files to process and load into the database.

  • tags.json contains a nested structure of labels that can be used to group tags and seach for your documents in the Lodestone web UI.

  • mapping.json is used to ensure that the elasticsearch container has a consistent data storage structue.

To overide these files, just setup a Docker volume binding to the specified file in the /lodestone/data/ directory in the webapp container.

Considerations

Lodestone is a very opinionated solution for personal document management. As such, there's a couple things you should know before even considering it.

  • Currently there's no user management. Lodestone is designed to run at home, on your trusted network. This may be reconsidered at a future date.

  • Limited support for file types

    • doc,docx,xls,xlsx, ppt, pptx - Microsoft Office Documents

    • pages, numbers, key - Apple iWork Documents

    • pdf

    • rtf

    • jpg, jpeg, png, tiff, tif

      If you think there are additional document types that may be useful to support, please open an issue.

What about..

As mentioned above, Lodestone isn't some magical new technology. EDMS and DMS systems have been around for a long time, but unfortunately they all seem to miss one or more features that I think are required for a modern filing cabinet.

Here's some of my research, but you should take a look at them yourselves.

Name Docker/Linux Web UI Modern UI Tagging Non-destructive OCR Watch Folder Email Import
MayanEDMS
Paperless

Contributing

If you're interested in contributing, please fill out the following Google Form:

https://forms.gle/u1RXnbocbFWqfxGb9

# run the following command in 1 terminal
docker-compose -f docker-compose.dev.yml up --force-recreate --build

# and the following in a different terminal
cd web/frontend && ng serve --configuration=dev

Place your documents in the /data/storage/documents directory, and the Filesystem Collector should automatically start processing them.

If you would like some test documents to play with safely, you can take a look at the AnalogJ/lodestone-test-docs repository.

find . -exec touch {} \;

Components

Name Software Version Docker Image
Elasticsearch Elasticsearch v7.2.1
Filesystem Publisher Go
Document/Thumbnail Processor Go
Web / Api Angular v8.x / ExpressJS v4.16
Storage minio 2019 (S3 compatible)
Queue RabbitMQ
OCR Tika

Future Development

Here's some of the feature's that we have targeted for future development. Check the Lodestone Project for in-progress development status.

  • Additional file types
  • Optional tag synchronization to cloud storage providers (Dropbox, Google Drive, etc)
  • Metadata Backup/Export system
  • (In-Progress) Email Collector (send emails to a customizable email address, and automatically parse into lodestone).
  • User management system

Logo

About

Personal Document Archiving (DMS, EDMS for Personal/Home Office use)

https://forms.gle/u1RXnbocbFWqfxGb9

License:GNU General Public License v3.0


Languages

Language:CSS 74.7%Language:TypeScript 11.6%Language:HTML 6.6%Language:JavaScript 5.6%Language:Dockerfile 1.2%Language:Shell 0.4%