sam-37 / filemanager-fastapi

Handle files? Ok.

Filemanager-Fastapi

Blazing fast filemanager using fastapi.

About The Project

Long story short, I needed microservice that would manage the files, so I ended up with writing Filemanager-Fastapi (FF) and I am not even complaining. Hope you will be able to use it in concrete needs. Have fun, and of course prs are welcome.

Here's what features FF has at this time:

• Uploading image file/files

• Downloading image file/files, from any image url

• Image file/files optimization/converting using Pillow-SIMD or FFMPEG

  • You can have both installed or you can choose any of engines depending your needs

• Uploading video file

• Downloading video file, from any video url

• Video file optimization/converting using FFMPEG

  • change in .env INSTALL_FFMPEG=false to INSTALL_FFMPEG=true
  • Then run

  docker-compose build api

• The bonus one Qrcode generator

• Uploading to local, Google Storage, or S3

• Live reloading on local development

• Self cleaning (temp files, Pillow-SIMD, FFMPEG)

• Serving files from local storage

  • The path starts from static folder for example:

  http://localhost/static/pictures/original/dcb8ac79618540688ea36e688a8c3635.png
  

• Easy Security using Bearer token

  • For security its recommended to make requests from your backend server, not from browser, as your key of FF can be tracked.

• Out of box documentation thanks to fast-api [/docs && /redoc paths are available]

• SSL secured reverse nginx proxy using gunicorn and uvloop

May optimize it a little therefore FF already is really fast, try by yourself :)

Built With

• FastAPI
• Docker
• pillow-simd
• FFMPEG
• Libcloud

Getting Started

To get a local copy up and running follow these simple example steps.

Before you start steps below make sure that you have created all .env files, .env-example s are provided.

Installation for local development

• Fast reloading included

1. Change docker-compose.dev.yml to docker-compose.yml

docker-compose build api

3. Start the service

docker-compose up

4. Enter your API at localhost/docs

5. Now you should be able to see the open api endpoints.

• Don't forget to authorize with FILE_MANAGER_BEARER_TOKEN that you should generate and paste in .env file.
  • If you forgot to generate FILE_MANAGER_BEARER_TOKEN here is one of the ways to generate secret key

    • openssl rand -base64 64

    • p.s You can create as much tokens as you want just separate them with ,

Installation for docker swarm

• If you’re trying things out on a local development environment, you can put your engine into swarm mode with docker swarm init.
• If you’ve already got a multi-node swarm running, keep in mind that all docker stack and docker service commands must be run from a manager node.

Set up a Docker registry

• Start the registry as a service on your swarm:

docker service create --name registry --publish published=5000,target=5000 registry:2

3. Build the image locally with docker-compose

• Decide wich docker-compose you are going to use there is .dev and .prod, when you decide rename to docker-compose.yml and continue

docker-compose build

4. Bring the app down

docker-compose down --volumes

5. Push the generated image to the registry

docker-compose push

6. Deploy the stack to the swarm

docker stack deploy --compose-file docker-compose.yml filemanager-fastapi

7. Check

docker stack services filemanager-fastapi

8. Result

ID                  NAME                      MODE                REPLICAS            IMAGE                                       PORTS
kkk5mmkgk6zf        filemanager-fastapi_api   replicated          1/1                 127.0.0.1:5000/filemenager-fastapi:latest   *:80->80/tcp

Docker Hub image

Available at (https://hub.docker.com/r/gujadoesdocker/filemanager-fastapi)

You need A+ ssl?

• No problem Filemanager-Fastapi comes with nginx and certbot configuration that guarantees A+ ssl.

• If you will need help let me know.

Size after development [with ffmpeg and pillow-simd]

• command

$ docker ps --size

Result

NAMES                         SIZE
filemanager-fastapi_nginx_1   126B (virtual 28.1MB)
filemanager-fastapi_api_1     310B (virtual 1.12GB)

FFMPEG 4

• If you like to use ffmpeg in your docker .env file change INSTALL_FFMPEG=false to INSTALL_FFMPEG=true
• Don't forget to change api .env IMAGE_OPTIMIZATION_USING to ffmpeg.
• LTS version at this time:

ffmpeg version 4.1.6-1~deb10u1 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 8 (Debian 8.3.0-6)
  libavutil      56. 22.100 / 56. 22.100
  libavcodec     58. 35.100 / 58. 35.100
  libavformat    58. 20.100 / 58. 20.100
  libavdevice    58.  5.100 / 58.  5.100
  libavfilter     7. 40.101 /  7. 40.101
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  3.100 /  5.  3.100
  libswresample   3.  3.100 /  3.  3.100
  libpostproc    55.  3.100 / 55.  3.100

• enjoy :)

Contributing

Contributions are what make the open source community such an amazing place to be learn, inspire, and create. Any contributions you make are greatly appreciated.

1. Fork the Project
2. Create your Feature Branch (git checkout -b feature/AmazingFeature)
3. Commit your Changes (git commit -m 'Add some AmazingFeature')
4. Push to the Branch (git push origin feature/AmazingFeature)
5. Open a Pull Request

Image optimization result

• Original

• Thumbnailed usind pillow-SIMD

• Thumbnailed usind FFMPEG

Generated Qrcode example

It is possible to edit configuration easily to generate Qrcode image on your needs with or without logo size you need color and etc. (For generating image is used Pillow-simd)

License

Distributed under the MIT License. See LICENSE for more information.

Contact

Twitter - @guja_py