DiscordChatExporter-frontend (DCEF)

Browse your Discord chat exports in a single discord like user interface. Try a demo in your browser :).

Features

browse multiple servers, channels, threads and forum posts in a single web based user interface
search messages in a server with autocomplete
optimized to handle really large exports well
customizability - change font and many other things in settings
self hosting option - you can host DCEF on your own server and share it with your friends. Web interface is mobile friendly too

Want to chat? Join Tyrrrz's discord server. I usually hang out in #dce-frontend channel

Quick start (Windows)

Export your data from Discord using DiscordChatExporter. Your export should be in JSON format, disable format markdown option and enable download assets and reuse assets options. If exporting using a CLI - use --media --reuse-media --markdown false --format Json command line options. Include threads and forum posts by adding --include-threads All option. More information in How to export data from Discord to view it in DCEF?
Download the latest release from releases page
Extract the archive
Move all your DiscordChatExporter exports to /exports/ folder (supported exports).
Run dcef.exe

Quick start (Linux)

Docker version is the best way to host the viewer on a server for others to use.

Export your data using DiscordChatExporter. Use --media --reuse-media --markdown false --format Json command line options.
Pull the image from docker hub

docker pull slada/dcef:main

Navigate to a folder with your exports

cd /path/to/your/exports

Run the container

docker run --restart=always --volume "$(pwd):/dcef/exports" --volume dcef_cache:/dcef/cache --name dcef -p 21011:21011 -it slada/dcef:main

Open http://127.0.0.1:21011/ in your browser

Note: arm based systems like Raspberry Pi or Apple M1 are not officially supported. Pull requests are welcome :)

Edit server configuration

If you want to select which discord servers are shown in the viewer or which users are hidden, you need to edit server configuration. Changes in this configuration are applied immediately - you don't need to restart DCEF. This configuration is enforced server-side on all users of the viewer.

Windows

run dcef.exe and wait for the user interface to appear
run configurator.exe
follow the instructions. Select actions by typing number and pressing enter.

Linux

dcef container needs to be running. Then run:

docker exec -it dcef /usr/bin/python3.11 /dcef/configurator.py

Other ways to run DCEF

Windows beta builds

If you want to try out the latest features, you can use beta builds. They are automatically built from the latest commit on main branch. You need to be logged in to Github account to download them.

Build docker image from source code on x86_64 architecture

You need docker and git installed. Then run:

git clone https://github.com/slatinsky/DiscordChatExporter-frontend
cd DiscordChatExporter-frontend
docker build -t dcef .

Then use the same instructions as for the Linux docker version, but replace in commands slada/dcef:main with dcef.

Build docker image from source code on Raspberry pi 4b

You need docker and git installed. Then run:

git clone https://github.com/slatinsky/DiscordChatExporter-frontend
cd DiscordChatExporter-frontend
docker build -t dcef -f Dockerfile.rpi4b

Then use the same instructions as for the Linux docker version, but replace in commands slada/dcef:main with dcef.

Upgrade guide

Upgrade windows binary release

Want to upgrade from previous version? Follow these steps:

Download the latest release from releases page.
Delete everything (except exports folder) in your discordchatexporter-frontend folder.
Move everything (except exports folder) from the new release to your discordchatexporter-frontend folder.

Upgrade docker image

cd path/to/your/exports/
docker rm dcef --force
docker image rm slada/dcef:main
docker pull slada/dcef:main
docker run --restart=always --volume "$(pwd):/dcef/exports" --volume dcef_cache:/dcef/cache --rm --name dcef -p 21011:21011 -it slada/dcef:main

Uninstall

Windows (binary release)

DCEF does not create any files outside of its folder, so you can just delete the folder to uninstall it.

Move your exports folder somewhere else if you want to keep your exports.

Linux (docker)

kill and delete the container

docker rm dcef --force

remove the volume

docker volume rm dcef_cache

remove the image

docker image rm slada/dcef:main

How to export data from Discord to view it in DCEF?

JSON exports are created using DiscordChatExporter.

JSON export format (--format Json) is required

Skipping markdown prerendering (--markdown false) is highly recommended, but not required

Downloading assets (--media --reuse-media) is highly recommended, but not required

Including threads (--include-threads All) is recommended for a full server export

Incremental export using DiscordChatExporter-incrementalBackup wrapper

I have created simple wrapper for DiscordChatExporter to simplify the process of incremental export for this frontend viewer. You declare what you want to export in a config file and the wrapper will automatically export only new messages since the last export.

For more information and instructions how to use, see slatinsky/DiscordChatExporter-incrementalBackup

Full export using GUI version of DiscordChatExporter

Make sure that export format is set to JSON and Format markdown is disabled. Optionally, you should also enable Download assets+Reuse assets option to download images, videos and other types of assets.

Recently Show threads option was added to the general settings of DiscordChatExporter - if you want to export threads. Note that fetching threads is slow and it may take a while to show them in the GUI.

Note: Exporting using CLI version of DiscordChatExporter is actually easier if you want to mass export everything.

Full server export using CLI version of DiscordChatExporter

Export all accessible channels, threads and forum posts in a server:

DiscordChatExporter.Cli.exe exportguild --token DISCORD_TOKEN -g SERVER_ID --media --reuse-media --markdown false --format Json --include-threads All --output OUTPUT_FOLDER_PATH

Export private messages using CLI version of DiscordChatExporter

Export all dms (sadly, exporting dms can't be done without selfboting):

DiscordChatExporter.Cli.exe exportdm --token DISCORD_TOKEN --media --reuse-media --markdown false --format Json --output OUTPUT_FOLDER_PATH

FAQ

No servers show up / troubleshooting steps

See logs (Windows dcef/logs.txt, Linux docker logs dcef) for more info.

[Windows only] Please check, if the top of the log contains line windows-runner: OK: All required ports are available.. DCEF needs ports 21011, 21013, 27017, 58000 to be available. If you have any of these ports occupied, DCEF won't start or won't work properly.

[All platforms] Find line found X json channel exports - if this number is 0, you don't have any valid exports in /exports/ folder.

[All platforms] Find if there is line preprocess done (at the end of lines processing <PATH_TO_JSON_FILE>) - if you see this line, preprocessing step finished successfully. If you don't see this line, please check if there is any stacktrace in the logs. Stacktrace looks like this:

Traceback (most recent call last):
  File "dcef/backend/preprocess/main_mongo.py", line 82, in <module>
    main(input_dir, output_dir)
  File "dcef/backend/preprocess/main_mongo.py", line 67, in main
    raise Exception("Example stack trace exception")
Exception: Example stack trace exception

Please report this issue with the stacktrace attached.

[Windows] If no other solution works for you, run Docker version of DCEF.

Some assets won't diplay on Windows / Windows path length limit

Files in /exports/ folder may exceed Windows path length limit of 260 characters. If you have any issues with loading your assets you can choose one of the following solutions:

move DCEF to a folder with shorter path
or run registry_tweaks/change_260_character_path_limit_to_32767.reg to increase the limit to 32767 characters (requires admin privileges) and restart your computer. To revert this change, run registry_tweaks/restore_260_character_path_limit.reg and restart your computer.

Assets won't display after moving them / how to clear cache

After you put your export to /exports/ folder, don't remove them. DCEF keeps track of assets and if you remove or move them, they won't show up in DCEF, because the old path would become invalid.

TL;DR - only adding new files to /exports/ folder is supported. If you want to remove or move files, you need to clear cache afterwards.

Clearing cache on Windows:

close DCEF
delete dcef/backend/mongodb/db folder
start DCEF again

Clearing cache on Linux:

kill DCEF container
remove dcef_cache volume (docker volume rm dcef_cache)
start DCEF container again

DCEF hangs and prints `Slow SessionWorkflow loop` to the console / long preprocessing time

Slow SessionWorkflow loop messages are completely normal - if you see them, you know that data is pushed to mongodb database and the process is not stuck. Just be patient and wait for the process to finish. If you have a lot of exports, it may take a while.

DCEF is not just an simple viewer. This process enriches your exports with additional data and stores them in a database for search and other features to work.

Impatient? Navigate to http://127.0.0.1:21011/ in your browser to see already processed exports.

DCEF won't run on M1 mac

This pull request may help you

DCEF is detected as a malware

DCEF is not a malware. It's a false positive. The project is open source, you can check the source code yourself.

The windows release is exactly the same as the ones built on Github's servers by github actions. Executables (nginx.exe, mongod.exe) are sourced from their official websites. I upload releases manually, but the zip is exactly the same as the last successful build on github actions.

Docker image is built on Github's servers github actions github actions too. This action directly uploads the image to docker hub.

But the project uses a lot of dependencies - it is susceptible to supply chain attack such as dependency hijacking. If you find anything suspicious in dependencies used, please let me know.

Tinfoil hat on? Replace nginx.exe, mongod.exe, msvcp140.dll and vcruntime140_1.dll with your own trusted copy. Then compile your own version from source code. The easiest way to compile is to run github action on your own fork.

Related discussion #13

For developers

Architecture choices

Setting up a development environment on Windows

Compile the Windows release from source code

Supporting other exporters

Thanks

Tyrrrz/DiscordChatExporter - for a great export tool
brussell98/discord-markdown - for discord markdown rendering library
themattman/mongodb-raspberrypi-binaries - unofficial mongodb builds for Raspberry Pi

And for other technologies used in this project - sveltekit, docker, nodejs, nvm, pyinstaller, nginx, mongodb

Related projects

Tyrrrz/DiscordChatExporter - exporter for your Discord chats
Roachbones/discordless - real time man-in-the-middle exporter
mlomb/chat-analytics - analytics for your Discord chats
slatinsky/DiscordChatExporter-incrementalBackup - incrementally export your Discord chats for DiscordChatExporter-frontend

License

GNU GENERAL PUBLIC LICENSE. See LICENSE for more details.

This product contains software provided by NGINX and its contributors.

DiscordChatExporter-frontend is not affiliated with Discord. Discord is a registered trademark of Discord Inc.

Contributing

Feel free to open issues and pull requests.

Short guide, how to contribute

Fork the repository
Create a new branch
Implement your changes
Commit and push your changes
Create a pull request

If you find this project useful, give it a star ⭐. Thank you!

tilderain / DiscordChatExporter-frontend