Worteks / RC2Matrix

Rocketchat to Matrix migration script

RC2Matrix

Rocketchat to Matrix/Synapse migration script. Workflow is largely inspired by https://git.verdigado.com/NB-Public/rocketchat2matrix.

*** WARNING : THIS IS WIP, USE AT YOUR OWN RISKS ***

Exporting RocketChat data

Currently, only public data should be exported. ./mongo_exportpublic.sh <dbname> <dbuser> <dbpass> must be run on the mongo server and will use mongoexport, mongosh and mongofiles to extract necessary data (all users, public rooms, messages in these public rooms only, attachments mentionned in these messages only, avatars of rooms and users).

Exporting private data (private chats, DMs) can be manually done via mongodb. Run the following on the server:

mongoexport --collection=rocketchat_message --db=<dbname> --out=rocketchat_messages.json -u <dbuser> -p <dbpass>  --sort='{ts: 1}'
mongoexport --collection=rocketchat_room --db=<dbname> --out=rocketchat_rooms.json -u <dbuser> -p <dbpass>
mongoexport --collection=users --db=<dbname> --out=rocketchat_users.json -u <dbuser> -p <dbpass>

Optionnaly, if you manually extracted private data, you can run ./mongofiles_exportall.sh <dbname> <dbuser> <dbpass> to export files and images. It will export files in a temporary directory, printed at the end of the script. This part is automatically done for public data during mongo_exportpublic.sh.

Please note that current features are only tested with the mongo_exportpublic.sh script focusing only on public data. Although most of the migration steps should also work with private data, it is currenlty not tested nor used.

Preparing Synapse server

On the Synapse server you need an admin account (user/pass) or directly its token.

You also need an application service. To register an application service, add the following to homeserver.yaml :

app_service_config_files:
- /your_path/rc2matrix.yaml

rc_joins:
  local:
    per_second: 1024
    burst_count: 2048
rc_joins_per_room:
  per_second: 1024
  burst_count: 2048
rc_message:
  per_second: 1024
  burst_count: 2048
rc_invites:
  per_room:
    per_second: 1024
    burst_count: 2048
  per_user:
    per_second: 1024
    burst_count: 2048
  per_issuer:
    per_second: 1024
    burst_count: 2048

You also need to create a rc2matrix.yaml (you need to customize the two tokens) :

url: null
as_token: ASecretASToken
hs_token: ASecretHSToken
id: rc2matrix
sender_localpart: rc2matrix
namespaces:
  users:  # List of users we're interested in
    - exclusive: false
      regex: ".*"
  aliases:
    - exclusive: false
      regex: ".*"
  rooms:
    - exclusive: false
      regex: ".*"

Running RC2Matrix

If you do not have your admin token, you can obtain it with ./rc2matrix.py -v -n <your matrix hostname> -u <user_admin> -p <pass_admin>. Token will be printed on the console.

Then, to import rooms, users and messages into Synapse : ./rc2matrix.py -v -n <your matrix hostname> -t <admin_token> -a <ASecretASToken> -i <your folder containing the exports> (the ASecretASToken is defined in rc2matrix.yaml).

In your folder containing the exports, you should have rocketchat_messages.json, rocketchat_rooms.json and rocketchat_users.json files, as well as a folder files/ containing the exported files.

You can remove -v for less verbose output, or add -k to not check certificates.

How does it work ?

First, the data exported from RC is three JSON files containing rooms, users, and messages. Contents are described on the RC website (rooms, messages). Files are not really JSON, but each line is a valid JSON, which can thus be processed sequentially.

On the Synapse side, we use both the Matrix client-server API, with the admin and application service account, and the Synapse Admin API, with the admin account. Some operations (such as creating rooms) are only authorized to the admin user, whereas some others (such as altering the timestamps of the messages or masquerading identities) are only authorized to the application service, hence these two needed tokens.

First the rooms are created (or retrieved if already existing). Then, users are added, without authentication method : they will have to authenticate through an external system. Finally, messages are posted on behalf of these users. Rooms settings (public, private, DM) should be quite similar to RC settings but there may be some unexpected cases. DM messages appear in dedicated rooms.

This script currently imports messages, threads, reactions, emojis and avatars in order to provide a usable migration path. Advanced formatting is only partially handled.

While the RC data will not be altered (there is just an export), Synapse data will obviously be altered. You should not run this script against an already used server, as it may have unexpected issues. You should run this script against a fresh Synapse server and carefully check the result.