Emojis in wiki pages not working when editing page

[ilpssun]

When I save a page and insert an emoji (e. g. „😃“), saving works fine for the first time. Loading the page then shows the correct emoji. When I edit the page, however, the emojis is broken and only shown as ?. When saving the page again, the emoji is also broken on the page. This is rather annoying. It seems to me that it’s not a fundamental problem in the database/architecture but rather a problem of the code that prepares the page for editing.

[Jacques Distler]

Hmm. Works for me, both with the default sqlite3 database and with mysql.

Details (both tested under Ruby 2.6):

production:
  adapter: sqlite3
  database: db/production.db.sqlite3

production:
  adapter: mysql
  database: XXXXX
  username: XXXXX
  password: XXXXXXX
  host: 127.0.0.1
  port: 3306
  encoding: utf8mb4

Note: old versions of MySQL, with the utf8 encoding have problems with Astral Plane characters. Recent versions use utf8mb4 by default, but it never hurts to make sure.

[ilpssun]

I will investigate later when I have access to my system again. As far as I recall, I set the encoding explicitly to utf8 because with utf8mb4 the migrations were producing errors about the constraint names being too long on MariaDB. Can that be true?

[Jacques Distler]

The 'utf8' encoding will definitely mangle astral-plane characters. If that's what you did, then then the problem is entirely one of your own creation.

I have not tried MariaDB, but I would have thought it was otherwise compatible with MySQL (which has no problem, AFAICT, with the migrations.

[ilpssun]

Yes, the wrong encoding was the issue. It was, however, not a straightforward fix. In the default configuration, converting the table schema_migrations fails with the error Specified key was too long; max key length is 767 bytes.

The fix was to use the following config for the MariaDB server:

[mysqld]
innodb_large_prefix=on
innodb_file_format=Barracuda

Also, I ensured that the table had ROW_FORMAT=DYNAMIC.

After that, a conversion to utf8mb4 was successful.

[Jacques Distler]

The fix was to use the following config for the MariaDB server ...

As always, some instructions on the wiki would probably help out some future users.