[RLM_ERR_INVALID_DATABASE] Realm cannot be decrypted after a migration from java-sdk to kotlin-sdk
brnmr opened this issue · comments
How frequently does the bug occur?
Always
Description
We've recently decided to migrate from java-sdk to to the kotlin-sdk and when the app is first started right after the migration, the encrypted realm file cannot be opened with the same key that has been used to encrypt the realm prior to the migration to the kotlin-sdk. The error we're getting is as follows:
[RLM_ERR_INVALID_DATABASE]: Failed to open Realm file at path '/data/user/0/com.example.app/files/default.realm': Realm file decryption failed (Decryption failed: 'unable to decrypt after 0 seconds (retry_count=4, from=i != bytes_read, size=4096)')
Here are details about the SDK versions:
// Java-sdk version prior to migration
implementation("io.realm:realm-gradle-plugin:10.11.1")
// Kotlin-sdk version after the migration
implementation ("io.realm.kotlin:library-base:1.11.0")
We've verified that the key used for the realm config is the same before and after the migration so there should be no issue with the key itself. In addition, we did tests with unencrypted realm and in this case we don't have issues with the migration.
In addition, we tried to open the realm file using Realm Studio by providing the encryption key. We get warning that the realm file is in an outdated format which I'm not sure is related to the issue but after choosing the Backup and upgrade
option, we are able to decrypt the realm database and see the content - everything is in there.
![Screenshot 2024-03-20 at 17 24 57](https://private-user-images.githubusercontent.com/7135956/314560891-0d183c81-424f-4a60-96f2-176b057bd0de.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MTk4NTEzNjksIm5iZiI6MTcxOTg1MTA2OSwicGF0aCI6Ii83MTM1OTU2LzMxNDU2MDg5MS0wZDE4M2M4MS00MjRmLTRhNjAtOTZmMi0xNzZiMDU3YmQwZGUucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDcwMSUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA3MDFUMTYyNDI5WiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9ZThiNDdmMzMwMTNkNzI2MTc3MWFhNGExODBlMDQ4NzRmNzg4NTgzMTQ1NmViMGQ2YmFmNmFhYjBmMjVjMjhiMyZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ._RCFbHFJGD93DBx1cvGGaznHBUElgq1JEGGVZgFF-n0)
Here is the code for generating the realm config:
private fun initRealmConfig(): RealmConfiguration? {
// Increment whenever you made changes to the Realm* database models
val schemaVersion: Long = 2
// Get encryption key
val realmKey = DBHelper.loadEncryptionKey(App.applicationContext())
realmKey?.let {
val realmConfig = RealmConfiguration.Builder(getSchemaClasses())
.schemaVersion(schemaVersion)
.migration(DBMigrationHelper.migrate())
.encryptionKey(it)
.build()
// Once we've used the key to generate a config, erase it in memory manually
Arrays.fill(realmKey, 0.toByte())
return realmConfig
}
return null
}
Stacktrace & log output
[RLM_ERR_INVALID_DATABASE]: Failed to open Realm file at path '/data/user/0/com.example.app/files/default.realm': Realm file decryption failed (Decryption failed: 'unable to decrypt after 0 seconds (retry_count=4, from=i != bytes_read, size=4096)')
Can you reproduce the bug?
Always
Reproduction Steps
- Run app version with java-sdk
- Migrate to kotlin-sdk
- Update the installed app version in step 1 with the migrated version
Problem
- Realm decryption fails
Version
1.11.0
What Atlas App Services are you using?
Local Database only
Are you using encryption?
Yes
Platform OS and version(s)
Android
Build environment
Android Studio version: Hedgehog | 2023.1.1 Patch 2
Android Build Tools version: com.android.tools.build:gradle:7.2.2
Gradle version: gradle-7.4-bin
➤ PM Bot commented:
Jira ticket: RKOTLIN-1047
Hi guys,
After further debugging I figured out that this line in the initialization of the RealmConfigration is causing the issue. I still have no clue why since this was working fine in the java-sdk and was taken from the official documentation here: https://www.mongodb.com/docs/realm/sdk/java/realm-files/encryption/
This is the line I am referring to:
// Once we've used the key to generate a config, erase it in memory manually
Arrays.fill(realmKey, 0.toByte())
So, when I delete this line and update to the app version with the migrated java-sdk to kotlin-sdk, the database is working fine and can be decrypted without issues.
Any ideas?
Hi @brnmr. The encryption key is actual not copied as part in the Kotlin RealmConfiguration, so clearing the data inbetween creating the configuration and opening the realm will cause the Realm.open
to use the cleared key. The key is also used to asynchronously open some internal background instances of the realm which causes it to be a bit tricky when the memory can be cleared, as the asynchronous operations are not guaranteed to happen before Realm.open
returns. The only safe way to know this at the moment is to wait for initialization of the internal realms by forcing an update through an empty transaction with:
realm.write { }
realm.asFlow().filterIsInstance<UpdatedRealm<*>>().first()
We are working on a fix to this with a callback to provide the key and a callback when it is safe to clear the key again. This should ensure that the key is in memory as little as possible. The progress of that can be track in #1636