Cannot access database after upgrading v3 to v5 (self host to docker) --> No account found using this email.

[homeusepb]

Version

5.0.4, 4.0.1, 3.4.2 (docker)

Details & Steps to reproduce

Hi! As per my discussion here (#318), I think this is now a bug not a discussion? Correct me if I am wrong I can close one. Also this issue seems related to (#278), except that user had a JSON backup but I am coming from v3 which had no backup ability.

After trying everything, I am still getting the error "No account found using this email." after trying to login, effectively I cannot access my data.

I had hosted 2fauth v3 (can't remember exact version but it was around 2022 I did) and didn't update it because once I did and I had issues with logo migrations so left it as is. Today I tried to finally move from old v3 self hosted to Docker v5 and I am unable to login.

1. Make a new directory in my /home/sydauhmsrv/DOCKER/CONTAINERS/2fauth. Directory /home/sydauhmsrv is owned by 1000 and has permission 770 infact. Same goes for /DOCKER/CONTAINERS. I have then set permission 2fauth to have 700 and 1000:1000 ownership. I made another directory now called "2fauth_data" (inside the main 2fauth) folder and set 1000:1000 with 700 permission.

2. Make a docker-compose.yml file with following information

version: "3"
services:
  2fauth:
    image: 2fauth/2fauth:latest
    container_name: 2fauth
    volumes:
      - /home/sydauhmsrv/DOCKER/CONTAINERS/2fauth/2fauth_data:/2fauth
    ports:
      - 8150:8000/tcp
    restart: unless-stopped

I have tried using the full path as above for the left host side and also the relative "./2fauth_data:/2fauth", but both have same issue.

3. Copy my "/var/www/2fauth/database" directory to "/home/sydauhmsrv/DOCKER/CONTAINERS/2fauth/2fauth_data/database" and also the "/var/www/2fauth/storage" directory to "/home/sydauhmsrv/DOCKER/CONTAINERS/2fauth/2fauth_data/storage". Then I also copied "/var/www/2fauth/database/database.sqlite" to "/home/sydauhmsrv/DOCKER/CONTAINERS/2fauth/2fauth_data/database.sqlite" since I believe docker uses the symbolic link directly to "./2fauth_data" without considering the whole database folder (but I copied that nonetheless).

4. I start up my docker-compose up, and I get 10-15 database migration messages all saying OK DONE. I also see this message "
   2fauth | Encryption keys already exist. Use the --force option to overwrite them.
   2fauth | Personal access client created successfully"

5. I assume this means it has found my existing database and keys? So then when I login to 2fauth GUI, I get the error "No account found using this email."

EDIT: I also tried to use "sqlite3" bash to SELECT * from users;. In the raw database from my /var/www/2fauth/database/database.sqlite, it correctly shows my user. In the new migrated database, it doesn't show my existing user at all, so somewhere while migrating my user is getting removed?

Expectation

Expected 2fauth is migrate properly and let me login to my existing database and not show "No account found using this email.".

Error & Logs

[2024-03-09 04:59:02] local.DEBUG: ReleaseRadarActivated event dispatched  
[2024-03-09 04:59:02] local.INFO: Release scan started  
[2024-03-09 04:59:03] local.NOTICE: App setting 'lastRadarScan' set to 1709960343  
[2024-03-09 04:59:03] local.NOTICE: App setting 'latestRelease' reset to default  
[2024-03-09 04:59:03] local.DEBUG: ReleaseRadarActivated event dispatched  


Do not see anything abnormal here.

Execution environment

Date: Sat, 09 Mar 2024 05:00:33 +0000
userAgent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36 Edg/122.0.0.0
Version: 5.0.4
Environment: local
Install path: /
Debug: true
Cache driver: file
Log channel: daily
Log level:
DB driver: sqlite
PHP version: 8.1.22
Operating system: Linux
interface: fpm-fcgi

Containerization

• Docker

Additional information

No response

[homeusepb]

Adding some extra information, I have just browsed the NEW database constructed by 2fauth docker-compose, it seems some tables are empty. The users table is fully empty, the twofaccounts table is empty and groups table is empty and maybe few others. So I am sure that the migrations are not happening properly.

I have manually modified the sqlite table myself and fixed the users table, which allowed me to login but I also need to copy over the "twofaccounts" table but that seems to not be working maybe it is encrypted? How can I copy over twofaccounts? I would assume copying the encrypted "account", "legacy_uri" and "secret" column off the table would still work but it doesn't.

[Bubka]

Hi,
this is not really a bug as this use case is not really covered (migrating from a selfhosted server to Docker). But anyway, let discuss it here.

Using Docker, the /2fauth directory on the host side should contain:

• The database.sqlite file
• The storage folder
• An installed file which contain a commit hash.

During the update process, this hash is compared to the commit hash used to build the image. If the hashes are different then db is updated by executing missing migrations. But if the installed file is missing then the whole db is reset and all migrations executed. I guess this is what happened in your case.

So try this:

• Copy your legacy db file to /2fauth. Its name must be database.sqlite
• Copy your storage folder to /2fauth
• Do not copy the legacy database folder to /2fauth. (this folder exists in the image)
• Create a file named installed which contain b4fe966 (this is the v3.4.1 hash) alongside these files.
• Start the container

You should see this line in Docker output:

Installed commit b4fe966 is different from program commit 342448b we are migrating...

(not sure about the 342448b ref, what is important is that you see the line)
After this line, I think you will see only one migration line (for 2023_12_06_131842_add_oauth_columns_to_user), not 10-15 as previously. You should then be able to sign in.

One more thing. Do not forget to pass your APP_KEY via docker-compose env vars if your db is encrypted

[homeusepb]

Hi!

I see, thanks for your help and it is now clearer.
However, I got impatient on Sunday ha-ha and I sort of used my own hacks. I just ended up opeing the sqlite database in a database reader and copied over the missing data and new user_id columns and it now seems to work all okay.

However the instructions above should help someone if they come this route!

Thank you for your help and wonderful app, wish you all the best!